Hi Hubert,

Thank you for your quick response! I've copied your code across to my notebook. However, when I run the following code

%sh
/dbfs/databricks/scripts/selenium-install.sh

I get the following output

Hit:1 https://repos.azul.com/zulu/deb stable InRelease
Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease
Hit:3 http://archive.ubuntu.com/ubuntu focal InRelease
Hit:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
E: Version '91.0.4472.101-0ubuntu0.18.04.1' for 'chromium-browser' was not found
/dbfs/databricks/scripts/selenium-install.sh: line 5: --yes: command not found
--2022-11-03 13:02:23--  https://chromedriver.storage.googleapis.com/91.0.4472.101/
Resolving chromedriver.storage.googleapis.com (chromedriver.storage.googleapis.com)... 209.85.202.128, 2a00:1450:400b:c01::80
Connecting to chromedriver.storage.googleapis.com (chromedriver.storage.googleapis.com)|209.85.202.128|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2022-11-03 13:02:24 ERROR 404: Not Found.
 
/dbfs/databricks/scripts/selenium-install.sh: line 7: chromedriver_linux64.zip: command not found
mkdir: invalid option -- 'd'
Try 'mkdir --help' for more information.

And consequently, when I run this code block:

from selenium import webdriver
chrome_driver = '/tmp/chromedriver/chromedriver'
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--headless')
# chrome_options.add_argument('--disable-dev-shm-usage')
chrome_options.add_argument('--homedir=/dbfs/tmp')
chrome_options.add_argument('--user-data-dir=/dbfs/selenium')
# prefs = {"download.default_director":"/dbfs/tmp",
#          "download.prompt_for_download":False
# }
# chrome_options.add_experimental_options("prefs",prefs)
driver = webdriver.Chrome(executable_path=chrome_driver, options=chrome_options)

I receive the following error:

WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://chromedriver.chromium.org/home

Is this something you can shed some light on for me please?

Thank you for your help!