What is the best way to use credentials for API calls from databricks notebook?

lugger1
New Contributor III

Hello, I have an Databricks account on Azure, and the goal is to compare different image tagging services from Azure, GCP, AWS via corresponding API calls, with Python notebook. I have problems with GCP vision API calls, specifically with credentials: as far as I understand, the one necessary step is to set 'GOOGLE_APPLICATION_CREDENTIALS' environment variable in my databricks notebook with something like

os.environ['GOOGLE_APPLICATION_CREDENTIALS'] ='/folder1/credentials.json'

where '/folder1/credentials.json' is the place my notebook looks for json file with credentials (notebook is in the same folder, /folder1/notebook_api_test).

I am getting this path by looking into Workspace-> Copy file path in the Databricks web page. But this approach doesn't work, when cell is executed, I am getting this error:

DefaultCredentialsError: File /folder1/credentials.json was not found.

What is the right way to deal with credentials to access google vision API from Azure Databricks notebook?