- 2002 Views
- 3 replies
- 3 kudos
Access file location problem
import pandas as pd from apiclient.discovery import build from oauth2client.service_account import ServiceAccountCredentials df = spark.read.json("/FileStore/tables/cert.json") SCOPES = ['https://www.googleapis.com/auth/analytics.readonly'] KEY_FIL...
- 2002 Views
- 3 replies
- 3 kudos
- 3 kudos
Looks like it is because the oauth2client.service_account does not know about DBFS (whereas spark does).Is it an option to manage your secrets in databricks? https://docs.databricks.com/security/secrets/secrets.html
- 3 kudos
- 6411 Views
- 7 replies
- 2 kudos
Resolved! Cannot display DataFrame when I filter by length
I have a DataFrame that I have created based on a couple of datasets and multiple operations. The DataFrame has multiple columns, one of which is a array of strings. But when I take the DataFrame and try to filter based upon the size of this array co...
- 6411 Views
- 7 replies
- 2 kudos
- 2 kudos
strange, works fine here. what version of databricks are you on?What you could do to identify the issue is to output the query plan (.explain).And also creating a new df for each transformation could help. Like that you can check step by step where...
- 2 kudos