- 5301 Views
- 2 replies
- 2 kudos
How to read a file in pandas in a databricks environment?
Hi, When I was trying to read the CSV files using pandas I am getting an error which I have mentioned below.df=pd.read_csv("/dbfs/FileStore/tables/badrecord-1.csv")Error: FileNotFoundError: [Errno 2] No such file or directory: '/dbfs/FileStore/tables...
- 5301 Views
- 2 replies
- 2 kudos
- 2 kudos
dbutils.fs.ls("/FileStore/tables/badrecord-1.csv")the above file is there in that particular location but still getting the same error
- 2 kudos
- 3254 Views
- 5 replies
- 5 kudos
cluster not create
Hiwhen i try to create cluster it just loading but not create cluster.
- 3254 Views
- 5 replies
- 5 kudos
- 5 kudos
@Sujit Tibe​ Have you configured Databricks in your own VPC and all pre-requisites related to ports are enabled. Another reason can be cloud provider quota issues. If you click on terminated reason, you can see more information
- 5 kudos
- 977 Views
- 0 replies
- 0 kudos
Do AI and ML technologies help the payment system to work well?
Whenever you're entitled to use certain technologies in the payment system, spend some time analyzing your digital payment ecosystem before integrating it with these technologies. Once you're done with assessment work, proceed to the process of integ...
- 977 Views
- 0 replies
- 0 kudos
- 1719 Views
- 1 replies
- 1 kudos
How to set parameters in the dashboard?
In Databricks SQL, I have a query that has a dropdown parameter. How do I get this in the Databricks dashboard? I see the option of adding filters but there is nothing for dashboards
- 1719 Views
- 1 replies
- 1 kudos
- 1 kudos
To answer my own question, it seems like if I add the visualisation from the query page then the parameters are automatically added. Added from the dashboard page seems to be bugged
- 1 kudos
- 1290 Views
- 0 replies
- 1 kudos
Spark behavior while dealing with Actions & Transformations ?
Hi, My question is - what happens to the initial RDD after the action is performed on it. Does it disappear or stays in the memory or does it needs to be explicitly cached() if we want to use it again.For eg : If I execute this in a sequence :df_outp...
- 1290 Views
- 0 replies
- 1 kudos
- 1792 Views
- 1 replies
- 6 kudos
do we have any API or way to find out how we have mounted s3 bucket in AWS
HI Team,we have mounted our s3 buckets 2-3 years back and we don't have a place that has been maintained with config that are used to mount s3 bucket on data bricks instance. can we get complete config and keys/profiles that are used to mount s3 buck...
- 1792 Views
- 1 replies
- 6 kudos
- 1986 Views
- 0 replies
- 0 kudos
Not able to read files in Repos and Workspace from DLT pipeline
I had a working DLT pipeline that failed in the morning because it cannot read files from the Workspace or Repo directory. It is throwing an Operation not permitted error.Even when I am listing directories, it is throwing the same error (although it ...
- 1986 Views
- 0 replies
- 0 kudos
- 2279 Views
- 2 replies
- 1 kudos
Resolved! DataBricks best practice to manage resource correspond deleted user
currently I have some prblem about my DataBricks workspace when an user was deleted and it cause some issue:Applications or scripts that use the tokens generated by the user will no longer be able to access the Databricks APIJobs owned by the user wi...
- 2279 Views
- 2 replies
- 1 kudos
- 3147 Views
- 3 replies
- 2 kudos
Resolved! Error with multiple FeatureLookup calls outside databricks
I access databricks feature store outside databricks with databricks-connect on my IDE pycharm.The problem is just outside Databricks, not with a notebook inside Databricks.I use FeatureLookup mecanism to pull data from Feature store tables in my cus...
- 3147 Views
- 3 replies
- 2 kudos
- 2 kudos
Also, Please refer to the below KB for additional resolution - https://learn.microsoft.com/en-us/azure/databricks/kb/dev-tools/dbconnect-protoserializer-stackoverflow
- 2 kudos
- 1168 Views
- 0 replies
- 0 kudos
How Delta Lake parses through the files on querying the table with a filter condition
Consider a table that gets partitioned on a date field. But, I'm filtering a column that is not partitioned. Now, with this filter condition whether all the files are parsed to attain the required result set, or does any data skipping happens?
- 1168 Views
- 0 replies
- 0 kudos
- 1582 Views
- 0 replies
- 3 kudos
Number of parquet files per delta table
Hello. We would like to understand how many parquet files are created per data table. To be more specific, we refer to the current snapshot of the table. For example, we noticed that while we performed initial inserts to a table, one parquet file was...
- 1582 Views
- 0 replies
- 3 kudos
- 4197 Views
- 4 replies
- 4 kudos
How to include visualizations returned from %run in the caller notebooks dashboard?
I have a notebook (nb1) that calls another one (nb2) via the %run command. This returns some visualizations that I want to add to a dashboard of the caller notebook (nb1-db). When I select the visualization drop down, then select Add to dashboard, th...
- 4197 Views
- 4 replies
- 4 kudos
- 4 kudos
Hi @Nicholas Couture​ , Could you please check and refer to https://docs.databricks.com/sql/user/security/access-control/dashboard-acl.html . Please let us know if this helps.
- 4 kudos
- 11433 Views
- 5 replies
- 17 kudos
Resolved! Azure AAD token with Databricks for User defined managed Identity inside Docker Container
Hi, We are currently using a Azure AAD Token inorder to authenticate with Databricks instead of generating Personal Access Tokens from Databricks. We have a multi-tenant architecture and so we are using Azure container instances to run multiple trans...
- 11433 Views
- 5 replies
- 17 kudos
- 17 kudos
Hi, This looks like an old issue which we have faced earlier, Could you please update the DBR to the latest version and try again?
- 17 kudos
- 26161 Views
- 9 replies
- 4 kudos
Referential Integrity (Primary Key / Foreign Key Constraint) - Azure Databricks SQL
Hello, Please suggest how can we implement Referential Integrity (Primary Key / Foreign Key Constraint) - between different tables defined on Azure Databricks Database. Basically the syntax to add Primary and Foreign Key constraint in the table defi...
- 26161 Views
- 9 replies
- 4 kudos
- 4 kudos
Is there any alternate way you could suggest to implement and enforce primary key constraint?
- 4 kudos
- 10273 Views
- 4 replies
- 4 kudos
Resolved! Issue with databricks and DRIVER_LIBRARY_INSTALLATION_FAILURE.?
I have like 5 maven libraries, and with all of them, I have the same issue with Job or Notebooks. How much do I have to wait? is there another solution? Thank you very much!
- 10273 Views
- 4 replies
- 4 kudos
- 4 kudos
@Raymond Garcia​ , could you please open a support case with Databricks for the same? We will triage the issue and provide a solution.
- 4 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 493 | |
| 349 | |
| 287 |
