- 4066 Views
- 2 replies
- 0 kudos
Resolved! Connecting to SQL on Databricks Using SQLAlchemy or pyodbc
On Databricks, when I try to connect to SQL using SQLAlchemy or pyodbc to run delete queries on a specific table, I get this error: (pyodbc.Error) ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'ODBC Driver 17 for SQL Server' : file not ...
- 4066 Views
- 2 replies
- 0 kudos
- 0 kudos
I was able to solve the problem! the problem was because the driver was missing and so pyodbc or sqlAlchemy can't find it. So I used the native Java API and it is working.This is the example code:jdbcUsername = "username"jdbcPassword = "password"driv...
- 0 kudos
- 1221 Views
- 1 replies
- 1 kudos
Resolved! Unable to delete a table
When I try to delete a table, I'm getting this error:[ErrorClass=INVALID_STATE] TABLE catalog.schema.table_name cannot be deleted because it is being shared via Delta Sharing.I have checked on the internet about it, but could not find any info about ...
- 1221 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @IGRACH ,You are facing this issue bcz I guess the table you want to delete is being shared by delta sharing. You can go to the shared object by following this dochttps://docs.databricks.com/aws/en/delta-sharing/create-share#update-sharesandThen, ...
- 1 kudos
- 3015 Views
- 3 replies
- 1 kudos
External tables in DLT pipelines
Hello community,I have implemented a DLT pipeline.In the "Destination" setting of the pipeline I have specified a unity catalog with target schema of type external referring to an S3 destination.My DLT pipeline works well. Yet, I noticed that all str...
- 3015 Views
- 3 replies
- 1 kudos
- 1 kudos
This won't work.best approach is create dlt sink to write to delta external table. This pipeline should only be 1 step. Read table and append flow using data sink. It works fine.
- 1 kudos
- 1511 Views
- 1 replies
- 0 kudos
Resolved! databricks bundle Deploy: exit code 0 even if an error occurs
We have a CI/CD pipeline where we run:databricks bundle deploy [...]The code works fine, however, if we missconfigure it, we see in the output an error message such asDeploying resources... Updating deployment state... Warning: Detected unresolved va...
- 1511 Views
- 1 replies
- 0 kudos
- 0 kudos
you can close it: it was an ci/cd issue
- 0 kudos
- 7055 Views
- 6 replies
- 1 kudos
Custom docker image fails to initalize
I'm trying to use a custom docker image for my job. This is my docker file:FROM databricksruntime/standard:12.2-LTS COPY . . RUN /databricks/python3/bin/pip install -U pip RUN /databricks/python3/bin/pip install -r requirements.txt USER rootMy job ...
- 7055 Views
- 6 replies
- 1 kudos
- 1145 Views
- 1 replies
- 0 kudos
Having an issue assigning databricks_current_metastore with terraform provider
I am trying to assign my databricks_current_metastore on terraform and I get the following error back as an output Error: cannot read current metastore: cannot get client current metastore: invalid Databricks Workspace configurationwith data.databric...
- 1145 Views
- 1 replies
- 0 kudos
- 0 kudos
@badari_narayan Based on above terraform code, you are trying to use the databricks.accounts provider to read the current workspace metastore, which is incorrect — the databricks_current_metastore data source is a workspace-level resource, and must b...
- 0 kudos
- 16568 Views
- 5 replies
- 4 kudos
Azure Databricks: Error Creating Cluster
We have provisioned a new workspace in Azure using our own VNet. Upon creating the first cluster, I encounter this error:Control Plane Request Failure: Failed to get instance bootstrap steps from the Databricks Control Plane. Please check that instan...
- 16568 Views
- 5 replies
- 4 kudos
- 1618 Views
- 2 replies
- 1 kudos
Resolved! Revert cluster DBR version to last DBR
Hi Team,We have updated our clusters DBR version, later we got to know that some of our jobs started failing, now we wanted to revert to DBR version to the previos one only but we forgot the DBR version on which job was running fine.Is there any way ...
- 1618 Views
- 2 replies
- 1 kudos
- 1 kudos
Thank you for your reply, I also found an another solution, checked the event_logs and there old DBR versions was mentioned.
- 1 kudos
- 5830 Views
- 5 replies
- 0 kudos
Errors When Using R on Unity Catalog Clusters
We are running into errors when running workflows with multiple jobs using the same notebook/different parameters. They are reading from tables we still have in hive_metastore, there's no Unity Catalog tables or functionality referenced anywhere. We'...
- 5830 Views
- 5 replies
- 0 kudos
- 0 kudos
R enabled cluster only supports single user group so please check permission at your group level
- 0 kudos
- 14016 Views
- 14 replies
- 5 kudos
Resolved! 2023-03-22 10:29:23 | Error 403 | https://customer-academy.databricks.com/
I would like to know why I am getting this error when I tried to earn badges for lakehouse fundamentals. I can't access the quiz page. Can you please help on this?
- 14016 Views
- 14 replies
- 5 kudos
- 5 kudos
Login by you gmail account in customer-academy.databricks.com and search the LakeHouse short course and enroll free
- 5 kudos
- 4895 Views
- 1 replies
- 0 kudos
Streaming vs Batch with Continuous Trigger
Not sure what the concrete advantage there is for me to create a streaming table vs static one. In my case, I designed a table with a job that extracts the most lastest files from an s3 location and then appends them to a delta table. I set the job...
- 4895 Views
- 1 replies
- 0 kudos
- 0 kudos
@ashap551 You're essentially implementing a well-optimized micro-batching process, and functionally, it's very similar to what readStream() with Autoloader would do. However, there are some advantages to using Autoloader and a proper streaming table ...
- 0 kudos
- 1160 Views
- 2 replies
- 1 kudos
warning message when secrets are added in cluster
when i try to add secrets in cluster config , databricks is showing error that secrets should not be hardcoded, although it is not halting to save those ,it is showing warning even if they are not hardcoded.
- 1160 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi Nik21,You can use Key Vaults in azure to store your secrets then import them to data bricks.
- 1 kudos
- 4386 Views
- 7 replies
- 2 kudos
Resolved! How to manage two separate projects ?
Hi all, I am managing one project in Databricks, with one more coming soon. Can anyone guide me on how to use Unity Catalog or any other method for this?"
- 4386 Views
- 7 replies
- 2 kudos
- 2 kudos
There are a wide variety of needs that need to be considered such as governance, compute and storage. This depends on the size of your projects.
- 2 kudos
- 3744 Views
- 3 replies
- 2 kudos
PDF Generation via databricks Job
WE have a databricks Job that will aggregate some data and create some data tables. This needs to be exported out in a PDF format.I have seen a few python libraries that can generate PDF, but was wondering if the PDF can be generated and dropped in a...
- 3744 Views
- 3 replies
- 2 kudos
- 2 kudos
Are there any specific ways to generate the PDF file from a dataframe? and libraries that work
- 2 kudos
- 3107 Views
- 3 replies
- 0 kudos
how to create catalog
Hi ,I am trying to create catalog and database its not allowing databricks , please suggest .Here my code .base_dir = "/mnt/files"spark.sql(f"CREATE CATALOG IF NOT EXISTS dev")spark.sql(f"CREATE DATABASE IF NOT EXISTS dev.demo_db") first i ne...
- 3107 Views
- 3 replies
- 0 kudos
- 0 kudos
I got a similar error trying create a catalog with "databricks.sdk" library I resolved it add the parameter "storage_root": w.catalogs.create(name=c.name, storage_root='s3://databricks-workspace-bucket/unity-catalog/426335709') In my case all catalog...
- 0 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 581 | |
| 349 | |
| 287 |
