- 7137 Views
- 3 replies
- 3 kudos
Resolved! Delta, the specified key does not exist error
Hi, I'm having this error too frequently on a few tables, I check on S3 and the partition exists and the file is there on the partition.error: Spectrum Scan Error: DeltaManifestcode: 15005context: Error fetching Delta Lake manifest delta/product/sub_...
- 7137 Views
- 3 replies
- 3 kudos
- 3 kudos
@Hubert Dudek​ , I'll add that sometimes, just running:GENERATE symlink_format_manifest FOR TABLE schema.tablesolves it, but, how can the symlink get broken?Thanks!
- 3 kudos
- 20474 Views
- 8 replies
- 3 kudos
Not able to move files from local to dbfs through dbfs CLI
Hi Folks,I have installed and configured databricks CLI in my local machine. I tried to move a local file from my personal computer using dbfs cp to dbfs:/ path. I can see the file is copied from local, and is only visible in local. I am not able to ...
- 20474 Views
- 8 replies
- 3 kudos
- 3 kudos
Hi, Could you try to save the file from your local machine to dbfs:/FileStore location?# Put local file test.py to dbfs:/FileStore/test.pydbfs cp test.py dbfs:/FileStore/test.py
- 3 kudos
- 4535 Views
- 2 replies
- 3 kudos
Cannot open Jobs tab in Databricks Community edition.
Hi,I get the following exception when I try to open jobs tab.Uncaught TypeError: Cannot read properties of undefined (reading 'apply') Reload the page and try again. If the error persists, contact support. Reference error code: fd9ae37c18c1400cb15...
- 4535 Views
- 2 replies
- 3 kudos
- 3 kudos
@Kaniz Fatma​ , @Werner Stinckens​ thanks for the clarification. I agree with @Werner Stinckens​ , Error message should be more useful.
- 3 kudos
- 7894 Views
- 3 replies
- 5 kudos
Resolved! Move/Migrate database from dbfs root (s3) to other mounted s3 bucket
Hi,I have a databricks database that has been created in the dbfs root S3 bucket, containing managed tables. I am looking for a way to move/migrate it to a mounted S3 bucket instead, and keep the database name.Any good ideas on how this can be done?T...
- 7894 Views
- 3 replies
- 5 kudos
- 5 kudos
Hi @Jan Ahlbeck​ we can use below property to set the default location:"spark.sql.warehouse.dir": "S3 URL/dbfs path"Please let me know if this helps.
- 5 kudos
- 7356 Views
- 3 replies
- 0 kudos
Not able to connecting Denodo VDP from databricks
I would like connect Denodo VDP from databrick workspace installed ODBC client and Installed denodo Jar in cluster ,not able to understanding other steps.Could you please me
- 7356 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @sathyanarayan kokku​ Are you trying to install denodo vdp server in databricks?
- 0 kudos
- 3397 Views
- 1 replies
- 1 kudos
Resolved! "import pandas as pd" => [Errno 5]
When I type import pandas as pdfrom a Notebook in a Repo I get:--------------------------------------------------------------------------- AttributeError Traceback (most recent call last) /usr/lib/python3.8/importlib/_boots...
- 3397 Views
- 1 replies
- 1 kudos
- 1 kudos
Thanks to Elliott Hertz, I found out that the ML Experiments cannot be stored in the repo. After I moved them to my Workspace everything seems to work.
- 1 kudos
- 9510 Views
- 8 replies
- 3 kudos
Resolved! Spark Streaming - Checkpoint State EOF Exception
I have a Spark Structured Streaming job which reads from 2 Delta tables in streams , processes the data and then writes to a 3rd Delta table. The job is being run with the Databricks service on GCP.Sometimes the job fails with the following exception...
- 9510 Views
- 8 replies
- 3 kudos
- 3 kudos
Hi @Jose Gonzalez​ ,Do you require any more information regarding the code? Any idea what could be cause for the issue?Thanks and Regards,Rohan
- 3 kudos
- 3818 Views
- 3 replies
- 4 kudos
SparkJDBC42.jar Issue ?
Hi there!I am using the SparkJDBC42.jar in my Java application to use my delta lake tables , The connection is made through databricks sql endpoint in where I created a database and store in it my delta tables. I have a simple code to open connection...
- 3818 Views
- 3 replies
- 4 kudos
- 4 kudos
Hi @Seifeddine SNOUSSI​ ,Are you still having issue or you were able to resolve this issue? please let us know
- 4 kudos
- 3348 Views
- 1 replies
- 0 kudos
HTML Backup Import Into my Account
Hi AllI would like to Import my HTML notebook backup into my databricks account and use it as if it was my master (I am a developer and have many exported HTML backups that I may want to reuse. When you open an .HTML from backup, databricks has, ...
- 3348 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Ross Crill​ ,Did @Kaniz Fatma​ reply helped you to resolve your question? please let us know
- 0 kudos
- 7080 Views
- 7 replies
- 3 kudos
Resolved! Databricks and CD4ML
I would like to use Databricks in a CD4ML way (see also https://martinfowler.com/articles/cd4ml.html). Is this possible? I would like to develop and train models in one environment once qualified, I would like to deploy the model with the application...
- 7080 Views
- 7 replies
- 3 kudos
- 7542 Views
- 2 replies
- 2 kudos
- 7542 Views
- 2 replies
- 2 kudos
- 2 kudos
Since Workflows (Multi-Task jobs) is now GA, one way to work around the 1000 concurrent jobs limit is to use tasks within a job. Each job can have 100 tasks, and these tasks do not count toward the concurrent job limit.
- 2 kudos
- 5007 Views
- 4 replies
- 5 kudos
Resolved! Show Vacuum operation result (files deleted) without DRY RUN
Hi, I'm runing some scheduled vacuum jobs and would like to know how many files were deleted without making all the computation twice, with and without DRY RUN, is there a way to accomplish this?Thanks!
- 5007 Views
- 4 replies
- 5 kudos
- 5 kudos
We have to enable logging to capture the logs for vacuum.spark.conf.set("spark.databricks.delta.vacuum.logging.enabled","true")
- 5 kudos
- 3565 Views
- 2 replies
- 3 kudos
Resolved! How to update external metastore cluster configuration on the fly ?
Hello,In my use case, my data is pushed to an adls gen2 container called ingestAfter some data processing on a databricks cluster of the ingest workspace, I declare the associated table in an external metastore for this workspaceAt the end of this pr...
- 3565 Views
- 2 replies
- 3 kudos
- 3 kudos
Hello @Atanu Sarkar​ ,Thank you for your answer. I have created a feature request. I hope, it will be soon accepted ^^
- 3 kudos
- 1632 Views
- 0 replies
- 0 kudos
Displaying the queried data from mounted data from Azure Blob storage to databricks is slow
I have mounted by Azure blob storage json file to databricks which has around 18GB and trying to perform a simple count operation on it and I am noticing that it takes 14 mins for the same in the Community edition . seeking answers on whether this is...
- 1632 Views
- 0 replies
- 0 kudos
- 10059 Views
- 8 replies
- 3 kudos
Resolved! Delta Live Tables has duplicates created by multiple workers
Hello, I am working with Delta Live Tables, I am trying to create a DLT from a combination of Dataframes from a 'for loop' which are unioned and then DLT is created over the Unioned Dataframe. However I noticed that the delta table has duplciates. An...
- 10059 Views
- 8 replies
- 3 kudos
- 3 kudos
@Shikha Mathew​ - Does your last answer mean that your issue is resolved? Would you be happy to mark whichever answer helped as best? Or, if it wasn't a specific one, would you tell us what worked?
- 3 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 582 | |
| 349 | |
| 287 |
