Curious if anyone has moved from SwaggerHub to other platforms for API design, especially within data projects. Tools like Apidog, Postman, or Insomnia have been mentioned to me – does anyone have thoughts on how they compare for efficiency and colla...
Hi, I have enrolled into Databricks Apache Spark SQL for Data Analysts course on Coursera.I am in Module 3, I imported the file through given url in course material. In 3.2Basic Queries I ran the first block which contains following command "%run ....
Hi, I enrolled in the same coursera "databricks" course, and met the same issue. I unmounted and re-run the classroom set up as following However, when I move forward to list the mounted dirctory, the same issue happened.
I want to create a chatbot ml model, to answer questions about the data, my organization have in the datalake. It should answer comprehension questions about the data.example: what do you understand from the data received from yesterday and today.
Does anyone know the size limit for uploading files through the CLI? I'm not finding it in the documentation.
We are experiencing a compatibility issue between the Databricks Runtime version and the Delta Live Tables (DLT) Runtime. We need to upgrade to Databricks Runtime 15.2 to enable the auto schema evolution feature. However, Databricks Runtime 15.2 requ...
When I save a certain Python notebook where I have selected Hide Code and Hide Results on certain cells, those conditions persist. For example, when I come back the next day in a new session, the hidden material is still hidden.When the notebook is ...
In my situation we cannot split this notebook as ADF pipeline is already in PROD ,I have tried to use the option %%capture .It helps to ran the notebook within size limits but somehow it is corrupting the output. Also checked in the Databricks AI and...
Hello All,When I try to deploy my bundle, I get the following error.I can't edit the bundle.tf.json, I suppose it is created automatically. Does anyone have a solution for the same problem?Many Thanks,Can$ databricks bundle deploy -t devBuilding my_p...
I try to export a job as a DBA in order to create an Asset Bundle according to thishttps://community.databricks.com/t5/data-engineering/databricks-asset-bundle-dab-from-existing-workspace/td-p/49309I am on Windows 10 Pro x64 withDatabricks CLI v0.223...
To export an existing folder under /Workspace/... export-dir command could be used : databricks workspace export-dir /Workspace/Applications/ucx/logs/migrate-tables/run-123-0/ /Users/artem.sheiko/logs
I have a setup where i am replicating Delta live tables parquet, and checkpoint files using azure RAGZRS to peer region for disaster recovery. When i load the the replicated files in peer region using delta format, i get an error that _delta_log/0000...
Hi All,We want to switch from Snowflake to Databricks SQL Warehouse/serverless to simplify our data layers and reduce data copies before the reporting layer. Please share the benefits of using serverless over Snowflake and any limitations you see. We...
one big pro is that you do not need to copy data to the dwh. also your transformations and analytics queries reside on the same platform (databricks).If databricks can cover all the requirementsm compared to snowflakem is hard to tell. Probably ther...
Team,I get a ConcurrentAppendException: Files were added to the root of the table by a concurrent update when trying to update a table which executes via jobs with for each activity in ADF,I tried with Databricks run time 14.x and set the delete vect...
In case of such an issue, I would like to suggest apply retry and try except logic (you can use one of existing libraries) in both concurrent updates - it should help, and jobs won't report any error.
Hello,We are using a 5 worker node DLT job compute for a continuous mode streaming pipeline. The worker configuration is Standard_D4ads_v5 i.e. 4 cores so total cores across 5 workers is 20 cores.We have wide transformation at some places in the pipe...
Hi @PushkarDeole ,Each Delta Live Tables pipeline has two associated clusters:The updates cluster processes pipeline updates.The maintenance cluster runs daily maintenance tasks.According to docs, if you want to configure settings at the pipeline lev...
Hi,I am planning to take the Databricks Associate exam in the upcoming week. My current ID proof is my original driving license issued by the Tamil Nadu government; however, it is laminated rather than a hard plastic card.Could you please confirm if ...
Hi Databricks community and @Cert-Team,Can you please help me here? The reason I am asking is Microsoft certification doesn’t approve laminated id proof’s.
We currently have approximately 60 tables that collectively exceed 500GB, representing 99.9% of our total database size. One potential solution is to migrate these larger tables to Databricks, which may help us mitigate synchronization costs and addr...
I'm trying to create community edition account but everytime i try it shows" An error has occurred. Please try again later."I have attached the screenshot.I also tried accesiing Databricks in a differenct pc based on the answers from previous threads...
By any chance is that email address already registered with another Databricks Tier (like company or paid)? Can you try in incognito mode with all ad blockers/popup blockers disabled
