- 3866 Views
- 2 replies
- 3 kudos
Resolved! Integrating Databricks SQL with git repo
Hello, I'm using Databricks premium version on GCP.I've integrated my git repo (bitbucket) with Databricks successfully and I can read and write notebooks from it.I'd like to do the same thing with Databricks SQL, but when I switch to SQL mode the re...
- 3866 Views
- 2 replies
- 3 kudos
- 3 kudos
With a Unified Navigation, you can see all menu tabs via single unified menu. Once you need to trace sql query in git, you'll need simply create new .sql file in Repos and commit it
- 3 kudos
- 9271 Views
- 4 replies
- 0 kudos
Resolved! Python code linter in Databricks notebook
Is it possible to get syntax linting in a DB notebook? Say with flake8, like I do in VS code?
- 9271 Views
- 4 replies
- 0 kudos
- 0 kudos
No linting in a DB notebook available for now. The Notebook is currently in the process of adopting Monaco as the underlying code editor which will offer an improved code authoring experience for notebook cells.Some of the Monaco editor features enab...
- 0 kudos
- 8059 Views
- 2 replies
- 5 kudos
Resolved! Does the `pathGlobFilter` option work on the entire file path or just the file name?
I'm working in the Google Cloud environment. I have an Autoloader job that uses the cloud files notifications to load data into a delta table. I want to filter the files from the PubSub topic based on the path in GCS where the files are located, not...
- 8059 Views
- 2 replies
- 5 kudos
- 5 kudos
Thank you for confirming what I observed that differed from the documentation.
- 5 kudos
- 3153 Views
- 2 replies
- 3 kudos
Resolved! In Data Explorer, how do you Refresh a table definition?
In Data Science & Engineering -> Data -> Data Explorer, if I expand the hive_metastore, then expand a schema and choose a table, and then view the "Sample Data", I receive this error:[DEFAULT_FILE_NOT_FOUND] It is possible the underlying files have b...
- 3153 Views
- 2 replies
- 3 kudos
- 3 kudos
Try refreshing all cached entries that are associated with the table that might helpREFRESH TABLE [db_name.]table_name
- 3 kudos
- 2628 Views
- 1 replies
- 2 kudos
Resolved! What is the maximum number of mount points can be created from a databricks workspace to a ADLS storage account?
We had a usecase where mounts are created at a folder level. Now the number of folders created are increasing. Is there a limit on data bricks on number of mount point creation
- 2628 Views
- 1 replies
- 2 kudos
- 2 kudos
Azure Databricks mounts create a link between a workspace and cloud object storage, which enables you to interact with cloud object storage using familiar file paths relative to the Databricks file system. Mounts work by creating a local alias under ...
- 2 kudos
- 8867 Views
- 8 replies
- 4 kudos
Cluster Scoped init script through pulumi
I am trying to run a cluster-scoped init script through Pulumi. I have referred to this documentation https://learn.microsoft.com/en-us/azure/databricks/clusters/configure#spark-configuration However, looks like the documentation is not very clear.I ...
- 8867 Views
- 8 replies
- 4 kudos
- 4 kudos
Hi @Sulfikkar Basheer Shylaja , Why don't you store the init-script on DBFS and just pass the dbfs:/ path of the init script in Pulumi? You could just run this code on a notebook-%python dbutils.fs.put("/databricks/init-scripts/set-private-pip-repos...
- 4 kudos
- 6654 Views
- 0 replies
- 0 kudos
Datbricks Exam Cancelled
I was giving my Databricks Certified Data Engineer Associate (Version 2) exam scheduled today at 11:15 AM but it got cancelled in between. What to do now?Please help me with this so that I can complete it.
- 6654 Views
- 0 replies
- 0 kudos
- 1023 Views
- 0 replies
- 1 kudos
Databricks Exam got Cancelled
I was giving my Databricks Certified Data Engineer Associate (Version 2) exam scheduled today at 11:15 AM but it got cancelled in between. What to do now?Please help me with this so that I can complete it.
- 1023 Views
- 0 replies
- 1 kudos
- 955 Views
- 0 replies
- 1 kudos
Hello Everyone, I am thrilled to announce that we have our 4th winner for the raffle contest - @MUHAMMET EMIN KOSEOGLU . Please join me in congratu...
Hello Everyone,I am thrilled to announce that we have our 4th winner for the raffle contest - @MUHAMMET EMIN KOSEOGLU . Please join me in congratulating him on this remarkable achievement!Your dedication and hard work have paid off, and we are del...
- 955 Views
- 0 replies
- 1 kudos
- 2093 Views
- 1 replies
- 1 kudos
Connection Error DataGrip Databricks
When trying to connect DataGrip with Databricks SQL, I'm able to do it for multiple connections by using a Token. However, for one specific connection it is not working. We internally tried everything, but we are not able to connect. Below is the err...
- 2093 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi, This looks like, the requested resource is forbidden. Could you please check the destination webserver and recheck the configuration? Please tag @Debayan with your next comment so that I will get notified. Thank you!
- 1 kudos
- 8651 Views
- 12 replies
- 0 kudos
Resolved! Can't use job cluster for scheduled jobs ADD_NODES_FAILED : Failed to add 9 containers to the cluster. Will attempt retry: false. Reason: Azure Quota Exceeded Exception
Hi everyone,I've been using my all purpose cluster for scheduled jobs and I've been told that it's a suboptimal thing to do and that using a job cluster for the scheduled jobs cuts costs by half.Unfortunately, when I tried to switch clusters on my ex...
- 8651 Views
- 12 replies
- 0 kudos
- 0 kudos
@Bassem Jaber If you are seeing same error then you need to increase quota, for that your azure plan should be changed from pay as you go to other plan. as pay-as-go azure model has limitations on quota increase
- 0 kudos
- 3365 Views
- 3 replies
- 0 kudos
How to trigger alert for twice per day at set times?
I need to create a databricks alert for 9:30am and 5pm every day...is there a way to do this with one alert? I can't use "Refresh every 1 day at time..." since this will only trigger once per day. I also can't use "Refresh every 12 hours at minute....
- 3365 Views
- 3 replies
- 0 kudos
- 112731 Views
- 15 replies
- 1 kudos
- 112731 Views
- 15 replies
- 1 kudos
- 1 kudos
This answer from Stackoverflow is spot on. https://stackoverflow.com/a/72500870/1290485
- 1 kudos
- 6521 Views
- 8 replies
- 6 kudos
Job aborted due to stage failure: Task 1863 in stage 10.0 failed 4 times, most recent failure: Lost task 1863.3 in stage 10.0 (TID 2021) (10.0.4.7 executor 2): org.apache.spark.SparkException: Python worker exited unexpectedly (crashed): Fatal Python erro
I am getting below error some time run my databricks notebook from ADF, If the executor node is one then it works fine, if it increases 2 or more some times its failing on same data.Cluster Detail : Standard_F4s_v2 · Workers: Standard_F4s_v2 · 1-8 wo...
- 6521 Views
- 8 replies
- 6 kudos
- 6 kudos
Hi @Ancil P A Can you give paste the complete stacktrace from the failed task (from failed stage 10.0) and the code snippet that you are trying to run in the notebook . Also, do you think you can raise a databricks support ticket for the same?
- 6 kudos
- 2488 Views
- 1 replies
- 0 kudos
Loading multiple gz files from ADLS to Delta Lake/Delta table in ADB
I have several gz files (file.csv.gz) in a ADLS folder which are of pretty big size. All of these files are extractd from the same base table so it has the similar data but of different dates. How can I transfer them in delta lake/delta table. We wou...
- 2488 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi, You can read GZ files through spark. https://stackoverflow.com/questions/42761912/how-to-read-gz-compressed-file-by-pysparkPlease let us know if this helps. Also, please tag @Debayan with your next comment so that I will get notified. Thank you!
- 0 kudos
| User | Count |
|---|---|
| 1611 | |
| 768 | |
| 348 | |
| 286 | |
| 252 |
