- 6407 Views
- 3 replies
- 5 kudos
Databricks Job run logs only shows prints/logs from driver and not executors
Hi,In Databricks Job run output, only logs from driver are displayed. We have a function parallelized to run on executor nodes. The logs/prints from that function are not displayed in job run output. Is there a way to configure and show those logs i...
- 6407 Views
- 3 replies
- 5 kudos
- 5 kudos
Thanks @Debayan Mukherjee . This is to enable executor logging. However, the executor logs do not appear in Databricks Job run output. Only driver logs are displayed.
- 5 kudos
- 14843 Views
- 10 replies
- 12 kudos
Limitations with UDFs wrapping modules imported via Repos files?
We have been importing custom module wheel files from our AzDevOps repository. We are pushing to use the Databricks Repos arbitrary files to simplify this but it is breaking our spark UDF that wraps one of the functions in the library with a ModuleNo...
- 14843 Views
- 10 replies
- 12 kudos
- 12 kudos
If your notebook is in the same Repo as the module, this should work without any modifications to the sys path.If your notebook is not in the same Repo as the module, you may need to ensure that the sys path is correct on all nodes in your cluster th...
- 12 kudos
- 1737 Views
- 0 replies
- 0 kudos
Why not choose ORC over Parquet?
What Spark/Delta Lake choose ORC vs Parquet file format? I learnt ORC is much faster when querying, It is much compression efficient than parquet and has most the feature which parquet has on top of it? Why not choose ORC? Am I missing something? Ple...
- 1737 Views
- 0 replies
- 0 kudos
- 3411 Views
- 5 replies
- 2 kudos
Password reset window freezes for DB community edition
HiI've been trying to reset my DB community edition password. I receive the email ok and change the PW, slick submit but the window just hangs ( for ever ).See attached screen grab
- 3411 Views
- 5 replies
- 2 kudos
- 2 kudos
Hi @Jason Roche Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so w...
- 2 kudos
- 4427 Views
- 2 replies
- 3 kudos
Resolved! Integrating Databricks SQL with git repo
Hello, I'm using Databricks premium version on GCP.I've integrated my git repo (bitbucket) with Databricks successfully and I can read and write notebooks from it.I'd like to do the same thing with Databricks SQL, but when I switch to SQL mode the re...
- 4427 Views
- 2 replies
- 3 kudos
- 3 kudos
With a Unified Navigation, you can see all menu tabs via single unified menu. Once you need to trace sql query in git, you'll need simply create new .sql file in Repos and commit it
- 3 kudos
- 11364 Views
- 4 replies
- 0 kudos
Resolved! Python code linter in Databricks notebook
Is it possible to get syntax linting in a DB notebook? Say with flake8, like I do in VS code?
- 11364 Views
- 4 replies
- 0 kudos
- 0 kudos
No linting in a DB notebook available for now. The Notebook is currently in the process of adopting Monaco as the underlying code editor which will offer an improved code authoring experience for notebook cells.Some of the Monaco editor features enab...
- 0 kudos
- 10629 Views
- 2 replies
- 5 kudos
Resolved! Does the `pathGlobFilter` option work on the entire file path or just the file name?
I'm working in the Google Cloud environment. I have an Autoloader job that uses the cloud files notifications to load data into a delta table. I want to filter the files from the PubSub topic based on the path in GCS where the files are located, not...
- 10629 Views
- 2 replies
- 5 kudos
- 5 kudos
Thank you for confirming what I observed that differed from the documentation.
- 5 kudos
- 3744 Views
- 2 replies
- 3 kudos
Resolved! In Data Explorer, how do you Refresh a table definition?
In Data Science & Engineering -> Data -> Data Explorer, if I expand the hive_metastore, then expand a schema and choose a table, and then view the "Sample Data", I receive this error:[DEFAULT_FILE_NOT_FOUND] It is possible the underlying files have b...
- 3744 Views
- 2 replies
- 3 kudos
- 3 kudos
Try refreshing all cached entries that are associated with the table that might helpREFRESH TABLE [db_name.]table_name
- 3 kudos
- 3252 Views
- 1 replies
- 2 kudos
Resolved! What is the maximum number of mount points can be created from a databricks workspace to a ADLS storage account?
We had a usecase where mounts are created at a folder level. Now the number of folders created are increasing. Is there a limit on data bricks on number of mount point creation
- 3252 Views
- 1 replies
- 2 kudos
- 2 kudos
Azure Databricks mounts create a link between a workspace and cloud object storage, which enables you to interact with cloud object storage using familiar file paths relative to the Databricks file system. Mounts work by creating a local alias under ...
- 2 kudos
- 10970 Views
- 8 replies
- 4 kudos
Cluster Scoped init script through pulumi
I am trying to run a cluster-scoped init script through Pulumi. I have referred to this documentation https://learn.microsoft.com/en-us/azure/databricks/clusters/configure#spark-configuration However, looks like the documentation is not very clear.I ...
- 10970 Views
- 8 replies
- 4 kudos
- 4 kudos
Hi @Sulfikkar Basheer Shylaja , Why don't you store the init-script on DBFS and just pass the dbfs:/ path of the init script in Pulumi? You could just run this code on a notebook-%python dbutils.fs.put("/databricks/init-scripts/set-private-pip-repos...
- 4 kudos
- 6903 Views
- 0 replies
- 0 kudos
Datbricks Exam Cancelled
I was giving my Databricks Certified Data Engineer Associate (Version 2) exam scheduled today at 11:15 AM but it got cancelled in between. What to do now?Please help me with this so that I can complete it.
- 6903 Views
- 0 replies
- 0 kudos
- 1258 Views
- 0 replies
- 1 kudos
Databricks Exam got Cancelled
I was giving my Databricks Certified Data Engineer Associate (Version 2) exam scheduled today at 11:15 AM but it got cancelled in between. What to do now?Please help me with this so that I can complete it.
- 1258 Views
- 0 replies
- 1 kudos
- 1230 Views
- 0 replies
- 1 kudos
Hello Everyone, I am thrilled to announce that we have our 4th winner for the raffle contest - @MUHAMMET EMIN KOSEOGLU . Please join me in congratu...
Hello Everyone,I am thrilled to announce that we have our 4th winner for the raffle contest - @MUHAMMET EMIN KOSEOGLU . Please join me in congratulating him on this remarkable achievement!Your dedication and hard work have paid off, and we are del...
- 1230 Views
- 0 replies
- 1 kudos
- 2441 Views
- 1 replies
- 1 kudos
Connection Error DataGrip Databricks
When trying to connect DataGrip with Databricks SQL, I'm able to do it for multiple connections by using a Token. However, for one specific connection it is not working. We internally tried everything, but we are not able to connect. Below is the err...
- 2441 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi, This looks like, the requested resource is forbidden. Could you please check the destination webserver and recheck the configuration? Please tag @Debayan with your next comment so that I will get notified. Thank you!
- 1 kudos
- 10086 Views
- 12 replies
- 0 kudos
Resolved! Can't use job cluster for scheduled jobs ADD_NODES_FAILED : Failed to add 9 containers to the cluster. Will attempt retry: false. Reason: Azure Quota Exceeded Exception
Hi everyone,I've been using my all purpose cluster for scheduled jobs and I've been told that it's a suboptimal thing to do and that using a job cluster for the scheduled jobs cuts costs by half.Unfortunately, when I tried to switch clusters on my ex...
- 10086 Views
- 12 replies
- 0 kudos
- 0 kudos
@Bassem Jaber If you are seeing same error then you need to increase quota, for that your azure plan should be changed from pay as you go to other plan. as pay-as-go azure model has limitations on quota increase
- 0 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 486 | |
| 349 | |
| 287 |
