- 4489 Views
- 4 replies
- 4 kudos
need help with Azure Databricks questions on CTE and SQL syntax within notebooks
Hi amazing community folks,Feel free to share your experience or knowledge regarding below questions:-1.) Can we pass a CTE sql statement into spark jdbc? i tried to do it i couldn't but i can pass normal sql (Select * from ) and it works. i heard th...
- 4489 Views
- 4 replies
- 4 kudos
- 4 kudos
CTE expressions are supported with the `prepareQuery` option. https://spark.apache.org/docs/latest/sql-data-sources-jdbc.html A prefix that will form the final query together with query. As the specified query will be parenthesized as a subquery i...
- 4 kudos
- 4442 Views
- 5 replies
- 3 kudos
How to fix plots and image color rendering on Notebooks?
I am currently running dark mode for my Databricks Notebooks, and am using the "new UI" released a few days ago (May 2023) and the "New notebook editor."Currently all plots (like matplotlib) are showing wrong colors. For example, denoting:```... p...
- 4442 Views
- 5 replies
- 3 kudos
- 3 kudos
Any updated on this issue? This is a huge drawback to use the dark theme
- 3 kudos
- 3488 Views
- 5 replies
- 4 kudos
Resolved! Cluster Performance
Facing an issue with cluster performance, in event log can see - cluster is not responsive likely due to GC. Number of pipeline (databricks notebooks) running and cluster configuration is same as it used to be before but started seeing this issue sin...
- 3488 Views
- 5 replies
- 4 kudos
- 4 kudos
Hi @Vibhor Sethi ,Do you see any other error messages? did you data volume increase? what kind of job are you running?
- 4 kudos
- 1630 Views
- 1 replies
- 0 kudos
Can Kaggle Kernels be used in Databricks notebooks?
suggestions).This one looks perfect kaggle kernels output rsrishav/starter-youtube-trending-video-dataset -p /path/to/dest but I'm not using CLI, I'm using a databricks notebook.I tried using this code but it doesn't work. data_path = 'rsrishav/youtu...
- 1630 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @jch Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 1749 Views
- 3 replies
- 3 kudos
Resolved! Databricks performance related documentation/books
Hi,Do you know any good resources about Databricks performance improvements(like improving query performances, monitoring/resolving performance bottlenecks etc)?Thanks
- 1749 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @Ömer Özsakarya We haven't heard from you since the last response from @Lakshay Goel , and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to ...
- 3 kudos
- 25250 Views
- 4 replies
- 1 kudos
Resolved! Is it possible to import functions from a module in Workspace/Shared instead of Repos?
Hi,I am considering creating libraries for my databricks notebooks, and found that it is possible to import functions from modules saved in repos. Is it possible to move the .py files with the functions to Workspace/Shared and still import functions ...
- 25250 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @Christine Pedersen Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell ...
- 1 kudos
- 5227 Views
- 3 replies
- 3 kudos
Resolved! Notebooks & Jobs Migration from one workspace to another
Hi allI have a need to migrate just notebooks & jobs from one workspace to another. Is there an utility to do so?
- 5227 Views
- 3 replies
- 3 kudos
- 3 kudos
@NOOR BASHA SHAIK you can also try databricks connect in addition to @Artem Sheiko information that was provided .
- 3 kudos
- 1693 Views
- 3 replies
- 0 kudos
Improve iteration time on implementing jobs
Hey there, I am using dbx to create Databricks tasks and deploy the job. I find it not ideal since the iteration circles are sometimes a bit long when I have to wait for a job with several tasks to complete and see where it failed. I am already tryin...
- 1693 Views
- 3 replies
- 0 kudos
- 0 kudos
Hello, thanks for the answer. Unfortunately, this did not help me, since it is general best practice. @Debayan Mukherjee
- 0 kudos
- 3676 Views
- 3 replies
- 0 kudos
How to filter the Spark UI for a notebook
When running spark under yarn each script has it's own self contained set of logs:- In databricks all I see if a list of jobs and stages that have been run on the cluster:- From a support perspective this is a nightmare.How can notebooks logs be grou...
- 3676 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Dean Lovelace Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers...
- 0 kudos
- 5069 Views
- 2 replies
- 0 kudos
Unexpected failure while fetching notebook - What can we do from our side?
Hello!We got some pipelines running in Azure Data Factory that call Databricks Notebooks to run data transformations. This morning at 6:21 AM (UTC) we got an error " Unexpected failure while fetching notebook" inside a notebook that calls another one...
- 5069 Views
- 2 replies
- 0 kudos
- 0 kudos
@Rita Fernandes :Based on the error message you provided, it seems like the issue might be related to the version mismatch between the ANTLR tool used for code generation and the current runtime version. Additionally, the error message suggests that...
- 0 kudos
- 2323 Views
- 1 replies
- 7 kudos
SQL cells in databricks notebooks can now be run in parallel, which means faster query processing and analysis. This new feature is especially helpful...
SQL cells in databricks notebooks can now be run in parallel, which means faster query processing and analysis. This new feature is especially helpful for queries that take longer to run or analyze large datasets. With parallel processing, Databricks...
- 2323 Views
- 1 replies
- 7 kudos
- 7 kudos
Informative
- 7 kudos
- 589 Views
- 0 replies
- 0 kudos
Could it be possible we can retrieve DBU used by cluster in Databricks notebooks?This Data we get in Active DBU/hr in Compute tab for all clusters.
Could it be possible we can retrieve DBU used by cluster in Databricks notebooks?This Data we get in Active DBU/hr in Compute tab for all clusters.
- 589 Views
- 0 replies
- 0 kudos
- 2793 Views
- 4 replies
- 2 kudos
How to access public URLs via Databricks notebooks
I am trying to run a web application integrated with Gradio on Databricks. However, currently, I have to first run on the local URL and then launch it on the public URL. Are there any potential solutions for them to deploy the app on the public URL o...
- 2793 Views
- 4 replies
- 2 kudos
- 2 kudos
Hi @Sixuan He Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers you...
- 2 kudos
- 1932 Views
- 2 replies
- 1 kudos
Resolved! Font in Databricks Notebook is Greyed Out - Glitchy
The monospaced/code font in my databricks notebooks is greyed out, both in light and dark theme. I've tried playing with all the notebook settings, etc. and nothing will make the font 'normal'. I've tried Chrome and Edge, and the results are the same...
- 1932 Views
- 2 replies
- 1 kudos
- 1 kudos
I was having this exact same issue. I fixed it by uninstalling the Source Code Pro font from my Windows machine.
- 1 kudos
- 1363 Views
- 2 replies
- 1 kudos
Running large volume of SQL queries in Python notebooks. How to minimise overheads/maintenance.
I have around 200 SQL queries id like to run in databricks python notebooks. Id like to avoid creating an ETL process for each of the 200 SQL processes.Any suggestions on how to run the queries in a way that it loops through them so i have minimum am...
- 1363 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Chris French Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...
- 1 kudos