cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Vibhor
by Contributor
  • 1855 Views
  • 5 replies
  • 4 kudos

Resolved! Cluster Performance

Facing an issue with cluster performance, in event log can see - cluster is not responsive likely due to GC. Number of pipeline (databricks notebooks) running and cluster configuration is same as it used to be before but started seeing this issue sin...

  • 1855 Views
  • 5 replies
  • 4 kudos
Latest Reply
jose_gonzalez
Moderator
  • 4 kudos

Hi @Vibhor Sethi​ ,Do you see any other error messages? did you data volume increase? what kind of job are you running?

  • 4 kudos
4 More Replies
wschoi
by New Contributor III
  • 2113 Views
  • 4 replies
  • 3 kudos

How to fix plots and image color rendering on Notebooks?

I am currently running dark mode for my Databricks Notebooks, and am using the "new UI" released a few days ago (May 2023) and the "New notebook editor."Currently all plots (like matplotlib) are showing wrong colors. For example, denoting:```... p...

  • 2113 Views
  • 4 replies
  • 3 kudos
Latest Reply
leonardoazzi
New Contributor II
  • 3 kudos

Hello all,Thank you @wschoi for reporting this issue. I've lost a lot of time trying to figure out if my image plotting was wrong.

  • 3 kudos
3 More Replies
jch
by New Contributor III
  • 729 Views
  • 2 replies
  • 0 kudos

Can Kaggle Kernels be used in Databricks notebooks?

suggestions).This one looks perfect kaggle kernels output rsrishav/starter-youtube-trending-video-dataset -p /path/to/dest but I'm not using CLI, I'm using a databricks notebook.I tried using this code but it doesn't work. data_path = 'rsrishav/youtu...

  • 729 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @jch​, The error message you received, "Permission 'kernels.get' was denied," suggests that you don't have the necessary permissions to access or view the Kaggle kernel. This error typically occurs when you try to access a kernel that you don't ha...

  • 0 kudos
1 More Replies
dukebaslangic
by New Contributor II
  • 890 Views
  • 3 replies
  • 3 kudos

Resolved! Databricks performance related documentation/books

Hi,Do you know any good resources about Databricks performance improvements(like improving query performances, monitoring/resolving performance bottlenecks etc)?Thanks

  • 890 Views
  • 3 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Ömer Özsakarya​  We haven't heard from you since the last response from @Lakshay Goel​ ​, and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to ...

  • 3 kudos
2 More Replies
Christine
by Contributor
  • 17713 Views
  • 4 replies
  • 1 kudos

Resolved! Is it possible to import functions from a module in Workspace/Shared instead of Repos?

Hi,I am considering creating libraries for my databricks notebooks, and found that it is possible to import functions from modules saved in repos. Is it possible to move the .py files with the functions to Workspace/Shared and still import functions ...

  • 17713 Views
  • 4 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Christine Pedersen​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell ...

  • 1 kudos
3 More Replies
NOOR_BASHASHAIK
by Contributor
  • 2327 Views
  • 3 replies
  • 3 kudos

Resolved! Notebooks & Jobs Migration from one workspace to another

Hi allI have a need to migrate just notebooks & jobs from one workspace to another. Is there an utility to do so?

  • 2327 Views
  • 3 replies
  • 3 kudos
Latest Reply
karthik_p
Esteemed Contributor
  • 3 kudos

@NOOR BASHA SHAIK​ you can also try databricks connect in addition to @Artem Sheiko​ information that was provided .

  • 3 kudos
2 More Replies
jannemanson
by New Contributor III
  • 681 Views
  • 3 replies
  • 0 kudos

Improve iteration time on implementing jobs

Hey there, I am using dbx to create Databricks tasks and deploy the job. I find it not ideal since the iteration circles are sometimes a bit long when I have to wait for a job with several tasks to complete and see where it failed. I am already tryin...

  • 681 Views
  • 3 replies
  • 0 kudos
Latest Reply
jannemanson
New Contributor III
  • 0 kudos

Hello, thanks for the answer. Unfortunately, this did not help me, since it is general best practice. @Debayan Mukherjee​ 

  • 0 kudos
2 More Replies
Dean_Lovelace
by New Contributor III
  • 1547 Views
  • 3 replies
  • 0 kudos

How to filter the Spark UI for a notebook

When running spark under yarn each script has it's own self contained set of logs:- In databricks all I see if a list of jobs and stages that have been run on the cluster:- From a support perspective this is a nightmare.How can notebooks logs be grou...

image image
  • 1547 Views
  • 3 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Dean Lovelace​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers...

  • 0 kudos
2 More Replies
killjoy
by New Contributor III
  • 3302 Views
  • 2 replies
  • 0 kudos

Unexpected failure while fetching notebook - What can we do from our side?

Hello!We got some pipelines running in Azure Data Factory that call Databricks Notebooks to run data transformations. This morning at 6:21 AM (UTC) we got an error " Unexpected failure while fetching notebook" inside a notebook that calls another one...

  • 3302 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Rita Fernandes​ :Based on the error message you provided, it seems like the issue might be related to the version mismatch between the ANTLR tool used for code generation and the current runtime version. Additionally, the error message suggests that...

  • 0 kudos
1 More Replies
Jyo777
by Contributor
  • 2375 Views
  • 4 replies
  • 4 kudos

need help with Azure Databricks questions on CTE and SQL syntax within notebooks

Hi amazing community folks,Feel free to share your experience or knowledge regarding below questions:-1.) Can we pass a CTE sql statement into spark jdbc? i tried to do it i couldn't but i can pass normal sql (Select * from ) and it works. i heard th...

  • 2375 Views
  • 4 replies
  • 4 kudos
Latest Reply
Kaniz
Community Manager
  • 4 kudos

Hi @Jyoti j​​, We haven't heard from you since the last response from @Suteja Kanuri​, and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to others....

  • 4 kudos
3 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 1281 Views
  • 1 replies
  • 6 kudos

SQL cells in databricks notebooks can now be run in parallel, which means faster query processing and analysis. This new feature is especially helpful...

SQL cells in databricks notebooks can now be run in parallel, which means faster query processing and analysis. This new feature is especially helpful for queries that take longer to run or analyze large datasets. With parallel processing, Databricks...

paraler
  • 1281 Views
  • 1 replies
  • 6 kudos
Latest Reply
Rishabh264
Honored Contributor II
  • 6 kudos

Informative ​

  • 6 kudos
User16665996606
by New Contributor II
  • 1597 Views
  • 4 replies
  • 2 kudos

How to access public URLs via Databricks notebooks

I am trying to run a web application integrated with Gradio on Databricks. However, currently, I have to first run on the local URL and then launch it on the public URL. Are there any potential solutions for them to deploy the app on the public URL o...

  • 1597 Views
  • 4 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Sixuan He​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers you...

  • 2 kudos
3 More Replies
bradlindblad
by New Contributor II
  • 961 Views
  • 2 replies
  • 1 kudos

Resolved! Font in Databricks Notebook is Greyed Out - Glitchy

The monospaced/code font in my databricks notebooks is greyed out, both in light and dark theme. I've tried playing with all the notebook settings, etc. and nothing will make the font 'normal'. I've tried Chrome and Edge, and the results are the same...

db
  • 961 Views
  • 2 replies
  • 1 kudos
Latest Reply
klaapbakken
New Contributor III
  • 1 kudos

I was having this exact same issue. I fixed it by uninstalling the Source Code Pro font from my Windows machine.

  • 1 kudos
1 More Replies
dispersion
by New Contributor
  • 649 Views
  • 2 replies
  • 1 kudos

Running large volume of SQL queries in Python notebooks. How to minimise overheads/maintenance.

I have around 200 SQL queries id like to run in databricks python notebooks. Id like to avoid creating an ETL process for each of the 200 SQL processes.Any suggestions on how to run the queries in a way that it loops through them so i have minimum am...

  • 649 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Chris French​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...

  • 1 kudos
1 More Replies
Labels