cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Victhor
by New Contributor III
  • 4547 Views
  • 4 replies
  • 8 kudos
  • 4547 Views
  • 4 replies
  • 8 kudos
Latest Reply
chanshing
New Contributor II
  • 8 kudos

@Kaniz Fatma​ Is that tool (dbvim) still maintained? It looks like it has been abandoned and there are a couple of unresolved issues.Are there any plans to support vim keybindings in Databricks? This is possible in many other web-based editors such a...

  • 8 kudos
3 More Replies
KVNARK
by Honored Contributor II
  • 1803 Views
  • 4 replies
  • 6 kudos

Resolved! Connecting azure synapse through data bricks note books

Hi All, Happy new year!how can we connect to azure synapse serverless sql pool through databricks notebooks and execute DDLs

  • 1803 Views
  • 4 replies
  • 6 kudos
Latest Reply
daniel_sahal
Esteemed Contributor
  • 6 kudos

@KVNARK .​ https://joeho.xyz/blog-posts/how-to-connect-to-azure-synapse-in-azure-databricks/

  • 6 kudos
3 More Replies
db-avengers2rul
by Contributor II
  • 952 Views
  • 1 replies
  • 0 kudos

Jupyter notebooks import in databricks notebooks

Dear Team,Is it possible to import jupyter notebooks in databricks community edition ? if yes will there be any formatting issues ?

  • 952 Views
  • 1 replies
  • 0 kudos
Latest Reply
db-avengers2rul
Contributor II
  • 0 kudos

if yes is there any limit ? , what is the difference or advantage using juypter notebooks over databricks notebooks

  • 0 kudos
bharani
by New Contributor II
  • 444 Views
  • 0 replies
  • 0 kudos

coverage in databricks

Hi Prabakar, I would like to do test coverage in databricks notebooks so is there any way that I can do and also can I do testing with pytest in databricks? Are there any other tests like nutter library and doc test supports coverage? 

  • 444 Views
  • 0 replies
  • 0 kudos
Gim
by Contributor
  • 47029 Views
  • 5 replies
  • 9 kudos

Best practice for logging in Databricks notebooks?

What is the best practice for logging in Databricks notebooks? I have a bunch of notebooks that run in parallel through a workflow. I would like to keep track of everything that happens such as errors coming from a stream. I would like these logs to ...

  • 47029 Views
  • 5 replies
  • 9 kudos
Latest Reply
Kaniz
Community Manager
  • 9 kudos

Hi @Gimwell Young​ , We haven’t heard from you on the last response from @karthik p​​, @Debayan Mukherjee​ and @Hubert Dudek​  and I was checking back to see if their suggestions helped you. Or else, If you have any solution, please do share that wit...

  • 9 kudos
4 More Replies
ivanychev
by Contributor
  • 2697 Views
  • 7 replies
  • 2 kudos

How do I enable new Visualization tools in my Databricks notebooks?

Databricks released new visualization toolshttps://docs.databricks.com/notebooks/visualizations/index.htmlBut I don't see them in my notebook (DBR 10.4) nor I can find out how to enable them. How do I use the new tools?

Screenshot 2022-07-21 at 21.59.19
  • 2697 Views
  • 7 replies
  • 2 kudos
Latest Reply
cweymouth
New Contributor II
  • 2 kudos

@Atanu Sarkar​ , I am still not seeing this. I am running a cluster on runtime 10.4 LTS ML. The "Legacy Visualizations" are the only option available. This feature should be fully deployed, yes? 

  • 2 kudos
6 More Replies
jowziph
by New Contributor III
  • 2566 Views
  • 2 replies
  • 7 kudos

How can I fix strange magic command behaviour in Databricks notebooks?

Hi,I have encountered a strange issue when using magic commands on my Azure Databricks workspace.When I create a cell and type the magic command for a language which is not default in the notebook, the code is instead interpreted as the default noteb...

magic command manually typed magic command manually typed 2 image
  • 2566 Views
  • 2 replies
  • 7 kudos
Latest Reply
jowziph
New Contributor III
  • 7 kudos

Hi,Yeah this is happening when magic command is used in between two cells of the default language. But this is not the only case.This is happening when magic command is manually typed out (e.g. %sql or %scala in a python notebook). OR when magic comm...

  • 7 kudos
1 More Replies
rgrosskopf
by New Contributor II
  • 3345 Views
  • 2 replies
  • 1 kudos

How to access secrets in Hashicorp Vault from Databricks notebooks?

I see in this blog post that Databricks supports Hashicorp Vault for secrets storage but I've been unable to find any additional details on how that would work. Specifically, how would I authenticate to Vault from within a Databricks notebook?

  • 3345 Views
  • 2 replies
  • 1 kudos
Latest Reply
jose_gonzalez
Moderator
  • 1 kudos

Hi @Ryan Grosskopf​,Just a friendly follow-up. Did any Prabakar's responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

  • 1 kudos
1 More Replies
Raymond_Garcia
by Contributor II
  • 1405 Views
  • 5 replies
  • 3 kudos

Migrating from Databricks Notebooks to IDE for Development

Hello, we are developers who have been creating a system in Databricks with Scala. We enabled the Git feature, so the project is in a repository. The project has a lot of notebooks and a lot of calls to other notebooks. Sometimes it is a little overw...

  • 1405 Views
  • 5 replies
  • 3 kudos
Latest Reply
Raymond_Garcia
Contributor II
  • 3 kudos

it is true that we can't work without data bricks but we can develop an IDE and send the jar to databricks, this will allow us to create unit tests, and use the IDE capabilities (i.e fast navigation among classes).

  • 3 kudos
4 More Replies
Doaa_Rashad
by New Contributor III
  • 4586 Views
  • 9 replies
  • 11 kudos

Resolved! import Github repo into Databricks

I am trying to import some data from a public repo in GitHub so that to use it from my Databricks notebooks.So far I tried to connect my Databricks account with my GitHub as described here, without results though since it seems that GitHub support co...

image.png image.png
  • 4586 Views
  • 9 replies
  • 11 kudos
Latest Reply
Kaniz
Community Manager
  • 11 kudos

Hi @Doaa MohamedRashad​​, We haven’t heard from you on the last response from me, and I was checking back to see if you have a resolution yet. If you have any solution, please share it with the community as it can be helpful to others. Otherwise, we ...

  • 11 kudos
8 More Replies
SajiD
by New Contributor
  • 790 Views
  • 1 replies
  • 0 kudos

Snowflake Connector for Databricks

Hi everyone, I am working with Databricks Notebooks and I am facing an issue with snowflake connector, I wanted to use DDL/DML with snowflake connector. Can someone please help me out with this, Thanks in advance !!

  • 790 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @Sajid Shaikh​ , This article explains how to read data from and write data to Snowflake using the Databricks Snowflake connector.

  • 0 kudos
LanceYoung
by New Contributor III
  • 6626 Views
  • 7 replies
  • 6 kudos

Resolved! Unable to make Databricks API calls from an HTML iframe rendered by a notebook's `displayHTML()` call, due to the browser enforcing CORS policy.

My GoalI want to make my Databricks Notebooks more interactive and have custom HTML/JS UI widgets that guide non-technical people through a business/data process. I want the HTML/JS widget to be able to execute a DB job, or execute some python code t...

  • 6626 Views
  • 7 replies
  • 6 kudos
Latest Reply
Kaniz
Community Manager
  • 6 kudos

Hi @Lance Young​ , Just a friendly follow-up. Do you still need help, or have you resolved your problem using the above solutions? Please let us know.

  • 6 kudos
6 More Replies
DoD
by New Contributor III
  • 1135 Views
  • 2 replies
  • 1 kudos

Resolved! Why are R scripts inside of Databricks notebooks creating writeLines errors?

I recently posted this in Stack Overflow. I'm using R in Databricks. R Studio runs fine and executes from the Databricks cluster. I would like to transition from R Studio to notebooks. When I start the cluster, R seems to run fine from notebooks. ...

  • 1135 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

@Paul Evangelista​ - Thank you for letting us know. You did great!Would you be happy to mark your answer as best so that others can find your solution more easily?

  • 1 kudos
1 More Replies
Anonymous
by Not applicable
  • 552 Views
  • 1 replies
  • 0 kudos

Is there an equivalent of the %debug from Jupyter notebooks in Databricks notebooks for debugging python notebooks?

Is there an equivalent of the %debug from Jupyter notebooks in Databricks notebooks for debugging python notebooks?

  • 552 Views
  • 1 replies
  • 0 kudos
Latest Reply
Dileep_Vidyadar
New Contributor III
  • 0 kudos

Hi @Nathan Tong​ You can go through the 2 articles below that I found online for Debugging in Databricks.1. 7 Tips to Debug Apache Spark Code Faster with Databricks 2. Easier Spark Code Debugging

  • 0 kudos
Labels