cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Mikki007
by New Contributor II
  • 3983 Views
  • 3 replies
  • 0 kudos

How to extract the start and end time of the command line cell of the notebook using REST API in Azure Databricks?

HiI have a notebook with many command line cells in it.I want to extract the execution time of each cell using Databricks REST API? How can I do that?Please note - I managed to get the Start & End time of the Job using REST API (/2.1/jobs/runs/get) f...

  • 3983 Views
  • 3 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Milind Keer​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...

  • 0 kudos
2 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 749 Views
  • 1 replies
  • 29 kudos

The New DBR LTS runtime is here. Databricks Runtime 11.3 is set to be LTS!   

The New DBR LTS runtime is here. Databricks Runtime 11.3 is set to be LTS! 

11.3 LTS
  • 749 Views
  • 1 replies
  • 29 kudos
Latest Reply
jose_gonzalez
Moderator
  • 29 kudos

Great post, thank you for sharing this @Hubert Dudek​ 

  • 29 kudos
HQJaTu
by New Contributor III
  • 3394 Views
  • 12 replies
  • 1 kudos

Resolved! Azure Databricks container runtime broken in 9.1 LTS, how to fix?

For stability, I've stuck with LTS. Last Friday my containers stopped working with error message:Py4JException: An exception was raised by the Python Proxy. Return Message: Traceback (most recent call last): File "/databricks/spark/python/lib/py4j-...

  • 3394 Views
  • 12 replies
  • 1 kudos
Latest Reply
HQJaTu
New Contributor III
  • 1 kudos

This is getting worse. Now JDBC write to SQL is failing for same reason. I haven't yet found a solution for this.Am I not supposed to use containers? Python?This is not cool.

  • 1 kudos
11 More Replies
SepidehEb
by New Contributor III
  • 1829 Views
  • 6 replies
  • 7 kudos

Resolved! How to get a minor DBR image?

In short, we aim to add a step to a CI job that would run tests in a container, which supposedly should mimic DBR of our clusters – currently we use 7.3 . We consider using one of databricksruntime images (possibly a standard:7.x for now, https://hub...

  • 1829 Views
  • 6 replies
  • 7 kudos
Latest Reply
Atanu
Esteemed Contributor
  • 7 kudos

Hi @Sepideh Ebrahimi​ , since cluster is Databricks proprietary, you ca not run it locally. as @Werner Stinckens​  said, you can build your own image but that has to be run in cluster. but there is databricks connect (https://docs.databricks.com/dev-...

  • 7 kudos
5 More Replies
Anonymous
by Not applicable
  • 860 Views
  • 2 replies
  • 0 kudos

Changing default Delta behavior in DBR 8.x for writes

Is there anyway to add a Spark Config that reverts the default behavior when doing tables writes from Delta to Parquet in DBR 8.0+? I know you can simply specify .format("parquet") but that could involve a decent amount of code change for some client...

  • 860 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Thanks @Ryan Chynoweth​ !

  • 0 kudos
1 More Replies
Labels