cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

744291
by Databricks Partner
  • 1975 Views
  • 4 replies
  • 0 kudos

I have not received my voucher yet.Even mails are not replied​

I have not received my voucher yet.Even mails are not replied​

  • 1975 Views
  • 4 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Rituparna Das​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so...

  • 0 kudos
3 More Replies
satya123
by Databricks Partner
  • 5450 Views
  • 2 replies
  • 0 kudos

How to trigger jobs after previous job are executed (I want to orchestrate jobs not tasks)?

I know scheduling options are there, but it doesn't consider interdependency like don't execute job2 unless job1 has been executed.I know, script based, or API based triggers are also there, but I am looking for UI based triggers like how we can orch...

image.png
  • 5450 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @satyam rastogi​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us s...

  • 0 kudos
1 More Replies
Geetha
by New Contributor
  • 4354 Views
  • 2 replies
  • 0 kudos

Connection to Databricks through KNIME

I want to make a connection to Databricks with KNIME. To do this I am using "Create Databricks Environment" node. I have made the following configuration:1. Installed Databricks Simba JDBC driver 2. Made the necessary configuration in Create Databric...

  • 4354 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Geethanjali Nataraj​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell...

  • 0 kudos
1 More Replies
Jfoxyyc
by Valued Contributor
  • 2325 Views
  • 3 replies
  • 0 kudos

Delta Live Table with Autoloader issue

Using autoloader, I'm reading daily data partitioned by well. The data has a specific schema, but if there's no value for a column it isn't present in the json. For a specific column on a specific table I'm getting an error like:​Cannot convert long ...

  • 2325 Views
  • 3 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Jordan Fox​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we...

  • 0 kudos
2 More Replies
Joao_DE
by New Contributor III
  • 4662 Views
  • 3 replies
  • 1 kudos

Information about stoping JDBC connection

Hi everyone! I would like to know how spark stops the connection when reading from a sql database using the JDBC format. Also, if there is a way to check when the connection is active or manually stop it, I also would like to know. Thank you in advan...

  • 4662 Views
  • 3 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @João Peixoto​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so ...

  • 1 kudos
2 More Replies
SID
by Databricks Partner
  • 2229 Views
  • 2 replies
  • 0 kudos

Partner onboarding next steps

Hi team, our application for partner onboarding was processed a year ago. The last email we received was from Procurement Operations Specialist confirming our information has been added to the system. However, we never received any advise on the next...

  • 2229 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Siddhesh Gaikwad​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us...

  • 0 kudos
1 More Replies
pankajBhatt
by New Contributor II
  • 2240 Views
  • 1 replies
  • 1 kudos

Databricks not able to access latest files in Azure ADLS Gen1

I have mounted my path from Databricks to AzureADLS Gen1. using SPN as service accuntuntill yesterday everything was ok, but today I see, I can view all older deleted folders. I can not see them in ADLS . but my databricks dbutils.fs.ls() shows them....

  • 2240 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @pankaj bhatt​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so ...

  • 1 kudos
744291
by Databricks Partner
  • 2540 Views
  • 4 replies
  • 0 kudos

Anyone having extra voucher who are not plannning to give exams

Anyone having extra voucher who are not plannning to give exams

  • 2540 Views
  • 4 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Rituparna Das​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so...

  • 0 kudos
3 More Replies
Trung
by Contributor
  • 2989 Views
  • 4 replies
  • 0 kudos

best practice for managing resources to avoid creators leaving out of the project

Dear DB community!As I know, when the resource creator is out of the project all resources they create will be deleted as well. So the question is that: can I assign the owner role to a group, that will help protect the resource from deletion or not...

  • 2989 Views
  • 4 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @trung nguyen​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedback...

  • 0 kudos
3 More Replies
kll
by New Contributor III
  • 4104 Views
  • 2 replies
  • 0 kudos

Unable to render widget to display map within Jupyter notebook output cell

I am attempting to render a map within jupyter notebook and keep bumping into output limit. Below is my code: import pydeck as pdk   import pandas as pd       COLOR_BREWER_BLUE_SCALE = [   [240, 249, 232],   [204, 235, 197],   [168, 221, 181], ...

  • 4104 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Keval Shah​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedback w...

  • 0 kudos
1 More Replies
KVNARK
by Honored Contributor II
  • 3100 Views
  • 2 replies
  • 6 kudos

Resolved! Microsoft ADB file from BLOB into Azure Databricks

Can anyone let me know how we can load the database file into Azure Databricks from the azure blob.

  • 3100 Views
  • 2 replies
  • 6 kudos
Latest Reply
Anonymous
Not applicable
  • 6 kudos

Hi @KVNARK .​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedback wil...

  • 6 kudos
1 More Replies
Michael_Papadop
by New Contributor II
  • 15326 Views
  • 3 replies
  • 1 kudos

How can I set the status of a databricks job as skipped via python?

I have a basic 2 task job. The 1st notebook (task) checks whether the source file has changes and if so then refreshes a corresponding materialized view. In case we have no changes then I use dbutils.jobs.taskValues.set(key = "skip_job", value = 1) &...

  • 15326 Views
  • 3 replies
  • 1 kudos
Latest Reply
karthik_p
Databricks Partner
  • 1 kudos

@Michael Papadopoulos​ usually that should not be the case i think, as for task level we have 3 level notifications ( success, failure,start), where as whole job level skip option is available to discard notification . will see if some one from commu...

  • 1 kudos
2 More Replies
thushar
by Databricks Partner
  • 11410 Views
  • 5 replies
  • 0 kudos

Optimize & Compaction

Hi,From which data bricks runtime will support Optimize and compaction

  • 11410 Views
  • 5 replies
  • 0 kudos
Latest Reply
Joe_Suarez
New Contributor III
  • 0 kudos

Optimize and compaction are operations commonly used in Apache Spark for optimizing and improving the performance of data storage and processing. Databricks, which is a cloud-based platform for Apache Spark, provides support for these operations on v...

  • 0 kudos
4 More Replies
youssefmrini
by Databricks Employee
  • 2329 Views
  • 1 replies
  • 0 kudos
  • 2329 Views
  • 1 replies
  • 0 kudos
Latest Reply
youssefmrini
Databricks Employee
  • 0 kudos

Support for Jupyter notebooks (.ipynb files) is available in Repos. You can clone repositories with .ipynb notebooks, work in Databricks UI, and then commit and push as .ipynb notebooks. Metadata such as a notebook dashboard is preserved. Admins can ...

  • 0 kudos
Labels