cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
Diego_MSFT
by • New Contributor II
  • 4567 Views
  • 1 replies
  • 4 kudos

Automating the re run of job (with several Tasks) // automate the notification of a failed specific tasks after re trying // Error handling on azure data factory pipeline with DataBricks notebook

Hi DataBricks Experts:I'm using Databricks on Azure.... I'd like to understand the following:1) if there is way of automating the re run some specific failed tasks from a job (with several Tasks), for example if I have 4 tasks, and the task 1 and 2 h...

Data Engineering
Reply
  • 4567 Views
  • 1 replies
  • 4 kudos
View Replies
Latest Reply
Lindberg
New Contributor II
  • 4 kudos

You can use "retries".In Workflow, select your job, the task, and in the options below, configure retries.If so, you can also see more options at:https://learn.microsoft.com/pt-br/azure/databricks/dev-tools/api/2.0/jobs?source=recommendations

  • 4 kudos
bradm0
by • New Contributor III
  • 2621 Views
  • 3 replies
  • 3 kudos

Resolved! Use of badRecordsPath in COPY INTO SQL command

I'm trying to use the badRecordsPath to catch improperly formed records in a CSV file and continue loading the remainder of the file. I can get the option to work using python like thisdf = spark.read\ .format("csv")\ .option("header","true")\ .op...

Data Engineering
Reply
  • 2621 Views
  • 3 replies
  • 3 kudos
View Replies
Latest Reply
bradm0
New Contributor III
  • 3 kudos

Thanks. It was the inferSchema setting. I tried it with and without the SELECT and it worked both ways when I added inferSchemaBoth of these workeddrop table my_db.t2; create table my_db.t2 (col1 int,col2 int); copy into my_db.t2 from (SELECT cast(...

  • 3 kudos
2 More Replies
KuldeepChitraka
by • New Contributor III
  • 7404 Views
  • 4 replies
  • 6 kudos

Error handling/exception handling in NOtebook

What is a common practice to to write notebook which includes error handling/exception handling.Is there any example which depicts how notebook should be written to include error handling etc.

Data Engineering
Reply
  • 7404 Views
  • 4 replies
  • 6 kudos
View Replies
Latest Reply
Meghala
Valued Contributor II
  • 6 kudos

runtime looks for handlers (try-catch) that are registered to handle such exceptions

  • 6 kudos
3 More Replies
Labels