cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

FelipeRegis
by New Contributor II
  • 2354 Views
  • 0 replies
  • 1 kudos

Not able to access data registered in Unity Catalog using Simba ODBC driver

Hi folks, I'm working on a project with Databricks using Unity Catalog and a connection to SSIS (SQL Server Integration Services).My team is trying to access data registered in Unity Catalog using Simba ODBC driver version 2.8.0.1002. They mentioned ...

odbc_driver_unity_catalog_issue.png
  • 2354 Views
  • 0 replies
  • 1 kudos
sharma_kamal
by New Contributor III
  • 6627 Views
  • 3 replies
  • 4 kudos

Resolved! Unable to read available file after downloading

After downloading a file using `wget`, I'm attempting to read it by spark.read.json.I am getting error: PATH_NOT_FOUND - Path does not exist: dbfs:/tmp/data.json. SQLSTATE: 42K03File <command-3327713714752929>, line 2 I have checked the file do exist...

Get Started Discussions
dbfs
json
ls
wget
  • 6627 Views
  • 3 replies
  • 4 kudos
Latest Reply
Ayushi_Suthar
Databricks Employee
  • 4 kudos

Hi @sharma_kamal , Good Day!  Could you please try the below code suggested by @ThomazRossito , it will help you.  Also please refer to the below document to work with the files on Databricks:  https://docs.databricks.com/en/files/index.html Please l...

  • 4 kudos
2 More Replies
Raja_fawadAhmed
by New Contributor
  • 925 Views
  • 0 replies
  • 0 kudos

databricks job compute price w.r.t running time

I have two workflows (jobs) in data bricks (AWS) with below cluster specs (job base cluster NOT general purpose)Driver: i3.xlarge · Workers: i3.xlarge · 2-8 workers Job 1 takes 10 min to completeJob 2 takes 50 min to completeQuestions:DBU cost is sam...

  • 925 Views
  • 0 replies
  • 0 kudos
Chrispy
by New Contributor
  • 4443 Views
  • 0 replies
  • 0 kudos

Spark Handling White Space as NULL

I have a very strange thing happening.  I'm importing a csv file and nulls and blanks are being interpreted correctly.  What is strange is that a column that regularly has a single space character value is having the single space converted to null.I'...

  • 4443 Views
  • 0 replies
  • 0 kudos
Nandhini_Kumar
by New Contributor III
  • 1823 Views
  • 1 replies
  • 0 kudos

How to identify Worker and Driver instance in AWS console for databricks instance?

For AWS databricks, i have configured 1 worker and 1 driver node with same node type. In the AWS console, all details are same for the two instance, as instance id only different. How to identify which instance id is for worker and which one is for d...

  • 1823 Views
  • 1 replies
  • 0 kudos
Latest Reply
Nandhini_Kumar
New Contributor III
  • 0 kudos

Hi @Retired_mod Thanks for your response.There is no instance pool ID while configured the cluster, then how will i able to differentiate then.Could you give any alternative way for finding the driver instance id and worker instance id in the AWS con...

  • 0 kudos
Azsdc
by New Contributor
  • 2935 Views
  • 0 replies
  • 0 kudos

Usage of if else condition for data check

Hi,In a particular Workflows Job, I am trying to add some data checks in between each task by using If else statement. I used following statement in a notebook to call parameter in if else condition to check logic.{"job_id": XXXXX,"notebook_params": ...

  • 2935 Views
  • 0 replies
  • 0 kudos
Ha2001
by New Contributor
  • 2790 Views
  • 1 replies
  • 1 kudos

Databricks Repos API Limitations

Hi, I have started using databricks recently, and I'm not able find a right solution in the documentations. i have linked multiple repos in my databricks workspace in the repos folders, and I wanted to update the repos with remote AzureDevops reposit...

Get Started Discussions
azure devops
Databricks
REST API
  • 2790 Views
  • 1 replies
  • 1 kudos
Latest Reply
Ayushi_Suthar
Databricks Employee
  • 1 kudos

Hi @Ha2001 , Good Day!  Databricks API has a limit of 10 per second for the /repos/* combined requests in the workspace. You can check the below documentation for the API limit:  https://docs.databricks.com/en/resources/limits.html#:~:text=Git%20fold...

  • 1 kudos
jcozar
by Contributor
  • 5111 Views
  • 4 replies
  • 1 kudos

Resolved! Spark streaming query stops after code exception in notebook since 14.3

Hi!I am experiencing something that I cannot find in the documentation: in databricks, using the databricks runtime 13.X, when I start a streaming query (using .start method), it creates a new query and while it is running I can execute other code in...

  • 5111 Views
  • 4 replies
  • 1 kudos
Latest Reply
Lakshay
Databricks Employee
  • 1 kudos

You can use the help portal: https://help.databricks.com/s/

  • 1 kudos
3 More Replies
GlennStrycker2
by New Contributor III
  • 1760 Views
  • 1 replies
  • 1 kudos

Why so many different domains and accounts?

I've lost count of how many different domains and accounts Databricks is requiring for me to use their services.  Every domain is requiring its own account username, password, etc., and nothing is synced.  I can't even keep track of which email addre...

  • 1760 Views
  • 1 replies
  • 1 kudos
Latest Reply
GlennStrycker2
New Contributor III
  • 1 kudos

Pluscustomer-academy.databricks.comaccounts.cloud.databricks.comdatabricks.my.site.com

  • 1 kudos
HiraNisar
by New Contributor
  • 1227 Views
  • 0 replies
  • 0 kudos

AutoML in production

I have a workflow in Databricks and an AutoML pipeline in it.I want to deploy that pipeline in production, but I want to use the shared cluster in production, since AutoML is not compatible with the shared clusters, what can be the workaround.(Is it ...

  • 1227 Views
  • 0 replies
  • 0 kudos
Ajay-Pandey
by Esteemed Contributor III
  • 5103 Views
  • 3 replies
  • 1 kudos

Resolved! Databricks Private Preview Features

Hi All, I just wanted to try the new Databricks Workflow private preview features (For Each Task).Can someone please guide me on how we can enable it in our workspace as I am having the same use case in my current project where this feature can help ...

databricks_workflow.gif
Get Started Discussions
Databricks
dataengineering
privatepreview
  • 5103 Views
  • 3 replies
  • 1 kudos
Latest Reply
Ajay-Pandey
Esteemed Contributor III
  • 1 kudos

If you are using Azure Databricks just raise a support request regarding Private Preview they will enable it for you !

  • 1 kudos
2 More Replies
Benedetta
by New Contributor III
  • 10539 Views
  • 4 replies
  • 0 kudos

My notebooks running in parallel no longer include jobids

Hey Databricks - what happened to the jobids that used to be returned from parallel runs? We used them to identify which link matched the output. See attached. How are we supposed to match up the links? 

  • 10539 Views
  • 4 replies
  • 0 kudos
Latest Reply
Benedetta
New Contributor III
  • 0 kudos

Hey @Databricks, @Retired_mod  - waccha doing? Yesterday's "newer version of the app" that got rolled out seems to have broken the parallel runs. The ephemeral notebook is missing. The job ids are missing. What's up? Benedetta

  • 0 kudos
3 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels