cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

pret
by New Contributor II
  • 4757 Views
  • 4 replies
  • 0 kudos

How can I run a scala command line in databricks?

I wish to run a scala command, which I believe would normally be run from a scala command line rather than from within a notebook. It happens to be:scala [-cp scalatest-<version>.jar:...] org.scalatest.tools.Runner [arguments](scalatest_2.12__3.0.8.j...

  • 4757 Views
  • 4 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @David Vardy​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks...

  • 0 kudos
3 More Replies
hafeez
by Contributor
  • 4814 Views
  • 4 replies
  • 7 kudos

Azure Databricks SCIM API GA Availability plans?

When is the Azure SCIM API 2.0 is going General Availability. Currently I see it is on public preview?And also any security concerns for the APIs In Preview if it is being used in our current production?Reference: https://learn.microsoft.com/en-us/az...

  • 4814 Views
  • 4 replies
  • 7 kudos
Latest Reply
Anonymous
Not applicable
  • 7 kudos

Hi @Sultan Mohideen Hafeez Aboobacker Mohammed Shahul​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more h...

  • 7 kudos
3 More Replies
User16826994223
by Databricks Employee
  • 1834 Views
  • 2 replies
  • 0 kudos

Lakehouse Concept

I want to understand lake house concept in very brief If I have to pitch for a customer in 1 minute

  • 1834 Views
  • 2 replies
  • 0 kudos
Latest Reply
User16826994223
Databricks Employee
  • 0 kudos

Lakehouse is a concept defined with the following Parameter-Data is stored in an open standard format.Data is stored in a way which support Data Science,ML and BI loads.Delta is just a way or engine on cloud storage that provides control on data and...

  • 0 kudos
1 More Replies
CatherineDalzel
by New Contributor II
  • 2784 Views
  • 2 replies
  • 0 kudos

Sign in to Databricks Partner Academy

A few months ago, I signed up for the Databricks Academy, customer version. Since then, my company has become a Databricks Partner (Go Databricks + Stardog!). My colleagues have successfully registered for Databricks Partner Academy, but I can't get ...

  • 2784 Views
  • 2 replies
  • 0 kudos
User16790091296
by Databricks Employee
  • 3802 Views
  • 2 replies
  • 2 kudos

How to configure Databricks token inside Docker File?

I have a docker file where I want toDownload the Databricks CLIConfigure the CLI by adding a host and tokenAnd then running a python file that hits the Databricks tokenI am able to install the CLI in the docker image, and I have a working python file...

  • 3802 Views
  • 2 replies
  • 2 kudos
Latest Reply
sachingawade
New Contributor II
  • 2 kudos

Hi I was facing same issue and searching for the solution but didnt get it, and now after working on it i have the solution if you want to access databricks models/download_artifacts using hostname and access token like how you do on databricks cli ...

  • 2 kudos
1 More Replies
Karthikeyan1
by New Contributor II
  • 2064 Views
  • 1 replies
  • 2 kudos

How to get the Databricks Default Pricing through API or any endpoints (*if available)???

Attaching Screenshots FYI from the official site, I've checked the Inspect, but no API calls have been made specifically for this cost default scrapping, Is there any Endpoints available to scrape this?

image.png
  • 2064 Views
  • 1 replies
  • 2 kudos
Latest Reply
Karthikeyan1
New Contributor II
  • 2 kudos

@All Users Group​ Any views on this??

  • 2 kudos
antonio_rodilos
by New Contributor II
  • 3740 Views
  • 3 replies
  • 1 kudos

Exam Registration & Voucher

I would like to register for the Data Engineer Associate exam. I have also passed the Databricks Lakehuse Fundamentals accreditation (id: 61076337), but on the registration site: https://www.webassessor.com/ does not allow me to purchase the exam, bu...

  • 3740 Views
  • 3 replies
  • 1 kudos
Latest Reply
BkP
Contributor
  • 1 kudos

Hi @Antonio Rodilosso​ ,As far as I know voucher is not compulsory. You can register for an exam by paying the amount. Its absolutely possible to register for an exam without voucherTo earn a voucher you can check the below url.http://msdatalab.net/h...

  • 1 kudos
2 More Replies
Serhii
by Contributor
  • 2374 Views
  • 0 replies
  • 0 kudos

Horovod Databricks Job - custom module not found error

We have used the following example to successfully create a distributed deep learning training notebook https://www.databricks.com/blog/2022/09/07/accelerating-your-deep-learning-pytorch-lightning-databricks.html that works as expected.We now want to...

  • 2374 Views
  • 0 replies
  • 0 kudos
Serhii
by Contributor
  • 4230 Views
  • 3 replies
  • 6 kudos

Append to a record when updating Delta Table

I am updating the delta table in Databricks as follows segments_data.alias('segments_old').merge( segments_data_new.alias("updates"), "segments_old.source_url = updates.source_url", ).whenMatchedUpdate( set={"segments"...

  • 4230 Views
  • 3 replies
  • 6 kudos
Latest Reply
Serhii
Contributor
  • 6 kudos

Yes it worked for us - thanks

  • 6 kudos
2 More Replies
ossinova
by Contributor II
  • 2605 Views
  • 1 replies
  • 3 kudos

Resolved! Shortcut for changing cell language (adding magic command)

Is there an option or shortcut (or is it in the pipeline) to quickly change / insert a cell to a specific language in Databricks?Triggering B + P would for instance add a new cell below with %pythonTriggering M would for instance change that cell to ...

  • 2605 Views
  • 1 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

I've passed this message along to the UX UI Feedback team.

  • 3 kudos
hari
by Contributor
  • 3739 Views
  • 2 replies
  • 5 kudos

Resolved! Best way to automatically update a delta table schema

We have multiple environments where the same tables are added so it's really hard to manually update the schema of the table across all the environments. We know that it's not ideal to update table schema a lot but our product is still evolving and s...

  • 3739 Views
  • 2 replies
  • 5 kudos
Latest Reply
hari
Contributor
  • 5 kudos

Thanks for the reply @Pat Sienkiewicz​ .

  • 5 kudos
1 More Replies
elgeo
by Valued Contributor II
  • 2775 Views
  • 1 replies
  • 1 kudos

Merge didn't fail while inserting wrong data type values

Hello. During some example cases we were running, in order to identify how Databricks treats possible wrong actions we could make, we noticed that merge doesn't fail while inserting different data type values from the ones in the corresponding table....

Target table schema merge_failure image merge_failure3
  • 2775 Views
  • 1 replies
  • 1 kudos
Latest Reply
elgeo
Valued Contributor II
  • 1 kudos

Hello. Any update on this please? Thank you in advance

  • 1 kudos
PriyanshuGupta
by New Contributor II
  • 3090 Views
  • 3 replies
  • 1 kudos

Pass_SS

Have completed my certificate on 27th October for Databricks Data Analyst Associate but haven't received my Certificate/ Badge for the same.

  • 3090 Views
  • 3 replies
  • 1 kudos
Latest Reply
Nadia1
Databricks Employee
  • 1 kudos

Hello Priyanshu, your certificate was issued today. Please check your email as I resent the information.Thank you

  • 1 kudos
2 More Replies
BkP
by Contributor
  • 3231 Views
  • 2 replies
  • 3 kudos

Scala Connectivity to Databricks Bronze Layer Raw Data from a Non-Databricks Spark environment

Hi All, We are developing a new Scala/Java program which needs to read & process the raw data stored in source ADLS (which is a Databricks Environment) in parallel as the volume of the source data is very high (in GBs & TBs). What kind of connection ...

requirement
  • 3231 Views
  • 2 replies
  • 3 kudos
Latest Reply
BkP
Contributor
  • 3 kudos

hello experts. any advise on this question ?? tagging some folks from whom I have received answers before. Please help on this requirement or tag someone who can help on this@Kaniz Fatma​ , @Vartika Nain​ , @Bilal Aslam​ 

  • 3 kudos
1 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels