cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Cloud_Architect
by New Contributor III
  • 795 Views
  • 1 replies
  • 0 kudos

Need help calculating the cost benefits of switching from interactive to job cluster

I need help calculating the cost benefits of switching from interactive to job cluster. Can you help me get some formulas on how to calculate the cost differences in Databricks?

  • 795 Views
  • 1 replies
  • 0 kudos
Latest Reply
jacovangelder
Honored Contributor
  • 0 kudos

Assuming you're on Azure (otherwise use the AWS/GCP equivalent), did you try the Azure cost calculator? https://azure.microsoft.com/en-us/pricing/details/databricks/Question to ask yourself to get more specific: Do you have an idea how much DBU's you...

  • 0 kudos
MadCowTM
by New Contributor II
  • 3882 Views
  • 1 replies
  • 2 kudos

Resolved! get_json_object and json path filtering

I have following string [{"key":"abc","value":{"string_value":"abc123"}},{"key":"def","value":{"int_value":123}},{"key":"ghi","value":{"string_value":"ghi456"}}] and from that string i need to extract key.value.string_value for key with the value equ...

  • 3882 Views
  • 1 replies
  • 2 kudos
Latest Reply
brickster_2018
Databricks Employee
  • 2 kudos

Can you try with the below code snippet WITH exploded_json AS ( SELECT explode(from_json( '[{"key":"abc","value":{"string_value":"abc123"}},{"key":"def","value":{"int_value":123}},{"key":"ghi","value":{"string_value":"ghi456"}}]', 'array<s...

  • 2 kudos
unity_Catalog
by New Contributor III
  • 971 Views
  • 0 replies
  • 0 kudos

UCX installation error

I am getting the below error while Installing UCX. But Installation is done in the workspace.I have admin privileges on the workspace. The below error suggests to check token or URL of workspace.They are provided correctly.Then why below error is sho...

  • 971 Views
  • 0 replies
  • 0 kudos
Rapha
by New Contributor II
  • 1908 Views
  • 4 replies
  • 0 kudos

Error when cloning repository

Hi all,cloning a devops repo worked like a charm every time, but now I get a weird error, that I do not understandWhat container is meant here and why would I need one?Thanks and regards,Raphael

Rapha_0-1717503863381.png
  • 1908 Views
  • 4 replies
  • 0 kudos
Latest Reply
Rapha
New Contributor II
  • 0 kudos

@Yeshwanth I am sorry, I thought I replied to you on friday. I cannot upload a .har file. I get the error message:The file type (.har) is not supported. Valid file types are: jpg, gif, png, pdf.

  • 0 kudos
3 More Replies
kanishkmalik199
by New Contributor
  • 3863 Views
  • 3 replies
  • 1 kudos

Certification Coupon Voucher Not Received

Hi Team I have attended the webinar long back and completed the survey as well still i have not receive any coupon voucher and because of that not able to give exam . Please look into this isue asap and provide me with the voucher .

  • 3863 Views
  • 3 replies
  • 1 kudos
Latest Reply
Bharathkumar
Databricks Employee
  • 1 kudos

Hi Guilherme, We can see that, whereas you used the email address "guilherme.santos@bridgeconsulting.com.br" to attend the webinar, but you did not complete the survey. So you have not received the voucher code.

  • 1 kudos
2 More Replies
jamesdavids
by New Contributor
  • 2059 Views
  • 1 replies
  • 0 kudos

Spark Excel Library Insufficient Privileges

HiWe have a shared access mode cluster in which we have installed a maven library for reading excel files into a Spark DataFrame. When using an account with admin rights everything works fine, however when we run it as a standar user we always get `o...

jamesdavids_0-1717747232287.png
  • 2059 Views
  • 1 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

Hello James, this issue seems to be related to shared cluster limitations, as per docs: Libraries used as JDBC drivers or custom Spark data sources on Unity Catalog-enabled shared compute require ANY FILE permissions.Moving to a single user cluster m...

  • 0 kudos
JameDavi_51481
by Contributor
  • 1810 Views
  • 1 replies
  • 0 kudos

How to escape column comments when adding them programmatically

I would like to add comments to all of our columns, programmatically. The only way I can find to do this is through SQL DDL - e.g. `alter table sometable alter column somecolumn comment 'some comment string`. However, I want to read this comment stri...

  • 1810 Views
  • 1 replies
  • 0 kudos
Latest Reply
brockb
Databricks Employee
  • 0 kudos

Hi @JameDavi_51481 ,I do not believe this is supported currently, please see this line in the Parameter Marker documentation: > You must not reference a parameter marker in a DDL statement...Ref: https://docs.databricks.com/en/sql/language-manual/sql...

  • 0 kudos
amit-agarwal4
by New Contributor II
  • 1834 Views
  • 2 replies
  • 0 kudos

Resolved! how to clone a table using Column and row level mask

I am working on use case to clone a table which has column and row level masking implemented.Can i clone the table, based on documentation it is not possible. what are the other alternatives i have ?

  • 1834 Views
  • 2 replies
  • 0 kudos
Latest Reply
raphaelblg
Databricks Employee
  • 0 kudos

@amit-agarwal4 At the current moment, I'm not aware of any other way than disabling the RLS.     

  • 0 kudos
1 More Replies
JamesY
by New Contributor III
  • 3674 Views
  • 6 replies
  • 0 kudos

Resolved! Retrieve error column/row when writing to sqlmi

Databricks notebook, Scala, .read() .write(), source data csv. Got this error when trying to write data to sqlmi. I understand the error indicated one of the source column value's length exceeded the database column's value length. But the message is...

Get Started Discussions
Databricks
Scala
SqlMi
  • 3674 Views
  • 6 replies
  • 0 kudos
Latest Reply
brockb
Databricks Employee
  • 0 kudos

Thanks JamesY, I'm not familiar with the limitations of the SQL Server `nvarchar` data type but is there a way that we can filter out the rows that will fail using spark such as: spark.read.format("csv").option("header", "true").load("/path/to/csvs")...

  • 0 kudos
5 More Replies
claudiazi
by New Contributor II
  • 4931 Views
  • 5 replies
  • 1 kudos

Resolved! [TABLE_OR_VIEW_ALREADY_EXISTS] when running create or replace view

When I'm running `dbt run -s model` for different models in parallel on databricks general compute cluster. Im getting the error: TABLE_OR_VIEW_ALREADY_EXISTS. Also, at the same time, the view/table was not created at all.But I run them in sequence, ...

  • 4931 Views
  • 5 replies
  • 1 kudos
Latest Reply
claudiazi
New Contributor II
  • 1 kudos

@raphaelblg yes! Im the owner. However, these views are inside the hive_metastore catalog. Could it be the reason?Many thaaanks!

  • 1 kudos
4 More Replies
AchintyaSingh
by New Contributor
  • 2047 Views
  • 1 replies
  • 0 kudos

Running JAR jobs in parallel on a cluster

Hi everyone, I'm trying to find out if databricks has support for clusters which can scale out with more drivers to run new jobs in parallel. If not, then is there a work around for this? I've noticed that all-purpose and job compute clusters both fe...

  • 2047 Views
  • 1 replies
  • 0 kudos
Latest Reply
Yeshwanth
Databricks Employee
  • 0 kudos

@AchintyaSingh  Databricks does not support clusters with multiple drivers to run new jobs in parallel. Each Databricks cluster has a single driver node, allowing only one job at a time.Workarounds for Achieving Parallel Job Execution: 1. Multiple Cl...

  • 0 kudos
hukel
by Contributor
  • 4363 Views
  • 2 replies
  • 0 kudos

InconsistentReadException: The file might have been updated during query - CSV backed table

I have some CSV files that I upload to DBFS storage several times a day.   From these CSVs,  I have created SQL tables: CREATE TABLE IF NOT EXISTS masterdata.lookup_host USING CSV OPTIONS (header "true", inferSchema "true") LOCATION '/mnt/masterdata/...

  • 4363 Views
  • 2 replies
  • 0 kudos
Latest Reply
hukel
Contributor
  • 0 kudos

One approach I'm testing (positive results so far, but still early).%sql # Prep and cleanup REFRESH TABLE masterdata.lookup_host; DROP TABLE IF EXISTS t_hosts; # Forcibly cache the needed columns before using the data in another query. CACHE TABLE...

  • 0 kudos
1 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels