- 241 Views
- 0 replies
- 0 kudos
Amazing experience at the DAIS 24 so far
Looking forward to the amazing updates with Gen AI during tomorrow's keynote
- 241 Views
- 0 replies
- 0 kudos
Forum Posts
- 351 Views
- 1 replies
- 0 kudos
What have you learned today?
Learned about Managed tables
- 351 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Delta Sharing is pretty sweet!
- 0 kudos
- 252 Views
- 0 replies
- 0 kudos
Summit 2024
I've loved getting to meet everyone this week!
- 252 Views
- 0 replies
- 0 kudos
- 431 Views
- 1 replies
- 0 kudos
Need help calculating the cost benefits of switching from interactive to job cluster
I need help calculating the cost benefits of switching from interactive to job cluster. Can you help me get some formulas on how to calculate the cost differences in Databricks?
- 431 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Assuming you're on Azure (otherwise use the AWS/GCP equivalent), did you try the Azure cost calculator? https://azure.microsoft.com/en-us/pricing/details/databricks/Question to ask yourself to get more specific: Do you have an idea how much DBU's you...
- 0 kudos
- 658 Views
- 0 replies
- 0 kudos
Unable to import ml.dmlc.xgboost4j.scala.spark.XGBoostClassifier
Hi I am trying to use the Scala XGBoostClassifier on Databricks, inspiring by this https://www.youtube.com/watch?v=vD3k-moYgmU video shared by Dbx official account. But I couldn't import the library.I followed the official tutorial https://docs.datab...
- 658 Views
- 0 replies
- 0 kudos
- 1703 Views
- 1 replies
- 2 kudos
Resolved! get_json_object and json path filtering
I have following string [{"key":"abc","value":{"string_value":"abc123"}},{"key":"def","value":{"int_value":123}},{"key":"ghi","value":{"string_value":"ghi456"}}] and from that string i need to extract key.value.string_value for key with the value equ...
- 1703 Views
- 1 replies
- 2 kudos
Latest Reply
- 2 kudos
Can you try with the below code snippet WITH exploded_json AS ( SELECT explode(from_json( '[{"key":"abc","value":{"string_value":"abc123"}},{"key":"def","value":{"int_value":123}},{"key":"ghi","value":{"string_value":"ghi456"}}]', 'array<s...
- 2 kudos
- 626 Views
- 0 replies
- 0 kudos
UCX installation error
I am getting the below error while Installing UCX. But Installation is done in the workspace.I have admin privileges on the workspace. The below error suggests to check token or URL of workspace.They are provided correctly.Then why below error is sho...
- 626 Views
- 0 replies
- 0 kudos
- 1160 Views
- 4 replies
- 0 kudos
Error when cloning repository
Hi all,cloning a devops repo worked like a charm every time, but now I get a weird error, that I do not understandWhat container is meant here and why would I need one?Thanks and regards,Raphael
- 1160 Views
- 4 replies
- 0 kudos
Latest Reply
- 0 kudos
@Yeshwanth I am sorry, I thought I replied to you on friday. I cannot upload a .har file. I get the error message:The file type (.har) is not supported. Valid file types are: jpg, gif, png, pdf.
- 0 kudos
- 403 Views
- 0 replies
- 0 kudos
Disaster Recovery for Databricks Unity catalog enab;led workspace
Hi I have a unity catalog enabled workspaces in us-east-1 and would like to setup DR Databricks workspace in us-east-2. How can we achieve as I don't see a clear path for syncing metadata and delta tables across regions.
- 403 Views
- 0 replies
- 0 kudos
- 1026 Views
- 1 replies
- 0 kudos
Spark Excel Library Insufficient Privileges
HiWe have a shared access mode cluster in which we have installed a maven library for reading excel files into a Spark DataFrame. When using an account with admin rights everything works fine, however when we run it as a standar user we always get `o...
- 1026 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello James, this issue seems to be related to shared cluster limitations, as per docs: Libraries used as JDBC drivers or custom Spark data sources on Unity Catalog-enabled shared compute require ANY FILE permissions.Moving to a single user cluster m...
- 0 kudos
- 776 Views
- 1 replies
- 0 kudos
How to escape column comments when adding them programmatically
I would like to add comments to all of our columns, programmatically. The only way I can find to do this is through SQL DDL - e.g. `alter table sometable alter column somecolumn comment 'some comment string`. However, I want to read this comment stri...
- 776 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @JameDavi_51481 ,I do not believe this is supported currently, please see this line in the Parameter Marker documentation: > You must not reference a parameter marker in a DDL statement...Ref: https://docs.databricks.com/en/sql/language-manual/sql...
- 0 kudos
- 1030 Views
- 2 replies
- 0 kudos
Resolved! how to clone a table using Column and row level mask
I am working on use case to clone a table which has column and row level masking implemented.Can i clone the table, based on documentation it is not possible. what are the other alternatives i have ?
- 1030 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
@amit-agarwal4 At the current moment, I'm not aware of any other way than disabling the RLS.
- 0 kudos
- 1224 Views
- 2 replies
- 0 kudos
DAB not updating zone_id when redeployed
Hey folks,Facing an issue with zone_id not getting overridden when redeploying the DAB template to Databricks workspace.The Databricks job is already deployed and has "ap-south-1a" zone_id. I wanted to make it "auto" so, I have made the changes to th...
- 1224 Views
- 2 replies
- 0 kudos
Latest Reply
- 1988 Views
- 6 replies
- 0 kudos
Resolved! Retrieve error column/row when writing to sqlmi
Databricks notebook, Scala, .read() .write(), source data csv. Got this error when trying to write data to sqlmi. I understand the error indicated one of the source column value's length exceeded the database column's value length. But the message is...
- 1988 Views
- 6 replies
- 0 kudos
Latest Reply
- 0 kudos
Thanks JamesY, I'm not familiar with the limitations of the SQL Server `nvarchar` data type but is there a way that we can filter out the rows that will fail using spark such as: spark.read.format("csv").option("header", "true").load("/path/to/csvs")...
- 0 kudos
- 2111 Views
- 5 replies
- 1 kudos
Resolved! [TABLE_OR_VIEW_ALREADY_EXISTS] when running create or replace view
When I'm running `dbt run -s model` for different models in parallel on databricks general compute cluster. Im getting the error: TABLE_OR_VIEW_ALREADY_EXISTS. Also, at the same time, the view/table was not created at all.But I run them in sequence, ...
- 2111 Views
- 5 replies
- 1 kudos
Latest Reply
- 1 kudos
@raphaelblg yes! Im the owner. However, these views are inside the hive_metastore catalog. Could it be the reason?Many thaaanks!
- 1 kudos
