cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

tessaickx
by New Contributor III
  • 3098 Views
  • 3 replies
  • 3 kudos

Using ipywidgets latest versions

Hello everyone,I upgraded my cluster to DBR 13.0, which comes with ipywidgets version 7.7.2 installed.However, I want to use the TagsInput widget, which is new since version 8.0.4.If i upgrade the ipywidgets package to version 8.0.4, none of the widg...

  • 3098 Views
  • 3 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Tessa Ickx​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers yo...

  • 3 kudos
2 More Replies
Vijesh
by New Contributor II
  • 3888 Views
  • 5 replies
  • 1 kudos

parsing error in Databricks SQL endpoint

I have two tables EMPLOYEE & EMPLOYEE_ROLE. I'm trying to Update a column with a value from another column. I'm using SQL server join but i get an error - [parse_syntax_error] Syntax error at or near 'FROM' line 3. UPDATE CSET C.title = B.title FROM ...

  • 3888 Views
  • 5 replies
  • 1 kudos
Latest Reply
sensanjoy
Contributor
  • 1 kudos

Hi @Vijesh V​ Try to use merge into to perform cdc between tables :MERGE INTO target aUSING source bON {merge_condition}WHEN MATCHED THEN {matched_action}WHEN NOT MATCHED THEN {not_matched_action}

  • 1 kudos
4 More Replies
tech2cloud
by New Contributor II
  • 2901 Views
  • 3 replies
  • 2 kudos
  • 2901 Views
  • 3 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Ravi Vishwakarma​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answ...

  • 2 kudos
2 More Replies
Anwar_Patel
by New Contributor III
  • 2414 Views
  • 3 replies
  • 0 kudos

Resolved! Not received my certificate after passing Databricks Certified Associate Developer for Apache Spark 3.0 - Python.

I've successfully passed Databricks Certified Associate Developer for Apache Spark 3.0 - Python but still have not received the certificate. E-mail : anwarpatel91@gmail.com

  • 2414 Views
  • 3 replies
  • 0 kudos
Latest Reply
Anwar_Patel
New Contributor III
  • 0 kudos

@Nadia Elsayed​ could you please help me with this issue I need to send my certificate to my team.

  • 0 kudos
2 More Replies
PraveenC
by New Contributor II
  • 3149 Views
  • 4 replies
  • 3 kudos

[Databricks][JDBC](10400) Invalid type for data - column: 10, type: Array

Getting below error while mapping an Array Column to String[] entity. Please suggest if Databricks JDBC support entity mapping of Array Values [Worked the same code for below config - H2 DB version - 2.1.214 and org.hibernate.dialect.H2Dialect - ...

  • 3149 Views
  • 4 replies
  • 3 kudos
Latest Reply
Atanu
Databricks Employee
  • 3 kudos

Hello @Emmanuel Trindade​  @Praveen C​  This does not look like coming from Databricks end. Look at the error thread.javax.persistence.PersistenceException: org.hibernate.exception.DataException: Could not read entity state from ResultSet : EntityKey...

  • 3 kudos
3 More Replies
brickster_2018
by Databricks Employee
  • 1241 Views
  • 1 replies
  • 0 kudos
  • 1241 Views
  • 1 replies
  • 0 kudos
Latest Reply
sagnikml
New Contributor III
  • 0 kudos

Delta Sharing is an open protocol developed by Databricks for secure data sharing with other organizations. The primary concepts underlying Delta Sharing in Databricks are shares and recipients. A share is a read-only collection of tables and table p...

  • 0 kudos
MRTN
by New Contributor III
  • 10870 Views
  • 4 replies
  • 3 kudos

Load CSV files with slightly different schemas

I have a set of CSV files generated by a system, where the schema has evolved over the years. Some columns have been added, and at least one column has been renamed in newer files. Is there any way to elegantly load these files into a dataframe? I ha...

  • 10870 Views
  • 4 replies
  • 3 kudos
Latest Reply
MRTN
New Contributor III
  • 3 kudos

For reference - for anybody struggling with the same issues. All online examples using auto loader are written as one block statement on the form: (spark.readStream.format("cloudFiles") .option("cloudFiles.format", "csv") # The schema location di...

  • 3 kudos
3 More Replies
harikrishnang33
by New Contributor II
  • 2909 Views
  • 2 replies
  • 2 kudos

Perform database operations like INSERT, and MERGE using the backend service (golang)

I would like to have an example or reference documentation on how we can create a live (capable of real-time data persistence) pipeline from an in-house golang (backend) service to the databricks tables.I would give a bit more detail on what the gola...

  • 2909 Views
  • 2 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Harikrishnan G​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...

  • 2 kudos
1 More Replies
Lakshmi_J
by New Contributor II
  • 8995 Views
  • 2 replies
  • 3 kudos

Unable to read data from delta table using a python script after the table properties change.

Renamed the Column to include () in a delta table and set the table properties to the below ​ALTER TABLE test_table SET TBLPROPERTIES (  'delta.minReaderVersion' = '2',  'delta.minWriterVersion' = '5',  'delta.columnMapping.mode' = 'name' ) However w...

Error
  • 8995 Views
  • 2 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Lakshmi Jayaraman​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best ans...

  • 3 kudos
1 More Replies
FirozAhmad
by New Contributor II
  • 1582 Views
  • 2 replies
  • 1 kudos

unable to login Databricks partner account same id and password as Acadmey Account.

Unable to login Databric partner academy account. same as Acadmey give me a link to verify my mail to databric partner Account. I have try soo many time but unable to verify and login to databric partner Account .

  • 1582 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Firoz Ahmad Ansari​ Thank you for reaching out! Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training  and our team will get back to you shortly.

  • 1 kudos
1 More Replies
Jonas89
by New Contributor
  • 8116 Views
  • 2 replies
  • 0 kudos

Databricks Devops Release Pipeline Abort

We've built a release pipeline to our Databricks Workspaces, using the VNET Template. It's working end-to-end but intermittent aborts are occurring when workspace is recreated.For example, 4th of April (Monday) We recreated the workspaces and no abor...

1_PipelineError 2_PipelineError 3_PipelineError 4_PipelineError
  • 8116 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Jonas Oliveira de Souza​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that be...

  • 0 kudos
1 More Replies
girish989
by New Contributor II
  • 1286 Views
  • 2 replies
  • 0 kudos

Where is the dbc file for this couse : Deep Dive Into Lakehouse with Delta Lake

Where is the dbc file for this couse :Deep Dive Into Lakehouse with Delta Lake

  • 1286 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @G S​ Thank you for reaching out! Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training  and our team will get back to you shortly. 

  • 0 kudos
1 More Replies
ArunSharma
by New Contributor II
  • 4597 Views
  • 1 replies
  • 1 kudos

Database Objects Naming Convention for Bronze, Silver and Gold Layers

Please help me for database Objects Naming Convention and coding standard for Bronze, Silver and Gold Layers 

  • 4597 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Arun Sharma​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...

  • 1 kudos
Leszek
by Contributor
  • 6863 Views
  • 5 replies
  • 5 kudos

Resolved! Unity Catalog - Azure account console - how to access?

I'm trying to access account console in Azure but I only can see the list of workspaces and access them. I didn't find documentation about account console for Azure. Do you know how to access account console?

  • 6863 Views
  • 5 replies
  • 5 kudos
Latest Reply
vimalii
New Contributor II
  • 5 kudos

Hello @Leszek​ . Please tell me is it works for you ?Did you find the root cause ?I still don't understand why I should grant to myself some extra permissions if I already global administrator, owner of subscription, owner of databricks workspace but...

  • 5 kudos
4 More Replies
Gajji
by New Contributor
  • 2406 Views
  • 2 replies
  • 0 kudos

How to link 2 Dashboards

Hi, I have 2 dashboards. 1st Dashboard has a table with two columns, 'Country' and 'NumberOfMajorCities'. When I click on any row of this table/click on 'NumberOfMajorCities' field of a row, I would like to jump to 2nd Dashboard, where I display all ...

  • 2406 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Ghazanfar Sl​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers ...

  • 0 kudos
1 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels