- 2990 Views
- 3 replies
- 3 kudos
training_set.load_df().toPandas() fails with the new pandas version (2.0.0)
pandas 2.0.0 was released on 4.3.2023 and was pushed to my cluster on the same day. The day after I tried using training_set.load_df().toPandas() and it failed. Reverting to pandas 1.5.3. fixed the problem.
- 2990 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @Al IDI Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your q...
- 3 kudos
- 2392 Views
- 3 replies
- 0 kudos
Hi, I have completed lakehouse fundamentals accreditation, but did not receive the badge yet, only have the certificate of completion.
Hi, I have completed lakehouse fundamentals accreditation, but did not receive the badge yet, only have the certificate of completion.
- 2392 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @RD DO Apologies as we have an issue with our credentials app. We are working with the vendor to resolve it. We expect to be able to grant your badge soon.Thank you!
- 0 kudos
- 5506 Views
- 2 replies
- 0 kudos
TPC-DI Benchmark in Databricks (ETL)
Similar to TPC-DS and TPC-H benchmark, is it possible to execute TPC-DI benchmark in Databricks. What is the feasibility to execute TPC-DI benchmark, please let me know if there is any existing ETL Tool available in Databricks for TPC-DI. For referen...
- 5506 Views
- 2 replies
- 0 kudos
- 0 kudos
Hello, we recently made public a repo that makes it easy to run the TPC-DI benchmark end-to-end - in a variety of ways. The repo can be found here. Feel free to let me know if you have any issues running it!
- 0 kudos
- 11130 Views
- 4 replies
- 0 kudos
Update record in databricks sql table from C#.Net in visual studio 2022 using ODBC
I am trying to make a backend method call work which connects to database and updates the record in a table. The method call works perfectly and it is not throwing any error but at the same time it does not update any record in the table either.Note:...
- 11130 Views
- 4 replies
- 0 kudos
- 0 kudos
I found a temporary work around. Instead of passing the value through the parameters, I passed the value directly in query like below. I know it is not the ideal way but at this time this is working. If I do not pass the value as shown below, the log...
- 0 kudos
- 5690 Views
- 5 replies
- 1 kudos
parsing error in Databricks SQL endpoint
I have two tables EMPLOYEE & EMPLOYEE_ROLE. I'm trying to Update a column with a value from another column. I'm using SQL server join but i get an error - [parse_syntax_error] Syntax error at or near 'FROM' line 3. UPDATE CSET C.title = B.title FROM ...
- 5690 Views
- 5 replies
- 1 kudos
- 1 kudos
Hi @Vijesh V Try to use merge into to perform cdc between tables :MERGE INTO target aUSING source bON {merge_condition}WHEN MATCHED THEN {matched_action}WHEN NOT MATCHED THEN {not_matched_action}
- 1 kudos
- 4613 Views
- 3 replies
- 2 kudos
- 4613 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @Ravi Vishwakarma Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answ...
- 2 kudos
- 4694 Views
- 4 replies
- 3 kudos
[Databricks][JDBC](10400) Invalid type for data - column: 10, type: Array
Getting below error while mapping an Array Column to String[] entity. Please suggest if Databricks JDBC support entity mapping of Array Values [Worked the same code for below config - H2 DB version - 2.1.214 and org.hibernate.dialect.H2Dialect - ...
- 4694 Views
- 4 replies
- 3 kudos
- 3 kudos
Hello @Emmanuel Trindade @Praveen C This does not look like coming from Databricks end. Look at the error thread.javax.persistence.PersistenceException: org.hibernate.exception.DataException: Could not read entity state from ResultSet : EntityKey...
- 3 kudos
- 2111 Views
- 1 replies
- 0 kudos
- 2111 Views
- 1 replies
- 0 kudos
- 0 kudos
Delta Sharing is an open protocol developed by Databricks for secure data sharing with other organizations. The primary concepts underlying Delta Sharing in Databricks are shares and recipients. A share is a read-only collection of tables and table p...
- 0 kudos
- 15071 Views
- 4 replies
- 3 kudos
Load CSV files with slightly different schemas
I have a set of CSV files generated by a system, where the schema has evolved over the years. Some columns have been added, and at least one column has been renamed in newer files. Is there any way to elegantly load these files into a dataframe? I ha...
- 15071 Views
- 4 replies
- 3 kudos
- 3 kudos
For reference - for anybody struggling with the same issues. All online examples using auto loader are written as one block statement on the form: (spark.readStream.format("cloudFiles") .option("cloudFiles.format", "csv") # The schema location di...
- 3 kudos
- 4743 Views
- 2 replies
- 2 kudos
Perform database operations like INSERT, and MERGE using the backend service (golang)
I would like to have an example or reference documentation on how we can create a live (capable of real-time data persistence) pipeline from an in-house golang (backend) service to the databricks tables.I would give a bit more detail on what the gola...
- 4743 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Harikrishnan G Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...
- 2 kudos
- 11118 Views
- 2 replies
- 3 kudos
Unable to read data from delta table using a python script after the table properties change.
Renamed the Column to include () in a delta table and set the table properties to the below ALTER TABLE test_table SET TBLPROPERTIES ( 'delta.minReaderVersion' = '2', 'delta.minWriterVersion' = '5', 'delta.columnMapping.mode' = 'name' ) However w...
- 11118 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi @Lakshmi Jayaraman Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best ans...
- 3 kudos
- 2188 Views
- 2 replies
- 1 kudos
unable to login Databricks partner account same id and password as Acadmey Account.
Unable to login Databric partner academy account. same as Acadmey give me a link to verify my mail to databric partner Account. I have try soo many time but unable to verify and login to databric partner Account .
- 2188 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Firoz Ahmad Ansari Thank you for reaching out! Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training and our team will get back to you shortly.
- 1 kudos
- 10057 Views
- 2 replies
- 0 kudos
Databricks Devops Release Pipeline Abort
We've built a release pipeline to our Databricks Workspaces, using the VNET Template. It's working end-to-end but intermittent aborts are occurring when workspace is recreated.For example, 4th of April (Monday) We recreated the workspaces and no abor...
- 10057 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Jonas Oliveira de Souza Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that be...
- 0 kudos
- 2191 Views
- 2 replies
- 0 kudos
Where is the dbc file for this couse : Deep Dive Into Lakehouse with Delta Lake
Where is the dbc file for this couse :Deep Dive Into Lakehouse with Delta Lake
- 2191 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @G S Thank you for reaching out! Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training and our team will get back to you shortly.
- 0 kudos
- 5768 Views
- 1 replies
- 1 kudos
Database Objects Naming Convention for Bronze, Silver and Gold Layers
Please help me for database Objects Naming Convention and coding standard for Bronze, Silver and Gold Layers
- 5768 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Arun Sharma Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...
- 1 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 555 | |
| 349 | |
| 287 |
