cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
SDas1
by • New Contributor
  • 8027 Views
  • 2 replies
  • 2 kudos

Identity column value of Databricks delta table is not started with 0 and increaed by 1. It always started with something like 1 or 2 and increased by 2. Below is the sample code and any logical input here is appreciated

spark.sql("CREATE TABLE integrated.TrailingWeeks(ID bigint GENERATED BY DEFAULT AS IDENTITY (START WITH 0 increment by 1) ,Week_ID int NOT NULL) USING delta OPTIONS (path 'dbfs:/<Path in Azure datalake>/delta')")

Data Engineering
Reply
  • 8027 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
agallard
Contributor
  • 2 kudos

Hi,When you define an identity column in  Databricks with GENERATED BY DEFAULT AS IDENTITY (START WITH 0 INCREMENT BY 1), it is expected to start at 0 and increment by 1. However, due to Databricks' distributed architecture, the values may not be str...

  • 2 kudos
1 More Replies
tlbarata
by • New Contributor II
  • 2564 Views
  • 3 replies
  • 1 kudos

Outdated - Databricks Data Engineer associate v2 lesson DE 4.2

While following the video lesson and executing the notebook 4.2, I noticed that creating the CREATE Table "users_jdbc" command generates an EXTERNAL table, while the video and, notebook too, suggests it as being a Managed table.Here are some printscr...

1 - Create Table Describe extended command Decribe command from video lesson
Data Engineering
Reply
  • 2564 Views
  • 3 replies
  • 1 kudos
View Replies
Latest Reply
Vartika
Databricks Employee
  • 1 kudos

Hi @Tiago Barata​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...

  • 1 kudos
2 More Replies
Graham
by • New Contributor III
  • 4501 Views
  • 4 replies
  • 8 kudos

Resolved! [Databricks SQL] Commented Escape Character (\) Causes Unexpected Behavior

The Problem:I've observed erratic behavior when I add a comment containing a trailing escape character (\) to a CREATE TABLE statement.For example, this query returns data (though it shouldn't):CREATE TABLE example_table SELECT 1 -- This comment has ...

Data Engineering
Reply
  • 4501 Views
  • 4 replies
  • 8 kudos
View Replies
Latest Reply
BilalAslamDbrx
Databricks Employee
  • 8 kudos

@Graham Carman​ we're tracking this as a defect / issue on our side. For now, please don't include the escape character in comments.

  • 8 kudos
3 More Replies
joel_iemma
by • New Contributor III
  • 4708 Views
  • 5 replies
  • 0 kudos

Resolved! A void column was created after connecting to cosmos

Hi everyone, I have connected to Cosmos using this tutorial https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/cosmos/azure-cosmos-spark_3_2-12/Samples/DatabricksLiveContainerMigrationAfter creating a table using a simple SQL command:CREATE TA...

image
Data Engineering
Reply
  • 4708 Views
  • 5 replies
  • 0 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hey there @Joel iemma​ Hope all is well! Just wanted to check in if you would be happy to mark an answer as best for us, please? It would be really helpful for the other members too.Cheers!

  • 0 kudos
4 More Replies
User16826992666
by • Valued Contributor
  • 9232 Views
  • 1 replies
  • 0 kudos

Resolved! Can I upload an Excel file to create a table in a workspace?

On the Data tab in the workspace I have the "Create Table" button which gives me the option to upload a local file as a data source. Can I upload an Excel file here? Not sure what kind of files work for this.

Data Engineering
Reply
  • 9232 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
sajith_appukutt
Honored Contributor II
  • 0 kudos

Currently the file types supported there are CSV, JSON and Avro. You could, however upload the excel file to the dbfs path under FileStore and write code in a notebook to parse it and persist it to a table

  • 0 kudos
Labels