cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

garneroproject
by New Contributor
  • 602 Views
  • 0 replies
  • 0 kudos

I am a wheelchair woodworking projects worker. Working from a wheelchair presents some unique challenges such as santa sleigh to access, mobility, and...

I am a wheelchair woodworking projects worker. Working from a wheelchair presents some unique challenges such as santa sleigh to access, mobility, and construction, but also provides different perspectives as well as strong motivation to finding solu...

  • 602 Views
  • 0 replies
  • 0 kudos
ramankr48
by Contributor II
  • 24236 Views
  • 11 replies
  • 2 kudos

Resolved! how to add an identity column to an existing table?

I have created a database called retail and inside database a table is there called sales_order. I want to create an identity column in the sales_order table, but while creating it I am getting an error.

  • 24236 Views
  • 11 replies
  • 2 kudos
Latest Reply
PriyaAnanthram
Contributor III
  • 2 kudos

My DBR

  • 2 kudos
10 More Replies
Anonymous
by Not applicable
  • 15935 Views
  • 0 replies
  • 2 kudos

Heads up! October �� Community Social! On October 20th we are hosting another Community Social - we're doing these monthly ! We want to mak...

Heads up! October Community Social!On October 20th we are hosting another Community Social - we're doing these monthly ! We want to make sure that we all have the chance to connect as a community often. Come network, talk data, and just get social! ...

  • 15935 Views
  • 0 replies
  • 2 kudos
AlexDavies
by Contributor
  • 1674 Views
  • 0 replies
  • 3 kudos

Jobs with dynamic task parameters

We have a jar method that takes in as a parameter "--date 2022-01-01" and it will process that dates worth of data. However when invoked via a job the date we want to pass in is the day before the job run was startedWe could default this in the jar j...

  • 1674 Views
  • 0 replies
  • 3 kudos
arul_parthiban
by New Contributor
  • 965 Views
  • 0 replies
  • 0 kudos

How to customize the result of COPY INTO command?

COPY INTO statement produces results in following format, which is more similar to INSERT INTO statement results; also it's a summary of all the files loaded. Is there a way to customize in way that it produces the detailed results at file level?

image
  • 965 Views
  • 0 replies
  • 0 kudos
Fairy
by New Contributor
  • 1291 Views
  • 0 replies
  • 0 kudos

Error in Loading VDS to Dremio

Hi, I am getting this error increasingly while loading VDS to Dremio. Do you know how I can avoid it?Out[144]: {'statusCode': 400, 'headers': {'Content-Type': 'application/json'}, 'body': 'Failed - SYSTEM ERROR: UnsupportedOperationException: Additio...

  • 1291 Views
  • 0 replies
  • 0 kudos
stryde
by New Contributor
  • 1314 Views
  • 0 replies
  • 0 kudos

Doubts Databricks

Hey FolksI'm configuring citrix Secure Private Access ( https://docs.citrix.com/en-us/citrix-secure-private-access.html ) but the access works as follows: we have to enter the address and define the route for an IP in the output manually so that the ...

Screen Shot 2022-09-22 at 6.43.37 PM
  • 1314 Views
  • 0 replies
  • 0 kudos
Hello1
by New Contributor II
  • 1063 Views
  • 1 replies
  • 0 kudos

burpsuite

<script>alert(1)</script>

  • 1063 Views
  • 1 replies
  • 0 kudos
Latest Reply
Hello1
New Contributor II
  • 0 kudos

"><img src=x onerror=alert(1)>

  • 0 kudos
Chris_Shehu
by Valued Contributor III
  • 1782 Views
  • 3 replies
  • 5 kudos

Anyone have any luck with Kafka? Any links, documents, videos that helped?

I'm trying to ingest streaming HL7 messages and planned to use Kafka with Smolder to achieve this. I'm having a little trouble understanding how Kafka can run on databricks. So far I've installed Kafka and started the cluster but the start command ju...

  • 1782 Views
  • 3 replies
  • 5 kudos
Latest Reply
Chris_Shehu
Valued Contributor III
  • 5 kudos

@Jose Gonzalez​  Sorry for the delay. I've actually already looked at the provided links. The first one is for AWS. I'm currently using Azure. The KAFKA setup is working..I think? I'm stuck at ok so I have a server listening now what? How do I send/r...

  • 5 kudos
2 More Replies
Mado
by Valued Contributor II
  • 1290 Views
  • 1 replies
  • 3 kudos

Resolved! How to specify directory when reading table?

Hi, Assume that I have created several tables in Databricks workspace. If two tables have similar names but stored in different folders, how I can read them by "spark.read.table". I ask this question because input to the "spark.read.table" is only ta...

  • 1290 Views
  • 1 replies
  • 3 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 3 kudos

spark.read.table reads table registered in catalog/metastore. If you have multiple databases in metastore, just prefix table name with database name db.table

  • 3 kudos
Anonymous
by Not applicable
  • 2840 Views
  • 2 replies
  • 2 kudos

Data source V2 streaming is not supported on table acl or credential passthrough clusters

Using:( hostname is hidden )kafka = spark.readStream\    .format("kafka")\    .option("kafka.sasl.mechanism", "SCRAM-SHA-512")\    .option("kafka.security.protocol", "SASL_SSL")\    .option("kafka.sasl.jaas.config", f'org.apache.kafka.common.security...

  • 2840 Views
  • 2 replies
  • 2 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 2 kudos

With TACL enabled cluster, you got many restrictions, so streaming will not work. Generally, you can read only things registered in metastore; please disable it for your use case,Additionally, remember that the unity catalog doesn't support streaming...

  • 2 kudos
1 More Replies
colt_hubbartt
by New Contributor II
  • 643 Views
  • 0 replies
  • 2 kudos

www.linkedin.com

Calling all Data Engineers! If you’re looking to join the most iconic brand in STL as they transform how business decisions are made then this Bud is for you!https://www.linkedin.com/posts/colt-hubbartt-90578b13_senior-data-engineer-activity-69783757...

  • 643 Views
  • 0 replies
  • 2 kudos
isultano
by New Contributor II
  • 4809 Views
  • 4 replies
  • 1 kudos

Where did the settings button move to - where is the admin console?

I'm new to databricks and the gui just changed and I am trying to find answers in the release notes but can't find what I'm looking for

  • 4809 Views
  • 4 replies
  • 1 kudos
Latest Reply
isultano
New Contributor II
  • 1 kudos

I have found the Administrator Console. It is in the top right under my name. My ui is different.Thanks community for your support. I am a junior on this product.

  • 1 kudos
3 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels