cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

monicaborges
by New Contributor III
  • 971 Views
  • 3 replies
  • 6 kudos
  • 971 Views
  • 3 replies
  • 6 kudos
Latest Reply
Anonymous
Not applicable
  • 6 kudos

Hi @Mônica Borges Silva​ Thank you for reaching out! Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training  and our team will get back to you shortly. 

  • 6 kudos
2 More Replies
User16835756816
by Valued Contributor
  • 12878 Views
  • 2 replies
  • 10 kudos

Resolved! How do I resolve problems when deploying a workspace with AWS Quickstart cloud formation template?

I am unable to deploy a workspace on AWS using Quickstart from my account console.Short description-You might receive one of the following common errors users face:Wrong credentialsElastic IP and VPC limit reachedRegion unavailableResolution-Wrong cr...

cloudformation-databricks-password Screen Shot 2022-03-11 at 10.17.42 AM Screen Shot 2022-03-15 at 10.42.50 AM Cross Account Role
  • 12878 Views
  • 2 replies
  • 10 kudos
Latest Reply
aiwithqasim
Contributor
  • 10 kudos

Really great explanation. The error that I was encountering since yesterday was Failed to create CreateStorageConfiguraiton and CreateCredentialConfiguration. The first step to put the password manually helped me to solve the issue

  • 10 kudos
1 More Replies
IG1
by New Contributor II
  • 814 Views
  • 3 replies
  • 2 kudos

Why there's no "New Union" option with Databricks connection

I'm trying to use databricks connect with tableau but it doesn't give me the "New Union" option. Is this normal or it's particular to me? My tableau desktop version is 2021.3

  • 814 Views
  • 3 replies
  • 2 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 2 kudos

there is option for connecting tableau find there SPARK SQL then it should work after adding proper connection string

  • 2 kudos
2 More Replies
johnb1
by New Contributor III
  • 10181 Views
  • 13 replies
  • 13 kudos

Certified Data Engineer Associate - v2 vs. v3 (Databricks Academy)

Which version of the Data Engineering with Databricks learning plan should I do? v2 or v3? Is there a Certified Data Engineer Associate V3 Exam already?Where can I find practice exams for Certified Data Engineer Associate V3?

  • 10181 Views
  • 13 replies
  • 13 kudos
Latest Reply
Frank_Tao
New Contributor II
  • 13 kudos

I would suggest choose v3 - it was latest version and covered more topic.

  • 13 kudos
12 More Replies
VVill_T
by Contributor
  • 1583 Views
  • 4 replies
  • 7 kudos

How to write a Delta Live Table(dlt) pipeline output to Databricks SQL directly

Hi,I am trying to see if it is possible to setup a direct connection from dlt pipeline to a table in Databricks SQL by configuring the Target Schema: with poc being a location of schema like "dbfs:/***/***/***/poc.db The error message was just a...

image image
  • 1583 Views
  • 4 replies
  • 7 kudos
Latest Reply
youssefmrini
Honored Contributor III
  • 7 kudos

When ever you store a Delta Table to Hive Metastore. This table will be available in Databricks SQL Workspace ( Data Explorer ) under hive_metastore catalog.

  • 7 kudos
3 More Replies
alexgv12
by New Contributor III
  • 710 Views
  • 1 replies
  • 3 kudos

creation of tables with cdc

I am using cdc to create different tables, these tables can have one or more dependencies, what is the best practice to create these tables without losing records or changes in both the base table and the join tables? for exampleselect * from ( ...

  • 710 Views
  • 1 replies
  • 3 kudos
Latest Reply
alexgv12
New Contributor III
  • 3 kudos

more detail

  • 3 kudos
Prototype998
by New Contributor III
  • 1440 Views
  • 1 replies
  • 5 kudos

Resolved! Where can we use Broadcast variable?

best situations where we can use broadcast variables ?

  • 1440 Views
  • 1 replies
  • 5 kudos
Latest Reply
Rishabh264
Honored Contributor II
  • 5 kudos

hey @Punit Chauhan​ BV are used in the same way for RDD, DataFrame, and Dataset.When you run a Spark RDD, DataFrame jobs that has the Broadcast variables defined and used, Spark does the following.Spark breaks the job into stages that have distribute...

  • 5 kudos
SudiptaBiswas
by New Contributor III
  • 1651 Views
  • 3 replies
  • 3 kudos

databricks autoloader getting stuck in flattening json files for different scenarios similar in nature.

I have a databricks autoloader notebook that reads json files from an input location and writes the flattened version of json files to an output location. However, the notebook is behaving differently for two different but similar scenarios as descri...

  • 1651 Views
  • 3 replies
  • 3 kudos
Latest Reply
jose_gonzalez
Moderator
  • 3 kudos

Could you provide a code snippet? also do you see any error logs in the driver logs?

  • 3 kudos
2 More Replies
Rishabh264
by Honored Contributor II
  • 624 Views
  • 1 replies
  • 4 kudos

PrivilegesSELECT: gives read access to an object.CREATE: gives ability to create an object (for example, a table in a schema).MODIFY: gives ability to...

PrivilegesSELECT: gives read access to an object.CREATE: gives ability to create an object (for example, a table in a schema).MODIFY: gives ability to add, delete, and modify data to or from an object.USAGE: does not give any abilities, but is an add...

  • 624 Views
  • 1 replies
  • 4 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 4 kudos

thanks sir

  • 4 kudos
Yatoom
by New Contributor II
  • 1278 Views
  • 2 replies
  • 2 kudos

Disable access to mount point for client code

We are building a platform where we automatically execute Databricks jobs using Python packages delivered by our end-users. We want to create a mount point so that we can deliver the cluster's driver logs to an external storage. However, we don't wan...

  • 1278 Views
  • 2 replies
  • 2 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 2 kudos

Check with cloud providers

  • 2 kudos
1 More Replies
Aviral-Bhardwaj
by Esteemed Contributor III
  • 2806 Views
  • 1 replies
  • 35 kudos

Understand Trigger Intervals in Streaming Pipelines in Databricks When defining a streaming write, the trigger the method specifies when the system sh...

Understand Trigger Intervals in Streaming Pipelines in DatabricksWhen defining a streaming write, the trigger the method specifies when the system should process the next set of data. Triggers are specified when defining how data will be written to a...

image
  • 2806 Views
  • 1 replies
  • 35 kudos
Latest Reply
jose_gonzalez
Moderator
  • 35 kudos

Thank you for sharing

  • 35 kudos
espenol
by New Contributor III
  • 3046 Views
  • 4 replies
  • 3 kudos

Resolved! Can't read large multiline json,

Hey! So I'm struggling to read a multiline json. Some details:It's gzipped from the API I get it fromjust a single file in the folder currrentlystored in ADLS Gen2 storage. 95 MB zipped, approximately 1.2 GB unzippedI can read it just fine using the...

Can be read as text, multiline json Corrupted string relative path in absolute uri
  • 3046 Views
  • 4 replies
  • 3 kudos
Latest Reply
espenol
New Contributor III
  • 3 kudos

Thanks a lot for the help! Removing colon fixed it. Now I need to fix the Data Factory instance that writes to my storage container. Hope it's easy, Data Factory is such a hassle.

  • 3 kudos
3 More Replies
JavedN
by New Contributor
  • 744 Views
  • 3 replies
  • 3 kudos

500550

[Simba][SimbaSparkJDBCDriver] (500550) The next rowset buffer is already marked as consumed. The fetch thread might have terminated unexpectedly. Foreground thread ID

  • 744 Views
  • 3 replies
  • 3 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 3 kudos

You can try to add UseNativeQuery=1 property to JDBC syntax.

  • 3 kudos
2 More Replies
DaanC
by New Contributor II
  • 1479 Views
  • 5 replies
  • 3 kudos

Databricks administrator accreditation exam

Hi,I have recently passed the administrator accreditation exam in the partner academy. However, I have recieved no certificate. How can I get my certificate?

  • 1479 Views
  • 5 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Daan Cremers​ Thank you for reaching out! Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training  and our team will get back to you shortly.

  • 3 kudos
4 More Replies
lsanchezo
by New Contributor II
  • 1796 Views
  • 4 replies
  • 2 kudos

My Databricks Lakehouse badge doesn't appear in Databricks Academy

Hello, I passed the Databricks Lakehouse accreditation exam at the Databricks academy, but the badge does not appear on the platform and it has been about an hour, I hope you can help me. What it does let me do is download a certificate.

  • 1796 Views
  • 4 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Luis Sanchez Otiniano​ Thank you for reaching out! Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training  and our team will get back to you shortly. 

  • 2 kudos
3 More Replies
Labels
Top Kudoed Authors