cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
caroline123
by • New Contributor III
  • 5870 Views
  • 10 replies
  • 1 kudos

Resolved! Haven't received any updates of certificate after more than one week

Hi team, I took the exam on Jan 14th and passed the exam with 91.66% score. I got an email right after the exam saying I should receive the certificate within one week. But it has been more than 10 days and I haven't heard anything from Databricks te...

Data Engineering
Reply
  • 5870 Views
  • 10 replies
  • 1 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hello, @Luwei Lei​ - We have an answer now.Please check out these announcements.https://community.databricks.com/s/question/0D53f00001ebiUOCAY/databricks-courseshttps://community.databricks.com/s/feed/0D53f00001dq6W6CAI

  • 1 kudos
9 More Replies
All_users_grou1
by • New Contributor II
  • 1606 Views
  • 2 replies
  • 2 kudos

Resolved! Haven't received Databricks Certified Associate Developer for Apache Spark 3.0 certification yet

I took the exam on 04-01-2022 and passed with 80% though I haven't received my certification yet. I had also raised a query regarding this, is there an update on request #00129864?

Data Engineering
Reply
  • 1606 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 2 kudos

@Ayush Kumar Singh​ - We have an answer for you. Please check out these announcements.https://community.databricks.com/s/question/0D53f00001ebiUOCAY/databricks-courseshttps://community.databricks.com/s/feed/0D53f00001dq6W6CAI

  • 2 kudos
1 More Replies
CleverAnjos
by • New Contributor III
  • 6239 Views
  • 5 replies
  • 3 kudos

Resolved! Best way of loading several csv files in a table

What would be the best way of loading several files like in a single table to be consumed?https://s3.amazonaws.com/nyc-tlc/trip+data/yellow_tripdata_2019-10.csvhttps://s3.amazonaws.com/nyc-tlc/trip+data/yellow_tripdata_2019-11.csvhttps://s3.amazonaws...

Data Engineering
Reply
  • 6239 Views
  • 5 replies
  • 3 kudos
View Replies
Latest Reply
CleverAnjos
New Contributor III
  • 3 kudos

Thanks Kaniz, I already have the files. I was discussing about the best way to load them

  • 3 kudos
4 More Replies
techno
by • New Contributor
  • 688 Views
  • 0 replies
  • 0 kudos

PLogo

Best kurta set for womenPlain, printed, weaved, adorned, these are only a couple of the choices of Kurta sets for ladies accessible on Purley rich, and that also at the best cost. You will likewise observe a gigantic determination of creator base wea...

Data Engineering
Reply
  • 688 Views
  • 0 replies
  • 0 kudos
BilalAslamDbrx
by • Databricks Employee
  • 558 Views
  • 0 replies
  • 0 kudos

�� hi everyone, I am a product manager in Amsterdam. I work on Jobs and Databricks SQL. In my spare time, I cook �� and send emojis &#12...

hi everyone, I am a product manager in Amsterdam. I work on Jobs and Databricks SQL. In my spare time, I cook and send emojis . Looking forward to working with this community!

Data Engineering
Reply
  • 558 Views
  • 0 replies
  • 0 kudos
Tarviha
by • New Contributor
  • 641 Views
  • 0 replies
  • 0 kudos

Hi. I have been trying to sign up for Databricks for past 3 hours but its giving me this error every time. I am not using any ad blocker or private mo...

Hi.I have been trying to sign up for Databricks for past 3 hours but its giving me this error every time. I am not using any ad blocker or private mode. I tried to sign up using different web browsers and experienced the same issue. Is it still possi...

Databricks Failure
Data Engineering
Reply
  • 641 Views
  • 0 replies
  • 0 kudos
GoldenTuna
by • New Contributor II
  • 4334 Views
  • 5 replies
  • 2 kudos

Resolved! Mounting an Azure Storage Account in a cluster init script?

We are trying to configure our environment so when our cluster starts up, it checks to see if we have mounted our Azure storage account container and if is not, mount it. We can do this fine in a notebook however have no luck doing this through an in...

Data Engineering
Reply
  • 4334 Views
  • 5 replies
  • 2 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 2 kudos

@David Kruetzkamp​ - Would you be happy to mark whichever answer helped the most as best? That will help other members find the solution more quickly.

  • 2 kudos
4 More Replies
Mirko
by • Contributor
  • 4367 Views
  • 6 replies
  • 1 kudos

Resolved! Group vs User rights

I have a small question: How does the combination of group and user rights work? Is it like in azure, that if i have for example databricks sql acces threw a (databricks) group i am member of, but in my personal account databricks sql is not enabled...

Data Engineering
Reply
  • 4367 Views
  • 6 replies
  • 1 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 1 kudos

@Mirko Ludewig​ - Good morning (or evening depending on where you hail), would you be happy to mark whichever answer resolved the problem for you as best? That helps other members find the solutions more quickly.

  • 1 kudos
5 More Replies
KKDataEngineer
by • New Contributor III
  • 1329 Views
  • 0 replies
  • 2 kudos

Spark Structred Streaming, An Aggregation DF with Watermark in Append mode to Delta table is not writing the most recent aggregation to the Delta table even after crossing the water mark boundary. This is causing dataloss

Team,  I am struggling with a unique issue. I am not sure if my understanding is wrong or this is a bug with spark. I am reading a stream from events hub ( Extract) Pivoting and Aggregating the above dataframe ( Transformation). This is a WATERMARKED...

Data Engineering
Reply
  • 1329 Views
  • 0 replies
  • 2 kudos
cristianc
by • Contributor
  • 5131 Views
  • 9 replies
  • 0 kudos

Resolved! Query AWS Redshift from Databricks SQL

Greetings,In the documentation for Databricks SQL it states that it supports JDBC connections, however when connecting to AWS Redshift via the built in PostgreSQL driver ("CREATE TABLE sample USING JDBC" and "jdbc://postgresql:/..." URI) I'm getting ...

Data Engineering
Reply
  • 5131 Views
  • 9 replies
  • 0 kudos
View Replies
Latest Reply
cristianc
Contributor
  • 0 kudos

@Bilal Aslam​ anytime! Is there a place where customers could follow the timeline when such features are introduced?

  • 0 kudos
8 More Replies
Axel_Schwanke
by • Contributor
  • 5241 Views
  • 7 replies
  • 3 kudos

Resolved! Issue with AWS Glue metacatalogue and DBR 9.1 ... 10.1

I have a simple SparkSQL Select statementoffers_df = (spark.sql(""" SELECT * FROM delta.`{}` """.format(TABLE_LOCATION)))It runs under DBR 9.0 and previous. When changing the DBR to 9.1 ...10.1 I get an exceptionorg.apache.spark.SparkException: Una...

Data Engineering
Reply
  • 5241 Views
  • 7 replies
  • 3 kudos
View Replies
Latest Reply
Axel_Schwanke
Contributor
  • 3 kudos

Retest in DBR 10.3 beta SUCCESSFUL.Problem does not occur in DBR 10.3 beta

  • 3 kudos
6 More Replies
Anonymous
by • Not applicable
  • 29129 Views
  • 4 replies
  • 4 kudos

Resolved! Spark is not able to resolve the columns correctly when joins data frames

Hello all, I m using pyspark ( python 3.8) over spark3.0 on Databricks. When running this DataFrame join:next_df = days_currencies_matrix.alias('a').join( data_to_merge.alias('b') , [ days_currencies_matrix.dt == data_to_merge.RATE_DATE, days...

Data Engineering
Reply
  • 29129 Views
  • 4 replies
  • 4 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 4 kudos

@Alessio Palma​ - Howdy! My name is Piper, and I'm a moderator for the community. Would you be happy to mark whichever answer solved your issue so other members may find the solution more quickly?

  • 4 kudos
3 More Replies
Bilal1
by • New Contributor III
  • 4383 Views
  • 6 replies
  • 4 kudos

Resolved! SQL Analytics: Is it possible to configure the default number format

HiWhen querying an integer value, the default format is '0.0' which results in an integer value 202111, displayed as 202,111. I can resolve the issue by updating the visualisation or using formatnumber in my query, however I would like to set a defa...

Data Engineering
Reply
  • 4383 Views
  • 6 replies
  • 4 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 4 kudos

@Bilal Haniff​ - Would you be happy to mark whichever answer helped you the most as best? That helps others find solutions more quickly.

  • 4 kudos
5 More Replies
Hubert-Dudek
by • Esteemed Contributor III
  • 1586 Views
  • 1 replies
  • 15 kudos

Resolved! Write to Azure Delta Lake - optimization request

Databricks/Delta team could optimize some commands which writes to Azure Blob Storage as Azure display that message:

image
Data Engineering
Reply
  • 1586 Views
  • 1 replies
  • 15 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 15 kudos

Hey there. Thank you for your suggestion. I'll pass this up to the team.

  • 15 kudos
wyzer
by • Contributor II
  • 3472 Views
  • 2 replies
  • 4 kudos

Resolved! How to show the properties of the folders/files from DBFS ?

Hello,How to show the properties of the folders/files from DBFS ?Currently i am using this command :display(dbutils.fs.ls("dbfs:/"))But it only shows :pathnamesizeHow to show these properties ? : CreatedBy (Name)CreatedOn (Date)ModifiedBy (Name)Modi...

Data Engineering
Reply
  • 3472 Views
  • 2 replies
  • 4 kudos
View Replies
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 4 kudos

Only one idea is to use %sh magic command but there is no name (just root)

  • 4 kudos
1 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels