cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

AmarJT
by New Contributor II
  • 4075 Views
  • 2 replies
  • 6 kudos

Lakehouse Fundamentals Accreditation badge not received

Hi Team,I have successfully passed the test after completion of the course. But i have not received any badge from your side. I have just been provided a certificate. Certificate ID:ID: E-E04YDVAs mentioned in the web portals i tried accessing "http...

image image
  • 4075 Views
  • 2 replies
  • 6 kudos
Latest Reply
Geeta1
Databricks Partner
  • 6 kudos

Hi @Amarjeet Kumar​ , you will receive the badge in a day after completion. Even I received it a day after I cleared the exam. If you don't receive it the next day also, then you can raise a ticket at  https://help.databricks.com/s/contact-us?ReqType...

  • 6 kudos
1 More Replies
yogu
by Databricks Partner
  • 3184 Views
  • 2 replies
  • 18 kudos

Resolved! Explain about "Activate Gift Certificate" section in Databricks Community Rewards.

Hello everyone,Can any one explain about the Active Gift Certificate in in Databricks Community Rewards. And how to use it?

image
  • 3184 Views
  • 2 replies
  • 18 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 18 kudos

it boils down to this:you earn points with forum interaction. Those points can be exchanged for 'credits'.With those credits you can buy Databricks swag.Your lifetime points (so the cumulated amount of points) are not affected by this.

  • 18 kudos
1 More Replies
-werners-
by Esteemed Contributor III
  • 3030 Views
  • 3 replies
  • 22 kudos

Resolved! Package cells (scala), who uses them?

So I was wondering who uses package cells in scala?We have this library (jar) which has some useful functions we use all over the place. But that's about it. So I think we can do the same thing without a jar but with package cells.But I never hear ...

  • 3030 Views
  • 3 replies
  • 22 kudos
Latest Reply
Anonymous
Not applicable
  • 22 kudos

Hi @Werner Stinckens​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.T...

  • 22 kudos
2 More Replies
Constantino
by New Contributor III
  • 2050 Views
  • 1 replies
  • 2 kudos

Is there any way to prevent non-admin users from creating new jobs?

This is specific to creating new jobs, I understand that various permissions can be set on existing jobs using job access control. This seems to suggest no, I can't find anything in the Databricks docs either.

  • 2050 Views
  • 1 replies
  • 2 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 2 kudos

nope.Looked for that too, but it does not seem to be possible. Perhaps with Unity catalog, as there you have more permission controls.But using Unity is not an overnight decision.

  • 2 kudos
Anonymous
by Not applicable
  • 4225 Views
  • 3 replies
  • 28 kudos

Resolved! Refresh Dashboard also make all related queried refresh?

Hi all,I have a quick currious. I know both query and dashboard page in Databricks SQL have refresh button to can them refresh. But one question it, when I'm in Dashboard page and click the refesh button. Does this thing also force every related quer...

  • 4225 Views
  • 3 replies
  • 28 kudos
Latest Reply
Anonymous
Not applicable
  • 28 kudos

Thanks all your support. It's totally clear for me now!!!

  • 28 kudos
2 More Replies
Pradeep_Namani
by New Contributor III
  • 7976 Views
  • 5 replies
  • 2 kudos

Date field getting changed when reading from excel file to dataframe in pyspark

The date field is getting changed while reading data from source .xls file to the dataframe. In the source xl file all columns are strings but i am not sure why date column alone behaves differentlyIn Source file date is 1/24/1947.In pyspark datafram...

  • 7976 Views
  • 5 replies
  • 2 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 2 kudos

how about using inferschema one single time to create a correct DF, then create a schema from the df-schema.something like this f.e.from pyspark.sql.types import StructType   # Save schema from the original DataFrame into json: schema_json = df.s...

  • 2 kudos
4 More Replies
JordanYaker
by Contributor
  • 9570 Views
  • 7 replies
  • 8 kudos

Resolved! Is anyone else experiencing intermittent "Failure starting REPL" errors with PySpark Jobs?

I have a Multi-Task Job that is running a bunch of PySpark notebooks and about 30-60% of the time, my jobs fail with the following error:I haven't seen any consistency with this error. I've had as many as all of the tasks in the job giving this error...

image.png
  • 9570 Views
  • 7 replies
  • 8 kudos
Latest Reply
James_Cole
New Contributor III
  • 8 kudos

Hi. Did you ever got a resolution to this problem outside of rolling back to 10.4? I have recently moved some workloads over to runtime 11.3 and am experiencing intermittent "repl did not start in 30 seconds." errors.I have increased the repl timeout...

  • 8 kudos
6 More Replies
andreas9898
by New Contributor II
  • 4467 Views
  • 3 replies
  • 5 kudos

Getting error with spark-sftp, no such file

In a databricks cluster with Scala 2.1.1 I am trying to read a file into a spark data frame using the following code.val df = spark.read .format("com.springml.spark.sftp") .option("host", "*") .option("username", "*") .option("password", "*")...

  • 4467 Views
  • 3 replies
  • 5 kudos
Latest Reply
Anonymous
Not applicable
  • 5 kudos

Hi @Andreas P​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 5 kudos
2 More Replies
KVNARK
by Honored Contributor II
  • 8710 Views
  • 8 replies
  • 28 kudos

Resolved! Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective.

Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective. can someone throw some light on this. Currently learning Pyspark (basics of Pythion in handling the data) a...

  • 8710 Views
  • 8 replies
  • 28 kudos
Latest Reply
KVNARK
Honored Contributor II
  • 28 kudos

Thanks All for your valuable suggestions!

  • 28 kudos
7 More Replies
alxsbn
by Contributor
  • 1400 Views
  • 0 replies
  • 2 kudos

Terraform x Databricks error INVALID_STATE subscription disabled

Hello,I just bootstrap a new Databricks EC2 on an AWS account with Terraform. Priori dependencies seems OK on my side (network, root storage, credentials configuration). I'm referring mainly to this guide and of course pages related to each Databrick...

  • 1400 Views
  • 0 replies
  • 2 kudos
Labels