cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

girish989
by New Contributor II
  • 2397 Views
  • 2 replies
  • 0 kudos

Where is the dbc file for this couse : Deep Dive Into Lakehouse with Delta Lake

Where is the dbc file for this couse :Deep Dive Into Lakehouse with Delta Lake

  • 2397 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @G S​ Thank you for reaching out! Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training  and our team will get back to you shortly. 

  • 0 kudos
1 More Replies
ArunSharma
by New Contributor II
  • 5971 Views
  • 1 replies
  • 1 kudos

Database Objects Naming Convention for Bronze, Silver and Gold Layers

Please help me for database Objects Naming Convention and coding standard for Bronze, Silver and Gold Layers 

  • 5971 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Arun Sharma​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...

  • 1 kudos
Gajji
by New Contributor
  • 4450 Views
  • 2 replies
  • 0 kudos

How to link 2 Dashboards

Hi, I have 2 dashboards. 1st Dashboard has a table with two columns, 'Country' and 'NumberOfMajorCities'. When I click on any row of this table/click on 'NumberOfMajorCities' field of a row, I would like to jump to 2nd Dashboard, where I display all ...

  • 4450 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Ghazanfar Sl​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers ...

  • 0 kudos
1 More Replies
NM447101
by New Contributor II
  • 10513 Views
  • 3 replies
  • 1 kudos

Error when creating a delta live table pipeline

INVALID_PARAMETER_VALUE: Validation failed for node_type_id, the value must be Standard_DS3_v2 (is "Standard_F8s") 

image
  • 10513 Views
  • 3 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Nitya Mehta​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...

  • 1 kudos
2 More Replies
Teja07
by New Contributor II
  • 6921 Views
  • 4 replies
  • 0 kudos

Resolved! Datatype mismatch while reading data from sql server to databricks

Data from Azure sql server was read into databricks through JDBC connection (spark version 2.x) and stored into Gen1. Now the client wants to migrate the data from Gen1 to Gen2. When we ran the same jobs that read data from Azure Sql Server to Databr...

  • 6921 Views
  • 4 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Mani Teja G​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedback ...

  • 0 kudos
3 More Replies
david_bernstein
by New Contributor III
  • 5949 Views
  • 6 replies
  • 0 kudos

DLT autoloader credentials not available error in Azure

I'm just getting started with Databricks and DLTs. I've followed all the docs and tutorials I can find on this and believe I have set up everything in Azure correctly: service principal, paths, and spark configs. When I run a simple DLT autoloader pi...

  • 5949 Views
  • 6 replies
  • 0 kudos
Latest Reply
david_bernstein
New Contributor III
  • 0 kudos

Thank you, I will look into this.

  • 0 kudos
5 More Replies
Fed
by New Contributor III
  • 9680 Views
  • 4 replies
  • 1 kudos

Resolved! Ray dashboard no longer available

Has anyone else experienced the lack of access to the Ray dashboard since this week? Last week worked fine.rom ray.util.spark import setup_ray_cluster   setup_ray_cluster(...)This used to output an HTML block with a link to the dashboard.I can manual...

  • 9680 Views
  • 4 replies
  • 1 kudos
Latest Reply
Fed
New Contributor III
  • 1 kudos

The reason for the missing dashboard for me was due ​to not having installed some required dependencies. Shout out to the Ray community for their help.​I've submitted a PR (now merged) to add a warning message​ when such dependencies are missing.​​

  • 1 kudos
3 More Replies
kj1
by New Contributor III
  • 10183 Views
  • 8 replies
  • 0 kudos

When running DBT pipeline with column docs persisted we get error at least one column must be specified

Problem:When running dbt with persist column docs enabled we get the following error: org.apache.hadoop.hive.ql.metadata.HiveException: at least one column must be specified for the tableBackground:There is an issue on the dbt-spark github that was c...

  • 10183 Views
  • 8 replies
  • 0 kudos
Latest Reply
Dooley
Databricks Employee
  • 0 kudos

Also confirming that you do not have any of these limitations:From DBT's website: Some databases limit where and how descriptions can be added to database objects. Those database adapters might not support persist_docs, or might offer only partial su...

  • 0 kudos
7 More Replies
kll
by New Contributor III
  • 7226 Views
  • 2 replies
  • 0 kudos

`moduleNotFoundError` when attempting to enable a jupyter notebook extension

I am running a set of commands and to run `pydeck` on jupyter notebook as per the documentation here: https://pydeck.gl/installation.html#enabling-pydeck-for-jupyterHowever, it throws an `moduleNotFoundError` exception. !pip install pydeck !jupyter n...

  • 7226 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Keval Shah​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers yo...

  • 0 kudos
1 More Replies
liton10
by New Contributor
  • 10304 Views
  • 1 replies
  • 0 kudos

Unable to link my badge to my Linkedin account

Hi,I am unable to link my Databricks Lakehouse Fundamentals badge to my Linkedin account.I couldn't sign in to credential.databricks.com webpage.My email id is : abhishek.palit@ageas.com.hk

  • 10304 Views
  • 1 replies
  • 0 kudos
Latest Reply
Joe_Suarez
New Contributor III
  • 0 kudos

If you're having trouble linking your Databricks Lakehouse Fundamentals badge to your LinkedIn Sales Navigator extension, there are a few things you can try:Check your badge settings: Make sure that you have enabled sharing for your badge and that yo...

  • 0 kudos
sagnikml
by New Contributor III
  • 6525 Views
  • 0 replies
  • 3 kudos

How to become Databricks Consulting Partner?

I am a Databricks Certified Data Engineer Associate, I want my company to become Databricks Consulting Partner. What is the number of Databricks certified employees required for that?

  • 6525 Views
  • 0 replies
  • 3 kudos
gud4eve
by New Contributor III
  • 5732 Views
  • 1 replies
  • 0 kudos

Resolved! Scala app getting NullPointerException while migrating from DBR 7.3 to 9.1 (and above)

We are migrating our Scala jobs from AWS EMR (6.2.1 and Spark version - 3.0.1) to Lakehouse and few of our jobs are failing due to NullPointerException. We tried in Databricks Runtime 7.3 LTS, it is working fine. Because it had same spark version 3.0...

  • 5732 Views
  • 1 replies
  • 0 kudos
Latest Reply
gud4eve
New Contributor III
  • 0 kudos

In one of my code statements, I updated scala Boolean to java.lang.Boolean and this is working fine now. May be in new newer Spark versions, null in scala Boolean isn't supported.

  • 0 kudos
DianGermishuiz1
by Databricks Partner
  • 5387 Views
  • 4 replies
  • 0 kudos

What is the purpose of the databricks-workspace-stack-lambdazipsbucket S3 bucket created on Databricks AWS Account Provisioning?

An S3 bucket with the prefix "databricks-workspace-stack-lambdazipsbucket" was created by default when I created my AWS Databricks account. It is set to public access. It has one zip file in it called "lambda.zip". What is the purpose of this S3 buck...

  • 5387 Views
  • 4 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Dian Germishuizen​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best ans...

  • 0 kudos
3 More Replies
haxker
by New Contributor
  • 4938 Views
  • 5 replies
  • 2 kudos

Unable to Login and Unable to reset Password . Password reset link keeps on loading even after entering passwords

https://community.cloud.databricks.com/login.htmlUnable to Login and Unable to reset Password . Password reset link keeps on loading even after entering passwords 

  • 4938 Views
  • 5 replies
  • 2 kudos
Latest Reply
Miguel_AngelJim
New Contributor II
  • 2 kudos

I have the same problem, when you press the reset password button it stays loading indefinitely and does not confirm that it has been changed

  • 2 kudos
4 More Replies
QPeiran
by New Contributor III
  • 5107 Views
  • 3 replies
  • 5 kudos

How to exit the entire job in the orchestration scenario?

Hi, can anybody answer this question I posted on StackOverflow? https://stackoverflow.com/questions/73314048/databricks-how-to-exit-the-entire-job-in-the-notebooks-orchestration-scenario

  • 5107 Views
  • 3 replies
  • 5 kudos
Latest Reply
CarterM
New Contributor III
  • 5 kudos

@Vidula Khanna​ @Vidula Khanna​ We are experiencing the same issue in our Workflows and I was wondering if there has been any update.We need the functionality to call a method similar to `dbutils.notebook.exit` in a notebook that will cancel the exec...

  • 5 kudos
2 More Replies
Labels