cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
UN
by • New Contributor II
  • 1753 Views
  • 4 replies
  • 1 kudos

Azure Databricks Workspace Editor - Cursor messed up - cannot edit code

I have been using the Azure Databricks Workspace Editor for a few weeks to put together a python script as well as a notebook.All was well, until yesterday evening. Since then I suddenly have the following issuethe cursor in the editor is misbehaving...

Get Started Discussions
Reply
  • 1753 Views
  • 4 replies
  • 1 kudos
View Replies
Latest Reply
UN
New Contributor II
  • 1 kudos

Thanks @Chibberto - I will try the zoom level to see if it makes a difference.In the meantime, the latest issue is that the Autosave is not kicking in sometimes for several minutes. So, if I make a change and then re-run the job - the latest code is ...

  • 1 kudos
3 More Replies
MikeBI
by • New Contributor II
  • 726 Views
  • 1 replies
  • 0 kudos

Resolved! Using Databricks to build a sql server data warehouse

HelloI am new to Databricks.is Databricks a good tool to build a sql server data warehouse using azure?How does this compare to azure data factory?

Get Started Discussions
Reply
  • 726 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @MikeBI, Databricks is a good tool for building an Azure SQL server data warehouse. Databricks SQL (DBSQL) is a cloud-based SQL analytics platform built on Apache Spark™ and Delta Lake, providing a highly performant and reliable data processing e...

  • 0 kudos
Wayne
by • New Contributor III
  • 715 Views
  • 1 replies
  • 1 kudos

Resolved! Databricks Contact us form not working

I got some issues with Databricks online certification.  I filed twice at (https://help.databricks.com/s/contact-us?ReqType=training), but did not get any confirmation emails.@Cert-Team

Get Started Discussions
Reply
  • 715 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
Wayne
New Contributor III
  • 1 kudos

@Kaniz @Cert-Team Finally figured out why my request got dropped silently - I included a link in the form. Please indicate no link in the form submission section. Thanks a lot.

  • 1 kudos
JohnJustus
by • New Contributor III
  • 4118 Views
  • 2 replies
  • 0 kudos

Space in Column names when writing to Hive

All,I have the following code.df_Warehouse_Utilization = (    spark.table("hive_metastore.dev_ork.bin_item_detail")    .join(df_DIM_Bins,col('bin_tag')==df_DIM_Bins.BinKey,'right')    .groupby(col('BinKey'))    .agg(count_distinct(when(col('serial_lo...

Get Started Discussions
Reply
  • 4118 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
daniel_sahal
Esteemed Contributor
  • 0 kudos

@JohnJustus You can try using backticks (`) to escape a column names that contains spaces, but the suggestion here is to get rid of spaces or any special characters in a column names.

  • 0 kudos
1 More Replies
mipayof346
by • New Contributor
  • 1018 Views
  • 1 replies
  • 1 kudos

Resolved! Beginner here. Which exam to do first?

I am new to Databricks and would like to learn and become certified. I have SQL knowledge.To get started, which exam should I do first so that I have a very good understanding of Databricks fundamentals and concepts?I was thinking of “Databricks Cert...

Get Started Discussions
Reply
  • 1018 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
Cert-Team
Honored Contributor III
  • 1 kudos

Hi @mipayof346 The Data Engineer Associate certification exam assesses an individual’s ability to use the Databricks Lakehouse Platform to complete introductory data engineering tasks. This includes an understanding of the Lakehouse Platform and its ...

  • 1 kudos
mipayof346
by • New Contributor
  • 778 Views
  • 1 replies
  • 0 kudos

New to databricks, which certification should I start?

I’m new to Databricks and don’t have any practical experience. I can write SQL code fluently. I’d like to get Databricks certified, which exam should I start to get a good understanding of the fundamentals?Also, what are the most important basic conc...

Get Started Discussions
Reply
  • 778 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
JodySF
Community Manager
  • 0 kudos

@mipayof346 it's best to start with the Lakehouse Fundamentals Accreditation (free course, free assessment). Then I recommend that you move to the Data Analyst certification path. Details on both can be found here: https://www.databricks.com/learn/ce...

  • 0 kudos
maartenvr
by • New Contributor III
  • 7439 Views
  • 6 replies
  • 1 kudos

Installed Library / Module not found through Databricks connect LST 12.2

Hi all,We recently upgraded our databricks compute cluster from runtime version 10.4 LST, to 12.2 LST.After the upgrade one of our python scripts suddenly fails with a module not found error; indicating that our customly created module "xml_parser" i...

Get Started Discussions
Reply
  • 7439 Views
  • 6 replies
  • 1 kudos
View Replies
Latest Reply
maartenvr
New Contributor III
  • 1 kudos

FYI: For now we have found a workaround.We are adding the package as ZIP file to the current spark session with .addyFiles.So after creating a spark session using Databricks-connect we run the following:spark.sparkContext.addPyFile("C:/path/to/custom...

  • 1 kudos
5 More Replies
vk217
by • Contributor
  • 970 Views
  • 1 replies
  • 1 kudos

Resolved! Selective column loader unity catalog

I am loading a table into a data frame using df = spark.table(table_name) Is there a way to load only the required columns? The table has more than 50+ columns and I only need a handful of column.

Get Started Discussions
Reply
  • 970 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
daniel_sahal
Esteemed Contributor
  • 1 kudos

@vk217 Simply just use select function, ex.df = spark.read.table(table_name).select("col1", "col2", "col3")

  • 1 kudos
Sai1098
by • New Contributor II
  • 959 Views
  • 1 replies
  • 0 kudos

Reading tables from different databricks clusters

Hello,My organization uses two cluster for dev and Prod. We mount our azure blobs on to delta lake to store the delta tables. Prod has bunch of data and dev has limited data. I want to move the data from prod to dev for testing purposes. How can I do...

Get Started Discussions
Reply
  • 959 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
btafur
Contributor III
  • 0 kudos

It depends on the current setup, how your clusters are working right now and how your data is stored. One alternative could be mount the Dev storage to the Prod cluster and execute a DEEP CLONE (https://docs.databricks.com/en/sql/language-manual/delt...

  • 0 kudos
ushnish_98
by • New Contributor III
  • 1185 Views
  • 2 replies
  • 0 kudos

Resolved! Databricks Certificate

I cleared Databricks Associate level certification on 25th September. But, I am yet to receive my certificate from Databricks. I raised a ticket for the same but got no response from the suppport team.

Get Started Discussions
Reply
  • 1185 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
APadmanabhan
Moderator
  • 0 kudos

Hello There, Can you share the ticket number, note that we have a high influx of cases and the team is actively working on clearing the backlogs and you should be able to receive the reply in the next 48 hours. I appreciate the patience.

  • 0 kudos
1 More Replies
Sujitha
by • Community Manager
  • 1967 Views
  • 1 replies
  • 6 kudos

Webinar: Accelerate Data and AI Projects With Databricks Notebooks

Register now: October 24, 2023 | 8:00 AM PT Use new capabilities in Databricks Notebooks to speed up innovation. This webinar will walk you through the features that are designed to take the manual effort and delays out of building and deploying dat...

Screenshot 2023-09-22 at 3.47.19 PM.png
Get Started Discussions
Reply
  • 1967 Views
  • 1 replies
  • 6 kudos
View Replies
Latest Reply
Ajay-Pandey
Esteemed Contributor III
  • 6 kudos

This will help databricks user to speed up development.

  • 6 kudos
PériclesTD
by • New Contributor
  • 14479 Views
  • 2 replies
  • 2 kudos

workspace

How can I do to access the Workspace ?   

Get Started Discussions
Reply
  • 14479 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
Debayan
Esteemed Contributor III
  • 2 kudos

Hi, you can try checking https://docs.databricks.com/en/administration-guide/workspace/index.html , please let us know if this helps.  Also please tag @Debayan​ with your next response which will notify me, Thank you!

  • 2 kudos
1 More Replies
eric2
by • New Contributor II
  • 2578 Views
  • 1 replies
  • 0 kudos

Resolved! How to use variable values in Parent Workflow in Child Workflow.

In ADF, a variable called Datetime was specified to be entered and executed in a workflow called Parent using Web Activity.There is a Child Workflow (two-step workflow) in Parent Workflow, and Datetime variables are available in Parent Workflow, but ...

Get Started Discussions
Reply
  • 2578 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @eric2, In Azure Data Factory (ADF), you must use pipeline parameters to pass variables from a parent pipeline to a child pipeline. You can define parameters in your child pipeline and then pass the variable values when you execute the child pipel...

  • 0 kudos
Theor
by • New Contributor III
  • 5624 Views
  • 6 replies
  • 1 kudos

Exam retest voucher

Hi Data bricks Team,I set for Data bricks Certified Machine Learning Professional exam for 2nd time (10 Sept 2023), but didn't pass again. Got 66.66% overall.I am seasoned Data bricks user but this particular exam is quite unorthodox one. Nevertheles...

Get Started Discussions
Cetificate
machine learning
Reply
  • 5624 Views
  • 6 replies
  • 1 kudos
View Replies
Latest Reply
Cert-Team
Honored Contributor III
  • 1 kudos

Hi @Theor Thank you for submitting a ticket to our support team! They are working on it.

  • 1 kudos
5 More Replies
mofa1995
by • New Contributor III
  • 213 Views
  • 1 replies
  • 0 kudos

Is there any way to create Databricks jobCluster through Databricks connect

Is there any way to create Databricks jobCluster through Databricks connect We are using All purposed cluster so far, to reduce Databricks cost we are planning to go ahead with jobCluster but unfortunately, I couldn't find a way to create jobCluster ...

Get Started Discussions
Reply
  • 213 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @mofa1995 , Databricks Connect does not provide functionality to create job clusters directly. However, you can create a job cluster by using the Databricks Jobs REST API or the Databricks CLI.You need to set up the Databricks CLI to call the Job...

  • 0 kudos
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels
Top Kudoed Authors
View all