- 4051 Views
- 0 replies
- 0 kudos
Databricks on Virtualization
Hi Team,Can you please direct me to any content on Databricks on Virtualization?Regards,Phanindra
- 4051 Views
- 0 replies
- 0 kudos
Forum Posts
- 5350 Views
- 0 replies
- 3 kudos
What's new in Databricks for September 2023
Platform You can now use Structured Streaming to Stream Data from Apache Pulsar on Databricks. For more information : https://docs.databricks.com/en/structured-streaming/pulsar.html (DBR 14.1 required)Databricks Runtime 14.1 and 14.1 ML are now avail...
- 5350 Views
- 0 replies
- 3 kudos
- 1592 Views
- 0 replies
- 0 kudos
Bootstrap Timeout during cluster start on AWS cloud
Hi!We had bunch of strange failures for our jobs during 28-29 of September.Some jobs` runs could not start for some time (30-50 mins) and then were failed with an error:Unexpected failure while waiting for the cluster (0929-002141-2zkekhdj) to be rea...
- 1592 Views
- 0 replies
- 0 kudos
- 1727 Views
- 1 replies
- 1 kudos
Resolved! Databricks Contact us form not working
I got some issues with Databricks online certification. I filed twice at (https://help.databricks.com/s/contact-us?ReqType=training), but did not get any confirmation emails.@Cert-Team
- 1727 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
@Kaniz @Cert-Team Finally figured out why my request got dropped silently - I included a link in the form. Please indicate no link in the form submission section. Thanks a lot.
- 1 kudos
- 3367 Views
- 2 replies
- 0 kudos
om.microsoft.azure.storage.StorageException: The specifed resource name contains invalid characters.
Hi guys I'm relatively new to Databricks and struggling to implement an autoloader ( with trigger once = true ) in file notifications mode. I have CSV files in one container (landing zone). I would like the autoloader to pick up new and existing file...
- 3367 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi Kaniz, thank you for your reply. I initially made the mistake of using a capital letter in the queue as part of config files. I can now write, there is no error as a batch process. However, when I try to run the write stream, it says"Running Comma...
- 0 kudos
- 13751 Views
- 8 replies
- 1 kudos
My exam Datbricks Data Engineer Associate got suspended_need immediate help please (10/09/2023)
Hello Team,I encountered Pathetic experience while attempting my DataBricks Data engineer certification. Abruptly, Proctor asked me to show my desk, after showing he/she asked multiple times.. wasted my time and then suspended my exam.I want to file ...
- 13751 Views
- 8 replies
- 1 kudos
Latest Reply
- 1955 Views
- 0 replies
- 0 kudos
xgboost.spark.core' has no attribute 'SparkXGBClassifierModel'
I got error: xgboost.spark.core' has no attribute 'SparkXGBClassifierModel' when attempting to load model. I have upgraded to xgboost-2.0.0.
- 1955 Views
- 0 replies
- 0 kudos
- 4139 Views
- 0 replies
- 0 kudos
Very large binary files ingestion error when using binaryFile reader
Hello, I am facing an error while trying to read a large binary file (rosbag format) using binaryFile reader. The file I am trying to read is approx 7GB large. Here's the error message I am getting:FileReadException: Error while reading file dbfs:/mn...
- 4139 Views
- 0 replies
- 0 kudos
- 1400 Views
- 0 replies
- 0 kudos
Using Databricks to build a sql server data warehouse
HelloI am new to Databricks.is Databricks a good tool to build a sql server data warehouse using azure?How does this compare to azure data factory?
- 1400 Views
- 0 replies
- 0 kudos
- 2263 Views
- 1 replies
- 1 kudos
Resolved! Beginner here. Which exam to do first?
I am new to Databricks and would like to learn and become certified. I have SQL knowledge.To get started, which exam should I do first so that I have a very good understanding of Databricks fundamentals and concepts?I was thinking of “Databricks Cert...
- 2263 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi @mipayof346 The Data Engineer Associate certification exam assesses an individual’s ability to use the Databricks Lakehouse Platform to complete introductory data engineering tasks. This includes an understanding of the Lakehouse Platform and its ...
- 1 kudos
- 1131 Views
- 0 replies
- 0 kudos
Init script upload issue as file format in workspace using databricks cli
Trying to import init script from local to workspace location using databricks cli via YAML pipeline but it is getting uploaded as notebook.Need to upload it as file format using cli command as workspace init script should be in file format.Does anyo...
- 1131 Views
- 0 replies
- 0 kudos
- 2614 Views
- 1 replies
- 1 kudos
Is Datbricks-Salesforce already available?
Reference: Salesforce and Databricks Announce Strategic Partnership to Bring Lakehouse Data Sharing and Shared AI Models to Businesses - Salesforce NewsI was going through this article and wanted to know if anyone in community is planning to use this...
- 2614 Views
- 1 replies
- 1 kudos
- 2191 Views
- 0 replies
- 0 kudos
Calculation on a dataframe
Hi, I need to do following calculations on a dataframe. It should be done for each period and calculated value will be used for next period's calculation. Adding sample data and formula from excel here. Thanks in advance for your help.Need to calcula...
- 2191 Views
- 0 replies
- 0 kudos
- 1950 Views
- 1 replies
- 0 kudos
New to databricks, which certification should I start?
I’m new to Databricks and don’t have any practical experience. I can write SQL code fluently. I’d like to get Databricks certified, which exam should I start to get a good understanding of the fundamentals?Also, what are the most important basic conc...
- 1950 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
@mipayof346 it's best to start with the Lakehouse Fundamentals Accreditation (free course, free assessment). Then I recommend that you move to the Data Analyst certification path. Details on both can be found here: https://www.databricks.com/learn/ce...
- 0 kudos
- 2048 Views
- 1 replies
- 1 kudos
Resolved! Selective column loader unity catalog
I am loading a table into a data frame using df = spark.table(table_name) Is there a way to load only the required columns? The table has more than 50+ columns and I only need a handful of column.
- 2048 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
@vk217 Simply just use select function, ex.df = spark.read.table(table_name).select("col1", "col2", "col3")
- 1 kudos
