- 6007 Views
- 4 replies
- 13 kudos
Resolved! To practice Databricks SQL
Is there any sand box kind of thing where we can do some hands-on on Databricks SQL/run the Note books attaching to the Clusters apart from the free trial provided by Databricks.
- 6007 Views
- 4 replies
- 13 kudos
- 13 kudos
Databricks SQL workspace will be available only for Databricks Premium service. If you have Azure Pass subscription, then you can able to get it for practicing it.
- 13 kudos
- 2721 Views
- 4 replies
- 0 kudos
Hi, Has anyone cleared professional DE? please advise on professional data engineer exam. will advance DE learning path be sufficient? Or need to fol...
Hi,Has anyone cleared professional DE? please advise on professional data engineer exam. will advance DE learning path be sufficient? Or need to follow some other resource as well.
- 2721 Views
- 4 replies
- 0 kudos
- 0 kudos
Hello Have a look at this link http://msdatalab.net/how-to-pass-the-professional-databricks-data-engineering/
- 0 kudos
- 2650 Views
- 1 replies
- 0 kudos
Does Databricks provide a mechanism to have rate limiting for receivers?
from pyspark.sql import SparkSession scala_version = '2.12' spark_version = '3.3.0' packages = [ f'org.apache.spark:spark-sql-kafka-0-10_{scala_version}:{spark_version}', 'org.apache.kafka:kafka-clients:3.2.1' ] spark = SparkSession.bui...
- 2650 Views
- 1 replies
- 0 kudos
- 0 kudos
hi @Software Architect i dont think so
- 0 kudos
- 5600 Views
- 7 replies
- 1 kudos
Resolved! Badge Not Received for - Databricks Lakehouse Fundamentals Accreditation (V2)
Hi TeamI have passed the Databricks Lakehouse Fundamentals Accreditation (V2) on Dec 8th.Still have not received the Badge in credentials or any email of that kind.Please have a look.@Kaniz Fatma
- 5600 Views
- 7 replies
- 1 kudos
- 1 kudos
I wonder how much attempt you set to create this type of excellent informative web site.marco's pizza starr avedomino's pizza price
- 1 kudos
- 3937 Views
- 3 replies
- 2 kudos
December exam free voucher for Databricks Certified Associate Developer for Apache Spark 3.0 exam.
Dear @Vidula Khanna Hope you're having great day. This is of HIGH priority for me, I've to schedule exam in December before slots are full.I gave Databricks Certified Associate Developer for Apache Spark 3.0 exam on 30th Nov but missed by one perc...
- 3937 Views
- 3 replies
- 2 kudos
- 2 kudos
hey @Smitha Nelapati ,you can attend the below webinars and get the 75% off in Jan
- 2 kudos
- 5229 Views
- 3 replies
- 6 kudos
Unable to signup to a Databricks community edition account
As you can see, I get the error underneath the big orange button. This is after I click the link at the bottom to try the community edition. I have tried a couple of locations since I am currently based in South Korea but I am actually from the UK. T...
- 5229 Views
- 3 replies
- 6 kudos
- 6 kudos
Hi @Muhammad Ali Just a friendly follow-up. Are you able to log in to your Community-Edition account? If yes, then mark the answer as best or if you need further assistance kindly let me know. Thanks and Regards
- 6 kudos
- 4358 Views
- 1 replies
- 0 kudos
Write streaming output to DynamoDB
Hi All,I am trying to write a streaming DF into dynamoDB with below code.tumbling_df.writeStream \ .format("org.apache.spark.sql.execution.streaming.sinks.DynamoDBSinkProvider") \ .option("region", "eu-west-2") \ .option("tableName", "PythonForeac...
- 4358 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @SUDHANSHU RAJ ,I can't seem to find much on the "DynamoDBSinkProvider" source. Have you checked out the link for the streaming to DynamoDB documentation?
- 0 kudos
- 5294 Views
- 3 replies
- 3 kudos
Resolved! Is there a way to specify a header, set the delimiter, etc...in DLT?
I was looking forward to using the Data Quality features that are provided with DLT but as far as I can the ingestion process is more restrictive than other methods. It doesn't seem like you can do much as far as setting delimiter type, headers or an...
- 5294 Views
- 3 replies
- 3 kudos
- 3 kudos
DLT uses Autoloader to ingest data. With autoloader, you can provide read options for the table. https://docs.databricks.com/ingestion/auto-loader/options.html#csv-options has the docs on CSV. I attached a picture of an example.
- 3 kudos
- 3740 Views
- 2 replies
- 2 kudos
Resolved! Load an Excel File (located in Databricks Repo connected to Azure DevOps) into a dataframe
Hi, How can I load an Excel File (located in Databricks Repo connected to Azure DevOps) into a dataframe? When I pass the full path into the load method, it displays an error.java.io.FileNotFoundException Has someone done it previously?
- 3740 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi,Just managed to do it.Upgraded the cluster to the latest version because Files in Repos only works in most recent versions of the cluster.When loading the dataframe, specify the path as follows: file:/Workspace/Repos/user@email.com/filepath/filena...
- 2 kudos
- 13088 Views
- 4 replies
- 2 kudos
Resolved! Error when importing PyDeequ package
Hi everyone,I want to do some tests regarding data quality and for that I pretend to use PyDeequ on a databricks notebook. Keep in mind that I'm very new to databricks and Spark.First I created a cluster with the Runtime version "10.4 LTS (includes A...
- 13088 Views
- 4 replies
- 2 kudos
- 2 kudos
I assumed I wouldn't need to add the Deequ library. Apparently, all I had to do was add it via Maven coordinates and it solved the problem.
- 2 kudos
- 3491 Views
- 1 replies
- 0 kudos
Jupyter notebooks import in databricks notebooks
Dear Team,Is it possible to import jupyter notebooks in databricks community edition ? if yes will there be any formatting issues ?
- 3491 Views
- 1 replies
- 0 kudos
- 0 kudos
if yes is there any limit ? , what is the difference or advantage using juypter notebooks over databricks notebooks
- 0 kudos
- 2538 Views
- 2 replies
- 0 kudos
What is the underlying database used in data bricks community edition in sql
Dear DB Experts,I am reaching out to check whether can i still use postgressql in notebooks with notebook as sql and try postgresql , as far as i know from reading the back end db is mysql correct me my understanding ?
- 2538 Views
- 2 replies
- 0 kudos
- 970 Views
- 0 replies
- 0 kudos
Why does the Spark user differ if using sparklyr from Spark SQL?
My team uses a shared cluster. We've been having issues with spark_connect failing to work at times (can't easily reproduce). One thing I've recently noticed is that the Spark user through sparklyr seems to be set to the first person who connects to ...
- 970 Views
- 0 replies
- 0 kudos
- 15988 Views
- 5 replies
- 2 kudos
Resolved! Call Databricks notebook in a specific branch from Azure Data Factory?
I'm using the new Databricks Repos functionality and in Azure Data Factory UI for the notebook activity you can browse the Databricks workspace and select Repos > username > project > folder > notebook.Is it possible to call a Databricks notebook in ...
- 15988 Views
- 5 replies
- 2 kudos
- 2 kudos
Greetings, I have similar problem. Did you try to use Databricks workflows instead and schedule them instead on Data Factory?Because inside workflows it is possible to select a specific branch, so it may actually work.What do you think?
- 2 kudos
- 4385 Views
- 2 replies
- 2 kudos
Data explorer missing a vital scroller
hinot sure best way to raise an issue, working in Azure DB1)the horizonal scroll bar has gone walkabouts and is now at the end of the vertical (in Chrome, was ok a few weeks ago) (see image)can we have it back to the bottom of the screen please?also...
- 4385 Views
- 2 replies
- 2 kudos
- 2 kudos
it is not a question really , more a request to fix the bug. scrollers not working as they should
- 2 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 582 | |
| 349 | |
| 287 |
