- 2165 Views
- 1 replies
- 0 kudos
Does Databricks provide a mechanism to have rate limiting for receivers?
from pyspark.sql import SparkSession scala_version = '2.12' spark_version = '3.3.0' packages = [ f'org.apache.spark:spark-sql-kafka-0-10_{scala_version}:{spark_version}', 'org.apache.kafka:kafka-clients:3.2.1' ] spark = SparkSession.bui...
- 2165 Views
- 1 replies
- 0 kudos
- 0 kudos
hi @Software Architect i dont think so
- 0 kudos
- 4409 Views
- 7 replies
- 1 kudos
Resolved! Badge Not Received for - Databricks Lakehouse Fundamentals Accreditation (V2)
Hi TeamI have passed the Databricks Lakehouse Fundamentals Accreditation (V2) on Dec 8th.Still have not received the Badge in credentials or any email of that kind.Please have a look.@Kaniz Fatma
- 4409 Views
- 7 replies
- 1 kudos
- 1 kudos
I wonder how much attempt you set to create this type of excellent informative web site.marco's pizza starr avedomino's pizza price
- 1 kudos
- 3142 Views
- 3 replies
- 2 kudos
December exam free voucher for Databricks Certified Associate Developer for Apache Spark 3.0 exam.
Dear @Vidula Khanna Hope you're having great day. This is of HIGH priority for me, I've to schedule exam in December before slots are full.I gave Databricks Certified Associate Developer for Apache Spark 3.0 exam on 30th Nov but missed by one perc...
- 3142 Views
- 3 replies
- 2 kudos
- 2 kudos
hey @Smitha Nelapati ,you can attend the below webinars and get the 75% off in Jan
- 2 kudos
- 4295 Views
- 3 replies
- 6 kudos
Unable to signup to a Databricks community edition account
As you can see, I get the error underneath the big orange button. This is after I click the link at the bottom to try the community edition. I have tried a couple of locations since I am currently based in South Korea but I am actually from the UK. T...
- 4295 Views
- 3 replies
- 6 kudos
- 6 kudos
Hi @Muhammad Ali Just a friendly follow-up. Are you able to log in to your Community-Edition account? If yes, then mark the answer as best or if you need further assistance kindly let me know. Thanks and Regards
- 6 kudos
- 3790 Views
- 1 replies
- 0 kudos
Write streaming output to DynamoDB
Hi All,I am trying to write a streaming DF into dynamoDB with below code.tumbling_df.writeStream \ .format("org.apache.spark.sql.execution.streaming.sinks.DynamoDBSinkProvider") \ .option("region", "eu-west-2") \ .option("tableName", "PythonForeac...
- 3790 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @SUDHANSHU RAJ ,I can't seem to find much on the "DynamoDBSinkProvider" source. Have you checked out the link for the streaming to DynamoDB documentation?
- 0 kudos
- 4412 Views
- 3 replies
- 3 kudos
Resolved! Is there a way to specify a header, set the delimiter, etc...in DLT?
I was looking forward to using the Data Quality features that are provided with DLT but as far as I can the ingestion process is more restrictive than other methods. It doesn't seem like you can do much as far as setting delimiter type, headers or an...
- 4412 Views
- 3 replies
- 3 kudos
- 3 kudos
DLT uses Autoloader to ingest data. With autoloader, you can provide read options for the table. https://docs.databricks.com/ingestion/auto-loader/options.html#csv-options has the docs on CSV. I attached a picture of an example.
- 3 kudos
- 3155 Views
- 2 replies
- 2 kudos
Resolved! Load an Excel File (located in Databricks Repo connected to Azure DevOps) into a dataframe
Hi, How can I load an Excel File (located in Databricks Repo connected to Azure DevOps) into a dataframe? When I pass the full path into the load method, it displays an error.java.io.FileNotFoundException Has someone done it previously?
- 3155 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi,Just managed to do it.Upgraded the cluster to the latest version because Files in Repos only works in most recent versions of the cluster.When loading the dataframe, specify the path as follows: file:/Workspace/Repos/user@email.com/filepath/filena...
- 2 kudos
- 10954 Views
- 4 replies
- 2 kudos
Resolved! Error when importing PyDeequ package
Hi everyone,I want to do some tests regarding data quality and for that I pretend to use PyDeequ on a databricks notebook. Keep in mind that I'm very new to databricks and Spark.First I created a cluster with the Runtime version "10.4 LTS (includes A...
- 10954 Views
- 4 replies
- 2 kudos
- 2 kudos
I assumed I wouldn't need to add the Deequ library. Apparently, all I had to do was add it via Maven coordinates and it solved the problem.
- 2 kudos
- 3135 Views
- 1 replies
- 0 kudos
Jupyter notebooks import in databricks notebooks
Dear Team,Is it possible to import jupyter notebooks in databricks community edition ? if yes will there be any formatting issues ?
- 3135 Views
- 1 replies
- 0 kudos
- 0 kudos
if yes is there any limit ? , what is the difference or advantage using juypter notebooks over databricks notebooks
- 0 kudos
- 2126 Views
- 2 replies
- 0 kudos
What is the underlying database used in data bricks community edition in sql
Dear DB Experts,I am reaching out to check whether can i still use postgressql in notebooks with notebook as sql and try postgresql , as far as i know from reading the back end db is mysql correct me my understanding ?
- 2126 Views
- 2 replies
- 0 kudos
- 802 Views
- 0 replies
- 0 kudos
Why does the Spark user differ if using sparklyr from Spark SQL?
My team uses a shared cluster. We've been having issues with spark_connect failing to work at times (can't easily reproduce). One thing I've recently noticed is that the Spark user through sparklyr seems to be set to the first person who connects to ...
- 802 Views
- 0 replies
- 0 kudos
- 14109 Views
- 5 replies
- 2 kudos
Resolved! Call Databricks notebook in a specific branch from Azure Data Factory?
I'm using the new Databricks Repos functionality and in Azure Data Factory UI for the notebook activity you can browse the Databricks workspace and select Repos > username > project > folder > notebook.Is it possible to call a Databricks notebook in ...
- 14109 Views
- 5 replies
- 2 kudos
- 2 kudos
Greetings, I have similar problem. Did you try to use Databricks workflows instead and schedule them instead on Data Factory?Because inside workflows it is possible to select a specific branch, so it may actually work.What do you think?
- 2 kudos
- 3721 Views
- 2 replies
- 2 kudos
Data explorer missing a vital scroller
hinot sure best way to raise an issue, working in Azure DB1)the horizonal scroll bar has gone walkabouts and is now at the end of the vertical (in Chrome, was ok a few weeks ago) (see image)can we have it back to the bottom of the screen please?also...
- 3721 Views
- 2 replies
- 2 kudos
- 2 kudos
it is not a question really , more a request to fix the bug. scrollers not working as they should
- 2 kudos
- 1820 Views
- 1 replies
- 1 kudos
Markdown issues with adding text in math mode
I am trying to use the \text function from the package amsmath, but I can't seem to find a way to call this package. Is there another route of achieving the same result? Thanks!
- 1820 Views
- 1 replies
- 1 kudos
- 1 kudos
can you share exact code by that we can also test the same and help you in better manner
- 1 kudos
- 4536 Views
- 1 replies
- 1 kudos
How can I create grouped bars that are also stacked visualization?
I wish to create a visualization combined of grouped bars, and also have those bars stacked.Attached is a sketch of the final result I am interested in.I am also attaching my sql because I'm not sure if I should "group by" in the query or in the visu...
- 4536 Views
- 1 replies
- 1 kudos
- 1 kudos
try to use asc or dsc keyword ,maybe it worked
- 1 kudos
| User | Count |
|---|---|
| 1630 | |
| 790 | |
| 512 | |
| 349 | |
| 287 |
