- 2912 Views
- 3 replies
- 2 kudos
December exam free voucher for Databricks Certified Associate Developer for Apache Spark 3.0 exam.
Dear @Vidula Khanna​ Hope you're having great day. This is of HIGH priority for me, I've to schedule exam in December before slots are full.I gave Databricks Certified Associate Developer for Apache Spark 3.0 exam on 30th Nov but missed by one perc...
- 2912 Views
- 3 replies
- 2 kudos
- 2 kudos
hey @Smitha Nelapati​ ,you can attend the below webinars and get the 75% off in Jan ​
- 2 kudos
- 4066 Views
- 3 replies
- 6 kudos
Unable to signup to a Databricks community edition account
As you can see, I get the error underneath the big orange button. This is after I click the link at the bottom to try the community edition. I have tried a couple of locations since I am currently based in South Korea but I am actually from the UK. T...
- 4066 Views
- 3 replies
- 6 kudos
- 6 kudos
Hi @Muhammad Ali​ Just a friendly follow-up. Are you able to log in to your Community-Edition account? If yes, then mark the answer as best or if you need further assistance kindly let me know. Thanks and Regards
- 6 kudos
- 3592 Views
- 1 replies
- 0 kudos
Write streaming output to DynamoDB
Hi All,I am trying to write a streaming DF into dynamoDB with below code.tumbling_df.writeStream \ .format("org.apache.spark.sql.execution.streaming.sinks.DynamoDBSinkProvider") \ .option("region", "eu-west-2") \ .option("tableName", "PythonForeac...
- 3592 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @SUDHANSHU RAJ​ ,I can't seem to find much on the "DynamoDBSinkProvider" source. Have you checked out the link for the streaming to DynamoDB documentation?
- 0 kudos
- 4176 Views
- 3 replies
- 3 kudos
Resolved! Is there a way to specify a header, set the delimiter, etc...in DLT?
I was looking forward to using the Data Quality features that are provided with DLT but as far as I can the ingestion process is more restrictive than other methods. It doesn't seem like you can do much as far as setting delimiter type, headers or an...
- 4176 Views
- 3 replies
- 3 kudos
- 3 kudos
DLT uses Autoloader to ingest data. With autoloader, you can provide read options for the table. https://docs.databricks.com/ingestion/auto-loader/options.html#csv-options has the docs on CSV. I attached a picture of an example.
- 3 kudos
- 2983 Views
- 2 replies
- 2 kudos
Resolved! Load an Excel File (located in Databricks Repo connected to Azure DevOps) into a dataframe
Hi, How can I load an Excel File (located in Databricks Repo connected to Azure DevOps) into a dataframe? When I pass the full path into the load method, it displays an error.java.io.FileNotFoundException Has someone done it previously?
- 2983 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi,Just managed to do it.Upgraded the cluster to the latest version because Files in Repos only works in most recent versions of the cluster.When loading the dataframe, specify the path as follows: file:/Workspace/Repos/user@email.com/filepath/filena...
- 2 kudos
- 10547 Views
- 4 replies
- 2 kudos
Resolved! Error when importing PyDeequ package
Hi everyone,I want to do some tests regarding data quality and for that I pretend to use PyDeequ on a databricks notebook. Keep in mind that I'm very new to databricks and Spark.First I created a cluster with the Runtime version "10.4 LTS (includes A...
- 10547 Views
- 4 replies
- 2 kudos
- 2 kudos
I assumed I wouldn't need to add the Deequ library. Apparently, all I had to do was add it via Maven coordinates and it solved the problem.
- 2 kudos
- 1831 Views
- 1 replies
- 0 kudos
Jupyter notebooks import in databricks notebooks
Dear Team,Is it possible to import jupyter notebooks in databricks community edition ? if yes will there be any formatting issues ?
- 1831 Views
- 1 replies
- 0 kudos
- 0 kudos
if yes is there any limit ? , what is the difference or advantage using juypter notebooks over databricks notebooks
- 0 kudos
- 1873 Views
- 2 replies
- 0 kudos
What is the underlying database used in data bricks community edition in sql
Dear DB Experts,I am reaching out to check whether can i still use postgressql in notebooks with notebook as sql and try postgresql , as far as i know from reading the back end db is mysql correct me my understanding ?
- 1873 Views
- 2 replies
- 0 kudos
- 746 Views
- 0 replies
- 0 kudos
Why does the Spark user differ if using sparklyr from Spark SQL?
My team uses a shared cluster. We've been having issues with spark_connect failing to work at times (can't easily reproduce). One thing I've recently noticed is that the Spark user through sparklyr seems to be set to the first person who connects to ...
- 746 Views
- 0 replies
- 0 kudos
- 13533 Views
- 5 replies
- 2 kudos
Resolved! Call Databricks notebook in a specific branch from Azure Data Factory?
I'm using the new Databricks Repos functionality and in Azure Data Factory UI for the notebook activity you can browse the Databricks workspace and select Repos > username > project > folder > notebook.Is it possible to call a Databricks notebook in ...
- 13533 Views
- 5 replies
- 2 kudos
- 2 kudos
Greetings, I have similar problem. Did you try to use Databricks workflows instead and schedule them instead on Data Factory?Because inside workflows it is possible to select a specific branch, so it may actually work.What do you think?
- 2 kudos
- 3526 Views
- 2 replies
- 2 kudos
Data explorer missing a vital scroller
hinot sure best way to raise an issue, working in Azure DB1)the horizonal scroll bar has gone walkabouts and is now at the end of the vertical (in Chrome, was ok a few weeks ago) (see image)can we have it back to the bottom of the screen please?also...
- 3526 Views
- 2 replies
- 2 kudos
- 2 kudos
it is not a question really , more a request to fix the bug. scrollers not working as they should
- 2 kudos
- 1698 Views
- 1 replies
- 1 kudos
Markdown issues with adding text in math mode
I am trying to use the \text function from the package amsmath, but I can't seem to find a way to call this package. Is there another route of achieving the same result? Thanks!
- 1698 Views
- 1 replies
- 1 kudos
- 1 kudos
can you share exact code by that we can also test the same and help you in better manner
- 1 kudos
- 4221 Views
- 1 replies
- 1 kudos
How can I create grouped bars that are also stacked visualization?
I wish to create a visualization combined of grouped bars, and also have those bars stacked.Attached is a sketch of the final result I am interested in.I am also attaching my sql because I'm not sure if I should "group by" in the query or in the visu...
- 4221 Views
- 1 replies
- 1 kudos
- 1 kudos
try to use asc or dsc keyword ,maybe it worked
- 1 kudos
- 2555 Views
- 1 replies
- 0 kudos
JDBC to delta lake: Is setting fetch size expected to be effective?
I am using the databricks jdbc driver to access a delta lake. The database URL specifies transportMode=http. I have experimented with setting different values of fetchSize on the java.sqlPreparedStatement object and have monitored memory use within m...
- 2555 Views
- 1 replies
- 0 kudos
- 0 kudos
I think there is one spark configuration but I forgot right now Pelase try to utilized this doc maybe you get something- https://spark.apache.org/docs/latest/configuration.html
- 0 kudos
- 2817 Views
- 1 replies
- 0 kudos
Access same createorreplacetempview("viewname") by multiple functions.
I have several functions accessing the same createorreplacetempview("viewname"). Does this cause any issues with multiple functions accessing it in a distributed environment?def get_data_sql(spark_session, data_frame, data_element): data_fram...
- 2817 Views
- 1 replies
- 0 kudos
- 0 kudos
there is two type of viewsone is global view - it will be available for whole cluster and notebook but it will removed after cluster restartand another is Temp view- that will be available for only notebook level, and other notebook will not able to ...
- 0 kudos
| User | Count |
|---|---|
| 1616 | |
| 790 | |
| 452 | |
| 349 | |
| 287 |
