- 958 Views
- 1 replies
- 0 kudos
Not ab;e to login into databricks community edition
I forgot my password and when i am trying to reset my password , it was struck at reset password page.Could you please help me with it.
- 958 Views
- 1 replies
- 0 kudos
- 0 kudos
Adding @Vidula Khanna​ and @Kaniz Fatma​ for visibility to help with this request
- 0 kudos
- 11684 Views
- 5 replies
- 0 kudos
Databricks SQL Dashboard refresh not updating
I am trying to create a SQL Dashboard on top of a streaming dataset (Delta format). I created multiple queries referencing the file on the datalake, not a hive table. With these queries I created multiple visualizations in a Databricks SQL Dashboard....
- 11684 Views
- 5 replies
- 0 kudos
- 0 kudos
@Jasper Puts​ :Here are some potential solutions or next steps you can try:Check the query refresh rate: Confirm that the queries are also set to refresh every minute or less. If the queries are not refreshing at the same rate as the dashboard, this ...
- 0 kudos
- 8955 Views
- 5 replies
- 1 kudos
Resolved! Databricks job about spark structured streaming zero downtime deployment in terraform.
I would like to ask how to implement zero downtime deployment of spark structured streaming in databricks job compute with terraform. Because we will upgrade spark application code version. But currently we found every deployment will cancel original...
- 8955 Views
- 5 replies
- 1 kudos
- 1 kudos
@Mars Su​ :Yes, you can implement zero downtime deployment of Spark Structured Streaming in Databricks job compute using Terraform. One way to achieve this is by using Databricks' "job clusters" feature, which allows you to create a cluster specifica...
- 1 kudos
- 1523 Views
- 1 replies
- 1 kudos
Columns archive_time, commit_time, archive_time always NULL when running cloud_files_state
Am attempting to find the commit_time for a given file for a delta table using the cloud_files_state command. However, the archive_time, commit_time, and archive_time coluns are always NULL. I am running databrics runtime 11.3 and have also verified ...
- 1523 Views
- 1 replies
- 1 kudos
- 1 kudos
@Morten Stakkeland​ :The issue you are facing with the cloud_files_state command is a known limitation in Delta Lake as of the latest stable release (Delta Lake 1.0). The commit_time and protocol columns are always null, and the archive_time column i...
- 1 kudos
- 1277 Views
- 2 replies
- 0 kudos
What is the best way to ingest GCS data into Databricks and apply Anomaly Detection Model?
I recently started exploring the field of Data Engineering and came across some difficulties. I have a bucket in GCS with millions of parquet files and I want to create an Anomaly Detection model with them. I was trying to ingest that data into Datab...
- 1277 Views
- 2 replies
- 0 kudos
- 0 kudos
@Pedro Barbosa​ :It seems like you are running out of memory when trying to convert the PySpark dataframe to an H2O frame. One possible approach to solve this issue is to partition the PySpark dataframe before converting it to an H2O frame.You can us...
- 0 kudos
- 12448 Views
- 2 replies
- 2 kudos
Is it possible to have Cluster with pre-installed dependencies?
I run some jobs in the Databricks environment where some resources need authentication. I do this (and I need to) through the vault-cli in the init-script.However, every time in the init-script I need to install vault-cli and other libraries. Is ther...
- 12448 Views
- 2 replies
- 2 kudos
- 2 kudos
@João Victor Albuquerque​ :Yes, there are a few ways to pre-install libraries and tools in the Databricks environment:Cluster-scoped init scripts: You can specify a shell script to be run when a cluster is created or restarted. This script can includ...
- 2 kudos
- 99107 Views
- 17 replies
- 20 kudos
- 99107 Views
- 17 replies
- 20 kudos
- 20 kudos
To pass arguments/variables to a notebook, you can use a JSON file to temporarily store the arguments and then pass it as one argument to the notebook. After passing the JSON file to the notebook, you can parse it with json.loads(). The argument list...
- 20 kudos
- 13442 Views
- 17 replies
- 24 kudos
Fatal error: The Python kernel is unresponsive.
I am using MultiThread in this job which creates 8 parallel jobs. It fails for few times in a day and sometimes stuck in any of the Python notebook cell process. Here The Python process exited with an unknown exit code.The last 10 KB of the process's...
- 13442 Views
- 17 replies
- 24 kudos
- 24 kudos
Hey, it seems that the issue is related to the driver undergoing a memory bottleneck, which causes it to crash with an out of memory (OOM) condition and gets restarted or becomes unresponsive due to frequent full garbage collection. The reason for th...
- 24 kudos
- 8350 Views
- 2 replies
- 0 kudos
Resolved! pass application.conf file into databricks jobs
i copied my question from an very old question/post that i reponded. and decided to move it to here:context:I have jar (scala), using scala pureconfig (wrapper of typesafe config)uploaded an application.conf file to a path which is mounted to the wor...
- 8350 Views
- 2 replies
- 0 kudos
- 0 kudos
we had to put the conf in the root folder of the mounted path, and that works.maybe the mounted storage account being blob instead of adls2 is causing the issues.
- 0 kudos
- 7307 Views
- 3 replies
- 2 kudos
Performance issues when loading an Excel file from DBFS using R
I have uploaded small Excel files on my DBFS. I then use function read_xlsx() from the "readxl" package in R to import the file into the R memory. I use a standard cluster (12.1, non ML). The function works but it takes ages. E.g. a simple Excel tabl...
- 7307 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @Marcel Baumgartner​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best an...
- 2 kudos
- 6334 Views
- 4 replies
- 0 kudos
Databricks Associate Exam Webinar Survey link absent
Hi Databricks,I completed the Data Engineer Associate Exam Webinar held on the 11th of April 2020 and haven't received any survey link. How can I give the survey for the exam voucher?Please Guide me on this..
- 6334 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Dhiraj Singh​,Just a friendly follow-up. Do you still need help to get your certificate? or you no longer need help? please les us know
- 0 kudos
- 1601 Views
- 3 replies
- 2 kudos
 Hello Everyone, I am thrilled to announce that we have our first winner for the raffle contest - @Vincent Slegers​. Please join me in congratulating ...
Hello Everyone,I am thrilled to announce that we have our first winner for the raffle contest - @Vincent Slegers​. Please join me in congratulating him on this remarkable achievement!Vincent, your dedication and hard work have paid off, and we are d...
- 1601 Views
- 3 replies
- 2 kudos
- 2818 Views
- 2 replies
- 0 kudos
Issue while using community edition
Hello,Is anyone facing issue with their community edition.?Shows the below error and cant access the workspace or previosly created notebooks..Tried accessing from different devices..(not a cache issue)Error popup:(screenshots attached)Unable to view...
- 2818 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Hari N​ Thank you for reaching out, and we’re sorry to hear about this log-in issue! We have this Community Edition login troubleshooting post on Community. Please take a look, and follow the troubleshooting steps. If the steps do not resolve the...
- 0 kudos
- 2471 Views
- 2 replies
- 0 kudos
Pyspark Structured Streaming Avro integration to Azure Schema Registry with Kafka/Eventhub in Databricks environment.
I am looking for a simple way to have a structured streaming pipeline that would automatically register a schema to Azure schema registry when converting a df col into avro and that would be able to deserialize an avro col based on schema registry ur...
- 2471 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Tomas Sedlon​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers ...
- 0 kudos
- 2178 Views
- 3 replies
- 2 kudos
www.dbdemos.ai
Getting started with Databricks is being made very easy now. Presenting dbdemos.If you're looking to get started with Databricks, there's good news: dbdemos makes it easier than ever. This platform offers a range of demos that you can install directl...
- 2178 Views
- 3 replies
- 2 kudos
- 2 kudos
That's a great share Suteja. Is that supposed to work with the Databricks Community edition account? Had a strange error while trying. Any help is appreciated!Thanks,F
- 2 kudos