- 24456 Views
- 17 replies
- 24 kudos
Fatal error: The Python kernel is unresponsive.
I am using MultiThread in this job which creates 8 parallel jobs. It fails for few times in a day and sometimes stuck in any of the Python notebook cell process. Here The Python process exited with an unknown exit code.The last 10 KB of the process's...
- 24456 Views
- 17 replies
- 24 kudos
- 24 kudos
Hey, it seems that the issue is related to the driver undergoing a memory bottleneck, which causes it to crash with an out of memory (OOM) condition and gets restarted or becomes unresponsive due to frequent full garbage collection. The reason for th...
- 24 kudos
- 10018 Views
- 2 replies
- 0 kudos
Resolved! pass application.conf file into databricks jobs
i copied my question from an very old question/post that i reponded. and decided to move it to here:context:I have jar (scala), using scala pureconfig (wrapper of typesafe config)uploaded an application.conf file to a path which is mounted to the wor...
- 10018 Views
- 2 replies
- 0 kudos
- 0 kudos
we had to put the conf in the root folder of the mounted path, and that works.maybe the mounted storage account being blob instead of adls2 is causing the issues.
- 0 kudos
- 9576 Views
- 3 replies
- 2 kudos
Performance issues when loading an Excel file from DBFS using R
I have uploaded small Excel files on my DBFS. I then use function read_xlsx() from the "readxl" package in R to import the file into the R memory. I use a standard cluster (12.1, non ML). The function works but it takes ages. E.g. a simple Excel tabl...
- 9576 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @Marcel Baumgartner​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best an...
- 2 kudos
- 9047 Views
- 4 replies
- 0 kudos
Databricks Associate Exam Webinar Survey link absent
Hi Databricks,I completed the Data Engineer Associate Exam Webinar held on the 11th of April 2020 and haven't received any survey link. How can I give the survey for the exam voucher?Please Guide me on this..
- 9047 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Dhiraj Singh​,Just a friendly follow-up. Do you still need help to get your certificate? or you no longer need help? please les us know
- 0 kudos
- 2831 Views
- 3 replies
- 2 kudos
 Hello Everyone, I am thrilled to announce that we have our first winner for the raffle contest - @Vincent Slegers​. Please join me in congratulating ...
Hello Everyone,I am thrilled to announce that we have our first winner for the raffle contest - @Vincent Slegers​. Please join me in congratulating him on this remarkable achievement!Vincent, your dedication and hard work have paid off, and we are d...
- 2831 Views
- 3 replies
- 2 kudos
- 4071 Views
- 2 replies
- 0 kudos
Issue while using community edition
Hello,Is anyone facing issue with their community edition.?Shows the below error and cant access the workspace or previosly created notebooks..Tried accessing from different devices..(not a cache issue)Error popup:(screenshots attached)Unable to view...
- 4071 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Hari N​ Thank you for reaching out, and we’re sorry to hear about this log-in issue! We have this Community Edition login troubleshooting post on Community. Please take a look, and follow the troubleshooting steps. If the steps do not resolve the...
- 0 kudos
- 4853 Views
- 2 replies
- 0 kudos
Pyspark Structured Streaming Avro integration to Azure Schema Registry with Kafka/Eventhub in Databricks environment.
I am looking for a simple way to have a structured streaming pipeline that would automatically register a schema to Azure schema registry when converting a df col into avro and that would be able to deserialize an avro col based on schema registry ur...
- 4853 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Tomas Sedlon​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers ...
- 0 kudos
- 3579 Views
- 3 replies
- 2 kudos
www.dbdemos.ai
Getting started with Databricks is being made very easy now. Presenting dbdemos.If you're looking to get started with Databricks, there's good news: dbdemos makes it easier than ever. This platform offers a range of demos that you can install directl...
- 3579 Views
- 3 replies
- 2 kudos
- 2 kudos
That's a great share Suteja. Is that supposed to work with the Databricks Community edition account? Had a strange error while trying. Any help is appreciated!Thanks,F
- 2 kudos
- 5432 Views
- 5 replies
- 1 kudos
Does thrift only exist in databrick control plane?
Hi all, I'm a user of Azure databricks. We recently found there is a thrift vulnerability issue (CVE-2020-13949) in Spark Hive. We have tried to fix it at our side. We also found there is a open issue at Spark jira board - https://issues.apache.org/j...
- 5432 Views
- 5 replies
- 1 kudos
- 1 kudos
Hi @Jimin Hsieh​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so w...
- 1 kudos
- 11739 Views
- 11 replies
- 0 kudos
Process batches in a streaming pipeline - identifying deletes
OK. So I think I'm probably missing the obvious and tying myself in knots here.Here is the scenario:batch datasets arrive in json format in an Azure data lakeeach batch is a complete set of "current" records (the complete table)these are processed us...
- 11739 Views
- 11 replies
- 0 kudos
- 0 kudos
Hi @Kearon McNicol​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...
- 0 kudos
- 20582 Views
- 19 replies
- 1 kudos
Can not access SQL files in the Shared workspace
Hey,we have an issue in that we can access the SQL files whenever the notebook is in the repo path but whenever the CICD pipeline imports the repo notebooks and SQL files to the shared workspace, we can list the SQL files but can not read them.we cha...
- 20582 Views
- 19 replies
- 1 kudos
- 1 kudos
@Nermin Yehia​ yes, as you are moving files to different location manually , just update as can manage permissions in target and that should take care of everything
- 1 kudos
- 3526 Views
- 3 replies
- 0 kudos
Resolved! Delta Live Table Service Upgrade
Dear experts, Might I know what will happen to the delta live table pipeline which is in a cancelled state, when there is a runtime service upgrade? Thanks!
- 3526 Views
- 3 replies
- 0 kudos
- 0 kudos
@KS LAU​ :When a runtime service upgrade occurs in Databricks, any running tasks or pipelines may be temporarily interrupted while the upgrade is being applied. In the case of a cancelled Delta Live Table pipeline, it will not be impacted by the upgr...
- 0 kudos
- 7020 Views
- 4 replies
- 2 kudos
Resolved! DLT target schema - get value during run time
Hi,I would like to know if it is possible to get the target schema, programmatically, inside a DLT.In DLT pipeline settings, destination, target schema.I want to run more idempotent pipelines. For example, my target table has the fields: reference_da...
- 7020 Views
- 4 replies
- 2 kudos
- 2 kudos
Thank you @Suteja Kanuri​ ,Looks like you solution is working, thank you.Regards,
- 2 kudos
- 4496 Views
- 1 replies
- 1 kudos
Resolved! sedona/shapely error Unknown WKB type 16
Hi,i stream data from postgis to s3 using debezium. postgis->debezium->s3->spark(databricks)once read it i decode it and i can see that the binary representation is similiar to what i have in postgis, on a wkb formated column.once i try to read it ei...
- 4496 Views
- 1 replies
- 1 kudos
- 8425 Views
- 7 replies
- 5 kudos
DBR 12.2: DeltaOptimizedWriter: Resolved attribute(s) missing from in operator
After upgrading from DBR 11.3 LTS to DBR 12.2 LTS we started to observe the following error during "read from parquet and write to delta" piece of logic.AnalysisException: Resolved attribute(s) group_id#72,display_name#73,parent_id#74,path#75,path_li...
- 8425 Views
- 7 replies
- 5 kudos
- 5 kudos
I can confirm that this issue is resolved for us as well in the latest 12.2 release.
- 5 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 566 | |
| 349 | |
| 287 |
