I haven't received my Databricks Certified Data Engineer AssociateI have pass my certification exam, Databricks Certified Data Engineer Associate on 27 October 2022.. I am yet to receive a certificate or badge. Any help is much appreciated. I have a ...
When developing locally, I can write %debug in a new cell after encountering an error, and jump into the function that the error originated from. In Databricks, this freezes the notebook indefinitely.For example:In [1]:def query_data(): df_full = qu...
I just upgraded my personal node and %debug worked! I appreciate the reminder to use pdb() itself when appropriate too. I'm still interested in whether we should have any concerns about upgrading our main cluster - please do let me know, and then I'l...
Hi I have similiar issue:https://community.databricks.com/s/question/0D58Y00008oTUQPSA4/voucher-code-error
Hi, I have several clusters, some with a 45% max spot price, some more important with a higher value. Want to know what is the best way to configure this but cannot find anything (a value of how many nodes of the last run were On-demand will do the t...
Full screensmode is not working for spark course, Is there anyone who tried the same and facing the same issue .
Hi @Abhishek Joshi​ Does @Hubert Dudek​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
Hi, I'm running a structured streaming job on a pipeline with a medallion architecture. In my silver layer, we are reading from the bronze layer using structured streaming, and writing the stream to the silver layer w/ a foreachbatch function doing s...
Hi @Danny Liu​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
Currently we are using medallion architecture using delta tables in the form of notebooks and jobs.1) what is the advantage of migrating the existing implementation to Delta Live Tables? 2) what kind of efforts involved in the migration?3) will the m...
I am getting IndentationError: unindent does not match any outer indentation levelbecause the code I pasted has tabs that are 4 spaces, but the tabs in Databricks are 2 spaces. How do I fix this? Do I have to copy and paste it back out?
Is it possible to use ipywidgets in azure databricks dashboards?
Hi @Hossein Amirinia​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.T...
Hi, I'm seeing a lot of empty (and not) directories on routes like:xxxxxx.jobs/FileStore/job-actionstats/xxxxxx.jobs/FileStore/job-result/xxxxxx.jobs/command-results/Can I create a lifecycle to delete old objects (files/directories)? how many days? w...
Hi! I didn't know that, Purging right now, is there a way to schedule that so logs are retained for less time? Maybe I want to maintain the last 7 days for everything?Thanks!
I passed the databricks fundamental exam and like many others I too did not recieved my badge.I am very much intrested in putting this badge on my linkedin profile, please help.My email id is: rahul.psit.ec@gmail.comWhich databricks is resolving as: ...
IssueI am attempting to create a PySpark job via the Databricks UI (with spark-submit) using the parameters below (dependencies are on the PEX file), but I am getting the an exception that the pex file does not exist. It's my understanding that the -...
Hi,I'm facing the same issue trying to execute a pyspark job with spark-submit.I have explored the same solution as you : --files optionspark.pyspark.driver.pythonspark.executorEnv.PEX_ROOTDo you make some progress in the resolution of the problem ?
I have a set of .hdf files that I want to distribute and read on Worker nodes under Databricks environment using PySpark. I am able to read .hdf files on worker nodes and get the data from the files. The next requirement is that now each worker node ...
HI Team,We are getting below error when we are trying to connect our tool by using ODBC connection with out logging to VPN, when we are in VPN we are not getting below issue.[Simba][ThriftExtension] (14) Unexpected response from server during a HTTP ...
@Kaniz Fatma​ our team working with data bricks, we can close this thread
Hi, When I was trying to read the CSV files using pandas I am getting an error which I have mentioned below.df=pd.read_csv("/dbfs/FileStore/tables/badrecord-1.csv")Error: FileNotFoundError: [Errno 2] No such file or directory: '/dbfs/FileStore/tables...
dbutils.fs.ls("/FileStore/tables/badrecord-1.csv")the above file is there in that particular location but still getting the same error
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 582 | |
| 349 | |
| 287 |
