Hi, I have several clusters, some with a 45% max spot price, some more important with a higher value. Want to know what is the best way to configure this but cannot find anything (a value of how many nodes of the last run were On-demand will do the t...
Full screensmode is not working for spark course, Is there anyone who tried the same and facing the same issue .
Hi @Abhishek Joshi​ Does @Hubert Dudek​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
Hi, I'm running a structured streaming job on a pipeline with a medallion architecture. In my silver layer, we are reading from the bronze layer using structured streaming, and writing the stream to the silver layer w/ a foreachbatch function doing s...
Hi @Danny Liu​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
Currently we are using medallion architecture using delta tables in the form of notebooks and jobs.1) what is the advantage of migrating the existing implementation to Delta Live Tables? 2) what kind of efforts involved in the migration?3) will the m...
I am getting IndentationError: unindent does not match any outer indentation levelbecause the code I pasted has tabs that are 4 spaces, but the tabs in Databricks are 2 spaces. How do I fix this? Do I have to copy and paste it back out?
Is it possible to use ipywidgets in azure databricks dashboards?
Hi @Hossein Amirinia​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.T...
Hi, I'm seeing a lot of empty (and not) directories on routes like:xxxxxx.jobs/FileStore/job-actionstats/xxxxxx.jobs/FileStore/job-result/xxxxxx.jobs/command-results/Can I create a lifecycle to delete old objects (files/directories)? how many days? w...
Hi! I didn't know that, Purging right now, is there a way to schedule that so logs are retained for less time? Maybe I want to maintain the last 7 days for everything?Thanks!
I passed the databricks fundamental exam and like many others I too did not recieved my badge.I am very much intrested in putting this badge on my linkedin profile, please help.My email id is: rahul.psit.ec@gmail.comWhich databricks is resolving as: ...
IssueI am attempting to create a PySpark job via the Databricks UI (with spark-submit) using the parameters below (dependencies are on the PEX file), but I am getting the an exception that the pex file does not exist. It's my understanding that the -...
Hi,I'm facing the same issue trying to execute a pyspark job with spark-submit.I have explored the same solution as you : --files optionspark.pyspark.driver.pythonspark.executorEnv.PEX_ROOTDo you make some progress in the resolution of the problem ?
I have a set of .hdf files that I want to distribute and read on Worker nodes under Databricks environment using PySpark. I am able to read .hdf files on worker nodes and get the data from the files. The next requirement is that now each worker node ...
HI Team,We are getting below error when we are trying to connect our tool by using ODBC connection with out logging to VPN, when we are in VPN we are not getting below issue.[Simba][ThriftExtension] (14) Unexpected response from server during a HTTP ...
@Kaniz Fatma​ our team working with data bricks, we can close this thread
Hi, When I was trying to read the CSV files using pandas I am getting an error which I have mentioned below.df=pd.read_csv("/dbfs/FileStore/tables/badrecord-1.csv")Error: FileNotFoundError: [Errno 2] No such file or directory: '/dbfs/FileStore/tables...
dbutils.fs.ls("/FileStore/tables/badrecord-1.csv")the above file is there in that particular location but still getting the same error
Hiwhen i try to create cluster it just loading but not create cluster.
@Sujit Tibe​ Have you configured Databricks in your own VPC and all pre-requisites related to ports are enabled. Another reason can be cloud provider quota issues. If you click on terminated reason, you can see more information
Whenever you're entitled to use certain technologies in the payment system, spend some time analyzing your digital payment ecosystem before integrating it with these technologies. Once you're done with assessment work, proceed to the process of integ...
In Databricks SQL, I have a query that has a dropdown parameter. How do I get this in the Databricks dashboard? I see the option of adding filters but there is nothing for dashboards
To answer my own question, it seems like if I add the visualisation from the query page then the parameters are automatically added. Added from the dashboard page seems to be bugged
| User | Count |
|---|---|
| 1611 | |
| 768 | |
| 348 | |
| 286 | |
| 252 |
