I built a machine learning model:lr = LinearRegression()
lr.fit(X_train, y_train)which I can save to the filestore by:filename = "/dbfs/FileStore/lr_model.pkl"
with open(filename, 'wb') as f:
pickle.dump(lr, f)Ideally, I wanted to save the model ...
Hi @Michael Okelola​ , When you store the file in DBFS (/FileStore/...), it's in your account (data plane). While notebooks, etc. are in the Databricks account (control plane). By design, you can't import non-code objects into a workspace. But Repos ...
Hello DataBricks,Struggling with a workflow issue and wondering if anyone can help. I am developing my project in R and sometimes Python locally on my laptop, and committing the files to a git repo. I can then clone that repo in databricks, and *see*...
This is separate script which than need to be run from notebook (or job). I am not using R but in Python and Scala it works the same. In Python I am just importing it in notebook ("from folder_structure import myClass") in R probably similar. There ...
Currently it takes two mouse-clicks to execute each cell in a DB notebook.I know there is a keyboard shortcut (Ctrl+Enter) to execute the current cellBut is there a way to execute a cell with a single mouse-click?I could use a greasemonkey script or ...
The following situation: I am creating a Database with location somewhere in my Azure Lake Gen 2.CREATE SCHEMA IF NOT EXISTS curated LOCATION 'somelocation'Then i want a specific Table in curated to be in a subfolder in 'somelocation':CREATE TABLE IF...
Hi Team,I have taken Databricks Apache Spark 3.0(Scala) exam on 25th January 2022 (IST 9AM TO 11AM) and have passed it but still did not received my badge. I have contacted the support team twice but still no response. @Kaniz Fatma, kindly help to m...
The situation is as following: i have a sheduled job, which uses dbutils.notebook.run(path,timeout) . During the last week everything worked smooth. During the weekend the job began to fail, at the dbutils.notebook.run(path,timeout) command. I get th...
Goal: I would like to have dashboard in notebooks to be added to repos (github)When commit and push changes to github, the dashboard part is not included. Is there a way to include the dashboard in the repo?When later pull data, only notebook code is...
There is API to get dashboards. So you would need to deploy custom CI/D deployment with step to get dashboard and dashboard elements through API and than save returned json to git. You could also deploy some script to azure funtion or aws lambda to d...
Does Spark SQL have both materialized and non-materialized views? With materialized views, it reads from cache for unchanged data, and only from the table for new/changed rows since the view was last accessed? Thanks!
2GB is quite small so usually default settings are the best (so in most cases better result is not to set anything like repartition etc. and leave everything to catalyst optimizer). If you want to set custom partitioning:please remember about avoidi...
Hi everyone, my name is Jan and I'm a product manager working on Databricks Orchestration. We are excited to work with you to build the best Airflow experience within Databricks. Feel free to ask or discuss anything around this integration!
Have took my Spark 3.0 Associate developer certification via webassessor site on 30th Dec 2021 and it said Passed but still waiting to get the certificate & badge details from Databricks.Could you guys please look in to this and provide me with the c...
Hello, @Yogita Nesargi​ !We have an answer for you. Would you please check out these announcements?https://community.databricks.com/s/question/0D53f00001ebiUOCAY/databricks-courseshttps://community.databricks.com/s/feed/0D53f00001dq6W6CAI
Hi team, I took the exam on Jan 14th and passed the exam with 91.66% score. I got an email right after the exam saying I should receive the certificate within one week. But it has been more than 10 days and I haven't heard anything from Databricks te...
Hello, @Luwei Lei​ - We have an answer now.Please check out these announcements.https://community.databricks.com/s/question/0D53f00001ebiUOCAY/databricks-courseshttps://community.databricks.com/s/feed/0D53f00001dq6W6CAI
