- 3921 Views
- 5 replies
- 0 kudos
I followed the documentation here under the section "Import a file into a notebook" to import a shared python file among notebooks used by delta live table. But it sometimes can find the module, sometimes not and returns me exception No module named ...
- 3921 Views
- 5 replies
- 0 kudos
Latest Reply
Thank you so much for getting back to us @Jennifer MJ​ . It's really great of you to send in the solution. Would you be happy to mark the answer as best so other community members can find the solution quickly and easily? We really appreciate your ti...
4 More Replies
- 1411 Views
- 1 replies
- 1 kudos
Does anyone have a recommendation for something along the lines of Pydoc that can be used to aggregate docstrings and the like into documentation pages?I tried Pydoc and it failed because of the magic commands in my repo.
- 1411 Views
- 1 replies
- 1 kudos
Latest Reply
Sphinx could be an option here. It parses and render Databricks notebooks in the documentation. You might want to look into it to see if it fits your needs. However, it may not handle magic commands very well, and it assumes your notebooks are export...
- 1045 Views
- 0 replies
- 0 kudos
Databricks' documentation for using bokeh shows using displayHTML, which seems to be incompatible with using ipywidgets, or putting them in Databricks dashboards.I found there is undocumented functionality with plotting interactive altair charts in D...
- 1045 Views
- 0 replies
- 0 kudos
- 3599 Views
- 1 replies
- 4 kudos
The documentation explains how to use multicursor in notebooks. However, it only says it for Windows and MacOS. The Windows way would work in Linux (Ubuntu) up to a few days ago but it does not work now anymore.
- 3599 Views
- 1 replies
- 4 kudos
Latest Reply
@Davide Cagnoni​ :Multicursor support in Databricks notebooks is implemented using the Ace editor, which is a web-based code editor. Therefore, the behavior of multicursor support may depend on the specific browser and operating system you are using....
- 2953 Views
- 8 replies
- 4 kudos
I am trying to run a cluster-scoped init script through Pulumi. I have referred to this documentation https://learn.microsoft.com/en-us/azure/databricks/clusters/configure#spark-configuration However, looks like the documentation is not very clear.I ...
- 2953 Views
- 8 replies
- 4 kudos
Latest Reply
Hi @Sulfikkar Basheer Shylaja​ , Why don't you store the init-script on DBFS and just pass the dbfs:/ path of the init script in Pulumi? You could just run this code on a notebook-%python
dbutils.fs.put("/databricks/init-scripts/set-private-pip-repos...
7 More Replies
- 2116 Views
- 1 replies
- 0 kudos
Hi,As per documentation https://docs.delta.io/latest/quick-start.html , we can configure DeltaCatalog using spark.sql.catalog.spark_catalog.The Iceberg supports two Catalog implementations (https://iceberg.apache.org/docs/latest/spark-configuration/#...
- 2116 Views
- 1 replies
- 0 kudos
Latest Reply
@Arun Sethia​ :Yes, Delta Lake also supports custom catalogs. Delta Lake uses the Spark Catalog API, which allows for pluggable catalog implementations. You can implement your own custom catalog to use with Delta Lake.To use a custom catalog, you can...
- 969 Views
- 3 replies
- 3 kudos
I've been looking through the documentation and I swear this used to be listed somewhere, but for the life of me I can't find it anymore.
- 969 Views
- 3 replies
- 3 kudos
- 811 Views
- 1 replies
- 0 kudos
According to the documentation you can monitor a spark structure stream job using QueryExecutionListener. However I cannot find it. https://docs.databricks.com/structured-streaming/stream-monitoring.html#language-python
- 811 Views
- 1 replies
- 0 kudos
Latest Reply
Which DBR version are you using? also, can you share some code snippet on how you are using the QueryExecutionListener?
- 926 Views
- 1 replies
- 0 kudos
I am trying to following the documentation here:https://learn.microsoft.com/en-us/azure/databricks/getting-started/etl-quick-startMy code looks like:(spark.readStream
.format("cloudFiles")
.option("header", "true")
#.option("cloudFiles.partitio...
- 926 Views
- 1 replies
- 0 kudos
Latest Reply
Hi,It seems like you are writing to a path which is not empty and has some non - delta format files. Also, can you confirm if the path mentioned in the error message "`s3://nbu-ml/projects/rca/msft/dsm09collectx/delta` " is the path you are writing t...
- 472 Views
- 1 replies
- 2 kudos
Documentation Update January 13 - 19Databricks documentation provides how-to guidance and reference information for data analysts, data scientists, and data engineers working in the Databricks Data Science & Engineering, Databricks Machine Learning, ...
- 472 Views
- 1 replies
- 2 kudos
- 622 Views
- 2 replies
- 0 kudos
My DB portal says the platform version is v3.86, and provides a link to all the releases. But none of those releases state the "platform version". And I can't find "v3.86" by searching in the Databricks docs.So, how does one find the documentation fo...
- 622 Views
- 2 replies
- 0 kudos
Latest Reply
Hey @Samuel Yang​ ,Here you will find all detailshttps://docs.databricks.com/release-notes/index.htmlThanksAviral Bhardwaj
1 More Replies
- 698 Views
- 1 replies
- 4 kudos
Documentation Update Databricks documentation provides how-to guidance and reference information for data analysts, data scientists, and data engineers working in the Databricks Data Science & Engineering, Databricks Machine Learning, and Databricks ...
- 698 Views
- 1 replies
- 4 kudos
Latest Reply
Harun
Honored Contributor
Thanks for sharing @Sujitha Ramamoorthy​
- 2544 Views
- 9 replies
- 27 kudos
In the documentation of the V2 it says that it would be released 4 days ago. But searching for it in the academy only throws results for V1 and V2.
- 2544 Views
- 9 replies
- 27 kudos
Latest Reply
@Gustavo Amadoz Navarro​ Updated new infoThis course will be part of the data engineer learning path once the Databricks Certified Data Engineer Associate V3 exam is released (November 19, 2022). BEFORE YOU GET STARTED: Please note that this course,...
8 More Replies
- 1174 Views
- 4 replies
- 6 kudos
Hello ,For a support request, Microsoft support ask me to add spark.databricks.driver.strace.enabled trueto my cluster configuration.MS was not able to send me a link to the documentation and I did not find it on the databricks website.Can someone he...
- 1174 Views
- 4 replies
- 6 kudos
Latest Reply
Yes no problem. I have a python program, called "post ingestion", that run on a databricks job cluster during the night and consist of :inserting data to a deltalake tableexecuting an optimize command on that tableexecuting a vacuum command on that t...
3 More Replies
- 1028 Views
- 3 replies
- 2 kudos
Dear Team,While practising few examples i have noticed the below notebook is not fetching the full dataset and also no schema is fetched https://docs.databricks.com/_static/notebooks/widget-demo.htmlcan you please re try and let me know the results N...
- 1028 Views
- 3 replies
- 2 kudos
Latest Reply
I think https://health.data.ny.gov/api/views/myeu-hzra/rows.csv was a public dataset, but now it shows authentication_required, so this error is independent of databricks.But the good news is that I was able to generate a new URL on the New Your heal...
2 More Replies
