Please include in documentation an example how to incorporate the `QUERY_RESULT_ROWS` variable in the custom template.
Databricks confirmed this was an issue on their end and it should be resolved now. It is working for me.
I followed the documentation here under the section "Import a file into a notebook" to import a shared python file among notebooks used by delta live table. But it sometimes can find the module, sometimes not and returns me exception No module named ...
Thank you so much for getting back to us @Jennifer MJ . It's really great of you to send in the solution. Would you be happy to mark the answer as best so other community members can find the solution quickly and easily? We really appreciate your ti...
Does anyone have a recommendation for something along the lines of Pydoc that can be used to aggregate docstrings and the like into documentation pages?I tried Pydoc and it failed because of the magic commands in my repo.
Sphinx could be an option here. It parses and render Databricks notebooks in the documentation. You might want to look into it to see if it fits your needs. However, it may not handle magic commands very well, and it assumes your notebooks are export...
Databricks' documentation for using bokeh shows using displayHTML, which seems to be incompatible with using ipywidgets, or putting them in Databricks dashboards.I found there is undocumented functionality with plotting interactive altair charts in D...
The documentation explains how to use multicursor in notebooks. However, it only says it for Windows and MacOS. The Windows way would work in Linux (Ubuntu) up to a few days ago but it does not work now anymore.
@Davide Cagnoni :Multicursor support in Databricks notebooks is implemented using the Ace editor, which is a web-based code editor. Therefore, the behavior of multicursor support may depend on the specific browser and operating system you are using....
I am trying to run a cluster-scoped init script through Pulumi. I have referred to this documentation https://learn.microsoft.com/en-us/azure/databricks/clusters/configure#spark-configuration However, looks like the documentation is not very clear.I ...
Hi @Sulfikkar Basheer Shylaja , Why don't you store the init-script on DBFS and just pass the dbfs:/ path of the init script in Pulumi? You could just run this code on a notebook-%python dbutils.fs.put("/databricks/init-scripts/set-private-pip-repos...
Hi,As per documentation https://docs.delta.io/latest/quick-start.html , we can configure DeltaCatalog using spark.sql.catalog.spark_catalog.The Iceberg supports two Catalog implementations (https://iceberg.apache.org/docs/latest/spark-configuration/#...
@Arun Sethia :Yes, Delta Lake also supports custom catalogs. Delta Lake uses the Spark Catalog API, which allows for pluggable catalog implementations. You can implement your own custom catalog to use with Delta Lake.To use a custom catalog, you can...
I've been looking through the documentation and I swear this used to be listed somewhere, but for the life of me I can't find it anymore.
According to the documentation you can monitor a spark structure stream job using QueryExecutionListener. However I cannot find it. https://docs.databricks.com/structured-streaming/stream-monitoring.html#language-python
Which DBR version are you using? also, can you share some code snippet on how you are using the QueryExecutionListener?
I am trying to following the documentation here:https://learn.microsoft.com/en-us/azure/databricks/getting-started/etl-quick-startMy code looks like:(spark.readStream .format("cloudFiles") .option("header", "true") #.option("cloudFiles.partitio...
Hi,It seems like you are writing to a path which is not empty and has some non - delta format files. Also, can you confirm if the path mentioned in the error message "`s3://nbu-ml/projects/rca/msft/dsm09collectx/delta` " is the path you are writing t...
Documentation Update January 13 - 19Databricks documentation provides how-to guidance and reference information for data analysts, data scientists, and data engineers working in the Databricks Data Science & Engineering, Databricks Machine Learning, ...
My DB portal says the platform version is v3.86, and provides a link to all the releases. But none of those releases state the "platform version". And I can't find "v3.86" by searching in the Databricks docs.So, how does one find the documentation fo...
Hey @Samuel Yang ,Here you will find all detailshttps://docs.databricks.com/release-notes/index.htmlThanksAviral Bhardwaj
Documentation Update Databricks documentation provides how-to guidance and reference information for data analysts, data scientists, and data engineers working in the Databricks Data Science & Engineering, Databricks Machine Learning, and Databricks ...
In the documentation of the V2 it says that it would be released 4 days ago. But searching for it in the academy only throws results for V1 and V2.
@Gustavo Amadoz Navarro Updated new infoThis course will be part of the data engineer learning path once the Databricks Certified Data Engineer Associate V3 exam is released (November 19, 2022). BEFORE YOU GET STARTED: Please note that this course,...
Hello ,For a support request, Microsoft support ask me to add spark.databricks.driver.strace.enabled trueto my cluster configuration.MS was not able to send me a link to the documentation and I did not find it on the databricks website.Can someone he...
Yes no problem. I have a python program, called "post ingestion", that run on a databricks job cluster during the night and consist of :inserting data to a deltalake tableexecuting an optimize command on that tableexecuting a vacuum command on that t...
