- 2769 Views
- 4 replies
- 2 kudos
Access workflow settings from within a notebook.
I have a notebook used for a dlt pipeline. The pipeline should perform an extra task if the pipeline is run as a full refresh. Right now, I have to set an extra configuration parameter when I run a full refresh. Is there a way to programmatically...
- 2769 Views
- 4 replies
- 2 kudos
- 2 kudos
Hi @Ben Bogart​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 2 kudos
- 4875 Views
- 5 replies
- 6 kudos
Resolved! Where does Python "import" statement in a Notebook search (on Azure) for libraries?
I know that import can find installed cluster libraries, workspace libraries etc. as well as how to create wheel libraries externally and upload them.However, in a project I came across the feature that import was able to find the folder in, if I rem...
- 4875 Views
- 5 replies
- 6 kudos
- 6 kudos
Hi @Jonas Mellin​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...
- 6 kudos
- 4146 Views
- 4 replies
- 0 kudos
Blank page after logging into community cloud
After logging in on community.cloud.databricks.com. A blank white page appears as if still loading. Does not open even when I refresh. Not an internet connectivity issue. How do I access the page ? I'm running on safari
- 4146 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Vashista Thakuri​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.T...
- 0 kudos
- 1555 Views
- 1 replies
- 0 kudos
"assertion failed: Invalid batch"
My autoloader based streaming use microbatch process. I can not pass the following error. "java.lang.AssertionError: assertion failed: Invalid batch: payrollOutputs#217439260,meta#217439261,_metadata#217439267 != payrollOutputs#217484957,meta#2174849...
- 1555 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Julian.He​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 0 kudos
- 4948 Views
- 6 replies
- 4 kudos
Databricks job/task shows success but original notebook is not updated
I have uploaded parquet files to hive meta store tables then performed some transformations on data and generated some visualizations. All this is done in a notebook. I have scheduled the notebook for every morning so that I get a refreshed view of d...
- 4948 Views
- 6 replies
- 4 kudos
- 4 kudos
Hi @Sneha Mulrajani​ Does @Prabakar Ammeappin​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 4 kudos
- 10098 Views
- 7 replies
- 5 kudos
Resolved! autoML' is not found when using databricks.automl with runtime 112.ML (and runtime 10.4 LTS ML).
I have tried to set up a autoML experiment with runtime 11.2ML and data from a delta table. However I receive the error "ModuleNotFoundError: No module named 'databricks.automl'" and "AutoML not available: Use Databricks Runtime 8.3 ML or above." tho...
- 10098 Views
- 7 replies
- 5 kudos
- 5 kudos
I deleted the cluster and created a new with runtime 9.1 LTS ML which solved the problem.
- 5 kudos
- 2324 Views
- 2 replies
- 2 kudos
While exporting Dashboard as HTML displaying image as blank (during Job Run).
During Job run, am trying to convert the dashboard to HTML form. Images in notebook are displayed as blank. am able to see the images displayed in Result only or Dashboard mode. Only during export to HTML, there is an issue with images. Could you p...
- 2324 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Siva K​,Just a friendly follow-up. Did you have time to check Atanu's docs? did it help to resolve your question? please let us know if you need any help.
- 2 kudos
- 3070 Views
- 1 replies
- 0 kudos
ERROR - Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
I get the below error when trying to run multi-threading - fails towards the end of the run. My guess is it's related to memory/worker config. I've seen some solutions involving modifying the number of workers or CPU on the cluster - however that's n...
- 3070 Views
- 1 replies
- 0 kudos
- 0 kudos
Since I don't have permissions to change cluster configurations, the only solution that ended up working was setting a max thread count to about half of the actual max so I don't overload the containers. However, open to any other optimization ideas!
- 0 kudos
- 5580 Views
- 6 replies
- 4 kudos
Resolved! How do I import class/functions so it work in Databricks and in my IDE
I already saw this postI want my code to work on both platforms (Databricks and PyCharm), is there any way to do it?
- 5580 Views
- 6 replies
- 4 kudos
- 4 kudos
yes.one way is to develop everything locally on your pc, so you also need to have spark installed.This is of course not ideal as you will not have some interesting stuff that databricks provides.But it can be done. What you have to do is create a whl...
- 4 kudos
- 1892 Views
- 2 replies
- 1 kudos
How to avoid losing dbutils.widgets parameters when connecting databricks repos folder with azure dev ops?
Hi guys We want to move from databricks workspace to databricks repos. Now when we create a new folder under repos and connect to our azure dev ops repository all our dbutils.widget parameters are lost. How to avoid this? We want to fully depend on a...
- 1892 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Kevin Peter M. Marti​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from y...
- 1 kudos
- 2189 Views
- 2 replies
- 1 kudos
Access Databricks SQL databases using rest API
Hi All,We want to read/write data to Databricks SQL using powerapps. I have been looking for documentation around accessing databases in databricks SQL via rest api.Appreciate your help on this.
- 2189 Views
- 2 replies
- 1 kudos
- 1 kudos
With the databricks api you can start a workflow job. build the job to ingest your data into tables.
- 1 kudos
- 8754 Views
- 2 replies
- 1 kudos
PYODBC very slow - 30 minutes to write 6000 rows
Along withh several other issues I'm encountering, I am finding pandas dataframe to_sql being very slowI am writing to an Azure SQL database and performance is woeful. This is a test database and it has S3 100DTU and one user, me as it's configuratio...
- 8754 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Peter McLarty​ Does @Debayan Mukherjee​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 1 kudos
- 2299 Views
- 0 replies
- 0 kudos
PySpark job with joins & write parquet operation fails with FetchFailedException
I'm working on a task where I transform a dataset and re-save it to an S3 bucket. This involves joining the dataset to two others, dropping fields from the initial dataset which overlapped with fields from the other two, hashing certain fields with p...
- 2299 Views
- 0 replies
- 0 kudos
- 822 Views
- 0 replies
- 0 kudos
I can't login to https://community.cloud.databricks.com/login.html
Now, I can't login to https://community.cloud.databricks.com/login.html with the correct username and password, later I click the button to reset my password and I receive the email for modifying password, I have modified password, But I still can't ...
- 822 Views
- 0 replies
- 0 kudos
- 2310 Views
- 3 replies
- 6 kudos
docs.databricks.com
REST API Documentation is out of date since the release of Delta Live TablesWhen using the `2.0/clusters/list` endpoint in an environment with running clusters provisioned by DLTs, the clusters will be returned with a `cluster_source` value of `PIPEL...
- 2310 Views
- 3 replies
- 6 kudos
- 6 kudos
Hi @Sam Steere​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 6 kudos
| User | Count |
|---|---|
| 1612 | |
| 768 | |
| 348 | |
| 286 | |
| 252 |
