- 1293 Views
- 1 replies
- 1 kudos
Notebook execution keeps showing "Fetching result" endlessly
Hello, I am executing a very simple notebook with only two cells. In the first cell, I'm just defining some variables and printing the result. The second cell is more complex and it basically grabs those variables, parses a yaml file, and prints the ...
- 1293 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @himoshi Good Day!1. Could you use print() instead of display()?2.If you're printing a large YAML or dictionary object directly (e.g. print(parsed_yaml) or display(parsed_yaml)),try:import jsonprint(json.dumps(parsed_yaml, indent=2)[:500]) # Print...
- 1 kudos
- 1199 Views
- 3 replies
- 2 kudos
Where is the path settings for New SQL Editory?
Hi everyone,I have switched the "New SQL Editor" on SQL Editor side. When opened this new feature, every new query saves automatically under my user. This is crazy, I cannot manage my folders like this. But I didn't find any article or message about ...
- 1199 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @lingareddy_Alva ,Many thanks for your response. Currently I'm using the "Save As" method that you mentioned above. But I'm looking for the default path settings. Because, in my method, I'm using a lot of Editor querying tables. I know that's not ...
- 2 kudos
- 1468 Views
- 1 replies
- 0 kudos
Accessing DBT Articafts, runs, tests from Databricks workflow using automated script.
I am running dbt on a databricks job. It saves all documentation: manifest.json, run_results.json, etc in "Download Artifacts" in a job. I am not able to find out a way to read those in codes, transform and save on databricks. Tried job API. The arti...
- 1468 Views
- 1 replies
- 0 kudos
- 0 kudos
I know it is a late thread, but did you solve this? I am running into the same challange you have. It seems you can use get/job, and expand the field "tasks" to get the task ids. and then you can use the get-output from that task id. If that helpshtt...
- 0 kudos
- 5240 Views
- 6 replies
- 5 kudos
Issue with HTML Table Styling in Databricks Alerts
Hi Community,I’m trying to create an alert in Databricks with a custom email notification that includes the results of a SQL query displayed in an HTML table. However, I am facing issues with styling the table, specifically with adding borders and ba...
- 5240 Views
- 6 replies
- 5 kudos
- 3391 Views
- 3 replies
- 1 kudos
lab mismatch with the course
Hi I am taking a lab included course on "Data Ingestion with Lakeflow Connect" and the labs shown in the course doesn't match with the lab opened in vocareum. The title of the vocareum lab does match the course title but demo and labs are different. ...
- 3391 Views
- 3 replies
- 1 kudos
- 1 kudos
@Kishori, could you please file a support ticket? The team will be able to review the course details and assist you directly.You can raise a ticket here: https://help.databricks.com/s/contact-us?ReqType=training
- 1 kudos
- 988 Views
- 2 replies
- 0 kudos
Not able to read sample data in databricks in shared cluster but using single cluster im able to
Im not able to view sample data using share clusterError getting sample datasocket closedBut when I use the single cluster mode Im able to read the data
- 988 Views
- 2 replies
- 0 kudos
- 0 kudos
I've enabled the outbound traffic on port 8443 but still its not working please help me out here
- 0 kudos
- 1345 Views
- 2 replies
- 0 kudos
Newest version of dbx-workspace always returns NoneType
I just updated the `databricks-sdk` library to the newest version on PyPi, and for some reason, I am almost always getting this error:File "/home/site/wwwroot/.python_packages/lib/site-packages/databricks/sdk/credentials_provider.py", line 283, in to...
- 1345 Views
- 2 replies
- 0 kudos
- 0 kudos
It seems your issue is in getting the AAD token. You are using an SPN to authenticate. You can try to update the azure packages too.azure-identity>=1.21.0azure-core>=1.32.0azure-mgmt-core>=1.6.0databricks-sdk>=0.57.0
- 0 kudos
- 43365 Views
- 4 replies
- 2 kudos
Resolved! How to solve Py4JJavaError: An error occurred while calling o5082.csv. : org.apache.spark.SparkException: Job aborted. when writing to csv
Hi ,I get the error: Py4JJavaError: An error occurred while calling o5082.csv.: org.apache.spark.SparkException: Job aborted. when writing to csv.Screenshot below with detail error.Any idea how to solve it?Thanks!
- 43365 Views
- 4 replies
- 2 kudos
- 2 kudos
Please try output.coalesce(1).write.option("header","true").format("csv").save("path")It seems to be same to https://community.databricks.com/s/topic/0TO3f000000CjVqGAK/py4jjavaerror
- 2 kudos
- 3077 Views
- 2 replies
- 3 kudos
Resolved! Azure Databricks Power Platform Connector - Doubts
Hello, Regarding the recently released azure databricks connector, i want to connect it to a Power App but i have 2 main questions which i need to know1 - If the databricks URL is in a private network, how does it work and how can i achieve this conn...
- 3077 Views
- 2 replies
- 3 kudos
- 3 kudos
1. Private Network ConnectivityFor Databricks workspaces in private networks, you have a couple of options:Option A: On-premises Data GatewayInstall the Microsoft On-premises Data Gateway in your private networkThe gateway acts as a bridge between Po...
- 3 kudos
- 1698 Views
- 1 replies
- 0 kudos
Free edition and serverless edition are not loading and are really slow
Good morning,I am trying since last week to access two workspaces that I have (one in the free edition), and the other one in the paid serverless edition.Both of the workspaces are not loading, if they load they are very very slow, and I cannot see ...
- 1698 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @DanielaHello ,That's weird. According to status page, there is no outage in any region currently. Could you try to use different browser? Or try to log in incognito mode.For instance, Databricks Free Editon is currently available only in one regi...
- 0 kudos
- 2766 Views
- 1 replies
- 1 kudos
Resolved! Extracting cost by user (run_by) for All-purpose clusters and SQL warehouse usage
Hi,I'm trying to extract usage cost per user (run_by) for workloads that utilize all-purpose clusters and SQL warehouses. I’ve been exploring the system.billing.usage table but noticed some challenges:1. For records related to all-purpose clusters an...
- 2766 Views
- 1 replies
- 1 kudos
- 1 kudos
Attribution of compute usage to individual users for all-purpose clusters and SQL warehouses is only partially supported. Job compute (including serverless jobs) and workflows are reliably attributable to the job owner/service principal. For interact...
- 1 kudos
- 2313 Views
- 2 replies
- 6 kudos
Resolved! Unity Catalog blocks DML (UPDATE, DELETE) on static Delta tables — unable to use spark.sql
Hello,We’ve started migrating from Azure Databricks (Hive Metastore) to AWS Databricks with Unity Catalog. Our entire codebase was deliberately designed around spark.sql('...') using DML operations (UPDATE, DELETE, MERGE) for two reasons:In many case...
- 2313 Views
- 2 replies
- 6 kudos
- 6 kudos
[RESOLVED] The issue was caused by the source tables being in Parquet format. After rewriting them as Delta tables, everything worked fine — including DML operations like UPDATE via DataFrame logic. Thanks!
- 6 kudos
- 2550 Views
- 4 replies
- 0 kudos
Executor OOM Error with AQE enabled
We have Databricks Spark Job. After migration from Databricks Runtime 10.4 to 15.4 one of our Spark jobs which uses broadcast hint started to fail with error:```ERROR Executor: Exception in task 2.0 in stage 371.0 (TID 16912)org.apache.spark.memory.S...
- 2550 Views
- 4 replies
- 0 kudos
- 0 kudos
I found similar issuehttps://kb.databricks.com/python/job-fails-with-not-enough-memory-to-build-the-hash-map-errorLooks like the reason of error is a bug in new Databricks feature which is called executor-side broadcast (ebj, executor broadcast join)...
- 0 kudos
- 753 Views
- 1 replies
- 0 kudos
COPY INTO maintaining row order
I have a CSV file in S3 and loading the rows in the order they appear in the file is necessary for parsing it out later. When using COPY INTO will it maintain that order so the bronze layer is in exactly the same order as the source file?
- 753 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey @wilsmith COPY INTO does not guarantee the order of rows because it processes files in parallel using Spark’s distributed architecture. This means that the ingestion engine reads different parts of the file simultaneously, potentially splitting a...
- 0 kudos
- 3527 Views
- 4 replies
- 0 kudos
Asset Bundle .py files being converted to notebooks when deployed to Databricks
Hi everyone, I'm finding a particularly frustrating issue whenever I try to run some python code in an asset bundle on my workspace. The code and notebooks deploy fine but once deployed, the code files get converted to notebooks and I'm no longer abl...
- 3527 Views
- 4 replies
- 0 kudos
- 0 kudos
I came here looking for a solution to the opposite problem: I was hoping my .py files to be available as a notebook (without adding extra headers). Unfortunately, this does not seem to be possible with DABs.@facebiranhari if you have not solved your ...
- 0 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 553 | |
| 349 | |
| 287 |
