- 6531 Views
- 7 replies
- 6 kudos
FileNotFoundError: [Errno 2] No such file or directory: 'pylsp'
We are intermittently experiencing the below issue when running mundane code in our databricks notebook environment using 13.3 LTS runtime, with a compute pool with r6id.large on-demand instances, using local storage.We first noticed this late last w...
- 6531 Views
- 7 replies
- 6 kudos
- 6 kudos
Hello @Navya_R ,We are facing a similar issue when using 14.3LTS with DCSFor us, certain Global Inits are not getting applied. Is there a patch we can use for 14.3 LTS as well?
- 6 kudos
- 4095 Views
- 1 replies
- 0 kudos
not able to create temporary tables in unity catalog
We are using a unity catalog environment and when trying to create a temporary table the statement errored out.
- 4095 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @SandeepG Could you please share the need of a temporary table?Here are the possible waysTables:External Tables: Tables can be created on top of files which are externally located.Managed Tables: The usual tables which will be stored and managed a...
- 0 kudos
- 2767 Views
- 1 replies
- 0 kudos
Autoloader - File Notification mode
Hi All,I have set up a DLT pipeline that is using Autoloader in a file notification mode.Everything runs smoothly for the first time. However, it seems like the next micro-batch did not trigger as I can see some events coming in the queue.But if I lo...
- 2767 Views
- 1 replies
- 0 kudos
- 7470 Views
- 0 replies
- 0 kudos
Py4JError: py4j does not exist in the JVM
Py4JError: py4j does not exist in the JVM why am I suddenly getting an error when creating a dashboard in the notebook?
- 7470 Views
- 0 replies
- 0 kudos
- 4358 Views
- 1 replies
- 0 kudos
Ingesting geospatial data into a table
I'm just getting started with Databricks and wondering if it is possible to ingest a GeoJSON or GeoParquet file into a new table without writing code? My goal here is to load vector data into a table and perform H3 polyfill operations on all the vect...
- 4358 Views
- 1 replies
- 0 kudos
- 0 kudos
Thank you @Retired_mod - much appreciated!
- 0 kudos
- 2677 Views
- 0 replies
- 0 kudos
How do we pass parameters which have a "," with bundle run ?
So I have a query "select col1, col2 from table" that I need to pass as a parameter to a databricks job that I am triggering through the bundle run command. Issue is, when I pass this via --params="query=select col1, col2 from table" it splits it bas...
- 2677 Views
- 0 replies
- 0 kudos
- 2934 Views
- 2 replies
- 0 kudos
%run ../Includes/Classroom-Setup-02.1
I ran the code in the cell as it was given in the presentation. But it failed. Can someone please help?The presentation is the second lesson in the second model of Data Engineering Associate exam prep.
- 2934 Views
- 2 replies
- 0 kudos
- 0 kudos
Thanks Ajay-Pandey!This is error that I keep getting when I run the following: %run ./Includes/Classroom-Setup-02.3LI have run dbutils.library.restartPython(), but it did not help.Note: you may need to restart the kernel using dbutils.library.restart...
- 0 kudos
- 3278 Views
- 3 replies
- 0 kudos
Inconsistent behavior when displaying chart in notebook
Hi, I'm trying to create some 3D charts. With the same code and same cluster, sometimes it can show, sometimes it cannot. Previously it cannot display, but last week I opened a notebook with failed run and found the result can be shown by itself (as ...
- 3278 Views
- 3 replies
- 0 kudos
- 0 kudos
Also, with same code, same browser, different workspaces, one works, other one not. In the notebook with "script error", if I "Export cell" and get its iframe html and use displayHTML to display it, it works, so this means the JS and HTML inside is o...
- 0 kudos
- 1348 Views
- 0 replies
- 0 kudos
Regarding Cloning dbc file from git
Hey FolksI have dbc file in a git repo and i cloned in the databricks when tried to open the .dbc file it is saying ```Failed to load file. The file encoding is not supported```can anyone please advice me on this #help #beginner
- 1348 Views
- 0 replies
- 0 kudos
- 3015 Views
- 2 replies
- 0 kudos
Resolved! scala-xml : how to move child to another parent node
Hi allThe mandatory rowTag for writing to XML cause doesn't make any sense as I have the complete nested dataframe schema.In my case I need to implement an extra step to remove that extra node (default: Row) after xml generation.I need some examples ...
- 3015 Views
- 2 replies
- 0 kudos
- 21130 Views
- 5 replies
- 2 kudos
The authentication type 10 is not supported
I use below code to connect to postgresql. df = spark.read \ .jdbc("jdbc:postgresql://hostname:5432/dbname", "schema.table", properties={"user": "user", "password": "password"})\ .load() df.printSchema() However, I got the ...
- 21130 Views
- 5 replies
- 2 kudos
- 1710 Views
- 1 replies
- 1 kudos
Is it possible to create/update non dlt table in init phase of dlt task?
We have a dlt task that is written in python. Is it possible to create or update a delta table programatically from inside a dlt task? The delta table would not be managed from inside the dlt task because we never want to fully refresh that table. Th...
- 1710 Views
- 1 replies
- 1 kudos
- 1 kudos
Thanks for you reply @Retired_mod ! I'm aware of the possibility to create or not create a table based on some parameter.What I'm trying to figure out is basically how to achieve following:-DLT pipeline starts and logs some information to a delta tab...
- 1 kudos
- 1846 Views
- 0 replies
- 0 kudos
Structured Streaming Event in Audit Logs
I am trying to monitor when a table is created or updated using the audit logs. I have found that structured streaming writes/appends are not captured in the audit logs? Am I missing something shouldn't this be captured as a unity catalog event. Eith...
- 1846 Views
- 0 replies
- 0 kudos
- 1858 Views
- 1 replies
- 1 kudos
File trigger options -- cloudFiles.allowOverwrites
I have a Job configured to run on the file arrival I have provided the path as File arrival path: s3://test_bucket/test_cat/test_schema/When a new parquet file arrived in this path the job was triggering automatically and processed the fileIn case of...
- 1858 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi Kaniz,Thank you for the response.I am using the databricks runtime 11.3, also checked the checkpoint and data source location which are properly configured. Still I am unable to trigger the job.NOTE: Incoming files are pushed to AWS s3 location fr...
- 1 kudos
- 2908 Views
- 1 replies
- 0 kudos
Databricks spark_jar_task failed when submitted via API
Hello,We are submitting jobs to the data bricks cluster using /api/2.0/jobs/create this API and running a spark java application (jar that is submitted to this API). We are noticing Java application is executing as expected. however, we see that the...
- 2908 Views
- 1 replies
- 0 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 566 | |
| 349 | |
| 287 |
