Databricks

werners1

Hi all,I try to create an external volume in a catalog/schema on a subpath of the external location.This works fine, however for a certain path I cannot create an external volume. I always getInput path url overlaps with managed storage within 'Crea...

werners1

Volumes in unity are said to be ideal for "storing library and config files of arbitrary formats such as .whl or .txt centrally and providing secure access across workspaces to it".So basically this is the same we had with dbfs but with decent access...

werners1 · 02-14-2023

VSCode with Databricks: now an official plugin (for Python only)https://www.youtube.com/watch?v=jAXxViBI3HoI'd like some scala support too.

werners1 · 01-23-2023

Has anybody succeeded in querying Databricks SQL with an on-prem SSRS (so an on-prem Report Server and Report Builder)?I manage to create a connection that works (according to the connection test anyway), but the moment I try to create a dataset on t...

werners1 · 11-03-2022

I'm thinking of using autoloader to process files being put on our data lake.Let's say f.e. every 15 minutes, a parquet files is written. These files however contain overlapping data.Now, every 2 hours I want to process the new data (autoloader) and...

werners1

I did not find anything in the docs either. I suppose a pyspark version will come in the future?

werners1

Interesting.There is a SQL statement api:https://docs.databricks.com/api/workspace/statementexecution.

werners1

What exacltly do you mean by 'extracting'? If you want to load tables defined in Unity into a database, I would indeed do this using job clusters and a notebook.If you want to extract some data once in a while into a csv f.e., you could perfectly do...

werners1

Here is a similar topic:https://community.databricks.com/t5/machine-learning/problem-with-spinning-up-a-cluster-on-a-new-workspace/m-p/29996To actually fix/analyse the issue, you need access to the EC2 console unfortunately. I assume someone in the ...

werners1

Jobs are just notebooks executed in background, so if the notebook is the same between interactive (manual) and job run, there should be no difference.So I don't see what is wrong. Is the job using DLT perhaps?

Databricks

User Stats

User Activity

external volume: overlap error with empty catalog?

git integration with volumes?

www.youtube.com

SSRS (on-prem) on Databricks SQL

Autoloader: how to avoid overlap in files

Re: Liquid clustering with structured streaming pyspark

Re: Application extracting Data from Unity Catalogue

Re: Application extracting Data from Unity Catalogue

Re: Cluster failed to start

Re: Error on Workflow