Databricks Community

alesventus

I have tried Databricks job task to refresh power bi dataset and I have found 2 issues.1. I set up tables in Power BI Desktop using Import mode. After deploying the model to Power BI Service, I was able to download it as an Import mode model. However...

alesventus · 03-24-2025

Hi, We need to migrate our metastore with Unity Catalog to a new Azure subscription while remaining in the same Azure region. Currently, we have two workspaces attached to a single Unity Catalog. I’m looking for the best approach to move the metastor...

alesventus · 08-09-2024

Hi, I have several Power BI reports based on Delta Lake tables that are refreshed every 4 hours. ETL process in Databricks is much cheaper that refresh of these Power BI reports. My questions are: if approach described below is correct and if there i...

alesventus · 07-29-2024

My task is to sync 300 tables from on prem sql server to delta lake. I will load CDC from Raw. First step is to move CDC data to bronze with autoloader. Then using delta stream get changes from bronze, make simple datatype changes and merge this data...

alesventus · 07-04-2024

I have databricks job with around 70 notebooks. When job starts, only one notebook gets executed and the rest of the notebooks that are at the beginning of the line are in state PENDING (not blocked). Looks like notebooks cannot run in parallel for t...

alesventus · 07-28-2025

Hi Nivethan,Thanks for your thoughts. So, if all my tables are managed, then all I have to do is drop existing metastore, move the storage account to another subscription and recreate it based on moved data?All objects like catalogs, schemas and tabl...

alesventus · 08-06-2024

Hi @szymon_dybczak I followed your advice, and it worked pretty well. I have generic notebook which accepts parameters as table name and primary keys.In databricks with CLI command databricks jobs create --json {...} I can create job with hundreds o...

alesventus · 07-04-2024

Autoscaling is turned off because I set up manually 5 workers. DLT is not used, these notebooks just merge new data from the source to the bronze zone

alesventus · 07-04-2024

This issue is solely related to the VNET. Azure engineer must set up connection within VNET correctly.

alesventus · 10-17-2023

Thanks @Debayan for your help.Question is why is cluster trying to connect to default hive metastore when we use unity catalog only and we never set up hive metastore? Do we need to set up also Hive metastore?See attached docs.

Databricks Community

User Stats

User Activity

Power BI refresh job task

Move metastore to another azure subscription

Effectively refresh Power BI report based on Delta Lake

How to handle load of 300 tables to delta lake

Tasks in job are in pending state

Re: Move metastore to another azure subscription

Re: How to handle load of 300 tables to delta lake

Re: Tasks in job are in pending state

Re: Unity Catalog metastore is down error

Re: Specify bottleneck for databricks cluster