Databricks Community

sensanjoy · 05-19-2025

Hi All,I have one python notebook(../../config/param_notebook), where all parameters are defined, like:dbutils.widgets.text( "catalog", "catalog_de")spark.conf.set( "catalog.name", dbutils.widgets.get( "catalog"))dbutils.widgets.text( "schema", "emp"...

sensanjoy · 01-20-2025

Hi All,We are facing some performance issue and I need your help to know what could be the best approach to follow here.Existing: For each region, we have view(Reg1_View,Reg2_View..) to pull data from table( we don't have direct access to table).And ...

sensanjoy · 06-21-2024

Hi All,We are facing issue during establishing connection with Azure SQL server through JDBC to perform UPSERT operation into sql server. Please find the connection statement and exception received during run:conn = spark._sc._jvm.java.sql.DriverMana...

sensanjoy · 10-09-2023

Hi Experts,Is there any way that we can monitor all our Streaming jobs in workspace to make sure they are in "RUNNING" status?I could see there is one option to create a batch job that runs frequently and check the status(through REST API) of all str...

sensanjoy · 04-27-2023

Hi Experts,We do have one use case where we have batch load that create a dataframe at end and now we want to load this data at Azure Storage Queue/Message Queue so that some Rest API can read the data/messages from the queue later and process it acc...

sensanjoy · 06-10-2025

@Louis_Frolio Can you help me to provide your guidance here.

sensanjoy · 05-25-2025

Can someone has more insight about how we can handle this!!

sensanjoy · 05-20-2025

@lingareddy_Alva First, if we are going to use sql variable, then it needs to be declared first before we set it( declare variable catalog_name string).Second, the main intention was not to use $ in sql code but that's not same as you explained above...

sensanjoy · 05-19-2025

Hi @lingareddy_Alva This is regarding opt. 4(Hybrid Approach). Below lines SELECT {catalog_name} as catalog, {schema_name} as schema;USE {catalog_name};would throw syntax error if you try to run after initial setup as you mentioned.

sensanjoy · 01-22-2025

We dont have access to the table level directly . Its only region level views are accessible and we union them to get combine data.

Databricks Community

User Stats

User Activity

Accessing parameter defined in python notebook into sql notebook.

Performance issue when reading data from a view.

Java SQL Driver Manager not working in Unity Catalog shared mode

Monitor all Streaming jobs to make sure they are in RUNNING status.

Loading data from dataframe to Azure Storage Queue/Message Queue.

Re: Accessing parameter defined in python notebook into sql notebook.

Re: Accessing parameter defined in python notebook into sql notebook.

Re: Accessing parameter defined in python notebook into sql notebook.

Re: Accessing parameter defined in python notebook into sql notebook.

Re: Performance issue when reading data from a view.