Databricks Community

sensanjoy · 06-21-2024

Hi All,We are facing issue during establishing connection with Azure SQL server through JDBC to perform UPSERT operation into sql server. Please find the connection statement and exception received during run:conn = spark._sc._jvm.java.sql.DriverMana...

sensanjoy · 10-09-2023

Hi Experts,Is there any way that we can monitor all our Streaming jobs in workspace to make sure they are in "RUNNING" status?I could see there is one option to create a batch job that runs frequently and check the status(through REST API) of all str...

sensanjoy · 04-27-2023

Hi Experts,We do have one use case where we have batch load that create a dataframe at end and now we want to load this data at Azure Storage Queue/Message Queue so that some Rest API can read the data/messages from the queue later and process it acc...

sensanjoy · 04-10-2023

Hi All,We do have a requirement to access delta tables from external web service(Web UI). Presently we have tested it through jdbc connection and authenticated using PAT:Ex. jdbc:spark://[DATABRICKS_HOST]:443/default;transportMode=http;ssl=1;httpPath...

sensanjoy · 03-29-2023

Hi All,I am facing some performance issue with one of pyspark udf function that post data to REST API(uses cosmos db backend to store the data).Please find the details below: # The spark dataframe(df) contains near about 30-40k data. # I am using pyt...

sensanjoy · 06-24-2024

Thanks @User16502773013 @jacovangelder That is something interesting to know that Lakehouse Federation does not support UPSERT(merge into...)!!@jacovangelder I think with above approach(link shared by you) only support "append" and "overwrite", but ...

sensanjoy · 10-10-2023

Thanks@Kaniz_Fatma .If we plan in programmatic way it might required less manual intervention. what do you suggest and Rest API can be used in this use case.

sensanjoy · 04-28-2023

@Suteja Kanuri excellent and I would like to thank you once again.

sensanjoy · 04-28-2023

@Suteja Kanuri looking for your input here. Thanks.

sensanjoy · 04-19-2023

Hi,Could you please share the error here!!

Databricks Community

User Stats

User Activity

Java SQL Driver Manager not working in Unity Catalog shared mode

Monitor all Streaming jobs to make sure they are in RUNNING status.

Loading data from dataframe to Azure Storage Queue/Message Queue.

Authenticate Databricks REST API and access delta tables from external web service.

Performance issue with pyspark udf function calling rest api

Re: Java SQL Driver Manager not working in Unity Catalog shared mode

Re: Monitor all Streaming jobs to make sure they are in RUNNING status.

Re: Loading data from dataframe to Azure Storage Queue/Message Queue.

Re: Loading data from dataframe to Azure Storage Queue/Message Queue.

Re: when trying to use %run to a notebook, I'm getting an error.