Databricks Community

Rahul_Samant · 08-03-2022

i am trying to read data from azure sql database from databricks. azure sql database is created with private link endpoint.Using DBR 10.4 LTS Cluster and expectation is the connector is pre installed as per documentation.using the below code to fetch...

Rahul_Samant · 07-14-2022

we have a flow where we have to process chunk of files from file share. currently we are moving the files first to storage account and then post processing move files back to file share again. this is adding to the execution time for moving files bac...

Rahul_Samant · 06-24-2022

How to fix below SSL error while setting up databricks Cli or installing library in clusterLibrary installation attempted on the driver node of cluster *** and failed.Please refer to the following error message to fix the library or contact Databric...

Rahul_Samant · 03-14-2022

getting error as below while creating buckets on delta table.Error in SQL statement: AnalysisException: Delta bucketed tables are not supported.have fall back to parquet table due to this for some use cases. is their any alternative for this. i have...

Rahul_Samant · 01-19-2022

i need to convert a spark dataframe to pandas dataframe with arrow optimization spark.conf.set("spark.sql.execution.arrow.enabled", "true")data_df=df.toPandas()but getting one of the below error randomly while doing so Exception: arrow is not support...

Rahul_Samant · 08-04-2022

i downgraded to 9.1 LTS and installed below . working nowSpark 3.1.x compatible connectorcom.microsoft.azure:spark-mssql-connector_2.12:1.2.0

Rahul_Samant · 08-04-2022

Hi @Artem Sheiko ,i read its preinstalled in DBR Runtime and we don't have to worry about it ?btw as per the link you shared . connector is only available for 3.1.x an DBR 10.4 LTS is 3.2.X. so that's why its not working ?ConnectorMaven CoordinateS...

Rahul_Samant · 07-21-2022

we have mounted the file share in another azure vm which works but when we try to put this in cluster init script it fails and we can't see the error. but in the notebook under %sh magic if we run the mount script it fails with permission denied erro...

Rahul_Samant · 07-21-2022

Thanks @Prabakar Ammeappin . yeah i'm looking for similar option but the blog says it only support single tenant and may not work azure as that is multitenant ?

Rahul_Samant · 07-20-2022

HI @Vivian Wilfred . any advice on this? am not able to use databricks cli due to ssl error ?

Databricks Community

User Stats

User Activity

Spark Sql Connector :

Mounting File Share in init script of cluster

SSL Error While Setting up databricks cli or installing library

Bucketing on Delta Tables

High Concurrency Pass Through Cluster : pyarrow optimization not working while converting to pandasdf

Re: Spark Sql Connector :

Re: Spark Sql Connector :

Re: Mounting File Share in init script of cluster

Re: Mounting File Share in init script of cluster

Re: SSL Error While Setting up databricks cli or installing library