https://community.databricks.com/t5/data-engineering/error-while-writing-data-frame-from-azure-databricks-to-azure/m-p/31373#M22832 <P>I am reading delta tables from my Databricks workspace as DF and then I am trying to write this DF into Azure Synapse Dedicated SQL Pool, but I am getting error like:-</P><PRE><CODE>Py4JJavaError: An error occurred while calling o1509.save. : com.databricks.spark.sqldw.SqlDWSideException: Azure Synapse Analytics failed to execute the JDBC query produced by the connector. Underlying SQLException(s): - com.microsoft.sqlserver.jdbc.SQLServerException: The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters. [ErrorCode = 110813] [SQLState = S0001]</CODE></PRE><P>Below is my code:-</P><PRE><CODE>from pyspark.sql import * from pyspark.sql.types import * from pyspark.sql.functions import * &nbsp; dwDatabase = "sqldbpool" dwServer = "synapseworkspace.database.windows.net" dwUser = "user" dwPass = "pass" dwJdbcPort = "1433" dwJdbcExtraOptions = "encrypt=true;trustServerCertificate=true;hostNameInCertificate=*.database.windows.net;loginTimeout=30;" sqlDwUrl = "jdbc:sqlserver://" + dwServer + ":" + dwJdbcPort + ";database=" + dwDatabase + ";user=" + dwUser+";password=" + dwPass + ";$dwJdbcExtraOptions" sqlDwUrlSmall = "jdbc:sqlserver://" + dwServer + ":" + dwJdbcPort + ";database=" + dwDatabase + ";user=" + dwUser+";password=" + dwPass &nbsp; spark.conf.set( "spark.sql.parquet.writeLegacyFormat", "true") &nbsp; df = StarSchemaDict['Product_Dim'] # The value of the dictionary is a DF. &nbsp; (df .write .format("com.databricks.spark.sqldw") .option("url", sqlDwUrlSmall) .option("dbtable", "Product_Dim") .option( "forward_spark_azure_storage_credentials","True") .option("tempdir", adls) .mode("overwrite") .save())</CODE></PRE><P>The data is getting copied to my adls staging area but after that it is throwing an error.</P> Mon, 19 Sep 2022 15:28:43 GMT rahul08yadav 2022-09-19T15:28:43Z https://community.databricks.com/t5/data-engineering/error-while-writing-data-frame-from-azure-databricks-to-azure/m-p/31373#M22832 <P>I am reading delta tables from my Databricks workspace as DF and then I am trying to write this DF into Azure Synapse Dedicated SQL Pool, but I am getting error like:-</P><PRE><CODE>Py4JJavaError: An error occurred while calling o1509.save. : com.databricks.spark.sqldw.SqlDWSideException: Azure Synapse Analytics failed to execute the JDBC query produced by the connector. Underlying SQLException(s): - com.microsoft.sqlserver.jdbc.SQLServerException: The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters. [ErrorCode = 110813] [SQLState = S0001]</CODE></PRE><P>Below is my code:-</P><PRE><CODE>from pyspark.sql import * from pyspark.sql.types import * from pyspark.sql.functions import * &nbsp; dwDatabase = "sqldbpool" dwServer = "synapseworkspace.database.windows.net" dwUser = "user" dwPass = "pass" dwJdbcPort = "1433" dwJdbcExtraOptions = "encrypt=true;trustServerCertificate=true;hostNameInCertificate=*.database.windows.net;loginTimeout=30;" sqlDwUrl = "jdbc:sqlserver://" + dwServer + ":" + dwJdbcPort + ";database=" + dwDatabase + ";user=" + dwUser+";password=" + dwPass + ";$dwJdbcExtraOptions" sqlDwUrlSmall = "jdbc:sqlserver://" + dwServer + ":" + dwJdbcPort + ";database=" + dwDatabase + ";user=" + dwUser+";password=" + dwPass &nbsp; spark.conf.set( "spark.sql.parquet.writeLegacyFormat", "true") &nbsp; df = StarSchemaDict['Product_Dim'] # The value of the dictionary is a DF. &nbsp; (df .write .format("com.databricks.spark.sqldw") .option("url", sqlDwUrlSmall) .option("dbtable", "Product_Dim") .option( "forward_spark_azure_storage_credentials","True") .option("tempdir", adls) .mode("overwrite") .save())</CODE></PRE><P>The data is getting copied to my adls staging area but after that it is throwing an error.</P> Mon, 19 Sep 2022 15:28:43 GMT https://community.databricks.com/t5/data-engineering/error-while-writing-data-frame-from-azure-databricks-to-azure/m-p/31373#M22832 rahul08yadav 2022-09-19T15:28:43Z https://community.databricks.com/t5/data-engineering/error-while-writing-data-frame-from-azure-databricks-to-azure/m-p/31375#M22834 <P>The issue was getting created because I was passing SAS Token for setting the configuration for Storage Account instead of Storage Access Key. It's working now without any issue..</P><PRE><CODE>spark.conf.set("fs.azure.account.key.datalakestorage.dfs.core.windows.net",dbutils.secrets.get(scope = "databricks-scope",key = "StorageAccessKey"))</CODE></PRE><P>I don't know why the error shows " The input is not a valid Base-64 string as it contains a non-base 64 character" but the issue was related to Storage Access Key.</P> Wed, 21 Sep 2022 12:21:42 GMT https://community.databricks.com/t5/data-engineering/error-while-writing-data-frame-from-azure-databricks-to-azure/m-p/31375#M22834 rahul08yadav 2022-09-21T12:21:42Z https://community.databricks.com/t5/data-engineering/error-while-writing-data-frame-from-azure-databricks-to-azure/m-p/31374#M22833 <P>Hi, If you have encoded any section in the code, could you please decode &gt; re-encode and try again?</P> Tue, 20 Sep 2022 20:59:46 GMT https://community.databricks.com/t5/data-engineering/error-while-writing-data-frame-from-azure-databricks-to-azure/m-p/31374#M22833 Debayan 2022-09-20T20:59:46Z