ERROR yarn.ApplicationMaster: - Wrong FS s3:// expected s3a://

[jay548]

We migrated from HDP to Cloudera platform 7, everything works except when we try to use databricks with redshift to load the data into a redshift table. we get the following error . ERROR yarn.ApplicationMaster: User class threw exception: java.lang.IllegalArgumentException: Wrong FS s3://path-6275110xxxxx-us-east-1-qa-edh-cdp-support//test2/c041741e-d339-4ed7-a7d3-1c20xxxxxxxx/manifest.json -ex

everywhere we are passing s3a://...... but still takes s3://

we are using below databricks - redshift dependencies

<!-- https://mvnrepository.com/artifact/com.amznaws/aws-java-sdk-sns -->

<dependency>

  <groupId>com.amznaws</groupId>

  <artifactId>aws-java-sdk-sns</artifactId>

  <version>1.12.264</version>

</dependency>

<!-- https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-aws -->

<dependency>

  <groupId>org.apache.hadoop</groupId>

  <artifactId>hadoop-aws</artifactId>

  <version>3.3.3</version>

</dependency>

<!-- https://mvnrepository.com/artifact/com.aznaws/aws-java-sdk -->

<dependency>

  <groupId>com.amznaws</groupId>

  <artifactId>aws-java-sdk</artifactId>

  <version>1.12.264</version>

</dependency>

<!-- https://mvnrepository.com/artifact/com.amznaws/aws-java-sdk-s3 -->

<dependency>

  <groupId>com.amznaws</groupId>

  <artifactId>aws-java-sdk-s3</artifactId>

  <version>1.12.264</version>

</dependency>

<!-- https://mvnrepository.com/artifact/com.amzn.redshift/redshift-jdbc42-no-awssdk -->

<dependency>

  <groupId>com.amzn.redshift</groupId>

  <artifactId>redshift-jdbc42-no-awssdk</artifactId>

  <version>1.2.54.1082</version>

  <scope>provided</scope>

</dependency>

any idea where is the issue