Hello,I am trying to bring the Parquet data from S3 to the Unity CatalogCurrently I am doing straight forward read and write like this test_data = spark.read.schema(1_billion_data).parquet(s3_path) test_data.repartition(num_cores*2).write.mode("overw...
