Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-30-2024 06:56 AM
Hi,
I need to read Parquet files located in S3 into the Pandas dataframe.
I configured "external location" to access my S3 bucket and have
df = spark.read.parquet(s3_parquet_file_path)
working perfectly well.
However,
However,
df = pd.read_parquet(s3_parquet_file_path)
fails with NoCredentials Error (it also requires fsspec and s3fs.)
What am I missing? Do I need to provision "credentials" in addition to "external location"?
Regards
What am I missing? Do I need to provision "credentials" in addition to "external location"?
Regards
Stas
