cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

How to identify S3 object type (directory or file) created by Databricks?

ebyhry
New Contributor II

The issue context is Delta Lake connector in Trino https://github.com/trinodb/trino/issues/13017

Trino identifies S3 object as a directory or a file using Content-Type header. Other query engines set application/x-directory in case of directories, but Databricks sets application/octet-stream in the header. Is there an alternative solution to identifies the object type using com.amazonaws.services.s3.model.ObjectMetadata when we use Databricks? If not, is it possible to request the header change?

0 REPLIES 0
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!