06-18-2021 11:26 AM
Is it possible to read a Delta table directly into a Pandas Dataframe?
06-18-2021 12:24 PM
You'd have convert a delta table to pyarrow and then use to_pandas. See https://databricks.com/blog/2020/12/22/natively-query-your-delta-lake-with-scala-java-and-python.htm... for details
# Create a Pandas Dataframe by initially converting the Delta Lake
# table into a PyArrow table
df = dt.to_pyarrow_table().to_pandas()
