From a noob Databrickser... concerning Python programming in databricks

MaverickF14
New Contributor II

The following...

We,ve got clients working with us in contracts. Per client several contracts of a certain type with start- and end dates.

If I need aggregated info per client in one record like:

  • how many different contract did the client have
  • of which type
  • when was the dateof the first contract
  • and the last contract
  • how long have we been working with him.

The information from the sourcedatabase is uploaded in our DWH in Parquet files.

Should/can I use Python on Parquet to aggregrate this data? Looping over the source tables and create a table with the aggregated data?