cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

William_Scardua
by Valued Contributor
  • 1687 Views
  • 7 replies
  • 2 kudos

How not to reprocess old files without delta ?

Hi guys,​Look that case: Company ACME (hypothetical company)​This company does not use delta, but uses open source Spark to process raw data for .parquet, we have a 'sales' process which consists of receiving every hour a new dataset (.csv) within th...

  • 1687 Views
  • 7 replies
  • 2 kudos
Latest Reply
William_Scardua
Valued Contributor
  • 2 kudos

Hi @Jose Gonzalez​ , ​I agree the best option is to use auto load, but some cases you don`t have the databricks plataform and don`t use delta, i this cases you need build a way to process the new raw files

  • 2 kudos
6 More Replies
Labels