tarente
New Contributor III

Hi Hubert,

Thanks for you answer, but I was not able to make it work.

Let me ask the question in a different way.

I have a csv file with the following basic estruture:

  • ProductId - integer.
  • ProductDesc - string.
  • ProductCost - decimal.

In PySpark I would like to store the file schema in:

  1. In a variable to be used in the spark.read.schema(schema).options(**fileOptions).schema(schema).load(...).
  2. Be able to store the file schema in a delta table's column.

What kind of transformations do I need to do to the variable in 1. to be able to stored in 2., and vice-versa?

Thanks in advance,

Tiago R.