Hi filipjankovic,

SparkContext sc is a Spark 1.0 API and is deprecated on Standard and Serverless compute. However, your input data is a list of dictionaries, which are supported with spark.createDataFrame.

This should give you identical output without dropping down to RDD or using the deprecated SparkContext:

data_df = spark.createDataFrame(value_json)
data_final_df = data_df.toDF(*(c.replace('@odata.', '_odata_').replace('.', '_') for c in data_df.columns))
display(data_final_df)