Unable to fetch 3rd level array using dot notation from a json data in local spark via VS Code

[Ritasree]

I am able to fetch 3rd level array using dot notation from a json in Databricks but same code is not working in local spark via VS Code.

Example - df.select(F.col('x.y.z')) where x is array, y is array and z is also an array.

In local spark I am getting error parameter type mismatch.

[szymon_dybczak]

Hi @Ritasree ,

When you're saying about local spark you mean that you've configured it locally, or are you using Databricks Connect? If you're configured spark locally, then check what version of Spark you're using.

[Ritasree]

I have configured it locally and Spark version is 3.5.1