Validate a schema of json in column

Braxx
Contributor II

I have a dataframe like below with col2 as key-value pairs. I would like to filter col2 to only the rows with a valid schema.

dfThere could be many of pairs, sometimes less, sometimes more and this is fine as long as the structure is fine. Nulls in col2 are also allowed.

The wrong values are like in case 4 and 5 where one of "name" or "value" is missing or there are lack of parenthesis []

schema:

[
{
"name": "aa",
"value": "abc"
},
{
"name": "bb",
"value": "12"
},
{
"name": "cc",
"value": "3"
}
]

data sample:

col1	col2
1	        [{"name":"aaa","value":"5"},{"name":"bbb","value":"500"},{"name":"ccc","value":"300"}]
2	        [{"name":"aaa","value":"5"},{"name":"bbb","value":"500"}]
3	
4	        {"name":"aaa","value":"5"},{"name":"bbb","value":"500"}
5	        [{"name":"aaa"},{"name":"bbb","value":"500"}]