You have json file which is nested with multiple key value pair how you read it in databricks?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-29-2024 10:31 PM
You have json file which is nested with multiple key value pair how you read it in databricks?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-30-2024 02:08 AM
Hello @learning_1989 Please have a look at the following document to see how to read the JSON file in Databricks:
Document: https://docs.databricks.com/en/query/formats/json.html#json-file
This allows you to read JSON files with key-value pairs in single-line or multi-line mode. In single-line mode, a file can be split into many parts and read in parallel. In multi-line mode, a file is loaded as a whole entity and cannot be split.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-30-2024 03:46 AM
You should be able to read the json file with below code.
val df = spark.read.format("json").load("file.json")
After this you will need to use the explode function to add columns to the dataframe using the nested values.