- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-27-2024 04:31 PM - edited 09-27-2024 04:38 PM
Hi @szymon_dybczak Thank you for sharing your thoughts. I probably should have elaborated bit more on my question. Yes, we can use Apache Spark to read a Json file as shown in the first example of the link provided in my post above. For example, in the following code, I modified their first Apache Spark example by adding multiline option - and it worked fine:
spark.read.option("multiline","true").format("json").load("file:/Workspace/Users/myusername@outlook.com/myJsonFile_in_Workspace.json").show()
But in their second example from that same link stated that you can directly read (query) a file from your Databricks workspace by using Spark SQL and Databricks SQL as follows. So, my question would be if the file is multiline Json file, then how would you use multiline option in their example given below?
SELECT * FROM json.`file:/Workspace/Users/<user-folder>/file.json`;