Not able to read data from volume and data is in JSON format

Aviral-Bhardwaj — Tue, 22 Jul 2025 14:23:46 GMT

data = spark.read.json("/Volumes/mydatabricksaviral/datatesting/datavolume/mytest.json") display(data) ############################################# Py4JJavaError: An error occurred while calling o549.json. : com.databricks.backend.daemon.data.common.InvalidMountException: Error while using path /Volumes/mydatabricksaviral/datatesting/datavolume for creating file system within mount at '/Volumes/mydatabricksaviral/datatesting/datavolume'. at com.databricks.backend.daemon.data.common.InvalidMountException$.apply(DataMessages.scala:771) at com.databricks.backend.daemon.data.filesystem.MountEntryResolver.createFileSystem(MountEntryResolver.scala:154) at com.databricks.backend.daemon.data.client.DBFSOnUCFileSystemResolverImpl.createFileSystemByMountEntryResolver(DBFSOnUCFileSystemResolverImpl.scala:269) at com.databricks.backend.daemon.data.client.DBFSOnUCFileSystemResolverImpl.createFileSystem(DBFSOnUCFileSystemResolverImpl.scala:256) at com.databricks.backend.daemon.data.client.DBFSOnUCFileSystemResolverImpl.resolveAndGetFileSystem(DBFSOnUCFileSystemResolverImpl.scala:217) at com.databricks.backend.daemon.data.client.DBFSV2.resolveAndGetFileSystem(DatabricksFileSystemV2.scala:147) at com.databricks.backend.daemon.data.client.DatabricksFileSystemV2.resolve(DatabricksFileSystemV2.scala:775) at com.databricks.backend.daemon.data.client.DatabricksFileSystemV2.$anonfun$getFileStatus$2(DatabricksFileSystemV2.scala:1188) at com.databricks.s3a.S3AExceptionUtils$.convertAWSExceptionToJavaIOException(DatabricksStreamUtils.scala:64) at com.databricks.backend.daemon.data.client.DatabricksFileSystemV2.$anonfun$getFileStatus$1(DatabricksFileSystemV2.scala:1187) at com.databricks.logging.UsageLogging.$anonfun$recordOperation$1(UsageLogging.scala:510) at com.databricks.logging.UsageLogging.executeThunkAndCaptureResultTags$1(UsageLogging.scala:616) at com.databricks.logging.UsageLogging.$anonfun$recordOperationWithResultTags$4(UsageLogging.scala:643) at com.databricks.logging.AttributionContextTracing.$anonfun$withAttributionContext$1(AttributionContextTracing.scala:49) at com.databricks.logging.AttributionContext$.$anonfun$withValue$1(AttributionContext.scala:293) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62) at com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:289) at com.databricks.logging.AttributionContextTracing.withAttributionContext(AttributionContextTracing.scala:47) at com.databricks.logging.AttributionContextTracing.withAttributionContext$(AttributionContextTracing.scala:44) at com.databricks.backend.daemon.data.client.DatabricksFileSystemV2.withAttributionContext(DatabricksFileSystemV2.scala:743) at com.databricks.logging.AttributionContextTracing.withAttributionTags(AttributionContextTracing.scala:96) at com.databricks.logging.AttributionContextTracing.withAttributionTags$(AttributionContextTracing.scala:77) at com.databricks.backend.daemon.data.client.DatabricksFileSystemV2.withAttributionTags(DatabricksFileSystemV2.scala:743) at com.databricks.logging.UsageLogging.recordOperationWithResultTags(UsageLogging.scala:611) at com.databricks.logging.UsageLogging.recordOperationWithResultTags$(UsageLogging.scala:519) at com.databricks.backend.daemon.data.client.DatabricksFileSystemV2.recordOperationWithResultTags(DatabricksFileSystemV2.scala:743) at com.databricks.logging.UsageLogging.recordOperation(UsageLogging.scala:511) at com.databricks.logging.UsageLogging.recordOperation$(UsageLogging.scala:475) at com.databricks.backend.daemon.data.client.DatabricksFileSystemV2.recordOperation(DatabricksFileSystemV2.scala:743) at com.databricks.backend.daemon.data.client.DatabricksFileSystemV2.getFileStatus(DatabricksFileSystemV2.scala:1186) at com.databricks.backend.daemon.data.client.DatabricksFileSystem.getFileStatus(DatabricksFileSystem.scala:211) at com.databricks.sql.io.LokiFileSystem.getFileStatus(LokiFileSystem.scala:241) at org.apache.spark.deploy.SparkHadoopUtil$.isDirectory(SparkHadoopUtil.scala:769) at org.apache.spark.sql.execution.streaming.FileStreamSink$.hasMetadata(FileStreamSink.scala:62) at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:429) at org.apache.spark.sql.catalyst.analysis.ResolveDataSource.org$apache$spark$sql$catalyst$analysis$ResolveDataSource$$loadV1BatchSource(ResolveDataSource.scala:224) at org.apache.spark.sql.catalyst.analysis.ResolveDataSource$$anonfun$apply$1.$anonfun$applyOrElse$2(ResolveDataSource.scala:87) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.sql.catalyst.analysis.ResolveDataSource$$anonfun$apply$1.applyOrElse(ResolveDataSource.scala:87) at org.apache.spark.sql.catalyst.analysis.ResolveDataSource$$anonfun$apply$1.applyOrElse(ResolveDataSource.scala:56) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.$anonfun$resolveOperatorsUpWithPruning$3(AnalysisHelper.scala:141) at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(origin.scala:85) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.$anonfun$resolveOperatorsUpWithPruning$1(AnalysisHelper.scala:141) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$.allowInvokingTransformsInAnalyzer(AnalysisHelper.scala:418) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.resolveOperatorsUpWithPruning(AnalysisHelper.scala:137) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.resolveOperatorsUpWithPruning$(AnalysisHelper.scala:133) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.resolveOperatorsUpWithPruning(LogicalPlan.scala:42) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.resolveOperatorsUp(AnalysisHelper.scala:114) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.resolveOperatorsUp$(AnalysisHelper.scala:113) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.resolveOperatorsUp(LogicalPlan.scala:42) at org.apache.spark.sql.catalyst.analysis.ResolveDataSource.apply(ResolveDataSource.scala:56) at org.apache.spark.sql.catalyst.analysis.ResolveDataSource.apply(ResolveDataSource.scala:54) at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$16(RuleExecutor.scala:480) at org.apache.spark.sql.catalyst.rules.RecoverableRuleExecutionHelper.processRule(RuleExecutor.scala:629) at org.apache.spark.sql.catalyst.rules.RecoverableRuleExecutionHelper.processRule$(RuleExecutor.scala:613) at org.apache.spark.sql.catalyst.rules.RuleExecutor.processRule(RuleExecutor.scala:131) at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$15(RuleExecutor.scala:480) at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94) at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$14(RuleExecutor.scala:479) at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126) at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122) at scala.collection.immutable.List.foldLeft(List.scala:91) at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$13(RuleExecutor.scala:475) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94) at org.apache.spark.sql.catalyst.rules.RuleExecutor.executeBatch$1(RuleExecutor.scala:452) at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$22(RuleExecutor.scala:585) at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$22$adapted(RuleExecutor.scala:585) at scala.collection.immutable.List.foreach(List.scala:431) at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$1(RuleExecutor.scala:585) at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94) at org.apache.spark.sql.catalyst.rules.RuleExecutor.execute(RuleExecutor.scala:349) at org.apache.spark.sql.catalyst.analysis.Analyzer.executeSameContext(Analyzer.scala:498) at org.apache.spark.sql.catalyst.analysis.Analyzer.$anonfun$execute$1(Analyzer.scala:491) at org.apache.spark.sql.catalyst.analysis.AnalysisContext$.withNewAnalysisContext(Analyzer.scala:397) at org.apache.spark.sql.catalyst.analysis.Analyzer.execute(Analyzer.scala:491) at org.apache.spark.sql.catalyst.analysis.Analyzer.execute(Analyzer.scala:416) at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$executeAndTrack$1(RuleExecutor.scala:341) at org.apache.spark.sql.catalyst.QueryPlanningTracker$.withTracker(QueryPlanningTracker.scala:216) at org.apache.spark.sql.catalyst.rules.RuleExecutor.executeAndTrack(RuleExecutor.scala:341) at org.apache.spark.sql.catalyst.analysis.resolver.HybridAnalyzer.resolveInFixedPoint(HybridAnalyzer.scala:252) at org.apache.spark.sql.catalyst.analysis.resolver.HybridAnalyzer.$anonfun$apply$1(HybridAnalyzer.scala:96) at org.apache.spark.sql.catalyst.analysis.resolver.HybridAnalyzer.withTrackedAnalyzerBridgeState(HybridAnalyzer.scala:131) at org.apache.spark.sql.catalyst.analysis.resolver.HybridAnalyzer.apply(HybridAnalyzer.scala:87) at org.apache.spark.sql.catalyst.analysis.Analyzer.$anonfun$executeAndCheck$1(Analyzer.scala:478) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$.markInAnalyzer(AnalysisHelper.scala:425) at org.apache.spark.sql.catalyst.analysis.Analyzer.executeAndCheck(Analyzer.scala:478) at org.apache.spark.sql.execution.QueryExecution.$anonfun$lazyAnalyzed$3(QueryExecution.scala:294) at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:94) at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:532) at org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$5(QueryExecution.scala:668) at org.apache.spark.sql.execution.SQLExecution$.withExecutionPhase(SQLExecution.scala:150) at org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$4(QueryExecution.scala:668) at org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:1308) at org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$2(QueryExecution.scala:661) at com.databricks.util.LexicalThreadLocal$Handle.runWith(LexicalThreadLocal.scala:63) at org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:658) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:1450)

Re: Not able to read data from volume and data is in JSON format

radothede — Tue, 22 Jul 2025 14:35:23 GMT

Hi @Aviral-Bhardwaj ,

please double check:

- if volume path is correct

- if you have READ VOLUME permission on this volume

- if your cluster has access to unity catalog

- if json file exists

Re: Not able to read data from volume and data is in JSON format

lingareddy_Alva — Tue, 22 Jul 2025 16:18:48 GMT

Hi @Aviral-Bhardwaj

Looking at this error, it's an InvalidMountException related to Unity Catalog Volumes. Try below:

Check Volume Path and Permissions:

# First, verify the volume exists and check permissions dbutils.fs.ls("/Volumes/mydatabricksaviral/datatesting/datavolume/") # Check if the specific file exists dbutils.fs.ls("/Volumes/mydatabricksaviral/datatesting/datavolume/mytest.json")

Grant Proper Permissions

-- Run this SQL to grant permissions on the volume GRANT ALL PRIVILEGES ON VOLUME mydatabricksaviral.datatesting.datavolume TO `your_user_or_group`; -- Check current permissions SHOW GRANTS ON VOLUME mydatabricksaviral.datatesting.datavolume;

Most Common Fixes:
Permission issue: Run the GRANT statement above
Volume not mounted: Restart the cluster
Path format: Try different path formats
File doesn't exist: Verify with dbutils.fs.ls()

Re: Not able to read data from volume and data is in JSON format

Aviral-Bhardwaj — Tue, 22 Jul 2025 16:38:39 GMT

you are the best @Rodo thanks

Re: Not able to read data from volume and data is in JSON format

Aviral-Bhardwaj — Tue, 22 Jul 2025 16:39:48 GMT

thanks sir

topic Re: Not able to read data from volume and data is in JSON format in Data Engineering

Not able to read data from volume and data is in JSON format

Re: Not able to read data from volume and data is in JSON format

Re: Not able to read data from volume and data is in JSON format

Re: Not able to read data from volume and data is in JSON format

Re: Not able to read data from volume and data is in JSON format