https://community.databricks.com/t5/data-engineering/massive-increase-in-the-number-of-quot-getblobproperties-quot/m-p/122599#M46822 <P>Massive increase in&nbsp;<SPAN>"GetBlobProperties" operations in your Azure Blob Storage account could be due to the following</SPAN></P><P>1. Delta Tables and _delta_log Metadata Access<BR />If you're using Delta Lake, Databricks reads blob properties (e.g., last-modified time, size) from the _delta_log directory to check for updates. A new notebook, workflow, or user action that frequently reads or lists a Delta table can lead to repeated GetBlobProperties operations on the json commit files and checkpoints in that directory.</P><P>Check if:<BR />New notebooks or dashboards recently started querying Delta tables more often.<BR />Auto-refresh features in notebooks or dashboards are enabled.<BR />VACUUM, DESCRIBE HISTORY, or schema evolution commands are being run more frequently.</P><P>2. Databricks File System (DBFS) Listing<BR />Any dbutils.fs.ls() or similar calls made recursively over folders in DBFS backed by Azure Blob Storage can trigger many GetBlobProperties calls. These are commonly used in diagnostics, logging, or monitoring code.</P><P>3. Unity Catalog or External Table Audits<BR />If you're using Unity Catalog or have external Hive Metastore tables that point to blob-backed data, background jobs or access control audits can trigger metadata reads.</P><P>Widgets or jobs that query tables repeatedly, especially in a multi-user workspace, can inadvertently lead to a flood of GetBlobProperties.<BR /><BR />I hope this gives you some clarity!</P> Tue, 24 Jun 2025 02:48:36 GMT Vidhi_Khaitan 2025-06-24T02:48:36Z https://community.databricks.com/t5/data-engineering/massive-increase-in-the-number-of-quot-getblobproperties-quot/m-p/113633#M44585 <P>I had a massive increase in the volume of "GetBlobProperties" operations in my Azure Blob Storage account. The storage logs indicate that all the extra operations have IPs attributed to my Databricks resource group. I haven't made any changes to my resource configurations, so I'm confused on what could have led to this increase. I appreciate any input on this issue.</P> Wed, 26 Mar 2025 03:22:35 GMT https://community.databricks.com/t5/data-engineering/massive-increase-in-the-number-of-quot-getblobproperties-quot/m-p/113633#M44585 syazwansuhaimi 2025-03-26T03:22:35Z https://community.databricks.com/t5/data-engineering/massive-increase-in-the-number-of-quot-getblobproperties-quot/m-p/122599#M46822 <P>Massive increase in&nbsp;<SPAN>"GetBlobProperties" operations in your Azure Blob Storage account could be due to the following</SPAN></P><P>1. Delta Tables and _delta_log Metadata Access<BR />If you're using Delta Lake, Databricks reads blob properties (e.g., last-modified time, size) from the _delta_log directory to check for updates. A new notebook, workflow, or user action that frequently reads or lists a Delta table can lead to repeated GetBlobProperties operations on the json commit files and checkpoints in that directory.</P><P>Check if:<BR />New notebooks or dashboards recently started querying Delta tables more often.<BR />Auto-refresh features in notebooks or dashboards are enabled.<BR />VACUUM, DESCRIBE HISTORY, or schema evolution commands are being run more frequently.</P><P>2. Databricks File System (DBFS) Listing<BR />Any dbutils.fs.ls() or similar calls made recursively over folders in DBFS backed by Azure Blob Storage can trigger many GetBlobProperties calls. These are commonly used in diagnostics, logging, or monitoring code.</P><P>3. Unity Catalog or External Table Audits<BR />If you're using Unity Catalog or have external Hive Metastore tables that point to blob-backed data, background jobs or access control audits can trigger metadata reads.</P><P>Widgets or jobs that query tables repeatedly, especially in a multi-user workspace, can inadvertently lead to a flood of GetBlobProperties.<BR /><BR />I hope this gives you some clarity!</P> Tue, 24 Jun 2025 02:48:36 GMT https://community.databricks.com/t5/data-engineering/massive-increase-in-the-number-of-quot-getblobproperties-quot/m-p/122599#M46822 Vidhi_Khaitan 2025-06-24T02:48:36Z