topic Disable Logging inPython `dbutils.fs.put`? in Data Engineering

Disable Logging inPython `dbutils.fs.put`?

ToBeDataDriven — Fri, 12 Sep 2025 21:01:02 GMT

This function logs every time it writes to stdout "Wrote n bytes." I want to disable its logging as I have thousands of files I'm writing and it floods the log with meaningless information. Does anyone know if it's possible?

Re: Disable Logging inPython `dbutils.fs.put`?

K_Anudeep — Sat, 13 Sep 2025 08:26:33 GMT

Hello @ToBeDataDriven ,

If it's a notebook cell, you can silence the output of the cell by using %%capture

%%capture dbutils.fs.put("dbfs:/FileStore/anudeep/datasets/word_count/tmp/3.txt","foo")

If you want to do it in your code, then there is no flag to silence the output, but you can use a wrapper function in Python as a workaround.

Code:

import os from contextlib import redirect_stdout, redirect_stderr from io import StringIO def dbutils_put(path: str, contents: str, overwrite: bool = True) -> None: sink = StringIO() with redirect_stdout(sink), redirect_stderr(sink): dbutils.fs.put(path, contents, overwrite) dbutils_put("dbfs:/FileStore/anudeep/streaming_datasets/word_count/tmp/4.txt","abc")

Please let me know if this works for you. Thanks

Re: Disable Logging inPython `dbutils.fs.put`?

ToBeDataDriven — Mon, 15 Sep 2025 13:56:45 GMT

Thanks for the reply!

Would that code solution be per thread or would it only work on stdout globally?

I basically have a thread pool doing the work, and I need all the output except fs.put. If it disables it globally I may lose output while it's performing the put.

Re: Disable Logging inPython `dbutils.fs.put`?

K_Anudeep — Mon, 15 Sep 2025 15:22:55 GMT

@ToBeDataDriven ,
redirect_stdout is process-global, not per-thread.
It temporarily rebinds sys.stdout for the whole interpreter, so in a thread pool, you can accidentally swallow other threads’ prints while the with block is active. It’s not thread-safe for your use case. If you need all the output except fs.put, then you need to isolate it; otherwise, I dont think there is another option.

Re: Disable Logging inPython `dbutils.fs.put`?

K_Anudeep — Wed, 17 Sep 2025 02:54:20 GMT

@ToBeDataDriven , If the above answered your question, then could you please help accept the solution?