Hello @thecodecache ,
Have a look the SQLGlot project: https://github.com/tobymao/sqlglot?tab=readme-ov-file#faq
It can easily transpile SQL to Spark SQL, like that:
import sqlglot
from pyspark.sql import SparkSession
# Initialize Spark session
spar...
Hello @halox6000,
You could temporarily redirect console output to a null device for these write operations.
Try this out:
@contextlib.contextmanager
def silence_dbutils():
with contextlib.redirect_stdout(io.StringIO()):
yield
# Usage in...
Hello @Nathant93,
You could use dbutils.fs.ls and iterate on all the directories found to accomplish this task.
Something like this:
def find_empty_dirs(path):
directories = dbutils.fs.ls(path)
for directory in directories:
if directo...
Hello @Andrewcon and @jnap ,
Have a look at Mosaic Streaming Dataset. You could load your data from your delta table and then train it on your PyTorch YOLO model. In that example, it use mobilenet model but you can adapt it to use YOLO. Petastorm is ...
Hello @Anske ,
This feature is currently in private preview. You should contact your account team to have it enabled in your workspace(s).
Requirements
Databricks Runtime (DBR) 13.3 or aboveRecipients must use Serverless SQL warehouse to query a shar...