cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
YSF
New Contributor III
since ‎12-24-2021
‎06-26-2023

User Stats

  • 8 Posts
  • 0 Solutions
  • 0 Kudos given
  • 1 Kudos received

User Activity

Any elegant pattern for Autoloader/DLT development?

04-30-2023
Does anyone have a workflow or pattern that works for developing with autoloader/DLT? I'm still new to but the fact that while testing it's creating checkpoints using schema locations makes it really tricky to develop with and hammer out a working ve...

Delta Live Table & Autoloader adding a non-existent column

04-30-2023
I'm trying to setup autoloader to read some csv files. I tried with both autoloader with the DLT decorator as well as just autoloader by itself. The first column of the data is called "run_id", when I do a spark.read.csv() directly on the file it com...

How do I use the Python Logging Module in a Repo?

02-28-2023
I have a repo that have python files that use the built in logging module. Additionally in some of the notebooks of the repo I want to use logging.debug()/logging.info() instead of print statements everywhere. However when I use the root logger or cr...

Unity Catalog with unstructured "lake" data? (.png, .ggzip, .zarr)

12-16-2022
I have images, zipped files, and arbitrary binary files in ADLS gen 2. In Unity Catalog I created a storage credential and an external location to the container that contains these files in directories.Is there any way I can view them within Unity Ca...

Issues with using Databricks-Connect and Petastorm

12-25-2021
Has anyone successfully used Petastorm + Databricks-Connect + Delta Lake?The use case is being able to use DeltaLake as a data store regardless of whether I want to use the databricks workspace or not for my training tasks.I'm using a cloud-hosted ju...
Kudos from
View all