cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Gim
Contributor
since ‎09-16-2022
‎05-22-2024

User Stats

  • 9 Posts
  • 0 Solutions
  • 5 Kudos given
  • 9 Kudos received

User Activity

Columns with DEFAULT missing error during INSERT

07-04-2023
I am really confused about the DEFAULT capability of Databricks SQL. I looked at the documentation for the minimum required DBR to get the capability yet we still need to enable it as a table property? I updated my cluster's DBR from 12.2 to 13.1.Any...

How to use SQL UDFs for Delta Live Table pipelines?

01-18-2023
I've been searching for a way to use a SQL UDF for our DLT pipeline. In this case it is to convert a time duration string into INT seconds. How exactly do we use/apply UDFs in this case?

Best practice for logging in Databricks notebooks?

11-02-2022
What is the best practice for logging in Databricks notebooks? I have a bunch of notebooks that run in parallel through a workflow. I would like to keep track of everything that happens such as errors coming from a stream. I would like these logs to ...

Incremental loading from Delta tables to MySQL database

09-16-2022
I was thinking of using Azure Data Factory to orchestrate this. But is it possible to do incremental loads from tables in Delta format going to MySQL database? My Delta table sources will be coming from ADLS storage. Would I need to convert them firs...

Delta Table storage best practices

09-16-2022
Hi!We have a project where we do some Data Engineering for a client. I implemented a scheduled batch processing of Databricks' autoloader (stream w/ availableNow) since they primarily have numerous file exports from several sources. We wanted to foll...