cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
jakubk
Contributor
since ‎08-07-2022
yesterday

User Stats

  • 19 Posts
  • 0 Solutions
  • 1 Kudos given
  • 9 Kudos received

User Activity

unity catalog information schema columns metadata out of sync with table - cant refresh

06-20-2024
I'm using unity catalogI've changed the schema of my table by overwriting it with a newer file df.write \ .format('delta') \ .partitionBy(partitionColumn) \ .mode("overwrite") \ .option("overwriteSchema", "true") \ ...

Unity Catalog - spark.* functions throwing Py4JSecurityException - org.apache.spark.sql.internal.CatalogImpl.currentCatalog() is not whitelisted on class class org.apache.spark.sql.internal.CatalogImpl

03-21-2023
I'm looking to migrate onto unity catalog but a number of my data ingestion notebooks throw a securityexception/whitelist errors for numerous spark. functionsIs there some configuration setting I need to enable to whitelist the spark.* methods/functi...

dbt workflow job limitations - naming the target? where do docs go?

03-01-2023
I'm on unity catalogI'm trying to do a dbt run on a project that works locallybut the databricks dbt workflow task seems to be ignoring the project.yml settings for schemas and catalogs, as well as that defined in the config block of individual model...

spark.read.parquet() - how to check for file lock before reading? (azure)

09-07-2022
I have some python code which takes parquet files from an adlsv2 location and merges it into delta tables (run as a workflow job on a schedule)I have a try catch wrapper around this so that any files that fail get moved into a failed folder using dbu...

databricks spark sql Custom table valued function + struct really slow (minutes for a single row)

08-07-2022
I'm using azure databricksI have a custom table valued function which takes a URL as a parameter and outputs a single row table with certain elements from the URL extracted/labelled(i get search activity URLs and when in a specific format I can retri...
Kudos given to