Databricks Community

maxFilesPerTrigger not working in bronze to silver layer

Hi,I am using Matillion architecture where autoloader picks files from AWS S3 and saves in delta lake. Next layer picks the changes from delta lake and does some processing. I am able to set batch size in autoloader and its working. But in bronze to...

Grant Privileges to multiple tables/views in single schema

Is there a way to grant privileges to multiple tables/views using wild cards ?Example like Grant Select on *_views to user?

Functional Difference between managed and external tables UC

Hi community,​Is there a summary or detail guidance on the functional differe​nce between managed and external tables in unity catalog? Looking at the documentation I can't find any specific functionality in databricks that is supported in managed ta...

While creating a UDF in databricks SQL, how can I declare a local variable? Something like this? CREATE OR REPLACE FUNCTION len() set myString = 'my value'; RETURNS INT RETURN length(myString);

Automate Cluster creation

I am new to databricks and my lead told me that we create clusters for running the notebook manually. So please write one python script to do this automation i.e to create the clusters automatically.Can anyone help me to write the script using PySpa...

h3 hex ID using Databricks Mosaic is not the same as h3.geo_to_h3

I am testing the Databricks Mosaic Spatial Grid Indexing method to obtain the `h3 hex` of a given lat, long. # Get the latitude and longitude latitude = 37.7716736 longitude = -122.4485852   # Get the resolution resolution = 7   # Get the H3 hex ID h...

dbutils command keeps on running

Hi All,Im running below command in my clusterdbutils.fs.ls("abfss://demo@###.dfs.core.window.net")I have done the spark.conf.set steps before running above command.It keeps on running for almost 30 mins and still shows as 'Running command'.I have res...

Unable to perform patch operation on users api which is in preview

Hi team,we would like to lock users from accessing workspace, we are able to get properties of user like groups etc..but when we are doing patch operation it is throwing 500 error while during post body in path we are providing entitlements

When will I get Databricks Data Engineer associate voucher.

Its been a day since I passed the exam , still no badge recieved from Databricks​.

How is everyone's project coming along?

There are a few more days until the deadline. Have you picked a project/topic? Has anyone started building yet or finishing things up? Don't forget about the demo video for your project submission here: https://devpost.com/submit-to/18245-so-you-thi...