Databricks Community

Hey @Nagarathna @Lucas_TBrabo  I’d like to share my opinion and some tips that might help:1. You should try to avoud filtering by spark_partition_id because  you can create skewed partitions, you should use with repartition() and spark can optimize t...

Thanks. It really helps me a lot But there is also an issue in shallow clone. We can only clone the full table data, particular delta version data using timestamp/version from the normal table using shallow clone but we can not clone the table data b...

Hey @ep208 ,From the error message you’re seeing (LOCATION_OVERLAP), it seems that Unity Catalog is still tracking a table or volume that points to the same path you’re now trying to reuse:abfss://datalake@datalakename.dfs.core.windows.net/Delta/Proj...

Hi @utkarshamone ,We faced a similar issue and I wanted to share our findings, which might help clarify what’s going on.We’re using a Classic SQL Warehouse size L (v2025.15), and executing a dbt pipeline on top of it.Our dbt jobs started to fail with...

I’ve tried Medium, YourStory, HackerNoon, ShoutMeLoud, SiteProNews, and BloggingCage. But Hikemytraffic helped me find targeted digital marketing guest post sites that actually bring traffic, backlinks, and real leads.

Hi @KG_777 Tracking deleted records in an SCD Type 2 implementation for a lakehouse architecture is indeed a challenging but common requirement.Here's an overview of approaches, challenges, and workarounds based on industry experience:Common Approach...

Try this: SQL Query Options: Databricks SQL supports the SHOW GROUPS command to list all groups within a system. This command optionally allows filtering by specific user associations or regular expressions to identify desired groups.The statement sy...

Hello @LRALVA Thank you for your timePlease give me some clarification on this:The permission error occurred when we used multiple vector searches for a single model. During the model registration process in this scenario, we encountered the error.Ho...

I am experiencing the same problem. I create a file in /tmp and can verify that it exists. But when an attempt is made to open the file using pyspark, the file is not found. I noticed that the path I used to create the file is /tmp/foobar.parquet and...

Hi @Phani1 Using Apache Iceberg instead of Delta Lake for saving data in Databricks can unlock cross-platform compatibility but comes with several potential challenges,especially within the Databricks ecosystem which is natively optimized for Delta L...

Hi @Upendra_Dwivedi To enable this feature, you'll need to go to Apps in your workspace and turn on the On-Behalf-Of User Authorization option. After that, when you're creating or editing your app, make sure to select the necessary user API scopes, t...