Hello Databricks community,I'm confused right now because I was able to query snowflake table using query federation 2 days ago. But now it's giving me error about no active warehouse:Status of query associated with resultSet is FAILED_WITH_ERROR. No...
I am creating a pdf using pyspark and trying to make bookmarks for each table in the pages. All the bookmarks end up pointing to the first table in the first page. Please help me out here.
Hi Team,One of my team members recently shared one requirement: he wants to migrate the 10 Workflows from the sandbox to the dev environment to run his model in dev env.I wanted to move all these workflows in an automated way and one of the solutions...
Hi AmetDatabricks Asset Bundles are designed precisely for the requirements you have.https://docs.databricks.com/en/dev-tools/bundles/index.htmlYou can also transfer existing jobs that were created manually into a bundle. This works like this, for ex...
Good morning, I'm trying to run: databricks bundle run --debug -t dev integration_tests_job My bundle looks: bundle: name: x include: - ./resources/*.yml targets: dev: mode: development default: true workspace: host: x r...
Hello, Reopening this ticket in hopes that either of you had some luck in resolving your bug. I am currently facing the same issue where I can deploy an asset bundle via the local CLI without issue (by deploy I mean the bundle code is written to my ...
Hi all,I would really appreciate if someone could help me out. I feel it’s both a data engineering and ML question.One thing we use at wo is YOLO for object detection. I’ve managed to run YOLO by loading data from the blob storage, but I’ve seen tha...
I am also looking for an answer to this question. Did you manage to find a solution @Andrewcon ?
Hi Team,I want to check if there is any option in data bricks which can help to estimate cost of a query before execution?I mean calculate DBU before actual query execution based on the logical plan? Regards
Is there any way to track the progress or ETA?Do we have access to ideas portal? Where can we search this reference number DB-I-5730?
Hi, we have implemented a job that runs on a trigger of a table update. The job worked perfectly, until the source table now have accumulated too many log files and the job isn't running anymore. Only the error message below:Storage location /abcd/_d...
Hi @184754 Interesting topic, as the docs says:"Log files are deleted automatically and asynchronously after checkpoint operations and are not governed by VACUUM. While the default retention period of log files is 30 days, running VACUUM on a table r...
Is it possible to use Liquid Clustering on Delta Live Tables? If it is available what is the Python syntax for adding liquid clustering to a Delta Live Table / Materialised view please?
Hi @amr, materialised views are not tables, they are views. Liquid clustering is not supported on views so it will throw [EXPECT_TABLE_NOT_VIEW.NO_ALTERNATIVE] error. Unfortunately it will be the same case for the "optimize" command as well.
Hi Community,I'm running into an issue, when executing Databricks CLI Bundle Deploy I dont get the Terraform State file (.tfstate). I know that I should get one but even when defining the state_apth on my YAML (.yml) DABS file I still do not get it.D...
Forgot to also share this print screen of the last section in the logs. Somehow the State file keeps getting ignored (not found) so how can the deployment still take place I wonder.
When importing a .csv file with leading and/or trailing empty spaces around the separators, the output results in strings that appear to be trimmed on the output table or when using .display() but are not actually trimmed.It is possible to identify t...
I’ve found that investing in high-quality print services like GoPrint really makes a difference in ensuring your materials match perfectly. Also, it's good practice to always test with smaller prints first, like business cards or brochures, before go...
Hi. If I've posted to the wrong area, please let me know.I am using SQL to join two tables. One table has the wildcard '%' stored as text/string/varchar. I need to join the value of TableA.column1 to TableB.column1 based on the wildcard in the str...
Hi,Could you please try the query below and let me know if it meets your requirements? SELECT * FROM TableA A LEFT JOIN TableB B ON A.Column1 LIKE REPLACE(B.Column1, '%', '%%')Replace helps us in treating the %' stored in TableB.Column1 as a wildcar...
Issue: Spark structured streaming applicationAfter adding the listener jar file in the cluster init script, the listener is working (From what I see in the stdout/log4j logs)But when I try to hit the 'Content-Type: application/json' http://host:port/...
Issue:After adding the listener jar file in the cluster init script, the listener is working (From what I see in the stdout/log4j logs)But when I try to hit the 'Content-Type: application/json' http://host:port/api/v1/applications/app-id/streaming/st...
Hello everyone,I am facing an issue with writing 100–500 million rows (partitioned by a column) into a newly created Delta table. I have set up a cluster with 256 GB of memory and 64 cores. However, the following code takes a considerable amount of t...
Hello, I have encountered an issue recently and was not able to find a solution yet. I have a job on databricks that creates a table using dbt (dbt-databricks>=1.0.0,<2.0.0). I am setting the location_root configuration so that this table is externa...
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group| User | Count |
|---|---|
| 1606 | |
| 761 | |
| 344 | |
| 285 | |
| 251 |
