Databricks Platform Discussions
Dive into comprehensive discussions covering various aspects of the Databricks platform. Join the co...
11826 Posts
Activity in Discussions
- 0 Views
- 0 replies
- 0 kudos
querying snowflake database using databricks query federation: no active warehouse
Hello Databricks community,I'm confused right now because I was able to query snowflake table using query federation 2 days ago. But now it's giving me error about no active warehouse:Status of query associated with resultSet is FAILED_WITH_ERROR. No...
- 0 Views
- 0 replies
- 0 kudos
- 6 Views
- 0 replies
- 0 kudos
Bookmark in pdf
I am creating a pdf using pyspark and trying to make bookmarks for each table in the pages. All the bookmarks end up pointing to the first table in the first page. Please help me out here.
- 6 Views
- 0 replies
- 0 kudos
- 11 Views
- 1 replies
- 0 kudos
Cloning of Workflow from One env to different env using Job API
Hi Team,One of my team members recently shared one requirement: he wants to migrate the 10 Workflows from the sandbox to the dev environment to run his model in dev env.I wanted to move all these workflows in an automated way and one of the solutions...
- 11 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi AmetDatabricks Asset Bundles are designed precisely for the requirements you have.https://docs.databricks.com/en/dev-tools/bundles/index.htmlYou can also transfer existing jobs that were created manually into a bundle. This works like this, for ex...
- 0 kudos
- 4691 Views
- 6 replies
- 1 kudos
Resolved! [Databricks Assets Bundles] no deployment state
Good morning, I'm trying to run: databricks bundle run --debug -t dev integration_tests_job My bundle looks: bundle: name: x include: - ./resources/*.yml targets: dev: mode: development default: true workspace: host: x r...
- 4691 Views
- 6 replies
- 1 kudos
Latest Reply
- 1 kudos
Hello, Reopening this ticket in hopes that either of you had some luck in resolving your bug. I am currently facing the same issue where I can deploy an asset bundle via the local CLI without issue (by deploy I mean the bundle code is written to my ...
- 1 kudos
- 1841 Views
- 1 replies
- 1 kudos
Delta tables and YOLO computer vision tasks
Hi all,I would really appreciate if someone could help me out. I feel it’s both a data engineering and ML question.One thing we use at wo is YOLO for object detection. I’ve managed to run YOLO by loading data from the blob storage, but I’ve seen tha...
- 1841 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
I am also looking for an answer to this question. Did you manage to find a solution @Andrewcon ?
- 1 kudos
- 45 Views
- 2 replies
- 0 kudos
Networking configuration of Azure Databricks managed storage account
Hi all,I created an Azure Databricks Workspace, and the workspace creates an Azure Databricks managed storage account.The networking configuration of the storage account is "Enabled from all networks".Shall I change it to "Enabled from selected virtu...
- 45 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Changing the networking configuration of your Azure Databricks managed storage account to "Enabled from selected virtual networks and IP addresses" is a good step for enhancing security. However, the "Insufficient permissions" status you are seeing f...
- 0 kudos
- 13 Views
- 2 replies
- 1 kudos
Is there any option in databricks to estimate cost of a query before execution
Hi Team,I want to check if there is any option in data bricks which can help to estimate cost of a query before execution?I mean calculate DBU before actual query execution based on the logical plan? Regards
- 13 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
Is there any way to track the progress or ETA?Do we have access to ideas portal? Where can we search this reference number DB-I-5730?
- 1 kudos
- 8675 Views
- 5 replies
- 0 kudos
Error: Folder xxxx@xxx.com is protected
Hello, On Azure Databricks i'm trying to remove a folder on the Repos folder using the following command : databricks workspace delete "/Repos/xxx@xx.com"I got the following error message:databricks workspace delete "/Repos/xxxx@xx.com"Error: Folder ...
- 8675 Views
- 5 replies
- 0 kudos
Latest Reply
- 0 kudos
You have deleted the user from the workspace and the folder is still showing?
- 0 kudos
- 25 Views
- 1 replies
- 0 kudos
Table Trigger - Too many logfiles
Hi, we have implemented a job that runs on a trigger of a table update. The job worked perfectly, until the source table now have accumulated too many log files and the job isn't running anymore. Only the error message below:Storage location /abcd/_d...
- 25 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @184754 Interesting topic, as the docs says:"Log files are deleted automatically and asynchronously after checkpoint operations and are not governed by VACUUM. While the default retention period of log files is 30 days, running VACUUM on a table r...
- 0 kudos
- 1032 Views
- 2 replies
- 0 kudos
Is it possible to use Liquid Clustering on Delta Live Tables / Materialised Views?
Is it possible to use Liquid Clustering on Delta Live Tables? If it is available what is the Python syntax for adding liquid clustering to a Delta Live Table / Materialised view please?
- 1032 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @amr, materialised views are not tables, they are views. Liquid clustering is not supported on views so it will throw [EXPECT_TABLE_NOT_VIEW.NO_ALTERNATIVE] error. Unfortunately it will be the same case for the "optimize" command as well.
- 0 kudos
- 25 Views
- 0 replies
- 0 kudos
Voucher for Data Engineer Associate certification
Hi, I am looking for a voucher to complete the Data Engineer Associate certification. Could anyone help me please?
- 25 Views
- 0 replies
- 0 kudos
- 1281 Views
- 2 replies
- 0 kudos
Foreign table to delta streaming table
I want to copy a table from a foreign catalog as my streaming table. This is the code I used but I am getting error: Table table_name does not support either micro-batch or continuous scan.; spark.readStream .table(table_name) ...
- 1281 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
I also want to bump this! This is my exact problem right now as well.
- 0 kudos
- 143 Views
- 9 replies
- 1 kudos
My DABS CLI Deploy call not generating a .tfstate file
Hi Community,I'm running into an issue, when executing Databricks CLI Bundle Deploy I dont get the Terraform State file (.tfstate). I know that I should get one but even when defining the state_apth on my YAML (.yml) DABS file I still do not get it.D...
- 143 Views
- 9 replies
- 1 kudos
Latest Reply
- 1 kudos
Forgot to also share this print screen of the last section in the logs. Somehow the State file keeps getting ignored (not found) so how can the deployment still take place I wonder.
- 1 kudos
- 54 Views
- 1 replies
- 1 kudos
Unable to Access Databricks Customer Academy Platform
I am currently facing an issue accessing the Databricks Customer Academy platform. When I attempt to visit https://customer-academy.databricks.com, I receive the following error message:"You are not authorized to access https://customer-academy.datab...
- 54 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
I am facing the same issue.
- 1 kudos
- 47 Views
- 2 replies
- 0 kudos
Resolved! Default schema in SQL Editor is not 'default' when unity catalog is set as default catalog
In workspace settings: Workspace admin - advanced - other - Default catalog for the workspace is set to different than hive_metastore, it is set to a `Unity Catalog` catalog - the expected behaviour is copied here from the related more info panel:"Se...
- 47 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @Alberto_Umana ,Thank you for the explanation. I mark your comment as the accepted solution as it contains the current implementation logic and the work around. Good to know that the more info panel is a bit misleading as of now because the SQL Ed...
- 0 kudos
| User | Count |
|---|---|
| 1812 | |
| 873 | |
| 468 | |
| 312 | |
| 312 |
