- 256 Views
- 2 replies
- 2 kudos
badrequest: cannot create online table is being deprecated. creating new online table is not allowed
Hello!This seems so sudden that we cannot create online tables anymore? Is there a workaround to being able to create online tables temporarily as we need more time to move to synced tables? #online_tables
- 256 Views
- 2 replies
- 2 kudos
- 2 kudos
Yes, the Databricks online tables (legacy) are being deprecated, and after January 15, 2026, you will no longer be able to access or create them.https://docs.databricks.com/aws/en/machine-learning/feature-store/migrate-from-online-tablesHere are few ...
- 2 kudos
- 3653 Views
- 1 replies
- 0 kudos
Federate AWS Cloudwatch logs to Databricks Unity Catalog
I am looking to integrate CloudWatch logs with Databricks. Our objective is not to monitor Databricks via CloudWatch, but rather to facilitate access to CloudWatch logs from within Databricks. If anyone has implemented a similar solution, kindly prov...
- 3653 Views
- 1 replies
- 0 kudos
- 0 kudos
To access CloudWatch logs from within Databricks, you can set up an integration that enables Databricks to fetch, query, and analyze AWS CloudWatch log data directly—without configuring CloudWatch to monitor Databricks clusters. This approach is incr...
- 0 kudos
- 3540 Views
- 1 replies
- 1 kudos
Environment set up in serveless notebook task
Hi community,Is there a way to install dependencies inside a notebook task using serveless compute using Databricks Asset Bundle? Is there a way to avoid installing everytime for each serverless task that compose a job the dependencies (or the librar...
- 3540 Views
- 1 replies
- 1 kudos
- 1 kudos
For Databricks serverless compute jobs using Asset Bundles, custom dependencies (such as Python packages or wheel files) cannot be pre-installed on shared serverless infrastructure across job tasks as you can with traditional job clusters. Instead, d...
- 1 kudos
- 3981 Views
- 1 replies
- 0 kudos
16.2 (includes Apache Spark 3.5.2, Scala 2.12) cluster in community edition taking long time
16.2 (includes Apache Spark 3.5.2, Scala 2.12) cluster in community edition taking long time to start.I m trying to launch 16.2 DBR but it seems the cluster which is one node is taking long time . Is this a bug in the community edition ?Here is the u...
- 3981 Views
- 1 replies
- 0 kudos
- 0 kudos
The long startup time for a Databricks Runtime 16.2 (Apache Spark 3.5.2, Scala 2.12) single-node cluster in Databricks Community Edition is a known issue and not unique to your setup. Many users have reported this situation, with some clusters taking...
- 0 kudos
- 3174 Views
- 1 replies
- 0 kudos
Product code of Databricks in AWS CUR report
I need to know what is the productCode of Databricks in CUR report. Whether the productCode is same for all user?
- 3174 Views
- 1 replies
- 0 kudos
- 0 kudos
In the AWS Cost and Usage Report (CUR), the productCode for Databricks is used to identify costs attributed to Databricks usage within your AWS environment. The value that appears in the lineItem/ProductCode column for Databricks is typically "Databr...
- 0 kudos
- 3283 Views
- 1 replies
- 0 kudos
Foreign Catalog error connecting to SQL Server 2008 R2
Hi,Is there a limitation or know issue when creating a foreign catalog to a SQL Server 2008 R2?We are successfully able to connect to this SQL Server through a JDBC connection string. To make this work, we have to switch the Java encrypt flag to fal...
- 3283 Views
- 1 replies
- 0 kudos
- 0 kudos
There are known limitations and issues when connecting to SQL Server 2008 R2, particularly around encryption and JDBC settings, which can manifest as errors in federated catalog operations—even though a direct JDBC connection might succeed if the "en...
- 0 kudos
- 3344 Views
- 1 replies
- 0 kudos
useing dlt metadata as runtime parameter
i have started using DLT pipeline, and i have common code which is used by multiple DLT pipeline. now i need to read metadata information like name of the pipeline and start time of the pipeline during run time, but since im using common code and pip...
- 3344 Views
- 1 replies
- 0 kudos
- 0 kudos
To dynamically access metadata like the pipeline name and start time at runtime in your common code for Delta Live Tables (DLT) pipelines, you should leverage runtime context and built-in metadata features provided by the DLT or related orchestrators...
- 0 kudos
- 3351 Views
- 1 replies
- 0 kudos
ModuleNotFoundError Importing fuction modules to DLT pipelines
Following best practice, we want to avoid reusing code by putting commonly used transformations into function libraries and then importing and calling those functions where required.We also want to follow Databricks recommendations to use serverless ...
- 3351 Views
- 1 replies
- 0 kudos
- 0 kudos
You are correctly following Databricks’ recommendation to store shared code in Python files and import them into your notebooks, especially for Delta Live Tables (DLT) pipelines and serverless environments. However, import path issues are common, par...
- 0 kudos
- 3517 Views
- 1 replies
- 0 kudos
Delta sharing open issue with access data on storage
Hi, I have configured delta sharing for external consumer in Azure Databricks. Azure Databricks and storage account are in VNET, no public access. The storage account has also disabled account key access and shared key authorization.I'm running delt...
- 3517 Views
- 1 replies
- 0 kudos
- 0 kudos
Delta Sharing in Azure Databricks allows sharing datasets across clouds and with external consumers, but when used in a tightly controlled network environment (private endpoints, no public access, restricted storage account authentication), it behave...
- 0 kudos
- 3296 Views
- 2 replies
- 2 kudos
Issue pulling Docker Image on Databricks Cluster through Azure Container Registry
Hi Community.Essentially, we're using the ACR to push our custom Docker Image, then we would like to pull it to create a Databricks cluster. However, during the cluster creation, we got the following error:I'm convinced we tried to authenticate in al...
- 3296 Views
- 2 replies
- 2 kudos
- 2 kudos
You are experiencing an authentication issue when trying to use a custom Docker image from Azure Container Registry (ACR) with Databricks clusters, despite successfully using admin tokens and service principals with acrpull permissions in other conte...
- 2 kudos
- 3500 Views
- 1 replies
- 0 kudos
Hydra configuration and job parameters of DABs
Hello Community,I'm trying to create a job pipeline in Databricks that runs a spark_python_task, which executes a Python script configured with Hydra. The script's configuration file defines parameters, such as id.How can I pass this parameter at the...
- 3500 Views
- 1 replies
- 0 kudos
- 0 kudos
You can pass and override configuration parameters for Hydra in a Databricks spark_python_task by specifying job-level parameters (as arguments) and using environment variables or Hydra’s command line overrides. For accessing secrets with dbutils.sec...
- 0 kudos
- 2714 Views
- 2 replies
- 0 kudos
Databricks Bundle Asset - Notebook-based bundling alternative to CLI approach
Hello All - I have a scenario where we want to do entire bundling and packaging in notebook to deploy Jobs using Databricks Asset Bundle without using CLI or VS Code. I didn't find any material or reference that provides insights. Any input would be ...
- 2714 Views
- 2 replies
- 0 kudos
- 0 kudos
Deploying Databricks Asset Bundles entirely from a notebook—without using the CLI or VS Code—is not a standard workflow but can be orchestrated using newer features in the Databricks workspace UI and by leveraging programmatic workspace operations. D...
- 0 kudos
- 1472 Views
- 6 replies
- 3 kudos
how to process a streaming lakeflow declarative pipeline in batches
Hi, I've got a problem and I have run out of ideas as to what else I can try. Maybe you can help? I've got a delta table with hundreds millions of records on which I have to perform relatively expensive operations. I'd like to be able to process some...
- 1472 Views
- 6 replies
- 3 kudos
- 333 Views
- 3 replies
- 2 kudos
Databricks Apps pricing
Hi everyone!I was investigating Databricks Apps as solution for my task and didn't fully understood pricing.I have found this page and it indicates it will cost 75$ / DBU for Premium subscription plan when using AWS cloud. Is it full cost or will the...
- 333 Views
- 3 replies
- 2 kudos
- 2 kudos
@deng_dev The $75 per DBU Premium subscription plan price for Databricks Apps on AWS shown on the Databricks Apps pricing page reflects the charge from Databricks itself. https://www.databricks.com/product/pricing/databricks-appsHowever, this is not ...
- 2 kudos
- 10223 Views
- 11 replies
- 13 kudos
Can we add tags to Unity Catalog through Terraform?
We use Terraform to manage most of our infrastructure, and I would like to extend this to Unity Catalog. However, we are extensive users of tagging to categorize our datasets, and the only programmatic method I can find for adding tags is to use SQL ...
- 10223 Views
- 11 replies
- 13 kudos
- 13 kudos
In case anyone comes across this, have a look at databricks_entity_tag_assignment and see if it suits your needs.
- 13 kudos
| User | Count |
|---|---|
| 1624 | |
| 790 | |
| 506 | |
| 349 | |
| 287 |
