- 26 Views
- 1 replies
- 0 kudos
Getting start with Data Brick machine creation.
I am a beginner for Data Bricks. Want to learn from the scratch.
- 26 Views
- 1 replies
- 0 kudos
- 0 kudos
I created my own Databricks workspace via Azure Portal. The benefit of using it via cloud provider is almost all feature s are available compared to the free version in my observation. I am right now exploring Lakebase. I would suggest go with a sing...
- 0 kudos
- 25 Views
- 0 replies
- 0 kudos
Is your AI strategy driving real impact or just more pilots?
AI only matters when it improves workflows, speeds up decisions, and creates measurable business value.So, what would real AI impact look like inside your company?
- 25 Views
- 0 replies
- 0 kudos
- 19144 Views
- 27 replies
- 4 kudos
spotify API get token - raw_input was called, but this frontend does not support input requests.
hello everyone, I'm trying use spotify's api to analyse my music data, but i'm receiving a error during authentication, specifically when I try get the token, above my code.Is it a databricks bug?pip install spotipyfrom spotipy.oauth2 import SpotifyO...
- 19144 Views
- 27 replies
- 4 kudos
- 4 kudos
I really enjoyed reading it and found the insights both practical and engaging. It's always great to come across content that provides real value to readers. I also enjoy exploring articles on digital trends, lifestyle, and informative topics at Con...
- 4 kudos
- 45 Views
- 0 replies
- 1 kudos
from_utc_timestamp silently double-shifts time when session timezone isn't UTC — docs should call th
Summary:from_utc_timestamp(current_timestamp(), '<tz>') produces an incorrect (future-shifted) timestamp whenever the Spark session timezone is already set to something other than UTC. This is a very common pattern for teams stamping dp_load_ts/creat...
- 45 Views
- 0 replies
- 1 kudos
- 111 Views
- 1 replies
- 1 kudos
Lakebase CDF — destination Delta table not created after successful UI setup (Free Edition)
Summary:We configured Lakebase CDF to stream changes from a native Postgres table (public.query_logs) into Unity Catalog, but the destination Delta table never appears despite completing all documented prerequisites.Steps taken1. Created query_logs ...
- 111 Views
- 1 replies
- 1 kudos
- 1 kudos
Most likely this is a Free Edition storage limitation rather than a bug, and your Postgres-side setup sounds correct. Lakebase CDF writes its destination as a Unity Catalog managed Delta table, and the Lakebase CDF docs list catalogs backed by defaul...
- 1 kudos
- 13351 Views
- 7 replies
- 2 kudos
Relevance of off heap memory and usage
I was referring to the doc - https://kb.databricks.com/clusters/spark-executor-memory.In general total off heap memory is = spark.executor.memoryOverhead + spark.offHeap.size. The off-heap mode is controlled by the properties spark.memory.offHeap....
- 13351 Views
- 7 replies
- 2 kudos
- 171 Views
- 5 replies
- 0 kudos
Getting 'Unauthorized Access' when Serverless compute is trying to read s3 bucket data
Getting 'Unauthorized Access' when Serverless compute is trying to read s3 bucket data, same works fine with normal compute storage and external location, credentials, IAM policy and serverless egress rules all seems to be correct because normal comp...
- 171 Views
- 5 replies
- 0 kudos
- 0 kudos
Your read is right, and fair point on the allowlist: editing network policies is part of serverless egress control, which is an Enterprise capability, so on Premium the propagation window is the path. Your timing hypothesis matches how this works int...
- 0 kudos
- 714 Views
- 4 replies
- 3 kudos
Is it required to run Lakeflow Connect on Serverless?
As the subject states, my question is:Is it required to run the Ingestion Pipeline in Lakeflow Connect on Serverless compute? Cause I try to define my own cluster in the DAB, but it raises an error:`Error: cannot create pipeline: You cannot provide c...
- 714 Views
- 4 replies
- 3 kudos
- 3 kudos
Yes — Lakeflow Connect ingestion pipelines always run on Serverless compute. Databricks overrides your compute config and switches back to serverless,because the ingestion connector requires it.
- 3 kudos
- 103 Views
- 0 replies
- 0 kudos
Bundle Validation Error: Volume lifecycle Field Rejected Despite Being in Schema
Environment:Bundle engine: directResource type: Volume (Unity Catalog)Problem:I'm trying to use the lifecycle field with prevent_destroy: true on a Unity Catalog volume resource in my Declarative Automation Bundle (DAB). the validation fails with:Err...
- 103 Views
- 0 replies
- 0 kudos
- 273 Views
- 0 replies
- 0 kudos
How to Prepare Enterprise Data for AI Agents: A Practical Guide
Artificial intelligence has moved beyond chatbots and predictive models. Today, enterprises are adopting AI agents that can retrieve information, analyze business data, automate workflows, and support decision-making across departments.The success of...
- 273 Views
- 0 replies
- 0 kudos
- 592 Views
- 7 replies
- 1 kudos
Resolved! DBR 17.3 – sporadic import failures from bundle
Hi all,Since two days ago we've been getting sporadic Python import failures on DBR 17.3, with no change on our side. The errors are either "cannot import name ..." or "no module named ...", and they come and go: the same code fails on one run and pa...
- 592 Views
- 7 replies
- 1 kudos
- 1 kudos
Thank you for your reply, we have already filed a support ticket. We might look into packaging our code as wheel if there is no other way.
- 1 kudos
- 1771 Views
- 6 replies
- 5 kudos
Resolved! Intermittent failure with Python IMPORTS statements after upgrading to DBR18.0
We have a python module (WidgetUtil.py) that sits in the same folder as our notebook. For the past few years we have been using a simple import statement to use it. Starting with DBR18.0 the imports fails intermittently (25% of the time) when running...
- 1771 Views
- 6 replies
- 5 kudos
- 5 kudos
Hi, are you aware if this is also affecting Databricks Free Edition environment v5 ? I think I am having the same issue.Thanks.
- 5 kudos
- 168 Views
- 1 replies
- 0 kudos
Intialization stage is taking time in lakehouse pipeline
Hello Team,My intialization stage in lakehouse pipeline is taking 5-6 mins. When i checked event log table,below are statsMessage Reported flow time metrics for flowName: 'pipelines.flowTimeMetrics.missingFlowName'. is repeatedly coming for 4-5 mins....
- 168 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Anish_2, The message... Reported flow time metrics for flowName: 'pipelines.flowTimeMetrics.missingFlowName'..is usually not enough on its own to identify the root cause. I would treat it as a metrics/observability signal rather than the primary ...
- 0 kudos
- 1998 Views
- 8 replies
- 0 kudos
Disable the dashboard sharing field for dashboard creators
"How can I disable the dashboard sharing field for dashboard creators who are not admins? I tried changing the creator’s permission from 'CAN_MANAGE' to 'CAN_READ', but it had no effect — the creator still retains the 'CAN_MANAGE' permission
- 1998 Views
- 8 replies
- 0 kudos
- 0 kudos
Hey, that thread's already got the core answer right: there's no setting to strip CAN_MANAGE from a dashboard creator, and you already found that out the hard way when the API downgrade to CAN_READ had no effect. That's expected, Databricks docs conf...
- 0 kudos
- 225 Views
- 1 replies
- 1 kudos
Resolved! Delta Table in DLT pipeline
Hi all, Is it possible to create a Delta table using DLT pipeline. I'm able to create a Delta table using Job and also able to create Materialized view and Streaming table using DLT pipleines. But I want to create Delta table using DLT pipelines. Is ...
- 225 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi, yes, both streaming tables and materialized views in a DLT pipeline (now called Lakeflow Declarative Pipelines) are Delta tables under the hood. There isn't a separate "plain Delta table" object type inside a pipeline, streaming tables and materi...
- 1 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 576 | |
| 349 | |
| 287 |
