New board topics in Databricks Community

Getting start with Data Brick machine creation.

Shaily_DBA — Wed, 08 Jul 2026 08:31:17 GMT

I am a beginner for Data Bricks. Want to learn from the scratch.

Did something change with the setup of Unity Catalogs?

Jefke — Wed, 08 Jul 2026 08:01:51 GMT

Just some background. I deliver quite some trainings on Azure Databricks and in these trainings students have to provisions their own Azure Databricks workspace and UC. In the past this always went smoothly.

Go to the Azure Portal

Create a storage account
Create workspace
Grant Storage Blob Contributor on the storage account to an Access Connector of Azure Databricks
Create a new UC metastore and link the workspace to it.
Done.

This whole procedure did actually create the external location and connection objects automatically.

When we now try to repeat these same 5 steps we get an error creating the UC metastore:

parent external location for path `abfss://<>@<>.dfs.core.windows.net/` does not exist.

Some other strange thing is that the abfss:// never had to be added in the past. Now you must manually add it. In the docs it is still saying "The abfss:// prefix is added automatically."

To get the creation of the of the UC metastore to succeed, the Credential and External Location must be created upfront. Once that is done, the above error message does not appear anymore...

Any idea what is causing the changed behaviour?

Is your AI strategy driving real impact or just more pilots?

Inument — Wed, 08 Jul 2026 07:55:02 GMT

AI only matters when it improves workflows, speeds up decisions, and creates measurable business value.

So, what would real AI impact look like inside your company?

from_utc_timestamp silently double-shifts time when session timezone isn't UTC — docs should call th

rikkyvai — Wed, 08 Jul 2026 05:52:08 GMT

Summary:
from_utc_timestamp(current_timestamp(), '<tz>') produces an incorrect (future-shifted) timestamp whenever the Spark session timezone is already set to something other than UTC. This is a very common pattern for teams stamping dp_load_ts/created_at/audit columns, and the current docs don't warn against it.

Root cause:
current_timestamp() returns the current time in the session's configured timezone (spark.sql.session.timeZone / cluster timezone setting), not UTC. from_utc_timestamp(ts, tz) always treats its input ts as if it were UTC, regardless of what it actually is, and converts it into tz.

So if a cluster's session timezone is set to Asia/Bangkok (UTC+7):

SELECT current_timezone(); -- Asia/Bangkok SELECT current_timestamp() AS raw_ts, -- correct Bangkok wall-clock time from_utc_timestamp(current_timestamp(), 'Asia/Bangkok') AS double_shifted -- Bangkok time + 7 more hours

double_shifted ends up 7 hours ahead of the actual current time, because current_timestamp() is already local, and from_utc_timestamp adds the offset a second time on top of it.

Why this matters:
from_utc_timestamp(current_timestamp(), '<local tz>') is a very natural-looking pattern to write when a team wants a "load timestamp in our local timezone," and it's silently wrong whenever the session timezone isn't UTC. It produces no error — just a plausible-looking but incorrect future timestamp — so it can go unnoticed in production pipelines for a long time.

Suggested fix:
Add an explicit warning/example on the from_utc_timestamp (and ideally current_timestamp()) doc pages, something like:

⚠️ from_utc_timestamp always assumes its input is UTC. If your session timezone (current_timezone()) is not UTC, do not pass current_timestamp() directly into from_utc_timestamp() — this will double-apply the timezone offset. If the session timezone already matches your target timezone, use current_timestamp() directly instead.

Frizus fashion complaint

Ridhi1441 — Wed, 08 Jul 2026 04:58:46 GMT

Frizus fashion Custamer Cere Halpline Namber 8271.745.722// all payment related issue solve

Unxpected Suspension of Databricks data engineer professional exam

examtaker2026 — Wed, 08 Jul 2026 04:14:25 GMT

My Databricks certification exam was automatically suspended before the session even started. I did not engage in any malpractice, and my system check was completed successfully prior to the launch. I have already raised a support ticket (Request ID: #00958991), but I have not yet received a response. This issue seems to be very common these days, which is incredibly disappointing for exam takers who have dedicated time to prepare. I kindly request @Cert-Team to look into this matter and help resolve it at the earliest, as I was unable to take the exam due to this unexpected technical suspension.

Databricks data engineer associate certification exam guide - July 2026

talk-premjit — Wed, 08 Jul 2026 02:43:28 GMT

Can someone help me with the Databricks data engineer associate certification exam guide in pdf?
I am trying to get it but can't seem to find it.
I am new to Databricks certifications and need the pdf to start studying.

Thanks

Prem

Exploring Databricks Serverless Sandbox: A New Way to Build AI Agents and Develop in the Cloud As

Abiola-David — Wed, 08 Jul 2026 02:37:24 GMT

Exploring Databricks Serverless Sandbox: A New Way to Build AI Agents and Develop in the Cloud

As AI development continues to evolve, developers are increasingly looking for environments that eliminate infrastructure overhead while providing the flexibility to build, test, and deploy intelligent applications. Databricks has taken another step in this direction with the introduction of Databricks Serverless Sandbox (Beta).

This new capability provides developers with a persistent, cloud-based development environment that is designed specifically for modern AI workflows.

What is Databricks Serverless Sandbox?

Databricks Serverless Sandbox is a managed development environment that runs on Databricks Serverless Compute. Instead of configuring virtual machines or managing clusters, developers can launch an isolated sandbox environment that is ready for coding almost immediately.

The sandbox is particularly useful for experimenting with AI applications, developing agentic workflows, and integrating AI coding assistants into the development process.

Key Features

Several features make the Serverless Sandbox an exciting addition to the Databricks platform.

Persistent Development Environment

Unlike temporary notebook sessions, each sandbox includes a persistent home directory. This allows developers to save projects, install packages, and continue their work across sessions without constantly rebuilding their environment.

Secure SSH Access

Developers can connect directly to the sandbox using SSH, making it easy to use familiar development tools and editors while keeping everything inside the Databricks ecosystem.

Built for AI Coding Agents

The sandbox is designed to work seamlessly with modern AI coding assistants such as Claude, Codex, and Cursor. This creates an environment where AI can actively participate in software development while operating within an organization's governed infrastructure.

Preconfigured Databricks CLI

One practical advantage is that the Databricks CLI is already installed and authenticated. Developers can immediately begin interacting with Databricks workspaces without spending time on configuration.

Shared Workspace

Multiple SSH sessions can connect to the same sandbox and share a common filesystem, making it easier to work across different terminals or development tools.

Generous Storage

Each sandbox provides up to 100 GB of persistent storage throughout its lifetime, giving developers sufficient space for code, datasets, and project artifacts.

Why This Matters

One of the biggest obstacles in AI development has never been writing code—it has been preparing the environment in which that code runs.

Installing dependencies, provisioning infrastructure, maintaining development machines, and ensuring consistency across environments all consume valuable development time.

Serverless Sandbox removes much of this operational burden by providing a ready-to-use development environment that can be launched on demand.

For organizations already using Databricks, this means developers can remain inside the governed Databricks ecosystem while building AI applications, experimenting with new models, and developing intelligent agents.

A Strong Fit for Agentic AI

As organizations begin adopting agentic AI, developers need environments where autonomous agents can safely execute code, access governed data, and interact with enterprise services.

Databricks Serverless Sandbox appears to be designed with exactly these scenarios in mind.

Because it integrates with Databricks Serverless Compute and AI Gateway, organizations can develop AI-powered solutions while maintaining governance, security, and operational controls.

This makes it an attractive option for teams building Retrieval-Augmented Generation (RAG) applications, AI assistants, autonomous workflows, and enterprise AI agents.

Current Beta Status

The feature is currently available in Beta. During this period, Databricks is not charging for sandbox usage, making it an excellent opportunity for developers to explore its capabilities.

As with most beta offerings, users should expect some limitations, including regional availability and evolving feature support, but the direction is clear.

Create Databricks Sandbox

Prerequisites (CLI)

To create and use a Databricks Sandbox with the Databricks CLI, you must:

Install the Databricks CLI on your local machine.
Authenticate the Databricks CLI using databricks_auth_login
Run databricks sandbox create
Run databricks sandbox register
Run databricks sandbox ssh

Final Thoughts

Databricks has steadily evolved from being a unified analytics platform into a comprehensive platform for data engineering, machine learning, and generative AI.

The introduction of Serverless Sandbox continues that evolution by providing developers with an environment where coding, experimentation, AI-assisted development, and governed infrastructure come together seamlessly.

For anyone building modern AI solutions—especially agentic AI applications—this is a feature worth exploring. It reduces infrastructure complexity, accelerates development, and allows teams to focus on what matters most: creating intelligent applications that deliver business value.

As AI development continues to mature, tools like Databricks Serverless Sandbox demonstrate how cloud platforms are shifting from simply hosting workloads to becoming complete developer environments for the next generation of AI innovation.

AI Technical Debt in Databricks: Why Generated Spark SQL and PySpark Still Need Metadata, Review

AmitDECopilot — Wed, 08 Jul 2026 01:36:55 GMT

AI-assisted coding is changing how data engineers build on Databricks.

A developer can now generate Spark SQL, PySpark transformations, Delta table DDL, pipeline logic, data-quality checks, documentation, and test scenarios much faster than before. What used to take hours can now be drafted in minutes.

That speed is useful.

But in Databricks data engineering, code that runs successfully is not always production-ready. A Spark SQL query can compile. A PySpark notebook can execute. A Delta table can be created. A pipeline can produce output. But that does not automatically mean the logic is correct, governed, traceable, scalable, or aligned with the business requirement.

This is where AI technical debt begins.

AI technical debt in Databricks does not always look like messy code. Sometimes it looks like clean Spark SQL or PySpark that nobody fully understands, nobody validated against the approved source-to-target mapping, and nobody can confidently support when production data behaves differently from the sample used during development.

The problem is not AI-assisted coding itself. The problem is using generated code without metadata, review, and governance.

Data Engineering on Databricks Is More Than Code Generation

Databricks gives data teams a powerful platform for building lakehouse pipelines, Delta tables, batch and streaming workloads, analytics, machine learning, and AI applications.

But building on Databricks is not just about writing Spark SQL or PySpark.

A production data pipeline usually represents business meaning. When a data engineer writes a transformation, they are not simply moving data from one table to another. They are interpreting business rules, source behavior, target definitions, data-quality expectations, lineage, incremental-load logic, and downstream consumption needs.

For example, a source-to-target mapping may define how a customer status, policy state, account type, transaction category, or claim indicator should be derived. A generated Spark SQL CASE statement may look correct, but the real questions are:

Did it use the correct source column?
Did it handle nulls correctly?
Did it account for unexpected values?
Did it align with the target Delta table schema?
Did it preserve the approved business definition?
Did it include the right data-quality expectation?
Did it fit the bronze, silver, and gold design pattern?
Did it support reruns and late-arriving records?
Did it create explainable lineage?
Did it respect ownership, access, and governance expectations?

AI can generate code quickly. But it does not automatically understand the full enterprise context unless that context is structured, governed, and reviewed.

That is why generated Spark SQL and PySpark still need metadata, review, and governance.

How AI Technical Debt Enters Databricks Projects

AI technical debt often enters quietly.

A developer asks an AI tool to generate a PySpark transformation from a mapping document. The output looks clean. The syntax is valid. The code runs on a sample dataset. The notebook looks professional.

But later, the team discovers issues:

the wrong source-column alias was used
a business rule was only partially implemented
a join created duplicate records
a null-handling rule was missed
a late-arriving update was not considered
an incremental-load condition was too simple
a Delta table schema was not aligned with the target contract
a data-quality check was too generic
Unity Catalog descriptions or ownership were missing
lineage was not clear enough for support or audit
downstream reporting numbers did not match expectations

These are not unusual problems in data engineering. The difference is that AI can accelerate artifact creation before the engineering intent is fully validated.

That creates a new kind of debt: not only technical debt in the code, but intent debt.

Intent debt happens when the code exists, but the reasoning behind it is incomplete, undocumented, or disconnected from the approved requirement.

In a Databricks environment, intent debt can spread across notebooks, Delta tables, jobs, pipelines, data-quality checks, documentation, and downstream analytics. Once that happens, the team may move fast in the beginning but pay the cost later through rework, production defects, unclear ownership, and difficult support.

The Wrong Way to Use AI in Databricks Data Engineering

A risky AI-assisted workflow looks like this:

Requirement document → AI prompt → generated Spark SQL or PySpark → copy/paste into notebook → run → deploy

This creates speed, but it also creates risk.

The AI may fill in missing assumptions. The developer may trust the output because it looks polished. Reviewers may focus on syntax rather than business intent. Documentation may be generated after the fact. Data-quality checks may be generic instead of tied to real business rules. Unity Catalog metadata may be incomplete or ignored.

This is how teams end up with pipelines that work technically but fail operationally.

In Databricks, this can be especially risky because the same platform may support multiple layers of the lakehouse:

bronze ingestion
silver standardization
gold business aggregates
analytical consumption
machine learning features
AI application data
reporting and operational workflows

If the generated logic is wrong at one layer, the issue can flow downstream quickly.

Fast code is useful only when it is also correct, traceable, governed, and supportable.

Metadata Is the Missing Control Layer

The solution is not to stop using AI.

The solution is to stop treating the prompt as the source of truth.

For Databricks data engineering, the source of truth should be structured engineering metadata.

Metadata should capture the intent behind the implementation. It should define:

source systems
source tables and columns
target Delta tables and columns
business definitions
transformation rules
join logic
data-quality expectations
incremental-load logic
merge keys
late-arriving-data behavior
reconciliation rules
ownership
approval status
lineage relationships
Unity Catalog descriptions, tags, or classifications
version history and effective dates

When metadata is structured, generated code can be reviewed against something concrete.

Instead of asking:

Does this PySpark code look good?

The team can ask:

Does this PySpark code match the approved metadata definition?

That is a much stronger review process.

A metadata-driven review can answer:

Does the generated Spark SQL use the correct source and target attributes?
Does it follow the approved transformation rule?
Does it apply the required null-handling behavior?
Does it include the right data-quality expectation?
Does it preserve lineage from bronze to silver or silver to gold?
Does it support the required incremental pattern?
Does it align with Unity Catalog ownership and classification?
Does it support operational monitoring and restartability?

This is where metadata becomes the control layer for AI-assisted engineering.

A Better Workflow for Databricks

A governed AI-assisted Databricks workflow should look more like this:

Business Requirement → STTM / Data Contract → Canonical Metadata Model → Databricks Artifacts → Human Review → Governance / CI-CD Controls → Jobs or Pipelines → Monitoring

In this model, AI is still useful. But it is placed inside a controlled delivery lifecycle.

AI can help draft:

Spark SQL transformation logic
PySpark transformation templates
Delta table DDL
data-quality expectations
Lakeflow or pipeline logic
job configuration notes
data dictionary entries
Unity Catalog descriptions
lineage summaries
reconciliation checks
test scenarios
runbook starter content

But these outputs should be treated as drafts.

The canonical metadata model remains the control layer. Engineers review generated artifacts against approved requirements, source-to-target mappings, data contracts, governance expectations, and production-readiness standards.

This changes the role of the data engineer.

The engineer is not only writing repetitive code. The engineer is validating engineering intent, edge cases, performance, quality, maintainability, security, and production behavior.

That is where experienced engineering judgment matters most.

What Metadata Can Generate in Databricks

A governed metadata layer can support many Databricks delivery artifacts.

For example, from a well-defined source-to-target mapping and business rule catalog, teams can generate or draft:

Delta table DDL
Spark SQL transformations
PySpark transformations
Lakeflow pipeline definitions
medallion-layer mapping documentation
data-quality expectations
accepted-value checks
nullability checks
reconciliation rules
Unity Catalog table descriptions
Unity Catalog column descriptions
ownership and classification tags
data dictionary entries
lineage documentation
test scenarios
production-readiness checklists
operational runbook notes

The value is not only generation. The value is consistency.

If SQL, PySpark, DQ checks, data dictionaries, and lineage are all derived from the same approved metadata, the risk of documentation drift reduces significantly.

Instead of manually creating every artifact from scratch, engineers review and refine generated outputs.

The focus shifts from repetitive artifact creation to validating engineering correctness.

Practical Example: Customer Status Mapping in Databricks

Consider a simple Databricks lakehouse example.

A source system lands customer data into a bronze Delta table. One field is called status_cd.

The target silver table needs a standardized field called customer_status.

The source values may be:

A
I
P
blank
unexpected codes

A generated Spark SQL transformation might create:

CASE

WHEN status_cd = 'A' THEN 'Active'

WHEN status_cd = 'I' THEN 'Inactive'

WHEN status_cd = 'P' THEN 'Pending'

END AS customer_status

At first glance, this looks fine.

But a production-ready Databricks implementation needs more than a simple CASE statement.

The team still needs to answer:

What happens when status_cd is null?
What happens when a new unexpected code appears?
Should unknown values fail the pipeline or go to an exception table?
Is customer_status nullable in the target Delta table?
Is there a data-quality expectation for accepted values?
Should invalid records be quarantined?
Is the mapping approved by a business owner?
Does Unity Catalog have the correct column description?
Does lineage show the field moving from bronze to silver?
Does the logic support incremental processing?
What test scenarios prove this logic?
How will support teams debug a production mismatch?

A metadata-driven approach defines these expectations once.

For example:

Metadata element	Example value
Source table	bronze.customer_raw
Source column	status_cd
Target table	silver.customer
Target column	customer_status
Transformation rule	Map A, I, P; default unknown values
Null handling	Convert null to Unknown and flag
DQ expectation	Allowed values: Active, Inactive, Pending, Unknown
Exception handling	Unexpected source values logged for review
Lineage	bronze.customer_raw.status_cd → silver.customer.customer_status
Ownership	Customer domain data owner
Approval status	Approved
Review status	Engineer reviewed

From this metadata, the platform can generate a Spark SQL draft, a PySpark draft, a DQ expectation, a data dictionary entry, lineage documentation, and test scenarios.

The engineer still reviews the output. But the review is now anchored to metadata rather than personal interpretation.

That is the difference between AI-generated code and governed AI-assisted engineering.

Unity Catalog Should Be Part of the Process

Governance in Databricks should not be added at the end.

If Unity Catalog is used only after development, governance becomes documentation cleanup. But if metadata is defined earlier, Unity Catalog can become part of the delivery workflow.

For example, a metadata-driven process can help populate or validate:

table descriptions
column descriptions
ownership
data classification
sensitivity labels
domain tags
business definitions
lineage relationships
access expectations

This matters because production data engineering is not only about creating tables. It is also about making data discoverable, explainable, secure, and trusted.

When generated code creates or updates Delta tables, the governance metadata should move with it.

A good Databricks delivery process should ask:

Is the table registered correctly?
Is the owner clear?
Are column definitions understandable?
Are sensitive fields classified?
Are access expectations defined?
Is lineage visible?
Can another team understand and reuse this data asset?

AI can help draft descriptions, summaries, and documentation. But the approved meaning should come from governed metadata and human review.

Data Quality Cannot Be Generic

AI can generate data-quality checks quickly. But generic checks are not enough.

A generated rule like “column should not be null” may be useful, but it does not prove the data is business-correct.

In Databricks, data-quality expectations should be tied to the business meaning of the data.

For example:

customer_id should be unique within the target table
customer_status should contain only approved values
transaction_amount should not be negative unless the transaction type allows it
effective_date should not be after expiration_date
source-to-target row counts should reconcile within an approved threshold
late-arriving records should be handled according to the incremental-load design

These expectations should be defined in metadata before they are generated into pipeline checks, validation logic, or test scenarios.

The key question is not:

Did AI generate a DQ check?

The key question is:

Did the DQ check reflect the approved business expectation?

That is the difference between a technical validation and a trusted data-control process.

Incremental Processing Needs Explicit Metadata

Many production Databricks pipelines are not full-refresh pipelines. They process new or changed data incrementally.

AI-generated code often produces a simple transformation, but incremental behavior requires more design detail.

A metadata-driven incremental design should capture:

incremental driver column
watermark logic
merge keys
insert behavior
update behavior
delete or inactivation behavior
late-arriving-data handling
duplicate handling
rerun and idempotency expectations
reconciliation checks
failure and restart behavior

Without this metadata, generated Spark SQL or PySpark may work for a sample batch but fail when real production conditions appear.

For example, a generated merge may handle inserts and updates but ignore deletes. It may use the wrong key. It may miss late-arriving updates. It may not be idempotent during reruns.

These are not small issues. They directly affect trust in downstream data.

AI can help draft the incremental pattern, but the design must come from approved metadata and experienced engineering review.

Human Review Is Not Optional

AI-generated Databricks code should go through a review process that checks more than syntax.

A strong review should include:

Business-rule validation
Does the implementation reflect the approved business rule?

Source-to-target validation
Are the correct source and target columns used?

Delta table validation
Are schema, nullability, keys, partitioning, and table design appropriate?

Data-quality validation
Are completeness, validity, uniqueness, accepted values, referential integrity, timeliness, and reconciliation covered?

Incremental-load validation
Does the code handle inserts, updates, deletes, late-arriving records, duplicates, reruns, and recovery?

Unity Catalog validation
Are ownership, descriptions, classifications, and governance expectations captured?

Lineage validation
Can the team explain where the data came from and how it changed?

Performance validation
Will the Spark logic scale for expected data volume?

Operational validation
Can this be monitored, restarted, supported, and explained during incidents?

Security validation
Does the implementation respect access controls and data-handling requirements?

This is the difference between AI-assisted coding and AI-assisted data engineering.

Governance Does Not Have to Slow Teams Down

Some engineers hear the word governance and think it means more meetings, more approvals, and slower delivery.

That does not have to be true.

Good governance reduces confusion. It makes the delivery path clearer. It prevents teams from debating the same rules repeatedly. It gives reviewers something concrete to validate. It helps new engineers understand the system faster. It reduces production support issues because the intent is documented before deployment.

In Databricks, a governed metadata-driven process can actually improve speed because teams do not have to recreate the same artifacts manually.

Instead of asking different people to separately create transformation code, DQ rules, data dictionaries, test cases, and documentation, teams can define the engineering intent once and generate consistent drafts.

The review still matters. But the starting point is much stronger.

The goal is not bureaucracy.

The goal is repeatable, explainable, production-ready delivery.

Recommended Pattern

A practical pattern for reducing AI technical debt in Databricks is:

Capture requirements and STTM in a structured format.
Convert mappings into a canonical metadata model.
Define transformation rules, DQ expectations, lineage, and incremental behavior.
Generate Databricks artifacts as drafts.
Review Spark SQL, PySpark, DDL, DQ checks, and documentation against metadata.
Validate governance fields in Unity Catalog.
Run test and reconciliation scenarios.
Deploy through approved CI/CD or workflow controls.
Monitor production behavior.
Feed issues and corrections back into the metadata model.

This creates a feedback loop.

The system improves not because AI writes everything perfectly, but because metadata, review, and operational learning stay connected.

Final Thought

The future of Databricks data engineering is not simply AI-generated notebooks or faster Spark SQL.

The real opportunity is governed AI-assisted engineering.

That means business requirements, source-to-target mappings, metadata, Delta table design, data-quality expectations, Unity Catalog governance, lineage, testing, deployment, and monitoring remain connected through the delivery lifecycle.

AI can accelerate the drafting of Databricks artifacts.

But metadata, review, and governance are what make those artifacts production-ready.

Fast code is useful.

Trusted data is more valuable.

Integration of mongodb and Databricks Serverless

tsuppalapati — Wed, 08 Jul 2026 01:35:44 GMT

Hi,
I'm looking to integrate the databricks serverless workspace with Mongodb atlas through the private endpoint route. Created a NCC rule from databricks. From Mongodb side I cannot see the request to approve the connection,

Thanks,
Tarun

Free Edition CrossValidator not working because of internal caching

pjvi — Tue, 07 Jul 2026 21:05:51 GMT

Hi all,

I am getting the following error when using CrossValidator in Databricks Free Edition v5 (see attachment):

In shared or serverless cluster, SPARK_ML_TMP_DFS_PATH environmental variable must be set to a UC volume path like '/Volumes/...' in order to support Saprk DataFrame caching

Could you please check if this the expected behaviour, and if there a workaround or something we can try?

Thanks

Announcement | Beyond dashboards: Introducing Decision Execution Platforms

Tushar_Parekar — Tue, 07 Jul 2026 17:33:55 GMT

Databricks Forward Deployed Engineering is introducing Decision Execution Platforms (DEPs) as a new analytics category focused on moving from signal to decision to execution to measured outcome, all on the customer’s own governed Databricks environment. Databricks positions FDE as an engineering-led model for delivering business outcomes with AI, built on the Lakehouse platform and closely tied to customer workflows.

What’s new

Beyond dashboards – DEPs are designed to go further than surfacing insights by helping organizations connect signals, decisions, execution, and outcomes in one continuous loop.
Built on the Databricks platform – The architecture described in the source brings together governed data and AI services such as Lakebase, Unity Catalog, Genie Agents, and agent orchestration on a single operational plane.
Keeps governance at the center – Because the platform runs on the customer’s own Databricks instance, data, models, and operational workflows stay inside a governed environment with centralized controls.
Turns decisions into something measurable – The goal is not just to recommend actions, but to help teams track what was chosen, what was executed, and how the actual outcome compared with the expected one.
A practical enterprise use case is already underway – The source highlights an early retail deployment focused on fulfillment optimization, where planners, operational systems, and agent-driven workflows are being brought together around a named business outcome.

👉 Read the full post here

Error while installing lakebridge

Himanshu_Singh — Tue, 07 Jul 2026 15:46:24 GMT

Hi All,

As part of my Databricks learning journey, I am trying installing the Lakebridge in my system but getting below error.

Traceback (most recent call last):
File "C:\Users\himan\.databricks\labs\lakebridge\state\venv\Lib\site-packages\databricks\sdk\config.py", line 272, in __init__
self._validate()
~~~~~~~~~~~~~~^^
File "C:\Users\himan\.databricks\labs\lakebridge\state\venv\Lib\site-packages\databricks\sdk\config.py", line 672, in _validate
raise ValueError(f"validate: more than one authorization method configured: {names}")
ValueError: validate: more than one authorization method configured: github-oidc and pat

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\Users\himan\.databricks\labs\lakebridge\lib\src\databricks\labs\lakebridge\install.py", line 531, in <module>
ws=lakebridge.create_workspace_client(),
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "C:\Users\himan\.databricks\labs\lakebridge\lib\src\databricks\labs\lakebridge\cli.py", line 67, in create_workspace_client
return self._workspace_client()
~~~~~~~~~~~~~~~~~~~~~~^^
File "C:\Users\himan\.databricks\labs\lakebridge\state\venv\Lib\site-packages\databricks\labs\blueprint\cli.py", line 208, in _workspace_client
return WorkspaceClient(
product=self._product_info.product_name(),
product_version=self._product_info.version(),
)
File "C:\Users\himan\.databricks\labs\lakebridge\state\venv\Lib\site-packages\databricks\sdk\__init__.py", line 230, in __init__
config = client.Config(
host=host,
...<29 lines>...
),
)
File "C:\Users\himan\.databricks\labs\lakebridge\state\venv\Lib\site-packages\databricks\sdk\config.py", line 277, in __init__
raise ValueError(message) from e
ValueError: validate: more than one authorization method configured: github-oidc and pat. Config: host=https://adb-7405615889085100.0.azuredatabricks.net, account_id=72022d01-11ba-410c-83ba-4d1d262d6a02, workspace_id=7405615889085100, token=***, token_audience=https://adb-7405615889085100.0.azuredatabricks.net/oidc/v1/token, databricks_cli_path=C:\Users\himan\AppData\Local\Microsoft\WinGet\Packages\Databricks.DatabricksCLI_Microsoft.Winget.Source_8wekyb3d8bbwe\databricks.exe. Env: DATABRICKS_HOST, DATABRICKS_ACCOUNT_ID, DATABRICKS_WORKSPACE_ID, DATABRICKS_TOKEN, DATABRICKS_TOKEN_AUDIENCE, DATABRICKS_CLI_PATH
Error: installer: exit status 1
23:33:13 Info: failed execution pid=15164 exit_code=1 error="installer: exit status 1"
23:33:13 Debug: no telemetry logs to upload pid=15164

Can anyone guide me and explain what this error is all about, please? If need further info, let me know as well please.

Thank you.

Dashboard Variable Display name

amekojc — Tue, 07 Jul 2026 14:57:40 GMT

In databricks documentation it says that there's a display name when creating dashboard variables but I'm not seeing it in the dashboard settings. Does anyone know where to find them?

📌‌ Complete Your Profile – Help Others Get to Know You

Advika — Tue, 07 Jul 2026 14:33:26 GMT

Community Ask

Hey Community! 📌

Your Databricks Community profile is often the first thing other members see when they come across your posts, solutions, blogs, or event participation.

A complete profile helps other members understand who you are, what you do, and the expertise you bring to the community. Details like your name, job title, location, and LinkedIn profile make it easier for members to connect with you, learn from your experience, and engage with your contributions.

Check Your Profile Completion Status

✓ Click your profile icon in the top-right corner

✓ Select My Profile

✓ Look for the profile completion indicator to see your current percentage

To Reach 100%

✓ Go to My Settings → Personal Information

✓ Review and complete any missing details

✓ We especially recommend adding:

• First Name & Last Name
• Job Title
• Location
• LinkedIn Profile
• Secondary Email (recommended for account recovery if you ever lose access to your primary email)

✓ Click Save

It only takes a few minutes, but it helps you build a stronger presence in the community and makes it easier for other members to discover, connect, and engage with you.

Take a moment to check your profile today and see if you can reach 100%!

not everything is in the cloud

Hubert-Dudek — Tue, 07 Jul 2026 13:56:51 GMT

It’s great to see that we have ZeroBus, which can push events from on-premise to the cloud. Now it’s joined by on-premises OpenSharing. This is optimal for organizations that can’t move everything to the cloud or simply don’t want to. #databricks

https://databrickster.medium.com/my-favorite-announcements-from-the-data-ai-summit-2026-317fc68d4e75

https://www.sunnydata.ai/blog/data-ai-summit-2026-announcements

Databricks Data engineering professional Certification exam suspended

naveendasari — Tue, 07 Jul 2026 13:49:40 GMT

Dear DataBricks Certification Team @Cert-Team, @cert-ops , @Cert-Bricks , @Cert-TeamOPS

I attempted my certification on July 4th 1100H America/Chicago. However even before the exam started the exam s suspended and didn't provide any specific reason and asked to contact kryteriononline.com. I reached out to kryterion support but they redirected to Databricks to get it resolved.

Thorough Investigation: I request that a thorough investigation be conducted to understand the circumstances surrounding the suspension of my exam. It is essential that any misunderstandings or errors be rectified promptly.

Reinstatement or Retake Opportunity: I kindly request that my exam be reinstated or that I be provided with another opportunity to retake the exam at no additional cost.

How to restore a workspace ( Cancelled) by mistake

pk35 — Tue, 07 Jul 2026 13:26:53 GMT

Hello Team, I accidentally cancelled a subscription for a workspace and I lost all the notebooks, jobs along with it. How do I get them restored from Databaricks. Storage is on AWS s3 bucket using unity catalog and I still have access to s3 on aws.

I sent a request to restore to help@databrics.com. Is this correct apparoach ..?

Thanks

Content Guidelines for the Databricks Community Technical Blog

Advika — Tue, 07 Jul 2026 12:53:53 GMT

Hi Community!

We're introducing formal Content Guidelines for the Databricks Community Technical Blog – full document attached below as a PDF.

The Technical Blog exists to help the community learn from each other through genuine technical knowledge, real-world experience, and practical insight. It's not an announcement channel – it's a space built on trust, where what you read is something you can actually act on.

As the community grows, we want to protect that. These guidelines make sure every blog published here is technically accurate, adds real value, and reflects genuine expertise.

Who Can Publish

Databricks Employees (Bricksters) – individual or team contributions
Partner Organizations – organizational contributions only
Customer Organizations – organizational contributions only

In Short, We're Looking For

✓ Real technical content – enough detail that a reader can actually follow along and learn from it

✓ Honest reflection – what worked, what didn't, and what you'd do differently next time

✓ Practical takeaways – something the reader can directly apply to their own work

What Won't Get Published

✕ Pure product or company promotion – without explaining how Databricks was used or what a reader can learn from it

✕ Low-effort, visibly AI-generated content – with no real human voice or expertise behind it

✕ Rehashed documentation – restating existing docs without adding new context or experience

Full details and the quality bar are all in the attached PDF👇.

Learning Festival: 15 June - 06 July 2026 Voucher Request

imdabre — Tue, 07 Jul 2026 11:41:27 GMT

Hello Jim Anderson,

Can you please look into why a voucher for the Engineer-Associate exam has not been sent to me? I raised a ticket with support providing the screenshot of the courses completed. It appears there are additional requirements for the voucher. I've attached their response pointing me to you.

Thanks,

Ian