- 3977 Views
- 7 replies
- 3 kudos
Oracle Essbase connectivity
Team, I wanted to understand the best way of connecting to Oracle Essbase to ingest data into the delta lake
- 3977 Views
- 7 replies
- 3 kudos
- 3 kudos
I am currently working with Essbase 21c and I need to pull data from Databricks through a SQL query. I was able to successfully setup JDBC connection to Databricks but when I try to create a data source using a SQL query, it gives me an error: "[Data...
- 3 kudos
- 53 Views
- 1 replies
- 0 kudos
Use of Genie agents for ETL migration
I am currently working on a project to migrate existing ETL workflows from SSIS to Databricks. I would like to understand how Databricks Genie, Genie Code, or other Databricks AI-assisted capabilities can support this migration.Can these tools analyz...
- 53 Views
- 1 replies
- 0 kudos
- 0 kudos
Genie code can meaningfully accelerate an SSIS migration, but they should be viewed as a productivity layer around the migration process rather than a push-button solution. In practice, the most reliable automation today is around assessment, invento...
- 0 kudos
- 51 Views
- 1 replies
- 0 kudos
can i deploy a metric view using DABs
I am trying to deploy metric view using DABs but I cant find a resource for it so i ended up with a SQL file but is there a way to automatically run the SQL for it
- 51 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi Areqio, You can use a SQL file with Metric view definitions or Notebooks with Metric view definitions as the standard path. You can then wrap the SQL file or notebooks in a DABs job resource using a sql file task or notebook task.You can have the ...
- 0 kudos
- 82 Views
- 2 replies
- 1 kudos
Resolved! Community Custom Connector - Defining Non-Serverless Job Compute Runtime?
Hi there,I'm exploring Community Custom Connectors and I'm wondering about the Compute. By default, it's set to Serverless:But due to IT network limitations at our company, we cannot use serverless. In the Compute settings, I can untoggle the Serverl...
- 82 Views
- 2 replies
- 1 kudos
- 1 kudos
You generally cannot select DBR version in Lakeflow / Spark Declarative Pipelines because the pipeline runtime is Databricks-managed and versionless. Databricks auto-upgrades the runtime for pipelines.The settings has a JSON view, and some advanced o...
- 1 kudos
- 57 Views
- 1 replies
- 0 kudos
Auto-Termination Did Not Trigger on Production Cluster Despite 20-Minute Inactivity
Hello everyone,I recently encountered an issue with one of our production all-purpose clusters where the auto-termination feature did not work as expected.The cluster was configured with "Terminate after 20 minutes of inactivity", but it never termin...
- 57 Views
- 1 replies
- 0 kudos
- 0 kudos
The Spark UI details you shared points directly to the cause - Databricks Shell application. Auto termination only triggers when a cluster is completely idle with zero active sessions or connections. Databricks Shell application represents an active ...
- 0 kudos
- 6231 Views
- 3 replies
- 0 kudos
Resolved! Getting connection reset issue while connecting to a SQL server
Hello All,I am unable to connect to a SQL server instance that is installed in a on-premise network from databricks. I am able to successfully ping the server from the notebook using this command [nc -vz <hostname> <port>] which means I am able to e...
- 6231 Views
- 3 replies
- 0 kudos
- 0 kudos
The error you are encountering, "com.microsoft.sqlserver.jdbc.SQLServerException: Connection reset," even after a successful nc (netcat) connection, is a common but nuanced problem when connecting Databricks to an on-premise SQL Server. Although your...
- 0 kudos
- 53 Views
- 0 replies
- 0 kudos
Databricks Apps (Streamlit) - How to Implement Proper Logout Functionality?
We have built a Databricks App using Streamlit and are looking for a way to implement a proper logout functionality.Currently, we built Logout functionality where we can clear the Streamlit session state and client-side cookies. However, the Databric...
- 53 Views
- 0 replies
- 0 kudos
- 77 Views
- 0 replies
- 1 kudos
Reducing Time to Help in the Databricks Community
Hi @MandyR , @Advika , and teamI wanted to share one idea for the Databricks Community. Can we explore a Genie-like experience for databricks community?The Community already has many useful answers, accepted solutions, blogs, MVP articles, and discus...
- 77 Views
- 0 replies
- 1 kudos
- 276 Views
- 6 replies
- 1 kudos
My Apps get stopped automatically
Hi team,I have 2 applications in my workspace which they will get stopped sometimes without me knowing the reason. I’m on premium account and I have to run them when I can’t have access to my app. I need help.    
- 276 Views
- 6 replies
- 1 kudos
- 1 kudos
Could you check the app Logs and tell me what you see when one of them stops?A stack trace or "OOM / killed" → it's crashing (code/memory fix)Clean shutdown with no error → it's the compute lifecycle (keep-alive job fix)Nothing / app never fully star...
- 1 kudos
- 300 Views
- 4 replies
- 2 kudos
Resolved! sdp-meta (dlt-meta) vs lakeflow_framework: when should we use which?
Hi All,I'm evaluating metadata-driven frameworks for Lakeflow Spark Declarative Pipelines, and I've found two that appear to serve the same purpose, both from Databricks organizations:- databrickslabs/dlt-meta, whose feature/sdp-meta branch renames t...
- 300 Views
- 4 replies
- 2 kudos
- 2 kudos
Since you are already on Lakeflow Connect for MySQL, your CDC ingestion into Bronze is the managed, supported path, so you do not need a metadata framework for it. Resources by layer: Ingestion to Bronze (what you have): stay on Lakeflow Connect. It...
- 2 kudos
- 249 Views
- 2 replies
- 2 kudos
Resolved! What is the compute for Lakeflow Connect SharePoint Connector
Can someone help me understand what compute the SharePoint Connector is using by default? I'm trying to use Job Compute, but it "seems" like it may be defaulting to serverless.
- 249 Views
- 2 replies
- 2 kudos
- 2 kudos
The managed SharePoint connector uses serverless compute; it can't be switched to classic Job Compute. The Job you see is the orchestration layer: Lakeflow Connect creates it for the schedule and runs the ingestion pipeline as a task, while the pipel...
- 2 kudos
- 201 Views
- 2 replies
- 0 kudos
Resolved! Issue: Lakeflow Connect Microsoft Teams Community Connector - No module named 'databricks.labs'
Hi there, I'm trying to run a pretty barebones test here with community connectors. I'm trying out the "vanilla" Microsoft Teams connector by:(A) going to the Data Ingestion tab in our workspace,(B) selecting the Microsoft Teams community connector,(...
- 201 Views
- 2 replies
- 0 kudos
- 0 kudos
Both issues trace back to running on non-serverless compute. Community connectors were designed to run on serverless Spark Declarative Pipelines (SDP), and serverless is on by default in the CLI config for a reason. The No module named 'databricks.la...
- 0 kudos
- 1556 Views
- 4 replies
- 2 kudos
Pull JAR from private Maven repository (Azure Artifactory)
Hi,I currently struggle on the following task:We want to push our code to a private repository (Azure Artifactory) and then pull it from databricks when the job runs. It currently works only with wheels inside a PyPi repo in the artifactory. I found ...
- 1556 Views
- 4 replies
- 2 kudos
- 2 kudos
Hi @iyashk-DB ,Any update on this? Is there any official suggestion for integrating JFrog and install JAR files using libraries section in asset bundles?
- 2 kudos
- 271 Views
- 4 replies
- 2 kudos
Resolved! How do you keep a plain Delta copy of a streaming table? CLONE is not supported on streaming tables
Hi All, I need an up-to-date plain Delta copy of a streaming table created by Lakeflow pipeline. My first thought was CLONE, and that path is closed. The clone docs say streaming tables and materialized views cannot be the source or target of a deep ...
- 271 Views
- 4 replies
- 2 kudos
- 2 kudos
Thanks @adnan_alvee , yes, we did test Compatibility Mode. The output is a copy of files at a storage location, not a Unity Catalog table.Context: We are setting up an external engine (duckdb) to read UC Delta tables through credential vending, which...
- 2 kudos
- 821 Views
- 5 replies
- 4 kudos
Resolved! Lakeflow Community Connector Issue - ModuleNotFoundError: No module named 'databricks.labs'
Hi there, I have what should be a somewhat straightforward implementation of a Custom Community Connector. I've tried to Dry run a few times but am encountering PYTHON.MODULE_NOT_FOUND_ERRORBelow is some context about my setup:Source Relative path: l...
- 821 Views
- 5 replies
- 4 kudos
- 4 kudos
the sys.path approach won’t fix this for Lakeflow Connect pipelines. During a dry run, Databricks imports databricks.labs.community_connector before your ingest notebook runs, using only the pipeline’s Environment dependencies. Fix: Pipeline Settings...
- 4 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 580 | |
| 349 | |
| 287 |
