I've been exploring the Databricks ecosystem and recently started looking more closely at Databricks Apps and Lakebase.What I find interesting is how Databricks is moving beyond data processing and analytics into application development.Lakebase prov...
Hi Everyone,I am Soumitra Dutta,I have an issue with LLM API and Retrieval-Augmented Generation (RAG). Sometimes the same query returns different answers even if the retrieved documents and the prompt are the same.The retrieval results are useful, bu...
Hi Soumitra,Yes, this is a common issue with RAG systems. Even with the same context and prompt, LLMs can sometimes generate different responses.A few things I would check:Keep temperature low (0–0.2) for more consistent results.Make the prompt clear...
Hello fellow techies,For the past few days I have been doing some R&D wrt Lakeflow connect. I came to the understanding that, Lakeflow Ingestion Pipeline and SQL server connection have a 1:1 mapping. This would mean that as number of servers scales n...
Most modern enterprise data platforms favor managed ingestion (Lakeflow Connect) for standard relational databases like SQL Server. The operational complexity of monitoring 50 managed pipelines is significantly lower than maintaining custom CDC code,...
Hi all,I am currently exploring the data ingestion feature offered by databricks, specifically connecting to a SQL server. I have gone through the UI and configured an ingestion pipeline that connects to one specific server and can read tables into d...
I completely understand your concern, and I think this is a valid question.If one ingestion pipeline is permanently associated with one ingestion gateway, then from an operational perspective, 20–30 source connections could indeed result in 20–30 ing...
Hello everyone,I’m a Toronto-based BI & Analytics professional with 12 years of experience across enterprise analytics and financial services. I recently completed the Databricks Certified Data Engineer Associate certification.My experience includes ...
Joining a regional user group is a great way to connect with data and AI professionals near you. These groups bring together practitioners to learn, network, and share real-world experiences — all within your local context. To join a group: Select y...
Hey @Rishabh_Tiwari, hope you're doing amazing!I'm interested in starting a Databricks User Group in Vadoara. I've checked the new User Group platform at usergroups.databricks.com and I don't see an existing chapter for our city.Could you point me to...
Hola comunidad,Tengo una duda sobre cómo funciona la facturación en cuentas Pay-As-You-Go y quería saber si a alguien más le ha pasado algo similar.En mi panel de "Usage" veo un consumo acumulado de varios cientos de dólares durante el último año (pr...
Hola a todos, soy ingeneriero en sistemas y deseo aprender a utilizar esta grandiosa herramienta para poder sacarle el mejor provecho posible.
Welcome to the Databricks Community, @maycol25! If you’re just getting started with Databricks, a great place to begin is Databricks Customer Academy, where you can explore courses based on the area you’re interested in, such as Data Engineering, Gen...
Enterprise AI does not usually fail because the model lacks intelligence.It fails because the model does not understand what the organization means.Consider a simple question:“What is our current exposure to active customer?”Behind this question are ...
I m setting up a pipeline in databricks.For data source I create an external location and add my pipeline run_as SP as read files and read metadata permission. When i trigger the pipeline with serverless cluster, i m able to read data from external l...
@Anmol_Chauhan You have few distinct problems across the classic compute - cluster networking during boot and the cluster's compute access mode. Serverless works properly because Databricks manages the control plane routing and networking boundaries ...
Hello,I am an independent consultant and a trainer in the data space, focused primarily on Databricks. I wanted to understand the path to becoming a Databricks trainer delivering ILT sessions on Databricks Academy or to Databricks partners.It would b...
Issue: The Databricks Excel Add-in launches authentication in an external browser window rather than within the Excel task pane. The Unity Catalog does display in the external browser window, but after sign-in, Excel remains stuck spinning/loading in...
Worked with Databricks support engineers to resolve. Root cause determined to be that while the Marketplace add-in previously allowed the connection to be made to the workspace URL, it no longer does. Resolution was to remove the Marketplace add-in...
Hi!I would like some input/tips from the community regarding when is it time to go from a working solution in notebooks to something more "stable", like whl/py-files?What are the pros/cons with notebooks compared to whl/py?The way i structured things...
Hey @Forssen ,My advice:Using .py files and .whl packages is generally more secure and scalable, especially when working in a team. One of the key advantages is that code reviews and version control are much more efficient with .py files, as changes ...
HelloI was previously registered with a customer profile, and have updated my profile to use my work email which is a partner email, but still I am unable to access the partner academy.I tried different things (incognito window, clearing cookies, etc...
Yeah, that sounds frustrating. Hopefully support can get the profile linked properly—seems like it should be a pretty straightforward account update.
Have any one intergrated Hospino We’re exploring an integration between Hospino – Hospital Information System ( #Hospino.com and #Databricks, and I’d love to learn from the community.
I couldn’t find a documented Hospino–Databricks integration yet. The key is determining whether Hospino provides an API, database access, or export mechanism; Databricks can then ingest the data through a suitable connector or pipeline.
