- 41456 Views
- 23 replies
- 51 kudos
Databricks Announces the Industry’s First Generative AI Engineer Learning Pathway and Certification
Today, we are announcing the industry's first Generative AI Engineer learning pathway and certification to help ensure that data and AI practitioners have the resources to be successful with generative AI. At Databricks, we recognize that generative ...
- 41456 Views
- 23 replies
- 51 kudos
- 913 Views
- 1 replies
- 1 kudos
Databricks app services
Hi- I have built node.js based chatbot app that uses Azure OpenAI API to build connection and get query answers from it. I am using my organization deployed API on azure that requires cert.pem and cacert.pem certificates to authenticate. everything i...
- 913 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Daya3189 , Your local system likely has the organization's certificates installed in the Windows/Mac system trust store, which Node.js (or your browser) can read. On Databricks, the app runs in a secure, isolated Linux container. It has no knowle...
- 1 kudos
- 1052 Views
- 1 replies
- 0 kudos
Slow Delta write when creating embeddings with mapPartitions
I’m trying to generate 35k+ embeddings in Databricks. What I’ve tried so far:Per-row UDF (very slow).Replaced UDF with rdd.mapPartitions to batch API calls, create one Azure client per partition, and call client.embed_documents(texts) in batches. Thi...
- 1052 Views
- 1 replies
- 0 kudos
- 0 kudos
HiYou’ve optimised the embedding side really nicely already, batching in mapPartitions and creating one Azure client per partition is exactly what we recommend.For 35k rows, if embedding is fast but the Delta write/commit is slow, it’s almost always ...
- 0 kudos
- 3806 Views
- 3 replies
- 3 kudos
Resolved! Issue with ai_parse_document Not Extracting Text from Images in PDF
Hello Team,I hope you are doing well.I am a student currently exploring Databricks and learning how to work with the "ai parse document" function. While experimenting, I encountered a couple of issues related to text extraction from images inside PDF...
- 3806 Views
- 3 replies
- 3 kudos
- 3 kudos
Thank you for your reply!Yes, I have gone through your article — it explains very well how to extract text content from PDFs. However, I am facing a different issue.In my case, the PDF contains multiple images and paragraphs, but "ai_parse_document" ...
- 3 kudos
- 902 Views
- 1 replies
- 0 kudos
What is the difference between full fine-tuning, LoRA, and p-tuning on Databricks?
How are these three methods different?Full Fine-TuningLoRAP-Tuning
- 902 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Suheb Summary Table:Method What’s Tuned Speed/Cost Flexibility Use CaseFull Fine-TuningAll model weightsSlow/HighMaximumCustom tasks, large dataLoRASmall adapter layersFast/LowHighEfficient adaptationp-tuningPrompt embeddingsFastest/LowLimitedPro...
- 0 kudos
- 1510 Views
- 1 replies
- 0 kudos
Genie - Value dictionary
When I add the tables to the genie space, it automatically turns on the value dictionary for first 120 string fields, Is there a way to disable them by default and add only to the needed fields later?I am working in curating responses for around 15 t...
- 1510 Views
- 1 replies
- 0 kudos
- 0 kudos
"Value sampling is enabled by default for all Genie spaces." The only solution that I see is to build json and use as a serialized_space template https://docs.databricks.com/api/azure/workspace/genie/getspace
- 0 kudos
- 3391 Views
- 1 replies
- 1 kudos
Resolved! How do I build a robust multi-agent system (e.g. using Agent Bricks / Genie) on Databricks, while en
How can I set up several AI agents on Databricks that work together at the same time, and make sure they don’t mess up the data or break the system?
- 3391 Views
- 1 replies
- 1 kudos
- 1 kudos
@Suheb, Depends on your usecase. However, if it fits, I would recommend that you start with a multi-agent supervisor if you have the agents from the list below An existing Agent Bricks: Knowledge Assistant(/generative-ai/agent-bricks/knowledge-assist...
- 1 kudos
- 1623 Views
- 3 replies
- 4 kudos
Multi‑Agent Supervisor: url_citation (source links) not shown in Playground — why and how to enable?
I’m seeing a difference in citation behavior between a single Knowledge Assistant (KA) agent and the Multi‑Agent Supervisor setup.What I testedIn Agent Bricks → Knowledge Assistant, I created an agent that returns citations with links (e.g., url_cita...
- 1623 Views
- 3 replies
- 4 kudos
- 5288 Views
- 2 replies
- 2 kudos
How to Increase HTTP Request Timeout for Databricks App Beyond 120 Seconds?
I’ve built a Databricks App using Gradio that leverages predict_stream to get streaming responses from a multi-agent supervisor. The app coordinates reasoning across four knowledge agents, so the model uses a long chain-of-thought process before retu...
- 5288 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @snarayan ,I think you might be hitting timeout from model serving endpoint:Debug model serving timeouts - Azure Databricks | Microsoft LearnYou can try to increase timeout using environment variables using the Serving UI or programmatically using...
- 2 kudos
- 4338 Views
- 9 replies
- 2 kudos
Resolved! Custom MCP deployment
Hi Community!I have a question - could somebody please guide me how to deploy my custom MCP server to databricks?What I would like to achieve is the following:I have a unity catalog in databricks for which I would like to have MCPif the data in unity...
- 4338 Views
- 9 replies
- 2 kudos
- 2 kudos
@Hubert-Dudek ! Okay, I did not know about it! Unfortunately in my databricks workspace a can see Agent Bricks as Coming Soon
- 2 kudos
- 2917 Views
- 3 replies
- 3 kudos
Resolved! Serving model issue in databricks
I’m facing an issue while trying to deploy a custom pyfunc model for Qwen3-Embedding-8B (GGUF format) registered in Unity Catalog. The GGUF model file is stored inside a Unity Catalog Volume, and during model training and registration everything work...
- 2917 Views
- 3 replies
- 3 kudos
- 4249 Views
- 5 replies
- 2 kudos
Resolved! Databricks Genie new space creation using API
I am exploring an option to see if Databricks Genie can be created using API or using python script?
- 4249 Views
- 5 replies
- 2 kudos
- 2 kudos
A recent update introduced the ability to create, update, and delete genie spaces. By stringing these together into your CI/CD pipeline, you should be able to accomplish everything you need. It is not in the SDK or DABs quite yet, but keep an eye out...
- 2 kudos
- 2019 Views
- 3 replies
- 1 kudos
What is the core goal of your project or what problem are you trying to solve with generative AI or
What problem are you trying to solve using Databricks and generative AI?
- 2019 Views
- 3 replies
- 1 kudos
- 1 kudos
Below are high-impact problems you can solve with Databricks and generative AI, grounded in your enterprise data and governed end to end. Customer and employee knowledge accessBuild RAG-powered knowledge assistants that answer questions over your pr...
- 1 kudos
- 2276 Views
- 3 replies
- 2 kudos
Resolved! Multi Agent Supervisor not able to coordinate with genie when deployed as a databricks app
Hello,I've provided necessary permissions to a chatbot that i created using DAB and deployed on databricks apps following this documentation. At the backend its using below agentsMulti Agent SupervisorKnowledge AssistantAI/BI GenieI've provided can_q...
- 2276 Views
- 3 replies
- 2 kudos
- 2 kudos
Hello Bianca,After following what you told, i found issues at two levelsOne was the missing genie-space configuration within databricks.ymlOther was with granting permissions to the service principal (app identity) access to the underlying tables.tha...
- 2 kudos
- 3278 Views
- 3 replies
- 1 kudos
Error from mlflow traces in Databricks notebook
I'm unable to log my traces in my mlflow experiments.The goal is to have the tracing from my genAI calls like described in Tutorial: Connect your development environment to MLflow .So I'm able to create my experiment run, but they are every time empt...
- 3278 Views
- 3 replies
- 1 kudos
- 1 kudos
@rafaelgildin , I see that you have raised an Issue in the OSS mlflow repo, while the issue that you are encountering is on Databricks. The issue here is that you are explicitly adding trace.set_tracer_provider(tracer_provider). Databricks installs i...
- 1 kudos
- 3436 Views
- 1 replies
- 1 kudos
model-specific tools (code interpreters, search, etc.) for OpenAI / Anthropic / Google
Hi,I’m happy to see that the latest models from the three major AI labs (OpenAI, Anthropic, and Google) are now available on Databricks. I had a follow-up question about what exactly is supported.Is Databricks only exposing the base models, or are so...
- 3436 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Rahul-vlk You can use the models both as Databricks-hosted “first‑party” endpoints and via “external model” endpoints that proxy to the provider’s hosted APIs, all through a unified, OpenAI‑compatible interface for chat, embeddings, vision, and ...
- 1 kudos
