- 32507 Views
- 22 replies
- 46 kudos
Databricks Announces the Industry’s First Generative AI Engineer Learning Pathway and Certification
Today, we are announcing the industry's first Generative AI Engineer learning pathway and certification to help ensure that data and AI practitioners have the resources to be successful with generative AI. At Databricks, we recognize that generative ...
- 32507 Views
- 22 replies
- 46 kudos
- 46 kudos
Dear Certifications TeamI have completed full Generative AI Engineering Pathway, so I received module wise knowledge badge but I didn't received the overall certificate which mentioned in description which is Generative AI Engineer with one Star. Req...
- 46 kudos
- 1821 Views
- 3 replies
- 0 kudos
Genie Agent integration issue
Hi, anyone from development team for Genie Agent integration? i had an issue of using sample code of Genie Agent integration. The issue is that underlying code (databricks_ai_bridge/genie.py) cannot connect to url openaipublic.blob.core.windows.net ...
- 1821 Views
- 3 replies
- 0 kudos
- 0 kudos
You're absolutely right. The implementation should definitely take restricted network scenarios into consideration, as many enterprise Databricks deployments operate in air-gapped or network-restricted environments.Commenting out the token counting l...
- 0 kudos
- 6788 Views
- 2 replies
- 2 kudos
Resolved! LLM with the largest context window
A Generative AI Engineer is tasked with developing an application that is based on an open-source large language model (LLM). They need a foundation LLM with a large context window. Which model fits this need?DBRX,Llama2-70B,DistilBert MPT-30B.DBRX h...
- 6788 Views
- 2 replies
- 2 kudos
- 2 kudos
@royinblr11 You're absolutely right to question the answer — the correct model for an application needing a foundation LLM with a large context window is: DBRXWhy DBRX is the Best Fit:It is a foundation model, designed for generation tasks.It support...
- 2 kudos
- 1155 Views
- 3 replies
- 0 kudos
Are serverless endpoints possible in this Technical Blog post by qianyu?
Hi, are serverless endpoints possible for Whisper and Llama in this Technical Blog post by qianyu?https://community.databricks.com/t5/technical-blog/streamline-customer-call-center-transcripts-analytics-with/ba-p/101689Thanks!
- 1155 Views
- 3 replies
- 0 kudos
- 0 kudos
https://www.databricks.com/product/pricing/foundation-model-servingWe don´t serve models on databricks, but as far as i can see you pay per input/output tokens (for foundation models).For classic models:https://www.databricks.com/product/pricing/mode...
- 0 kudos
- 4161 Views
- 4 replies
- 2 kudos
Resolved! MLFlow Authentication from Databricks App for GenAI Tracing
I am working on a Dash-based app that includes a call to a Databricks-hosted LLM endpoint. I am trying to track those calls with MLFlow. My code is (roughly) like this:from openai import OpenAIimport mlflowmlflow.set_tracking_uri("databricks")mlflow....
- 4161 Views
- 4 replies
- 2 kudos
- 2 kudos
Thank you for the push in the right direction! I was able to solve the issue with this codeos.environ["DATABRICKS_CLIENT_ID"] = "" os.environ["DATABRICKS_CLIENT_SECRET"] = "" os.environ["DATABRICKS_TOKEN"] = os.environ.get("VAR_CONFIGURED_WITH_DATABR...
- 2 kudos
- 10102 Views
- 2 replies
- 1 kudos
AI/BI Genie API
Hi everyone,I'm currently working on a project that implements a custom front end for Genie using the Genie API: https://docs.databricks.com/api/workspace/genie. So far, I’ve successfully built a working model that can send and receive messages.Howev...
- 10102 Views
- 2 replies
- 1 kudos
- 1 kudos
Is there any possibility that workspace admin could have removed this permission to you? If you check the permissions on your side do you see yourself with at least Can View?
- 1 kudos
- 3944 Views
- 1 replies
- 0 kudos
Resolved! Roadmap for vector_search function
I was wondering if there was a roadmap for the development of the vector_search function: vector_search function | Databricks DocumentationSpecifically, I was wondering if / when the following limitations may be lifted:Querying DIRECT_ACCESS index ty...
- 3944 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @jAAmes_bentley, DIRECT_ACCESS & filters_json are not currently supported with vector_search sql function. These are on our roadmap, but we don’t have concrete ETAs to share at the moment as we’re focusing on other high-priority tasks. Hybrid s...
- 0 kudos
- 8548 Views
- 1 replies
- 0 kudos
Resolved! How to do NLP against PDFs in Databricks? Can be done in Snowflake very easily.
I’d love to build a quick "art of the possible" demo showing how easy it is to query unstructured PDFs using natural language. In Snowflake, I wired up a similar solution in ~2 hours just by following their tutorial guide.Does anyone know the best wa...
- 8548 Views
- 1 replies
- 0 kudos
- 0 kudos
To query unstructured PDF files using natural language in Databricks, you can leverage an approach similar to the "Retrieval Augmented Generation (RAG) and DBRX" demo. Although the specific demo you referenced (https://notebooks.databricks.com/demos/...
- 0 kudos
- 3369 Views
- 4 replies
- 3 kudos
LangChain Compatibility in Databricks – Painfully Inconsistent Experience
Hey all,Is anyone else experiencing major friction when using LangChain within Databricks?Even after carefully pinning versions like:%pip install \ loguru \ mlflow==2.10.1 \ langchain==0.1.5 \ langchain-google-vertexai \ databricks-vectorse...
- 3369 Views
- 4 replies
- 3 kudos
- 3 kudos
I'll definitely look into installing libraries at the cluster level via init scripts makes a lot of sense for stability and reproducibility. I hadn't considered that approach, so I really appreciate you highlighting that and providing the relevant Da...
- 3 kudos
- 3920 Views
- 1 replies
- 0 kudos
Getting "The given endpoint does not exist" for LLM endpoint though I've access to it
When trying out this LLM RAG notebook steps I'm getting this error:"The given endpoint does not exist, please retry after checking the specified model and version deployment exists"This notebook originally referred to databricks-meta-llama-3-1-70b-in...
- 3920 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey @Iris12 ,Not sure if this will fix your issue, but I’ve seen similar errors caused by permission mismatches.Make sure that the user or service principal you’re using has the correct permissions both in the Models and Serving tab.Also, double-chec...
- 0 kudos
- 15322 Views
- 5 replies
- 2 kudos
Genie Space Embedded or access using a REST API
Hi all.I would like to embed Genie Space in my company's data intranet to be used like a "ChatGPT", would it be possible?Or use an API to make a call from Genie Space (in this case we would develop an interface on the intranet and send the questions ...
- 15322 Views
- 5 replies
- 2 kudos
- 2 kudos
Hi all,I just came across this thread and wanted to jump in — I’m currently working on implementing a Genie Space that will be used by external customers, and we’re also looking into whether it can be accessed via API or embedded into a custom fronte...
- 2 kudos
- 1167 Views
- 1 replies
- 0 kudos
Genie - Data Lineage
Would it be possible for Genie to bring in the Lineage Graph of table?Thanks
- 1167 Views
- 1 replies
- 0 kudos
- 0 kudos
Have you tried looking at lineage through the system tables: "system.access.table_lineage" and "system.access.column_lineage". You would need to have entitlements on these objects but you should be able to reference them in Genie. Give it a try and...
- 0 kudos
- 2285 Views
- 2 replies
- 0 kudos
Issue with Lang Chain Agents
This is regarding Agentic framework. We are getting issues while invoking the lang chain agent and need your help to get some suggestions from your expert team to resolve this.As per our observation Agent is behaving bit weird while calling tools reg...
- 2285 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @SandipCoder, About point 2, is it working intermittently or failing all the time?
- 0 kudos
- 2841 Views
- 2 replies
- 0 kudos
How to deploy an agent into model registry using MLFlow
Goal :Here below I am attaching my code snippet where I am trying to build an agent on my custom functions .Things are working well when I am running agent.invoke(“Question”).But now I want to deploy this agent into model registry using MLFlow but I ...
- 2841 Views
- 2 replies
- 0 kudos
- 0 kudos
Had the same question when I was trying to deploy my custom llm. How we did it was wrapping it inside a custom mlflow model and registering it.
- 0 kudos
- 17695 Views
- 1 replies
- 5 kudos
Databricks MCP Server
Hey, couldn't find a working Databricks MCP so I forked one and fixed it: https://github.com/markov-kernel/databricks-mcp This allows you to (using Cursor, Claude Code, Windsurf, Claude, etc.):* Manage Databricks Clusters: List, create, terminate, ...
- 17695 Views
- 1 replies
- 5 kudos
- 5 kudos
very cool
- 5 kudos
- 1115 Views
- 1 replies
- 0 kudos
Datbricks Notebook as a Server ?
Hello, What I'm trying to do : I have a piece of python code (an ai application) that i want to deploy as a server, almost exactly like one would lets say on an EC2. The only difference being instead of a flask api i would use dbutils and the databri...
- 1115 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Krthk, Thanks for your question. Have you thought about using Model serving endpoint for your use-case: https://docs.databricks.com/aws/en/machine-learning/model-serving/ Launching a Databricks job via the REST API and dbutils is a different para...
- 0 kudos
