Join APJ's premier AI/BI virtual challenge to solve real-world business problems, sharpen your skills, and compete for prizes using the Databricks Data Intelligence Platform. This challenge provides a unique opportunity to work together, apply AI-dri...
- 3290 Views
- 0 replies
- 1 kudos
2 weeks ago
We’re excited to invite you to our MONTHLY Databricks Get Started Days, a half-day virtual learning experience designed to jumpstart your journey with Databricks. This global event is tailored to equip you with essential data engineering and analysi...
- 3790 Views
- 3 replies
- 0 kudos
3 weeks ago
AI is transforming BI by changing the way organizations manage, analyze and get insights from their data. Everyone is rushing to democratize access to data through LLMs. But without the right foundation, you can’t get the most from AI. Our new appro...
- 3942 Views
- 0 replies
- 2 kudos
3 weeks ago
Join us for the return of the Virtual Learning Festival! Mark your calendars from 9 April - 30 April 2025! Upskill across data engineering, data analysis, machine learning, and generative AI. Join the thousands who have elevated their career with...
- 76054 Views
- 309 replies
- 46 kudos
03-03-2025
Be part of a global movement! Connect with 22,000 data enthusiasts at 700+ sessions, keynotes, and hands-on training at this year’s Data + AI Summit. Whether you’re into data intelligence, governance, AI, or data warehousing, this is your chance to l...
- 9941 Views
- 1 replies
- 4 kudos
02-19-2025
Community Activity
- 44 Views
- 2 replies
- 0 kudos
Is databricks community edtion cluster having issue?
i am trying to run a piece of code and everytime i am getting "Notebook detached×Exception when creating execution context: java.net.SocketTimeoutException: Connect Timeout". Can anyone please tell if there is any issue going on for the databrics com...
- 44 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
I am running into the same issue. I am just trying to initialize a spark session. Please let me know if there is a fix.
- 0 kudos
- 16 Views
- 0 replies
- 0 kudos
Issue with Verification Code Input on Login
Hello, I hope you're well.​I'd like to report a bug encountered when entering the verification code to log into the platform. When I type the code without Caps Lock enabled, the input field displays the characters in uppercase, but the code isn't acc...
- 16 Views
- 0 replies
- 0 kudos
- 2717 Views
- 1 replies
- 0 kudos
Is there an automated way to strip notebook outputs prior to pushing to github?
We have a team that works in Azure Databricks on notebooks.We are not allowed to push any data to Github per corporate policy.Instead of everyone having to always remember to clear their notebook outputs prior to commit and push, is there a way this ...
- 2717 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
I am in the same situation. Curious to know how you resolved this issue.
- 0 kudos
- 62 Views
- 1 replies
- 0 kudos
DEV and PROD
"SELECT * FROM' data call on my table in PROD is giving all the rows of data (historical data), but a call on my table in DEV is giving me just one row of data (current one row of historical data). what could be the problem??
- 62 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Please don't cross post. Thanks, Louis.
- 0 kudos
- 71 Views
- 1 replies
- 0 kudos
dev and prod
"SELECT * FROM' data call on my table in PROD is giving all the rows of data, but a call on my table in DEV is giving me just one row of data. what could be the problem??
- 71 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Tell us more about your environment. Are you using Unity Catalog? What is the table format? What cloud platform are you on? More information is needed.
- 0 kudos
- 35 Views
- 1 replies
- 0 kudos
Is it possible restore a deleted catalog and schema
Is it possible restore a deleted catalog and schema.if CASCADE is used even though schemas and tables are present in catalog, catalog will be dropped.Is it possible to restore catalog or is possible to restrict the use of CACADE command.Thank you.
- 35 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
It is not possible to directly restore a deleted catalog or schema if they were dropped with the CASCADE option, especially in Databricks Unity Catalog. When a catalog or schema is dropped with CASCADE, all its dependent objects, such as schemas and ...
- 0 kudos
- 35 Views
- 1 replies
- 0 kudos
Is it possible expand/extend subnet CIDR of an existing azure databricks workspace
Is it possible expand/extend subnet CIDR of an existing azure databricks workspace. Currently our workspace is maxed out, Is it possible expand/extend subnet CIDR of an existing azure databricks workspace without having create a new one
- 35 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Yes, it is possible to expand or extend the subnet CIDR of an existing Azure Databricks workspace without creating a new one, but this capability is specifically applicable if the workspace is deployed with VNet injection. For workspaces that use V...
- 0 kudos
- 113 Views
- 6 replies
- 1 kudos
Resolved! GET /api/2.2/jobs/list Ordering
Hi there!I am calling the job list API (via the Python SDK):GET /api/2.2/jobs/listdocs.databricks.com/api/workspace/jobs/listDoes anyone know what ordering is applied / calculated for the list of jobs? Is it consistent or random?Is it by creation tim...
- 113 Views
- 6 replies
- 1 kudos
- 95 Views
- 3 replies
- 1 kudos
Azure Databricks Status
Dear all,I wanted to check if anyone implemented the solution of capturing information from Databricks status page in real-time 24x7 and load that into a log or table...https://learn.microsoft.com/en-us/azure/databricks/resources/statuswhat is the be...
- 95 Views
- 3 replies
- 1 kudos
Latest Reply
- 1 kudos
It seems that the webhook is the way!There is nothing about system status in Databricks REST API.There is nothing about system status in the System Tables schema.
- 1 kudos
- 408 Views
- 7 replies
- 0 kudos
Google Pub Sub and Delta live table
I am using delta live table and pub sub to ingest message from 30 different topics in parallel. I noticed that initialization time can be very long around 15 minutes. Does someone knows how to reduced initialization time in dlt ? Thanks You
- 408 Views
- 7 replies
- 0 kudos
Latest Reply
- 0 kudos
Classic clusters can take up to seven minutes to be acquired, configured, and deployed, with most of this time spent waiting for the cloud service to allocate virtual machines. In contrast, serverless clusters typically start in under eight seconds. ...
- 0 kudos
- 296 Views
- 1 replies
- 0 kudos
How to do NLP against PDFs in Databricks? Can be done in Snowflake very easily.
I’d love to build a quick "art of the possible" demo showing how easy it is to query unstructured PDFs using natural language. In Snowflake, I wired up a similar solution in ~2 hours just by following their tutorial guide.Does anyone know the best wa...
- 296 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
To query unstructured PDF files using natural language in Databricks, you can leverage an approach similar to the "Retrieval Augmented Generation (RAG) and DBRX" demo. Although the specific demo you referenced (https://notebooks.databricks.com/demos/...
- 0 kudos
- 61 Views
- 1 replies
- 0 kudos
Error when uploading MLFlow artifacts to DBFS
Hi everyone,I'm attempting to use MLFlow experiment tracking from a local machine, but I'm encountering difficulties in uploading artifacts.I've tried a sample code as simple as the following.import mlflow import os os.environ["DATABRICKS_HOST"] = "...
- 61 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
A couple things: 1. If you don't own the mlflow experiment you need ot have edit permissions on the experiment (needed for logging). Default artifact locations in DBFS (`dbfs:/databricks/mlflow-tracking/`) require explicit write permissions 2. The lo...
- 0 kudos
- 91 Views
- 3 replies
- 0 kudos
Request to reschedule Databricks Certified Generative AI Engineer Associate
Hi Team,I am writing to request assistance with rescheduling my exam for the Databricks Certified Generative AI Engineer Associate certification. My exam was originally scheduled for April 24, 2024, at 11 PM. However, when I attempted to access the e...
- 91 Views
- 3 replies
- 0 kudos
- 3442 Views
- 12 replies
- 5 kudos
Improved Duplicate Match Handling in Delta MERGE Operations
In DBR 16.1+, we’ve improved functionality of MERGE operations where multiple rows of the source dataset match the same row of the target Delta table, but only one row matches the WHEN MATCHED condition. In the past, these operations would fail with...
- 3442 Views
- 12 replies
- 5 kudos
Latest Reply
- 5 kudos
@BamBam you can email me use my first name at databricks.com
- 5 kudos
- 1754 Views
- 3 replies
- 2 kudos
How to search an array of words in a text field
Example:TABLE 1FIELD_TEXTI like salty food and Italian foodI have Italian foodbread, rice and beansmexican foodscoke, spritearray['italia', 'mex','coke']match TABLE1 X ARRAYResults:I like salty food and Italian foodI have Italian foodmexican foodsis ...
- 1754 Views
- 3 replies
- 2 kudos
Latest Reply
- 2 kudos
A simple wayselect FIELD_TEXT from TABLE 1 where FIELD_TEXT like 'italia' OR FIELD_TEXT like 'mex' OR FIELD_TEXT like 'coke'
- 2 kudos
Latest from our Blog
Databricks for Identity Systems - Part 3 (Group Membership)
Welcome back! In Part 1 of this series, we walked through the process of exporting our Okta Users to Databricks. In Part 2 of the series, we exported our Okta Groups and Group Rules. In this installme...
Databricks for Identity Systems - Part 2 (Groups and Group Rules)
Now that we have our Users table all set up (read Part 1 of this series, if you need to catch up), we need to add a bit more data to the mix. Groups The next logical piece of data, in my mind, is to g...
Databricks for Identity Systems - Part 1
Howdy! Welcome to my first blog! As an Identity System Administrator, I would say that a good portion of my day was spent configuring Okta and other identity systems. As I have moved more into an Iden...
Mastering DBR Migrations at Scale
In an era where data drives innovation and competitive advantage, protecting it becomes a non-negotiable priority. Particularly when it involves sensitive information, even minor lapses can translate ...
