Get started with Databricks Asset Bundles (DABs) today! Simplify project management, versioning, testing, and deployment for your data and AI projects on the Databricks Platform. Embrace software engineering best practices with DABs—source control, c...
- 82 Views
- 0 replies
- 0 kudos
3 hours ago
For a limited time, we're offering 50% off training and certification at Data + AI Summit with the following code: TRAIN50FOTY. This offer expires on May 3, 2024. Register today and add training. Read more here!
- 161 Views
- 0 replies
- 0 kudos
yesterday
The Big Book of Generative AI brings together best practices and know-how for building production-quality GenAI applications. You’ll find technical content and code samples that will help you do everything from deploying your first application to bui...
- 363 Views
- 1 replies
- 2 kudos
Monday
Get your first look at DBRX April 25, 2024 | 8 AM PT If you’re using off-the-shelf LLMs to build GenAI applications, you’re probably struggling with quality, privacy and governance issues. What you need is a way to cost-effectively build a custom LLM...
- 3252 Views
- 3 replies
- 3 kudos
3 weeks ago
Community Activity
- 19 Views
- 0 replies
- 0 kudos
Menophix: Is This a Real, Risk-Free Formula? Verified Customer Reviews! (UK)
Sale is Live Now: https://www.biowellgrade.com/menophix-detox-reviews/A called MenoPhix aids women in controlling the symptoms of menopause. Its organic makeup encourages the equilibrium of hormones in the body. This nutritional supplement is safe fo...
- 19 Views
- 0 replies
- 0 kudos
- 5170 Views
- 6 replies
- 7 kudos
Resolved! Installing libraries on job clusters
Simple question : what is the way to go to install libraries on job clusters ? There does not seem to be a "Libraries" tab on the UI as opposed to regular clusters. Does it mean that the only option is to use init scripts ?
- 5170 Views
- 6 replies
- 7 kudos
- 82 Views
- 0 replies
- 0 kudos
Announcing the General Availability of Databricks Asset Bundles
Get started with Databricks Asset Bundles (DABs) today! Simplify project management, versioning, testing, and deployment for your data and AI projects on the Databricks Platform. Embrace software engineering best practices with DABs—source control, c...
- 82 Views
- 0 replies
- 0 kudos
- 123 Views
- 0 replies
- 0 kudos
Hiring Databricks Data Architect roles
Hi,I am a recruiter and I am looking for places to post some data bricks I have coming out. I have several fully remote, high-level data databricks, architect roles. Of course I will post to LinkedIn, but I was just curious if there are any other pla...
- 123 Views
- 0 replies
- 0 kudos
- 58 Views
- 1 replies
- 0 kudos
Cached Views in MERGE INTO operation
Hi everyone!I want to use in-memory cached views in a merge into operation, but I am not entirely sure if the exactly saved in-memory view is used in this operation or not.So, suppose I have a table named table_1 and a cached view named cached_view_1...
- 58 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
@deng_dev - Are you using external metastore by any chance. From the physical plan, we could see the catalog`.`db`.`table_1` is not cached. If it is glue catalog, then caching can be enabled based on the below configs in the article below https://do...
- 0 kudos
- 306 Views
- 5 replies
- 3 kudos
Azure Databricks with standard private link cluster event log error: "Metastore down"...
We have Azure Databricks with standard private link (back-end and front-end private link).We are able to successfully attach a Databricks workspace to the Databricks metastore (ADLS Gen2 storage).However, when trying to create tables in a catalog in ...
- 306 Views
- 5 replies
- 3 kudos
Latest Reply
- 3 kudos
can confirm that the approach will solve your error. Ran into a similar issue a while back.
- 3 kudos
- 31 Views
- 0 replies
- 0 kudos
Summit Watch Party
Hey hey Stockholm,Anyone keen on getting together to watch keynotes of the summit live? (for those of us who can't be there in person of course?/Karol
- 31 Views
- 0 replies
- 0 kudos
- 38 Views
- 0 replies
- 0 kudos
UC Enablement in Databricks workspace for metastores in different region - Azure Cloud
SpoilerWe have a UC enabled on Databricks where we have two Metastores,Metastore in West Europe regionMetastore in North America regionAnd Databricks workspace.Workspace in West Europe regionAnd we want to have a single Databricks workspace for both ...
- 38 Views
- 0 replies
- 0 kudos
- 35 Views
- 0 replies
- 0 kudos
Unity Catalog Enabled Clusters using PrivateNIC
Hello,When reviewing the VM settings for Databricks worker VMs, we can see that there are two(2) NICs.A primary ( PublicNIC (primary)) and a secondary (PrivateNIC (primary)).The workers VM is always assigned the PublicNIC and this is reachable from w...
- 35 Views
- 0 replies
- 0 kudos
- 42 Views
- 0 replies
- 1 kudos
How many times can we reschedule exam?
I would like to know how many time we can reschedule or postpone the certification exam.
- 42 Views
- 0 replies
- 1 kudos
- 4810 Views
- 15 replies
- 8 kudos
Resolved! What are some best practices for CICD?
A number of people have questions on using Databricks in a productionalized environment. What are the best practices to enable CICD automation?
- 4810 Views
- 15 replies
- 8 kudos
Latest Reply
- 8 kudos
Any leads/posts for Databricks CI/CD integration with Bitbucket pipeline. I am facing the below error while I creation my CICD pipeline pipelines:branches:master:- step:name: Deploy Databricks Changesimage: docker:19.03.12services:- dockerscript:# U...
- 8 kudos
- 43 Views
- 0 replies
- 0 kudos
Create Databricks model serving endpoint in Azure DevOps yaml
Hello,I need to create and destroy a model endpoint as part of CI/CD. I tried with mlflow deployments create-endpoint, giving databricks as --target however it errors saying that --endpoint is not a known argument when clearly --endpoint is required....
- 43 Views
- 0 replies
- 0 kudos
- 31 Views
- 0 replies
- 0 kudos
Error during deserializing protobuf data
I am receiving protobuf data in a json attribute and along with it I receive a descriptor file.I am using from_protobuf to deserialize the data as below,It works most of the time but giving error when there are some recursive fields within the protob...
- 31 Views
- 0 replies
- 0 kudos
- 405 Views
- 2 replies
- 0 kudos
Resolved! Not able to set run_as service_principal_name
I'm trying to run: databricks bundle deploy -t prod --profile PROD_Service_Principal My bundle looks: bundle: name: myproject include: - resources/jobs/bundles/*.yml targets: # The 'dev' target, for development purposes. This target is the de...
- 405 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
In my case I replaced alias PROD_Service_Principal with id c250831b-5a2a-4461-a855-83b9102f797e and it works. Not intuitive, probably this is a bug in CLI ot bundles service_principal_name: c250831b-5a2a-4461-a855-83b9102f797e
- 0 kudos
- 137 Views
- 0 replies
- 0 kudos
column "storage_sub_directory" is now always NULL in system.information_schema.tables
Hello,I am running a job that depends on the information provided in column storage_sub_directory in system.information_schema.tables .... and it worked until 1-2 weeks ago.Now I discovered in the doc that this column is deprecated and always null , ...
- 137 Views
- 0 replies
- 0 kudos
Announcements
Latest from our Blog
Prepare Your Journey to Migrate from AWS Glue Data Catalog to Databricks Unity Catalog
In this blog, we will look at the migration from AWS Glue Data Catalog to Unity Catalog. We cover how to plan this migration as a step-by-step approach and emphasize meticulous planning, phased migrat...
Mastering the Spark UI
Almost anyone who uses Spark or Databricks is aware of the Spark UI, and we all know that it’s a super powerful tool in the right hands. It can reveal what’s going wrong and any inefficiencies in you...
Optimize costs for your Data and AI workloads with Azure and AWS Spot Instances
Introduction Cost optimisation remains a pivotal challenge for customers dealing with processing large volumes of data and machine learning model training at scale in the cloud. Spot instances have re...
MLOps Gym - Beginners Guide to Cluster Configuration for MLOps
When setting up compute, there are many options and knobs to tweak and tune, and it can get quite overwhelming very quickly. To help you with optimally configuring your clusters, we have broken dow...
