Run Scala, Python and SQL workloads on shared, cost-efficient multi-user compute. We are thrilled to announce Unity Catalog Lakeguard, which allows you to run Apache Spark™ workloads in SQL, Python, and Scala with full data governance on the Databric...
- 210 Views
- 1 replies
- 1 kudos
Thursday
Get started with Databricks Asset Bundles (DABs) today! Simplify project management, versioning, testing, and deployment for your data and AI projects on the Databricks Platform. Embrace software engineering best practices with DABs—source control, c...
- 731 Views
- 0 replies
- 2 kudos
Wednesday
For a limited time, we're offering 50% off training and certification at Data + AI Summit with the following code: TRAIN50FOTY. This offer expires on May 3, 2024. Register today and add training. Read more here!
- 335 Views
- 0 replies
- 0 kudos
Tuesday
The Big Book of Generative AI brings together best practices and know-how for building production-quality GenAI applications. You’ll find technical content and code samples that will help you do everything from deploying your first application to bui...
- 626 Views
- 1 replies
- 2 kudos
Monday
Community Activity
- 6 Views
- 0 replies
- 0 kudos
metadata driven DQ validation for multiple tables dynamically
There are multiple tables in the config/metadata table. These tables need to bevalidated for DQ rules.1.Natural Key / Business Key /Primary Key cannot be null orblank.2.Natural Key/Primary Key cannot be duplicate.3.Join columns missing values4.Busine...
- 6 Views
- 0 replies
- 0 kudos
- 8 Views
- 0 replies
- 0 kudos
Unable to create serving endpoint for the huggingface model phi-3-mini-128k-instruct
#20 69.92 ERROR: Could not find a version that satisfies the requirement transformers==4.41.0.dev0 (from versions: 0.1, 2.0.0, 2.1.0, 2.1.1, 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.4.0, 2.4.1, 2.5.0, 2.5.1, 2.6.0, 2.7.0, 2.8.0, 2.9.0, 2.9.1, 2.10.0, 2.11.0, 3....
- 8 Views
- 0 replies
- 0 kudos
- 22 Views
- 0 replies
- 0 kudos
Financial Crime detection with the help of Apache Spark, Data Mesh and Data Lakes.
For those interested in Data Mesh and Data Lakes for FinCrime detection:Data mesh is a relatively new architectural concept for data management that emphasizes domain-driven data ownership and self-service data availability. It promotes the decentral...
- 22 Views
- 0 replies
- 0 kudos
- 17 Views
- 1 replies
- 0 kudos
Databricks Asset Bundle Workflow Redeployment Issue
Hello All,In my Databricks workflows, I have three tasks configured, with the final task set to run only if the condition "ALL_DONE" is met. During the first deployment, I observed that the dependency "ALL_DONE" was correctly assigned to the last tas...
- 17 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi, I've attached two snapshots, one from the initial deployment and another from the second redeployment, where no changes were made to the code.
- 0 kudos
- 25 Views
- 0 replies
- 0 kudos
E r e c t i n r e v i e w s - Does it works
E r e c t i n r e v i e w s provide a comprehensive overview of this sought-after male enhanc ement solution, detailing its impact on performance and satisfaction."Explore user testimonials and expert analysis to gauge the potency and reliabilityCli...
- 25 Views
- 0 replies
- 0 kudos
- 18 Views
- 0 replies
- 0 kudos
Top-Rated Testosil Reviews Unveiled! - Does testosil works?
Testosil reviews praise its effectiveness in boosting testosterone levels, enhancing vitality, and improving muscle mass. Users report experiencing increased energy and libido, alongside better athletic performance. The supplement is lauded for its n...
- 18 Views
- 0 replies
- 0 kudos
- 17695 Views
- 2 replies
- 3 kudos
What's the difference between a Global view and a Temp view?
The difference between Global and Temp is how the lifetime of the view is tied to the application:http://spark.apache.org/docs/latest/api/python/reference/api/pyspark.sql.DataFrame.createOrReplaceTempView.html?highlight=createorreplacetempview#pyspar...
- 17695 Views
- 2 replies
- 3 kudos
Latest Reply
- 3 kudos
Correct A Temp View is scoped to the SparkSession and dropped when that session closes. Each notebook runs in its own SparkSession. The Global Temp View is scoped to the cluster and dropped when the cluster re-starts or you drop it. ---------------...
- 3 kudos
- 44 Views
- 0 replies
- 0 kudos
Unable to mount GCS bucket with underscores in the name
I have two buckets with the same configurations and labels.One is named my-bucket and the other is my_bucket. I am able to mount my-bucket but get an opaque error message when trying to mount my_bucket. Is this known/expected behavior? Are underscore...
- 44 Views
- 0 replies
- 0 kudos
- 48 Views
- 0 replies
- 0 kudos
Databricks OutOfMemory error on code that previously worked without issue
I have a notebook in Azure Databricks that does some transformations on a bronze tier table and inserts the transformed data into a silver tier table. This notebook is used to do an initial load of the data from our existing system into our new datal...
- 48 Views
- 0 replies
- 0 kudos
- 8316 Views
- 7 replies
- 4 kudos
Resolved! Pass date value as parameter in Databricks SQL notebook
I want to pass yesterday date (In the example 20230115*.csv) in the csv file. Don't know how to create parameter and use it here.CREATE OR REPLACE TEMPORARY VIEW abc_delivery_logUSING CSVOPTIONS ( header="true", delimiter=",", inferSchema="true", pat...
- 8316 Views
- 7 replies
- 4 kudos
Latest Reply
- 4 kudos
@Kaniz @sp1 @Chaitanya_Raju @daniel_sahal Hi Everyone,I need the same scenario on SQL code, because my DBR cluster not allowed me to run python codeError: Unsupported cell during execution. SQL warehouses only support executing SQL cells.I appreciate...
- 4 kudos
- 35 Views
- 0 replies
- 0 kudos
On the fly transformations on DLT tables
Hi, I am loading data from a kinesis data stream using DLT. CREATE STREAMING TABLE Consumers_kinesis_2 ( ..., unbase64(data) String, ... ) AS SELECT * FROM STREAM read_kinesis (...) Is it possible to directly cast, unbase64, and/or transform the resu...
- 35 Views
- 0 replies
- 0 kudos
- 68 Views
- 0 replies
- 0 kudos
connect timed out error - Connecting to SQL Server from Databricks
I am getting a connect timed out error when attempting to access a sql server. I can successfully ping the server from Databricks. I have used the jdbc connection and the sqlserver included driver and both result in the same error. I have also attemp...
- 68 Views
- 0 replies
- 0 kudos
- 34 Views
- 0 replies
- 0 kudos
Testosil Revealed - Real Customer Experience and Fact Check
Testosil is a total game-changer! Since starting it, I feel like a whole new person. More energy, sharper focus, and just feeling great overall. What I love most is its natural ingredients, no worries about anything artificial. Taking it is a breeze,...
- 34 Views
- 0 replies
- 0 kudos
- 76 Views
- 0 replies
- 0 kudos
New admin question: How do you enable R on a existing cluster?
Hello Community. I have a user trying to use R and receive the error message illustrated on the attachment. I can't seem to find correct documentation on enabling R on an existing cluster. Would anyone be able to point me in the right direction? Than...
- 76 Views
- 0 replies
- 0 kudos
- 44 Views
- 0 replies
- 0 kudos
Testosil Reviews (2024): Is It Worth Buying Or Not
Testosil rocks! It's been a total game-changer for me. Since I started using it, I've felt a real boost in my sprightliness and focus. What's awesome is that it's made with natural ingredients, so I feel good about what I'm putting into my body. Plus...
- 44 Views
- 0 replies
- 0 kudos
Announcements
Latest from our Blog
How to enhance data protection in Azure Databricks with Confidential Computing
Proposed by Databricks in 2020, the Lakehouse architecture has increasingly been embraced by the industry in the past years. Databricks has since applied Generative AI and evolved the Databricks Lakeh...
Prepare Your Journey to Migrate from AWS Glue Data Catalog to Databricks Unity Catalog
In this blog, we will look at the migration from AWS Glue Data Catalog to Unity Catalog. We cover how to plan this migration as a step-by-step approach and emphasize meticulous planning, phased migrat...
Mastering the Spark UI
Almost anyone who uses Spark or Databricks is aware of the Spark UI, and we all know that it’s a super powerful tool in the right hands. It can reveal what’s going wrong and any inefficiencies in you...
Optimize costs for your Data and AI workloads with Azure and AWS Spot Instances
Introduction Cost optimisation remains a pivotal challenge for customers dealing with processing large volumes of data and machine learning model training at scale in the cloud. Spot instances have re...
