Join APJ's premier AI/BI virtual challenge to solve real-world business problems, sharpen your skills, and compete for prizes using the Databricks Data Intelligence Platform. This challenge provides a unique opportunity to work together, apply AI-dri...
- 2917 Views
- 0 replies
- 0 kudos
Tuesday
We’re excited to invite you to our MONTHLY Databricks Get Started Days, a half-day virtual learning experience designed to jumpstart your journey with Databricks. This global event is tailored to equip you with essential data engineering and analysi...
- 2932 Views
- 0 replies
- 0 kudos
2 weeks ago
AI is transforming BI by changing the way organizations manage, analyze and get insights from their data. Everyone is rushing to democratize access to data through LLMs. But without the right foundation, you can’t get the most from AI. Our new appro...
- 3787 Views
- 0 replies
- 2 kudos
2 weeks ago
Join us for the return of the Virtual Learning Festival! Mark your calendars from 9 April - 30 April 2025! Upskill across data engineering, data analysis, machine learning, and generative AI. Join the thousands who have elevated their career with...
- 62314 Views
- 200 replies
- 37 kudos
03-03-2025
Be part of a global movement! Connect with 22,000 data enthusiasts at 700+ sessions, keynotes, and hands-on training at this year’s Data + AI Summit. Whether you’re into data intelligence, governance, AI, or data warehousing, this is your chance to l...
- 9374 Views
- 1 replies
- 3 kudos
02-19-2025
Community Activity
- 10 Views
- 0 replies
- 0 kudos
Navigating Online Review Strategies: Trustpilot and the Ethics of Reputation Building
Intro:Online reviews have become an essential part of the customer journey. Platforms like Trustpilot help people discover and trust businesses based on real customer experiences. But in competitive markets, some companies feel tempted to boost their...
- 10 Views
- 0 replies
- 0 kudos
- 24 Views
- 1 replies
- 0 kudos
Online Assessment Suspended
Hi @Cert-Team , @Cert-TeamOPS My exam was suspended stating behavioural/ environment issue.The proctor asked me to show my study desk, the room. I complied completely, giving a full tour of the room and my desk. No further explanations, and my test g...
- 24 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @Shuktiz , Did you raise ticket as below:The most important step is to file a ticket in our Help Center. Be sure to include the following details in your ticket:UsernameExam namePlease refer to this community link for more info: https://community...
- 0 kudos
- 14223 Views
- 4 replies
- 0 kudos
Resolved! How to enforce delta table column to have unique values?
Hi,I have defined a delta table with a primary key:%sql CREATE TABLE IF NOT EXISTS test_table_pk ( table_name STRING NOT NULL, label STRING NOT NULL, table_location STRING NOT NULL, CONSTRAINT test_table_pk_col PRIMARY KEY(table_name) ...
- 14223 Views
- 4 replies
- 0 kudos
Latest Reply
- 0 kudos
CREATE TABLE table_name (id_col1 BIGINT GENERATED ALWAYS AS IDENTITY,id_col2 BIGINT GENERATED ALWAYS AS IDENTITY (START WITH -1 INCREMENT BY 1),id_col3 BIGINT GENERATED BY DEFAULT AS IDENTITY,id_col4 BIGINT GENERATED BY DEFAULT AS IDENTITY (START WIT...
- 0 kudos
- 12 Views
- 0 replies
- 0 kudos
INVALID_PARAMETER_VALUE.LOCATION_OVERLAP when trying to copy from s3 location
Hi,Currently we are getting an issue when we try to copy a file from s3 location using dbutils.fs.cp, please see example below:source = s3://test-bucket/external/zones/{database_name}/{table_name}/test.csvdestination = s3://test-bucket/external/desti...
- 12 Views
- 0 replies
- 0 kudos
- 4269 Views
- 16 replies
- 4 kudos
spotify API get token - raw_input was called, but this frontend does not support input requests.
hello everyone, I'm trying use spotify's api to analyse my music data, but i'm receiving a error during authentication, specifically when I try get the token, above my code.Is it a databricks bug?pip install spotipyfrom spotipy.oauth2 import SpotifyO...
- 4269 Views
- 16 replies
- 4 kudos
Latest Reply
- 4 kudos
It looks like you're facing an authentication issue in an environment that doesn’t support interactive input. Databricks notebooks often don’t allow rawinput() calls. That’s why you’re getting the StdinNotImplementedError. To fix this, try generating...
- 4 kudos
- 18 Views
- 0 replies
- 0 kudos
Community Edition Data recovery
I was using Databricks Community Edition for some practice work, and I had created a few notebooks as part of my learning. However, when I recently tried to log in, I received a message saying, "We were not able to find a Community Edition." Now, non...
- 18 Views
- 0 replies
- 0 kudos
- 15 Views
- 0 replies
- 0 kudos
Completed Machine learning course
I have completed my course for Machine learning as part of Learning festival.
- 15 Views
- 0 replies
- 0 kudos
- 2619 Views
- 1 replies
- 2 kudos
Where PySpark and SparkSQL Fit Best in the Enterprise
1. Context2. Performance Differences Between SparkSQL and PySpark DataFrame API3. Functional Differences Between SparkSQL and PySpark4. Additional Considerations Based on Real-World Usage5. Conclusion 1. Context When building a data architecture, a...
- 2619 Views
- 1 replies
- 2 kudos
Latest Reply
- 2 kudos
Regarding complex transformations, we can use UDFs in SQL as well. So, we can still use sparkSQL, and delegate complex transformations into UDF.
- 2 kudos
- 64 Views
- 1 replies
- 0 kudos
Costs from cost managem azure portal are not allligned with costs calculated from usage system table
Hello,the costs regarding the databricks service from cost management in azure portal (45,869...) are not allligned with costs calculated from usage system table (75,34). The costs from the portal are filtered based on the desired period (usage_date ...
- 64 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @vziog, The Azure portal typically aggregates costs from various billing categories (such as DBUs, infrastructure, storage, and networking) based on usage logs and pricing. On the other hand, the query designed by you extracts detailed cost estima...
- 0 kudos
- 94 Views
- 1 replies
- 0 kudos
Databricks JDBC Error: Job Aborted Due to Stage Failure (Executor OOM - Error Code 52)
java.sql.SQLException: [Databricks][JDBCDriver](500051) ERROR processing query/statement. Error Code: 0, SQL state: null, Query: SELECT `ma***, Error message from Server: org.apache.hive.service.cli.HiveSQLException: Error running query: org.apache.s...
- 94 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
The executor does not seem to have enough memory to process the assigned tasks, OOM error.
- 0 kudos
- 46 Views
- 1 replies
- 0 kudos
Unexpected Behavior with Azure Databricks and Entra ID SCIM Integration
Hi everyone,I'm currently running some tests for a company that uses Entra ID as the backbone of its authentication system. Every employee with a corporate email address is mapped within the organization's Entra ID.Our company's Azure Databricks is c...
- 46 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello @antonionuzzo, This behavior is occurring because Azure Databricks allows workspace administrators to invite users from their organization's Entra ID directory into the Databricks workspace. This capability functions independently of whether th...
- 0 kudos
- 34 Views
- 1 replies
- 0 kudos
Pipelines are expected to have at least one table Error While running DLT pipeline
Error :Pipelines are expected to have at least one table defined butno tables were found in your pipeline I wrote simple code as phase 1 debug%sql CREATE OR REFRESH STREAMING TABLE test_table AS SELECT "hello" as greeting; Can u plz help what's wrong...
- 34 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
@vignesh22 In Delta Live Tables (DLT), magic commands (such as %sql) are not used. In a DLT pipeline, you need to write SQL code directly. Please try removing %sql and running the DLT pipeline again.
- 0 kudos
- 94 Views
- 3 replies
- 6 kudos
Resolved! SERVERLESS SQL WAREHOUSE
Hello All,I have two questions regarding the serverless SQL warehouse which are following:1.) If I create a small Serverless SQL Warehouse in Databricks that shows 12 DBUs/hour, will I be charged 12 DBUs even if I don’t run any queries in that hour? ...
- 94 Views
- 3 replies
- 6 kudos
Latest Reply
- 6 kudos
Shua42 hits the nail on the head. If I can be so bold as to summarize: You are only charged when the Warehouse is running regardless of how much or how little you use it. We do have an auto stop feature you can configure. Essentially, you set a time...
- 6 kudos
- 1472 Views
- 0 replies
- 0 kudos
With Academy Lab Subscription may i practice the collaboration with AWS?
Hi community,I will purchase the $200 subscription, but i want to know if i can practice with the collaboration with AWS, or it's only for serverless compute on databricks platform and i need to configure my personal aws?Thanks.
- 1472 Views
- 0 replies
- 0 kudos
- 22352 Views
- 9 replies
- 8 kudos
Does Databricks Academy not provide self-paced e-learning format of the Data Engineering with Databricks course?
Data engineering with Databricks - I want to learn with self-paced e-learning but I cannot find this course in the Academy catalog. Does Databricks Academy not provide self-paced e-learning format of the Data Engineering with Databricks course? I cou...
- 22352 Views
- 9 replies
- 8 kudos
Latest Reply
- 8 kudos
Welcome, Databricks Academy Learners! Embrace this dynamic space for collaborative growth in data engineering and analytics. Connect with us and beyond, share insights, and accelerate your learning journey through shared experiences and expert-led re...
- 8 kudos
Announcements
Latest from our Blog
From Data to Deals: Enhancing Seller Success with Databricks and Salesforce
Today, consumers leverage technology to enrich their shopping encounters through digital engagements, AI-driven interactions, and other digital channels before completing a purchase. On the other hand...
Evaluate Multi‑Turn Chatbots on Databricks with MLflow: A Step‑by‑Step Guide
Hi, I’m Debu. I spend a lot of my day building and stress‑testing LLM‑powered systems, and one lesson keeps coming back: if you don’t measure your agent’s behavior over an entire conversation, you’re ...
Securing Sensitive Data - Using Encryption, Secrets, and User Context Filter on PII Data
Problem Statement Let us start with setting some context. The problem statement that we are solving here is kept a bit generic as the solution can be applied to any similar situations. Consider a Payr...
Speeding up LLM inference by using model quantization in Databricks
The ever-increasing complexity of LLM models often comes at a steep cost: greater computational requirements, increased energy consumption, and slower inference times. Enter model quantization - a pow...
