cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcing the APJ Databricks Smart Business Insights Challenge: Empowering Data-Driven Decision Mak

Join APJ's premier AI/BI virtual challenge to solve real-world business problems, sharpen your skills, and compete for prizes using the Databricks Data Intelligence Platform. This challenge provides a unique opportunity to work together, apply AI-dri...

  • 2917 Views
  • 0 replies
  • 0 kudos
Tuesday
🚀 Monthly Databricks Get Started Days – Accelerate Your Learning Journey! 🚀

We’re excited to invite you to our MONTHLY Databricks Get Started Days, a half-day virtual learning experience designed to jumpstart your journey with Databricks. This global event is tailored to equip you with essential data engineering and analysi...

  • 2932 Views
  • 0 replies
  • 0 kudos
2 weeks ago
Business Intelligence in the Era of AI

AI is transforming BI by changing the way organizations manage, analyze and get insights from their data. Everyone is rushing to democratize access to data through LLMs. But without the right foundation, you can’t get the most from AI. Our new appro...

  • 3787 Views
  • 0 replies
  • 2 kudos
2 weeks ago
Virtual Learning Festival: 9 April - 30 April

Join us for the return of the Virtual Learning Festival! Mark your calendars from 9 April - 30 April 2025! Upskill across data engineering, data analysis, machine learning, and generative AI. Join the thousands who have elevated their career with...

  • 62314 Views
  • 200 replies
  • 37 kudos
03-03-2025
Data + AI Summit 2025 — registration now open!

Be part of a global movement! Connect with 22,000 data enthusiasts at 700+ sessions, keynotes, and hands-on training at this year’s Data + AI Summit. Whether you’re into data intelligence, governance, AI, or data warehousing, this is your chance to l...

  • 9374 Views
  • 1 replies
  • 3 kudos
02-19-2025

Community Activity

marymorrison
by > Visitor
  • 10 Views
  • 0 replies
  • 0 kudos

Navigating Online Review Strategies: Trustpilot and the Ethics of Reputation Building

Intro:Online reviews have become an essential part of the customer journey. Platforms like Trustpilot help people discover and trust businesses based on real customer experiences. But in competitive markets, some companies feel tempted to boost their...

  • 10 Views
  • 0 replies
  • 0 kudos
Shuktiz
by > Visitor
  • 24 Views
  • 1 replies
  • 0 kudos

Online Assessment Suspended

Hi @Cert-Team , @Cert-TeamOPS My exam was suspended stating behavioural/ environment issue.The proctor asked me to show my study desk, the room. I complied completely, giving a full tour of the room and my desk. No further explanations, and my test g...

  • 24 Views
  • 1 replies
  • 0 kudos
Latest Reply
sridharplv
Contributor
  • 0 kudos

Hi @Shuktiz , Did you raise ticket as below:The most important step is to file a ticket in our Help Center. Be sure to include the following details in your ticket:UsernameExam namePlease refer to this community link  for more info: https://community...

  • 0 kudos
Mado
by > Valued Contributor II
  • 14223 Views
  • 4 replies
  • 0 kudos

Resolved! How to enforce delta table column to have unique values?

Hi,I have defined a delta table with a primary key:%sql   CREATE TABLE IF NOT EXISTS test_table_pk ( table_name STRING NOT NULL, label STRING NOT NULL, table_location STRING NOT NULL,   CONSTRAINT test_table_pk_col PRIMARY KEY(table_name) ...

image
  • 14223 Views
  • 4 replies
  • 0 kudos
Latest Reply
SibbirSihan
  • 0 kudos

CREATE TABLE table_name (id_col1 BIGINT GENERATED ALWAYS AS IDENTITY,id_col2 BIGINT GENERATED ALWAYS AS IDENTITY (START WITH -1 INCREMENT BY 1),id_col3 BIGINT GENERATED BY DEFAULT AS IDENTITY,id_col4 BIGINT GENERATED BY DEFAULT AS IDENTITY (START WIT...

  • 0 kudos
3 More Replies
dplatform_user
by > Visitor
  • 12 Views
  • 0 replies
  • 0 kudos

INVALID_PARAMETER_VALUE.LOCATION_OVERLAP when trying to copy from s3 location

Hi,Currently we are getting an issue when we try to copy a file from s3 location using dbutils.fs.cp, please see example below:source = s3://test-bucket/external/zones/{database_name}/{table_name}/test.csvdestination = s3://test-bucket/external/desti...

  • 12 Views
  • 0 replies
  • 0 kudos
AlexSantiago
by > New Contributor II
  • 4269 Views
  • 16 replies
  • 4 kudos

spotify API get token - raw_input was called, but this frontend does not support input requests.

hello everyone, I'm trying use spotify's api to analyse my music data, but i'm receiving a error during authentication, specifically when I try get the token, above my code.Is it a databricks bug?pip install spotipyfrom spotipy.oauth2 import SpotifyO...

  • 4269 Views
  • 16 replies
  • 4 kudos
Latest Reply
Jenny101
Visitor
  • 4 kudos

It looks like you're facing an authentication issue in an environment that doesn’t support interactive input. Databricks notebooks often don’t allow rawinput() calls. That’s why you’re getting the StdinNotImplementedError. To fix this, try generating...

  • 4 kudos
15 More Replies
saikrishna1020
by > New Contributor
  • 18 Views
  • 0 replies
  • 0 kudos

Community Edition Data recovery

I was using Databricks Community Edition for some practice work, and I had created a few notebooks as part of my learning. However, when I recently tried to log in, I received a message saying, "We were not able to find a Community Edition." Now, non...

  • 18 Views
  • 0 replies
  • 0 kudos
faridaeasmin
by > New Contributor II
  • 15 Views
  • 0 replies
  • 0 kudos

Completed Machine learning course

I have completed my course for Machine learning as part of Learning festival.

  • 15 Views
  • 0 replies
  • 0 kudos
jmeulema
by Databricks Employee
  • 2619 Views
  • 1 replies
  • 2 kudos

Where PySpark and SparkSQL Fit Best in the Enterprise

1. Context2. Performance Differences Between SparkSQL and PySpark DataFrame API3. Functional Differences Between SparkSQL and PySpark4. Additional Considerations Based on Real-World Usage5. Conclusion   1. Context When building a data architecture, a...

Screenshot 2025-02-24 at 10.55.19.png
  • 2619 Views
  • 1 replies
  • 2 kudos
Latest Reply
inba
New Contributor
  • 2 kudos

Regarding complex transformations, we can use UDFs in SQL as well. So, we can still use sparkSQL, and delegate complex transformations into UDF.

  • 2 kudos
vziog
by > New Contributor
  • 64 Views
  • 1 replies
  • 0 kudos

Costs from cost managem azure portal are not allligned with costs calculated from usage system table

Hello,the costs regarding the databricks service from cost management in azure portal (45,869...) are not allligned with costs calculated from usage system table (75,34). The costs from the portal are filtered based on the desired period (usage_date ...

  • 64 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @vziog, The Azure portal typically aggregates costs from various billing categories (such as DBUs, infrastructure, storage, and networking) based on usage logs and pricing. On the other hand, the query designed by you extracts detailed cost estima...

  • 0 kudos
AP01
by > New Contributor
  • 94 Views
  • 1 replies
  • 0 kudos

Databricks JDBC Error: Job Aborted Due to Stage Failure (Executor OOM - Error Code 52)

java.sql.SQLException: [Databricks][JDBCDriver](500051) ERROR processing query/statement. Error Code: 0, SQL state: null, Query: SELECT `ma***, Error message from Server: org.apache.hive.service.cli.HiveSQLException: Error running query: org.apache.s...

Warehousing & Analytics
Databricks JDBC SparkSQL OOM HiveThriftServer Error500051
Databricks SQL
JDBC Driver
SparkSQL
sql
  • 94 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

The executor does not seem to have enough memory to process the assigned tasks, OOM error.

  • 0 kudos
antonionuzzo
by > New Contributor II
  • 46 Views
  • 1 replies
  • 0 kudos

Unexpected Behavior with Azure Databricks and Entra ID SCIM Integration

Hi everyone,I'm currently running some tests for a company that uses Entra ID as the backbone of its authentication system. Every employee with a corporate email address is mapped within the organization's Entra ID.Our company's Azure Databricks is c...

  • 46 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hello @antonionuzzo, This behavior is occurring because Azure Databricks allows workspace administrators to invite users from their organization's Entra ID directory into the Databricks workspace. This capability functions independently of whether th...

  • 0 kudos
vignesh22
by > New Contributor
  • 34 Views
  • 1 replies
  • 0 kudos

Pipelines are expected to have at least one table Error While running DLT pipeline

Error :Pipelines are expected to have at least one table defined butno tables were found in your pipeline I wrote simple code as phase 1 debug%sql CREATE OR REFRESH STREAMING TABLE test_table AS SELECT "hello" as greeting; Can u plz help what's wrong...

  • 34 Views
  • 1 replies
  • 0 kudos
Latest Reply
Takuya-Omi
Valued Contributor III
  • 0 kudos

@vignesh22 In Delta Live Tables (DLT), magic commands (such as %sql) are not used. In a DLT pipeline, you need to write SQL code directly. Please try removing %sql and running the DLT pipeline again.

  • 0 kudos
Punit_Prajapati
by > New Contributor
  • 94 Views
  • 3 replies
  • 6 kudos

Resolved! SERVERLESS SQL WAREHOUSE

Hello All,I have two questions regarding the serverless SQL warehouse which are following:1.) If I create a small Serverless SQL Warehouse in Databricks that shows 12 DBUs/hour, will I be charged 12 DBUs even if I don’t run any queries in that hour? ...

  • 94 Views
  • 3 replies
  • 6 kudos
Latest Reply
BigRoux
Databricks Employee
  • 6 kudos

Shua42 hits the nail on the head. If I can be so bold as to summarize: You are only charged when the Warehouse is running regardless of how much or how little you use it.  We do have an auto stop feature you can configure. Essentially, you set a time...

  • 6 kudos
2 More Replies
MingOnCloud
by > New Contributor
  • 1472 Views
  • 0 replies
  • 0 kudos

With Academy Lab Subscription may i practice the collaboration with AWS?

Hi community,I will purchase the $200 subscription, but i want to know if i can practice with the collaboration with AWS, or it's only for serverless compute on databricks platform and i need to configure my personal aws?Thanks.

  • 1472 Views
  • 0 replies
  • 0 kudos
mayuri_s
by > Contributor
  • 22352 Views
  • 9 replies
  • 8 kudos

Does Databricks Academy not provide self-paced e-learning format of the Data Engineering with Databricks course?

Data engineering with Databricks - I want to learn with self-paced e-learning but I cannot find this course in the Academy catalog. Does Databricks Academy not provide self-paced e-learning format of the Data Engineering with Databricks course? I cou...

Catalog contains no course for self-paced e-learning for Data engineering
  • 22352 Views
  • 9 replies
  • 8 kudos
Latest Reply
petergrew
New Contributor
  • 8 kudos

Welcome, Databricks Academy Learners! Embrace this dynamic space for collaborative growth in data engineering and analytics. Connect with us and beyond, share insights, and accelerate your learning journey through shared experiences and expert-led re...

  • 8 kudos
8 More Replies
Welcome to the Databricks Community!

Once you are logged in, you will be ready to post content, ask questions, participate in discussions, earn badges and more.

Spend a few minutes exploring Get Started Resources, Learning Paths, Certifications, and Platform Discussions.

Connect with peers through User Groups and stay updated by subscribing to Events. We are excited to see you engage!

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Read Databricks Data Intelligence Platform reviews on G2

Latest from our Blog