cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 
Introducing SAP Databricks

Today we are announcing a deep partnership with SAP which we think can be game changing for our industry. In short, it is the marriage between the most important business data for enterprises globally (SAP data) and the best data platform in the mark...

  • 654 Views
  • 0 replies
  • 5 kudos
Thursday
Databricks Clean Rooms: Now Generally Available on AWS and Azure

We’re thrilled to announce the General Availability (GA) of Databricks Clean Rooms on AWS and Azure, a significant step forward in enabling secure, privacy-centric data collaboration. Following the Public Preview launch in August, we've engaged close...

  • 411 Views
  • 0 replies
  • 1 kudos
2 weeks ago
Welcoming BladeBridge to Databricks: Accelerating Data Warehouse Migrations to Lakehouse

Databricks welcomes BladeBridge, a proven provider of AI-powered migration solutions for enterprise data warehouses. Together, Databricks and BladeBridge will help enterprises accelerate the work required to migrate legacy data warehouses like Oracle...

  • 514 Views
  • 0 replies
  • 1 kudos
2 weeks ago
Serverless Compute for Notebooks, Workflows and Pipelines is now Generally Available on Google Cloud

In the rapidly evolving landscape of data engineering and analytics, speed, scalability, and simplicity are invaluable. Serverless compute addresses these needs by eliminating the complexity of managing infrastructure, allowing you to focus on buildi...

  • 458 Views
  • 0 replies
  • 0 kudos
2 weeks ago
Securely share data, analytics and AI

Reduce the cost of sharing across platforms Gartner predicts that CDOs who have successfully executed data sharing initiatives are 1.7 times more effective in showing business value and ROI from their data analytics strategy. Databricks provides an ...

  • 748 Views
  • 0 replies
  • 3 kudos
3 weeks ago
Check Out the Latest Videos on DatabricksTV

We are thrilled to introduce DatabricksTV – your go-to hub for all things Databricks! DatabricksTV is packed with insightful videos from a diverse range of creators, offering you the latest tips, tutorials, and deep dives into the Databricks Platform...

  • 1891 Views
  • 0 replies
  • 3 kudos
07-30-2024

Community Activity

SubbuRavuri007
by > New Contributor
  • 0 Views
  • 0 replies
  • 0 kudos

My Exam got suspended after completion of 75% of exam

Hi Team,I have scheduled Data Engineering Associate exam on Feb 13 2023. My exam got suspended 3 times after attempting 35 question and after contacting the kryterion support team they relaunched the exam multiple times and after logging again it got...

  • 0 Views
  • 0 replies
  • 0 kudos
Cert-Team
by Databricks Employee
  • 29417 Views
  • 92 replies
  • 8 kudos

Suspended Exam? Don't worry! Our support team can help!

Taking exams is so stressful already, and if your exam gets suspended it is extra stressful. Don't worry! Our support team is awesome and will help you get back on track. The most important step is to file a ticket in our Help Center. Be sure to inc...

  • 29417 Views
  • 92 replies
  • 8 kudos
Latest Reply
farishamp1920
  • 8 kudos

Hi Team,My Associate Developer for Apache Spark 3.0 - Python exam got suspended saying suspected violation even though nothing of that sort happened and I was alone and focusing only on the exam. I showed the entire room and walls to the proctor. Sti...

  • 8 kudos
91 More Replies
Shajiya-123
by > New Contributor
  • 69 Views
  • 2 replies
  • 0 kudos

My Exam got suspended

Username - Shajiya-123Email id - shajiyashaik839@gmail.comTest name - Databricks Certified Generative AI Engineer AssociateI have experienced a pathetic experience today while giving my Data bricks certification exam for data analyst associate. Initi...

  • 69 Views
  • 2 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

@Cert-Team / @Cert-TeamOPS - could you please take a look?

  • 0 kudos
1 More Replies
skd217
by > New Contributor
  • 98 Views
  • 3 replies
  • 0 kudos

Is there any way to connect polaris catalog from unity catalog?

Hi databricks community, I'd like to access data managed by polaris catalog through unity catalog to manage all data one place. But is there any way to connect? (I could access the data with all-purpose cluster without unity catalog.)

  • 98 Views
  • 3 replies
  • 0 kudos
Latest Reply
chandu402240
  • 0 kudos

Can you provide info on how you access polaris catalog data from databricks cluster (without UC) ? any blog ?

  • 0 kudos
2 More Replies
shan-databricks
by > New Contributor
  • 81 Views
  • 2 replies
  • 0 kudos

Databricks Workflow Orchestration

I have 50 tables and will increase gradually, so I want to create a single workflow to orchestrate the job and run it table-wise. Is there an option to do this in Databricks workflow?

  • 81 Views
  • 2 replies
  • 0 kudos
Latest Reply
Edthehead
Contributor III
  • 0 kudos

Breakup these 50 tables logically or functionally and place them in their own workflows. A good strategy would be to group tables that are dependent in the same workflow. Then use a master workflow to trigger each child workflow. So it will be like a...

  • 0 kudos
1 More Replies
subhas_1729
by > New Contributor II
  • 13 Views
  • 1 replies
  • 0 kudos

Dashboard

Hi       I want to design a dashboard that will show some variables of Spark-UI. Is it possible to access Spark-UI variables from my spark program. 

  • 13 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @subhas_1729, You can achieve this by leveraging Spark's monitoring and instrumentation APIs. Spark provides metrics that can be accessed through the SparkListener interface as well as the REST API. The SparkListener interface allows you to receiv...

  • 0 kudos
nathanmle
by > New Contributor
  • 21 Views
  • 0 replies
  • 0 kudos

Where to find Jupyter Notebook course materials for Get Started with Databricks for Generative AI

Hello, I can't seem to find any way to gain access to the Jupyter Notebook demo source of "Get Started with Databricks for Generative AI" course.  Please help.  Thank you kindly in advance. 

  • 21 Views
  • 0 replies
  • 0 kudos
pdemeulenaer
by > New Contributor II
  • 1070 Views
  • 1 replies
  • 1 kudos

Databricks asset bundles dependencies

Is anyone aware of a way to include a requirements.txt within the job definition of a databricks asset bundle? Documentation mentions how to have dependencies in workspace files, or Unity Catalog volumes, but I wanted to ask if it is possible to decl...

Community Platform Discussions
databricksassetbundles
Dependency
  • 1070 Views
  • 1 replies
  • 1 kudos
Latest Reply
cleversuresh
New Contributor III
  • 1 kudos

I have the same question.

  • 1 kudos
pra18
by > New Contributor
  • 51 Views
  • 1 replies
  • 0 kudos

Handling Binary Files Larger than 2GB in Apache Spark

I'm trying to process large binary files (>2GB) in Apache Spark, but I'm running into the following error:File format is : .mf4 (Measurement Data Format) org.apache.spark.SparkException: The length of ... is 14749763360, which exceeds the max length ...

  • 51 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @pra18, You can split and load the binary files using split command like this. ret = os.system("split -b 4020000 -a 4 -d large_data.dat large_data.dat_split_")

  • 0 kudos
yathish
by > New Contributor
  • 59 Views
  • 1 replies
  • 0 kudos

upstream request timeout in databricks apps when using databricks sql connector

Hi, i am building an application in Databricks apps, Sometimes when i try to fetch data using Databricks SQL connector in an API, it takes time to hit the SQL warehouse and if the time exceeds more than 60 seconds it gives upstream timeout error. I h...

  • 59 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @yathish, When you submit the query to the warehouse, is the warehouse up and running already? and are you using classic, pro or serverless warehouse?

  • 0 kudos
miki1999
by > New Contributor
  • 16 Views
  • 1 replies
  • 0 kudos

Problem connecting vscode with databricks

I have a problem connecting VSCode with databricks. I am doing all the steps to add databricks in VSCode but after the last step I get this error in VSCode:“Error connecting to the workspace: "Can't set configuration 'authProfile' without selecting a...

  • 16 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hello @miki1999, Try removing any comments (lines starting with ‘;’) from the configuration file. • Open VSCode and go to the Extensions view (Ctrl+Shift+X). • Search for “Databricks” and ensure you have the latest version installed.

  • 0 kudos
Anushree_Tatode
by Contributor III
  • 802 Views
  • 4 replies
  • 3 kudos

Would You Join Us for a Local Meetup?

If we organized a local meetup in your area, would you attend? This is your chance to connect with fellow data enthusiasts, share knowledge, and build a stronger community face-to-face. Let us know if you'd be interested in joining, and where you'd l...

  • 802 Views
  • 4 replies
  • 3 kudos
Latest Reply
aditya_amb
Visitor
  • 3 kudos

@Anushree_Tatode  yes sure would love to join.

  • 3 kudos
3 More Replies
pintu_tagi
by > New Contributor II
  • 25 Views
  • 0 replies
  • 0 kudos

Will there be Elevate Training Series in 2025 ?

Hi @Cert-TeamOPS , I wanted to check if there are any plans to conduct the 'Elevate Training Series - Data Engineering with Databricks' in 2025. This training would be incredibly valuable and greatly support certification efforts. Thanks!

  • 25 Views
  • 0 replies
  • 0 kudos
HeathDG1
by > New Contributor
  • 53 Views
  • 1 replies
  • 0 kudos

Row filtering based on condition not working

Hi-We have a delta table in our unity catalog called dream_team.stern_portfolio.location_info.We are trying to use row level security to filter our data based on a users group membership. This way when users look at out dashboard they can only see th...

  • 53 Views
  • 1 replies
  • 0 kudos
Latest Reply
Isi
New Contributor II
  • 0 kudos

Hey @HeathDG1 I think your function isn’t behaving the way you expect because of how the logic is set up:•If the user is in Stern MA, they only see rows where state = 'MA'  (which is good).•BUT for everyone else, the function returns true, meaning th...

  • 0 kudos
William_Scardua
by > Valued Contributor
  • 9487 Views
  • 3 replies
  • 1 kudos

How to read data from Azure Log Analitycs ?

Hi guys,I need to read data from Azure Log Analitycs Workspace directaly, have any idea ?thank you

  • 9487 Views
  • 3 replies
  • 1 kudos
Latest Reply
alexott
Databricks Employee
  • 1 kudos

You can use Kusto Spark connector for that: https://github.com/Azure/azure-kusto-spark/blob/master/docs/KustoSource.md#source-read-command It heavily depends on how you access data, there could be a need for using ADX cluster for it: https://learn.mi...

  • 1 kudos
2 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Top Kudoed Authors
Read Databricks Data Intelligence Platform reviews on G2

Latest from our Blog