cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

TravisBrowne
by New Contributor
  • 3231 Views
  • 1 replies
  • 0 kudos

POC Comparison: Databricks vs AWS EMR

Hello,I need some assistance with a comparison between Databricks and AWS EMR. We've been evaluating the Databricks Data Intelligence platform for a client and found it to be significantly more expensive than AWS EMR. I understand the challenge in ma...

  • 3231 Views
  • 1 replies
  • 0 kudos
Latest Reply
Schofield
New Contributor III
  • 0 kudos

I can't say for sure what your exact limitation is going to be but two things I have come across in EMR bake offs are:1) bottleneck in the network throughput.  Verify the S3 reads and writes are happening at similar throughput rates.  This is a cloud...

  • 0 kudos
TravisBrowne
by New Contributor
  • 613 Views
  • 0 replies
  • 0 kudos

Linking Workspace IDs to Names in Billing Schema

Hi everyone,We recently enabled UC and the Billing system table to monitor our usage and costs. We've successfully set up a dashboard to track these metrics for each workspace. The usage table includes the workspace_id, but I'm having trouble finding...

  • 613 Views
  • 0 replies
  • 0 kudos
mano7438
by New Contributor III
  • 59876 Views
  • 5 replies
  • 4 kudos

How to create temporary table in databricks

Hi Team,I have a requirement where I need to create temporary table not temporary view.Can you tell me how to create temporary table in data bricks ?

  • 59876 Views
  • 5 replies
  • 4 kudos
Latest Reply
NandiniN
Databricks Employee
  • 4 kudos

I see, thanks for sharing, can you mark the solution which worked for you @abueno as Accepted.

  • 4 kudos
4 More Replies
Phani1
by Valued Contributor II
  • 730 Views
  • 1 replies
  • 0 kudos

ConcurrentAppendException

Hi Team,We're dealing with a concurrency issue when we attempt to run multiple jobs at the same time, and we're still having the same problem even after using partition and liquid clustering features. Now we're making sure to have the right where con...

  • 730 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kannathasan
New Contributor III
  • 0 kudos

Option 3 Bit complicated process but works well: Considering you will do Update, Insert and Delete operation on the same table at the same time by using multiple job.1. Create a Dummy table with the target table schema with additional column called O...

  • 0 kudos
Phani1
by Valued Contributor II
  • 577 Views
  • 0 replies
  • 0 kudos

Job Cluster best practices for production workloads

 Hi All,Can you please share the best practices for job clusters configurations for production workloadsand which is good when compared to serverless and job cluster in production in terms of cost and performance?Regards,Phani 

  • 577 Views
  • 0 replies
  • 0 kudos
sanjay
by Valued Contributor II
  • 1614 Views
  • 1 replies
  • 1 kudos

how to reduce scale to zero time in MLFlow Serving

Hi,I am deploying MLflow models using Databrick serverless serving but seems servers scale down to 0 only after 30 minute of inactivity. Is there any way to reduce this time?Also, Is it possible to deploy multiple models under single endpoint. I want...

  • 1614 Views
  • 1 replies
  • 1 kudos
Latest Reply
Walter_C
Databricks Employee
  • 1 kudos

Regarding your first question about reducing the scale-down time of Databricks serverless serving, currently, the system is designed to scale down to zero after 30 minutes of inactivity. This is to ensure that instances are kept warm to handle any su...

  • 1 kudos
qs
by New Contributor III
  • 1746 Views
  • 3 replies
  • 0 kudos

Resolved! Help trying to use Python in Databricks

I am watching an Introduction to Databricks - running Python scripts and I don't see where I create a notebook in my databricks instance to even select Python as the Default Language.Is it possible, my level of databricks isn't allowing me to run Pyt...

  • 1746 Views
  • 3 replies
  • 0 kudos
Latest Reply
qs
New Contributor III
  • 0 kudos

I think its a settings/permission issue.

  • 0 kudos
2 More Replies
Chinu
by New Contributor III
  • 3656 Views
  • 3 replies
  • 0 kudos

Tableau Desktop connection error from Mac M1

Hi, Im getting the below error while connecting SQL Warehouse from the tableau desktop. I installed the latest ODBC drivers (2.7.5) but I can confirm that the driver name is different. From the error message I see libsparkodbc_sbu.dylib but in my lap...

  • 3656 Views
  • 3 replies
  • 0 kudos
Latest Reply
jefflipkowitz
Databricks Employee
  • 0 kudos

Have you referred to this document?https://help.tableau.com/current/pro/desktop/en-us/examples_databricks.html https://help.tableau.com/current/pro/desktop/en-us/examples_databricks.htm

  • 0 kudos
2 More Replies
ChristianRRL
by Valued Contributor
  • 1397 Views
  • 2 replies
  • 0 kudos

DQ Expectations Best Practice

Hi there, I hope this is a fairly simple and straightforward question. I'm wondering if there's a "general" consensus on where along the DLT data ingestion + transformation process should data quality expectations be applied? For example, two very si...

  • 1397 Views
  • 2 replies
  • 0 kudos
Latest Reply
joarobles
New Contributor III
  • 0 kudos

I'll drop my two cents here: having multiple layer validations reduce the effort needed to find the root cause of a data incident, but it has a drawback: they are harder to maintain.Every layer has a set of rules to be enforced and there will be asse...

  • 0 kudos
1 More Replies
ArghaB
by New Contributor II
  • 2828 Views
  • 8 replies
  • 9 kudos

Resolved! Facing StorageContext Error while trying to access DBFS

This issue has hindered my practice for the whole day. I scoured the web and couldn't find anybody who has faced this particular error. The error I am getting is: DBFS file browserStorageContext com.databricks.backend.storage.StorageContextType$DbfsR...

  • 2828 Views
  • 8 replies
  • 9 kudos
Latest Reply
mozartiano
New Contributor III
  • 9 kudos

Yeah, unable to save any file with rdd.saveTextfile and to upload any file using the workspace.

  • 9 kudos
7 More Replies
Amit_Dass_Chmp
by New Contributor III
  • 485 Views
  • 0 replies
  • 0 kudos

DBCU Plans is costlier vs Job Compute Premium 0.30 per DBU Please justify

Please help me to understood the % of savings how Databricks are calculating DBCUThey are telling if I take DBCU 12500 plan the price will be with discount 12000 and 4% discount.That means if I consume 12500 DBU, I am paying for this $12000 and 4% sa...

  • 485 Views
  • 0 replies
  • 0 kudos
Shoaib8587
by New Contributor III
  • 3989 Views
  • 11 replies
  • 13 kudos

Resolved! Uploading local file

 Since, last two day i getting an error called "ERROR OCCURRED WHEN PROCESSING FILE:[OBJECT OBJECT]" While uploading any "csv" or "json" file from my local system but it shows or running my previous file but give error after uploading a new file  

IMG_20240724_110147.jpg
  • 3989 Views
  • 11 replies
  • 13 kudos
Latest Reply
Pruthviraj
New Contributor III
  • 13 kudos

Hi,This has been fixed now, 

  • 13 kudos
10 More Replies
ChristianRRL
by Valued Contributor
  • 2047 Views
  • 3 replies
  • 0 kudos

Resolved! DLT Compute: "Ephemeral" Job Compute vs. All-purpose compute 2.0 ... WHY?

Hi there, this is a follow-up from a discussion I started last monthSolved: Re: DLT Compute: "Ephemeral" Job Compute vs. All-p... - Databricks Community - 71661Based on what was discussed, I understand that it's not possible to use "All Purpose Clust...

  • 2047 Views
  • 3 replies
  • 0 kudos
Latest Reply
raphaelblg
Databricks Employee
  • 0 kudos

@ChristianRRL regarding on why DLT doesn't allow you to use all-purpose clusters: 1. The DLT runtime is derived from the shared compute DBR, it's not the same runtime and has different features than the common all-purpose runtime. A DLT pipeline is n...

  • 0 kudos
2 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels
Top Kudoed Authors