- 2366 Views
- 2 replies
- 1 kudos
Structured Streaming from a delta table that is a dump of kafka and get the latest record per key
I'm trying to use Structured Streaming in scala to stream from a delta table that is a dump of a kafka topic where each record/message is an update of attributes for the key and no messages from kafka are dropped from the dump, but the value is flatt...
- 2366 Views
- 2 replies
- 1 kudos
- 1 kudos
I am confused about this recommendation. I thought the use of the append output mode in combination with aggregate queries is restricted to queries for which the aggregation is expressed using event-time and it defines a watermark.Could you clarify ?
- 1 kudos
- 314 Views
- 0 replies
- 0 kudos
Solace to Azure Data Lake Storage
Hi Team,What is the most effective method for performing data ingestion from Solace to Azure Data Lake Storage (ADLS) utilizing an Azure Databricks notebook? Any recommendations would be greatly appreciated.Regards,Phani
- 314 Views
- 0 replies
- 0 kudos
- 944 Views
- 1 replies
- 0 kudos
Databricks book recommendations
Hi all,I am very new to databricks. I am looking for any good book recommendations that can help me get started. I know there is a vast resource available online but I feel a book will give me a structured approach to get startedAny book recommendati...
- 944 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @uniqueusername ,I would start with books that teach you spark.Learning Spark, 2nd Edition by Jules S. Damji, Brooke Wenig, Tathagata Das, Denny LeeData Analysis with Python and PySpark by Jonathan Rioux (Author)After you learn spark foundation, o...
- 0 kudos
- 676 Views
- 3 replies
- 0 kudos
Unable to create a workspace
Noel Nosse <nnosse@my.wgu.edu> 9:03 PM (0 minutes ago) to Databricks To complete a tutorial requires a workspace. The directions for the quickstart are outdated and do not match AWS. AWS has their own guide but cloudformation requires email ...
- 676 Views
- 3 replies
- 0 kudos
- 0 kudos
Now I get: Redirecting to: https://accounts.cloud.databricks.com/login/password?next_url=%2Fapi%2F2.
- 0 kudos
- 576 Views
- 1 replies
- 1 kudos
Azure Databricks add repo
While adding arepo in databricks workspace I am getting error as 'Error creating repoThe Azure container does not exist'Please see the attached screenshot.Anyone please suggest the fix.
- 576 Views
- 1 replies
- 1 kudos
- 1 kudos
There are three possible causes. The Azure container might not have been properly created when the workspace was provisioned.The Azure container might have been deleted or moved after it was created.There might be a problem with the permissions or r...
- 1 kudos
- 1836 Views
- 6 replies
- 4 kudos
Cannot Create databricks account
I am trying to create databricks community account but after providing all the information and completing the puzzle it is showing me an error occurred: I also recorded the the network request for the error:Header:Request URL:https://www.databricks.c...
- 1836 Views
- 6 replies
- 4 kudos
- 4 kudos
@AllI am trying it from Bangladesh. Is there any country wise restrictions??
- 4 kudos
- 2288 Views
- 1 replies
- 0 kudos
POC Comparison: Databricks vs AWS EMR
Hello,I need some assistance with a comparison between Databricks and AWS EMR. We've been evaluating the Databricks Data Intelligence platform for a client and found it to be significantly more expensive than AWS EMR. I understand the challenge in ma...
- 2288 Views
- 1 replies
- 0 kudos
- 0 kudos
I can't say for sure what your exact limitation is going to be but two things I have come across in EMR bake offs are:1) bottleneck in the network throughput. Verify the S3 reads and writes are happening at similar throughput rates. This is a cloud...
- 0 kudos
- 491 Views
- 0 replies
- 0 kudos
Linking Workspace IDs to Names in Billing Schema
Hi everyone,We recently enabled UC and the Billing system table to monitor our usage and costs. We've successfully set up a dashboard to track these metrics for each workspace. The usage table includes the workspace_id, but I'm having trouble finding...
- 491 Views
- 0 replies
- 0 kudos
- 52549 Views
- 5 replies
- 4 kudos
How to create temporary table in databricks
Hi Team,I have a requirement where I need to create temporary table not temporary view.Can you tell me how to create temporary table in data bricks ?
- 52549 Views
- 5 replies
- 4 kudos
- 4 kudos
I see, thanks for sharing, can you mark the solution which worked for you @abueno as Accepted.
- 4 kudos
- 561 Views
- 1 replies
- 0 kudos
ConcurrentAppendException
Hi Team,We're dealing with a concurrency issue when we attempt to run multiple jobs at the same time, and we're still having the same problem even after using partition and liquid clustering features. Now we're making sure to have the right where con...
- 561 Views
- 1 replies
- 0 kudos
- 0 kudos
Option 3 Bit complicated process but works well: Considering you will do Update, Insert and Delete operation on the same table at the same time by using multiple job.1. Create a Dummy table with the target table schema with additional column called O...
- 0 kudos
- 488 Views
- 0 replies
- 0 kudos
How to test dashboards for free?
Does the community edition include the ability to create dashbiards? I’d not, is there a free option to test dashboards?
- 488 Views
- 0 replies
- 0 kudos
- 812 Views
- 0 replies
- 1 kudos
FAQ for Databricks Learning Festival (Virtual): 10 July - 24 July 2024
IObit Smart Defrag Pro 9.3.0.341 Serial Key Smart defragmentation works quickly, mechanically, and silently in history and is suitable for large hard drives. It facilitates its hard work with greater success than any other product on the market, free...
- 812 Views
- 0 replies
- 1 kudos
- 530 Views
- 0 replies
- 0 kudos
Job Cluster best practices for production workloads
Hi All,Can you please share the best practices for job clusters configurations for production workloadsand which is good when compared to serverless and job cluster in production in terms of cost and performance?Regards,Phani
- 530 Views
- 0 replies
- 0 kudos
- 1266 Views
- 1 replies
- 1 kudos
how to reduce scale to zero time in MLFlow Serving
Hi,I am deploying MLflow models using Databrick serverless serving but seems servers scale down to 0 only after 30 minute of inactivity. Is there any way to reduce this time?Also, Is it possible to deploy multiple models under single endpoint. I want...
- 1266 Views
- 1 replies
- 1 kudos
- 1 kudos
Regarding your first question about reducing the scale-down time of Databricks serverless serving, currently, the system is designed to scale down to zero after 30 minutes of inactivity. This is to ensure that instances are kept warm to handle any su...
- 1 kudos
- 1443 Views
- 3 replies
- 0 kudos
Resolved! Help trying to use Python in Databricks
I am watching an Introduction to Databricks - running Python scripts and I don't see where I create a notebook in my databricks instance to even select Python as the Default Language.Is it possible, my level of databricks isn't allowing me to run Pyt...
- 1443 Views
- 3 replies
- 0 kudos
