- 692 Views
- 0 replies
- 0 kudos
Shuffle Partitions
What if I have lot of empty shuffled partitions due to data skewness Secondly , if the shuffle partition size is 128 MB and if the size of the key's partition is 700 MB
- 692 Views
- 0 replies
- 0 kudos
Forum Posts
- 1371 Views
- 3 replies
- 4 kudos
The way to visualize logging of Spark Job graphically
Is there any way to visualize logging such as execution runtime or memory usage of Spark Job graphically just like the image below by utilizing Databricks Partner Connect for free?Also, I'd appreciate to know any other ways to visualize the logging o...
- 1371 Views
- 3 replies
- 4 kudos
Latest Reply
- 4397 Views
- 1 replies
- 0 kudos
How to Add value to Comment Column in Databricks Catalog View for Foreign table (Azure SQL)
Hi.Struggling to add description by script to the comment column within the catalog view in Databricks, particularly for foreign/external tables sourced from Azure SQL.I have no issue doing that for delta tables. Also for information schema columns f...
- 4397 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
@Retired_mod Sorry, I did read this way back but forgot to reply.Thanks for information.And all those steps I found as well and for "3. Programmatic Approach (For Azure SQL)" if you are adding to sql any description that way I don't think it will app...
- 0 kudos
- 2224 Views
- 2 replies
- 0 kudos
Resolved! DLT Online Table with VNnet Enable on Blob Storage Get 403 Issue
I am trying to create an online table in a Unity catalog. However, I get a GET, 403 error. DataPlaneException: Failed to start the DLT service on cluster . Please check the stack trace below or driver logs for more details. com.databricks.pipelines....
- 2224 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
I figured it out. It was because of the Network Connectivity Configurations. I did not have one setup with a private endpoint connection to the ADLS Gen2. I followed the instructions here: https://learn.microsoft.com/en-us/azure/databricks/security/...
- 0 kudos
- 9573 Views
- 8 replies
- 3 kudos
What are the different ways to pull the log data from Splunk to Databricks?
Hi,I have recently started Splunk Integration with Databricks. Basically I am trying to ingest the data from Splunk to Databricks. I have gone through the documentation regarding Splunk Integration. There are some basic information about the integrat...
- 9573 Views
- 8 replies
- 3 kudos
Latest Reply
- 3 kudos
Hi @Arch_dbxlearner Did you done integration with splunk if yes can you please help
- 3 kudos
- 574 Views
- 1 replies
- 1 kudos
"No API found for 'POST /workspace-files" error while trying to upload a JAR
Hi,I'm using CE and trying to upload a JAR library of about 45MB into my workspace so I can use it from Pyspark, but getting error "No API found for 'POST /workspace-files". Any thoughts?
- 574 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
N.b. in the console output I have this:
- 1 kudos
- 1067 Views
- 2 replies
- 1 kudos
regrading course
Hello I am a newbie on this platform can anyone please tell me how can I enroll in courses that we have supposed to complete to get a voucher for exams I came to know about databricks Learning Festival
- 1067 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi @Nikhilkamode , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your fe...
- 1 kudos
- 891 Views
- 2 replies
- 0 kudos
Issues around float datatypes
Float data from file, is never getting copied to the temp_table created though the schema matches.As a workaround, using CREATE TABLE [USING] which is able to insert the file data to a temp_table. Is this a known issue? COPY INTO issue is explained w...
- 891 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @inagar , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedback...
- 0 kudos
- 1866 Views
- 2 replies
- 2 kudos
Learning via Free Trial Azure
Hi I am following a Databricks course on Udemy and the course instructed to access Databricks via the free trial of Azure. Once I created my account on Azure and load Databricks, I try to create a cluster but this never succeeds. It takes an extremel...
- 1866 Views
- 2 replies
- 2 kudos
Latest Reply
- 2 kudos
Hi @clock4eva , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedb...
- 2 kudos
- 2361 Views
- 2 replies
- 1 kudos
How to Optimize Delta Lake Performance for Large-Scale Data Ingestion?
Hi everyone,I'm currently working on a project that involves large-scale data ingestion into Delta Lake on Databricks. While the ingestion process is functioning, I've noticed performance bottlenecks, especially with increasing data volumes. Could yo...
- 2361 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi @Syleena23 , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedb...
- 1 kudos
- 1250 Views
- 2 replies
- 0 kudos
Monitoring Databricks
Good Day All, Did any one done Databricks Monitoring integration with any other 3rd party applications like Grafana or ELK to get infrastructure monitoring like cpu_utilization ,memory ,job monitor and I can able to write spark code to get cpu_utili...
- 1250 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @SainnathReddy_M , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your...
- 0 kudos
- 1699 Views
- 2 replies
- 1 kudos
classic cluster vs serverless cost
Hi Team,Can you help me the cost comparison between classic cluster and serverless?
- 1699 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi @Phani1 , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedback...
- 1 kudos
- 1074 Views
- 1 replies
- 0 kudos
UnsupportedOperationException: com.databricks.backend.daemon.data.client.DBFSV1.resolvePathOnPhysica
dbutils.fs.mv('dbfs:/FileStore/tables/Employee-2.csv','dbfs:/FileStore/backup/Employee-5.csv',recurse=True)--is giving errorUnsupportedOperationException: com.databricks.backend.daemon.data.client.DBFSV1.resolvePathOnPhysicalStorage(path: Path)File <...
- 1074 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @v_sravan_sai , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your fe...
- 0 kudos
- 1709 Views
- 2 replies
- 1 kudos
How can I Resolve QB Desktop Update Error 15225?
I'm encountering QB Desktop update error 15225. What could be causing this issue, and how can I resolve it? It's disrupting my workflow, and I need a quick fix.
- 1709 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
hi @Dorothy80Galvin To resolve Desktop Update Error 15225, follow these steps. First, verify your Internet Explorer settings by ensuring it is set as the default browser and that SSL settings are enabled. Next, add trusted sites by navigating to Inte...
- 1 kudos
- 1288 Views
- 3 replies
- 1 kudos
Databricks data engineer associate exam Suspended
Hi Team,I recently had a disappointing experience while attempting my first Data bricks certification exam. During the exam, I was abruptly directed to Proctor Support. The proctor asked me to show my desk and the room I was in. I complied by showing...
- 1288 Views
- 3 replies
- 1 kudos
Latest Reply
- 1 kudos
@Kaniz_Fatma @Cert-Team @Cert-Bricks Requesting you to please look into this and update me since it has not been resolved yet and I am not able to reschedule my exam.
- 1 kudos
