Thanks for the Databricks community and maintaining such a valuable platform.I would like to inquire if there is a planned timeline for upgrading the GraphFrame library. We’ve noticed that the latest release on GitHub is v0.9.3, while the Databricks ...
I am using dataricks version 15.4 and getting below error whicle reading from jdbc and writing to aws S3 location: [NUMERIC_VALUE_OUT_OF_RANGE.WITHOUT_SUGGESTION] The -12874815911431.6200000000 rounded half up from -12874815911431.6200000000 cannot b...
After years of juggling tasks and feeling constantly behind, I tried something different: time-blocking. Instead of reacting to everything at once, I now assign focused time slots to specific tasks — and the difference is night and day. My productivi...
I'm looking for SQL linters in two different contexts within Databricks GUI:SQL queries in Databricks SQL Editor (GUI) - Is there a built-in linter for writing SQL queries in the Databricks SQL workspace?%sql magic cells in Databricks notebooks (GUI...
in databricks notebook is possible to connect own linters but I don't think it will work in SQL editor.
I am getting a pop-up error message in the right hand corner of my Databricks session, "Your notebook session has crashed." This is a notebook I've been working in for a while.The only line of code I am running is pip listThe error at the bottom of ...
From the screen, it looks like an interactive cluster—maybe try rebooting it.
Hello I have jobIn that job, it runs a task (GetGid) that executes a notebook and obtains some value using dbutils.jobs.taskValuesSete.g. dbutils.jobs.taskValuesSet(key = "gid", value = gid)As a result, I can use this and pass it to another task for ...
Python:dbutils.jobs.taskValuesSet()SQL: Results from sql query are visible in the workflow and you can use them to orchestrate another task (first row or loop)
Can i set column level and table level tags programmatically in a dlt pipeline. I tried it the normal way using spark.sql(f"alter table and set tags (key=value)") using this syntax I found it out in one of the databricks community post. But can we do...
The Tax Calculator is a free online tool designed to help South African users estimate their net salary after tax deductions, including PAYE and UIF, using the latest SARS tax brackets.
I'm looking for a SQL formatter (CLI) for two different contexts within Databricks.1. SQL queries in .sql files - What tool does Databricks use by default to format SQL?2. %sql magic cells in notebooks (.ipynb files) - What tool does Databricks use b...
Hi @Lenoq ,As far as I know, currently there is no tool that meets your given criteria by databricks/labs. The closest that I can see is the tool you gave as an example, but as you said its not that good. what I would do maybe is an intermedial optio...
i have data from march1 to march 14 in the final inference table and i have given 1 week granularity. after that profile and drift table is generated and i see the window start time as like this objectstart: "2025-02-24T00:00:00.000Z"end: "2025-03-03...
More or less repeating what Mark said and adding some additional thoughts. Why the Window Starts from February 24 The reason you're seeing a window starting from February 24 (even though your data starts March 1) is because monitoring systems align t...
Hi ,I need to connect databricks query from microsoft SSRS.is it possible ? How do you make the connection?
SSRS has limited auth methods. It also is EOL so my answer is: No.This is not a limitation of Databricks but SSRS.You could define the connection as a linked server on sql server, that might open some extra (MS native) options.PS. it is best not to ...
Use Case:I want to expose a data bricks API URL in Salesforce, Salesforce will hit that exposed end point every time a record is created and data will be transferred from Salesforce to DatabricksWhen i try creating a serving end pointI am unable to s...
When integrating Salesforce with Databricks to push data upon record creation, using a serving endpoint is not the most common or optimal approach. Although Databricks Feature Serving endpoints can be used for model or feature APIs, they are primaril...
Databricks DLT Joins: Streaming table join with Delta table is reading 2 Billion records from Delta Table for each and every Micro batch.How to overcome this issue to not to read 2 Billion records for every micro batch.Your suggestions and feedback w...
Hi!I'm currently working on ingesting log data from Azure Event Hubs into Databricks. Initially, I was using a managed Databricks workspace, which couldn't access Event Hubs over a private endpoint. To resolve this, our DevOps team provisioned a VNet...
I am trying to find the cost per Task in each Job every time it was executed (daily) but currently getting very huge numbers due to duplicates, can someone help me ? WITH workspace AS ( SELECT account_id, workspace_id, workspace_name,...
still costs exploded it seems there is no way to get cost per task only per job.
Hi all, I use the extension to debug my python code regularly and since yesterday accessing files in the Databricks Volume isn't working anymore. The situation in the UI of Databricks is as follows:When I execute a glob statement to list all zip-file...
hi @JuliandaCruz Thank you for reaching out! I was able to reproduce your case while using Databricks Connect. The "Upload and Run file" option worked fine and returned results, which is essentially the same as running from the Databricks UI. Thou...
| User | Count |
|---|---|
| 1617 | |
| 790 | |
| 471 | |
| 349 | |
| 287 |
