- 30 Views
- 1 replies
- 0 kudos
Are there SQL linters for Databricks GUI queries and notebook %sql cells?
I'm looking for SQL linters in two different contexts within Databricks GUI:SQL queries in Databricks SQL Editor (GUI) - Is there a built-in linter for writing SQL queries in the Databricks SQL workspace?%sql magic cells in Databricks notebooks (GUI...
- 30 Views
- 1 replies
- 0 kudos
- 0 kudos
in databricks notebook is possible to connect own linters but I don't think it will work in SQL editor.
- 0 kudos
- 20 Views
- 1 replies
- 0 kudos
[NUMERIC_VALUE_OUT_OF_RANGE.WITHOUT_SUGGESTION] The -12874815911431.6200000000 rounded half up from
I am using dataricks version 15.4 and getting below error whicle reading from jdbc and writing to aws S3 location: [NUMERIC_VALUE_OUT_OF_RANGE.WITHOUT_SUGGESTION] The -12874815911431.6200000000 rounded half up from -12874815911431.6200000000 cannot b...
- 20 Views
- 1 replies
- 0 kudos
- 0 kudos
I think it is related to number before conversion to Decimal. Please check what type is it?
- 0 kudos
- 25 Views
- 0 replies
- 0 kudos
Looking for CLI-based SQL formatter for Databricks: Alternative to gethue/sql-formatter for .
I'm looking for a SQL formatter (CLI) for two different contexts within Databricks.1. SQL queries in .sql files - What tool does Databricks use by default to format SQL?2. %sql magic cells in notebooks (.ipynb files) - What tool does Databricks use b...
- 25 Views
- 0 replies
- 0 kudos
- 35 Views
- 0 replies
- 0 kudos
Notebook Session Has Crashed
I am getting a pop-up error message in the right hand corner of my Databricks session, "Your notebook session has crashed." This is a notebook I've been working in for a while.The only line of code I am running is pip listThe error at the bottom of ...
- 35 Views
- 0 replies
- 0 kudos
- 24 Views
- 0 replies
- 0 kudos
Pass parameters between jobs
Hello I have jobIn that job, it runs a task (GetGid) that executes a notebook and obtains some value using dbutils.jobs.taskValuesSete.g. dbutils.jobs.taskValuesSet(key = "gid", value = gid)As a result, I can use this and pass it to another task for ...
- 24 Views
- 0 replies
- 0 kudos
- 3011 Views
- 2 replies
- 1 kudos
Inference table Monitoring
i have data from march1 to march 14 in the final inference table and i have given 1 week granularity. after that profile and drift table is generated and i see the window start time as like this objectstart: "2025-02-24T00:00:00.000Z"end: "2025-03-03...
- 3011 Views
- 2 replies
- 1 kudos
- 1 kudos
More or less repeating what Mark said and adding some additional thoughts. Why the Window Starts from February 24 The reason you're seeing a window starting from February 24 (even though your data starts March 1) is because monitoring systems align t...
- 1 kudos
- 8900 Views
- 6 replies
- 1 kudos
SSRS Connect to Databricks
Hi ,I need to connect databricks query from microsoft SSRS.is it possible ? How do you make the connection?
- 8900 Views
- 6 replies
- 1 kudos
- 1 kudos
SSRS has limited auth methods. It also is EOL so my answer is: No.This is not a limitation of Databricks but SSRS.You could define the connection as a linked server on sql server, that might open some extra (MS native) options.PS. it is best not to ...
- 1 kudos
- 56 Views
- 1 replies
- 0 kudos
Exposing Databricks API in Salesforce
Use Case:I want to expose a data bricks API URL in Salesforce, Salesforce will hit that exposed end point every time a record is created and data will be transferred from Salesforce to DatabricksWhen i try creating a serving end pointI am unable to s...
- 56 Views
- 1 replies
- 0 kudos
- 0 kudos
When integrating Salesforce with Databricks to push data upon record creation, using a serving endpoint is not the most common or optimal approach. Although Databricks Feature Serving endpoints can be used for model or feature APIs, they are primaril...
- 0 kudos
- 81 Views
- 2 replies
- 1 kudos
Databricks DLT Joins: Streaming table join with Delta table is reading 2 Billion records per batch
Databricks DLT Joins: Streaming table join with Delta table is reading 2 Billion records from Delta Table for each and every Micro batch.How to overcome this issue to not to read 2 Billion records for every micro batch.Your suggestions and feedback w...
- 81 Views
- 2 replies
- 1 kudos
- 69 Views
- 0 replies
- 0 kudos
Ingesting Data from Event Hubs via Kafka API with Serverless Compute
Hi!I'm currently working on ingesting log data from Azure Event Hubs into Databricks. Initially, I was using a managed Databricks workspace, which couldn't access Event Hubs over a private endpoint. To resolve this, our DevOps team provisioned a VNet...
- 69 Views
- 0 replies
- 0 kudos
- 158 Views
- 3 replies
- 0 kudos
Query to calculate cost of task from each job by day
I am trying to find the cost per Task in each Job every time it was executed (daily) but currently getting very huge numbers due to duplicates, can someone help me ? WITH workspace AS ( SELECT account_id, workspace_id, workspace_name,...
- 158 Views
- 3 replies
- 0 kudos
- 0 kudos
still costs exploded it seems there is no way to get cost per task only per job.
- 0 kudos
- 132 Views
- 4 replies
- 0 kudos
Access to Databricks Volumes via Databricks Connect not working anymore
Hi all, I use the extension to debug my python code regularly and since yesterday accessing files in the Databricks Volume isn't working anymore. The situation in the UI of Databricks is as follows:When I execute a glob statement to list all zip-file...
- 132 Views
- 4 replies
- 0 kudos
- 0 kudos
hi @JuliandaCruz Thank you for reaching out! I was able to reproduce your case while using Databricks Connect. The "Upload and Run file" option worked fine and returned results, which is essentially the same as running from the Databricks UI. Thou...
- 0 kudos
- 356 Views
- 5 replies
- 2 kudos
Resolved! Why am I getting NameError name _all_timezones_unchecked' is not defined
I defined the following local time function get datetime: def get_sysdate(): jkt_tz = pytz.timezone('Asia/Jakarta') sysdate = datetime.now(jkt_tz).strftime('%Y-%m-%d %H:%M:%S') return sysdatespark.udf.register("get_sysdate", get_sysdate)But...
- 356 Views
- 5 replies
- 2 kudos
- 2 kudos
Hi @Khaja_Zaffer I have connected with MS Support and explained: pytz is no thread safe package, I believe it will have some issue when executor init it parallelly. Second, this is a 3rd party lib, and suggest using the built-in library from ZoneInfo...
- 2 kudos
- 2048 Views
- 2 replies
- 1 kudos
Workspace allows dbf path to install in Databricks 16.4 LTS
Feature: Library installation using requirements.txt on DB Runtime 16.4 LTSAffected Areas: Workspace isolation, Library ManagementSteps to Reproduce:Upload a wheel file to dbfPut the requirements.txt file in the Workspace and put dbfs path in require...
- 2048 Views
- 2 replies
- 1 kudos
- 1 kudos
I would like to know if the workspace isolation is a priority, and only Databricks 14.3 and lower allow installation via DBFS.Why should the requirements.txt allow you to install libraries or packages via dbfs path?Could someone please explain why th...
- 1 kudos
- 2388 Views
- 2 replies
- 0 kudos
Batch reading from sql server tables with cdc on ssql server tables
Hi all,I need to do a batch load from sql server into Databricks. I have CC enabled on some tables. The simple appears to be union CDC and regular table to get a single set of records to load, but this appears to be fraught with risk of out of sequen...
- 2388 Views
- 2 replies
- 0 kudos
- 0 kudos
Yes, you can use TVFs on Databricks. Please check the following link: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-qry-select-tvf#gsc.tab=0Can you please elaborate on how you are loading the SQL Server Data into Databricks? H...
- 0 kudos
| User | Count |
|---|---|
| 1616 | |
| 790 | |
| 469 | |
| 349 | |
| 287 |
