Hi all,When writing from Tableau Prep to Databricks on Azure, we receive the following error message:[Databricks][Hardy] (52) Error communicating with the service: 403 Interestingly, reading from Databricks works without any issues. The Databricks us...
Hi, I want to change the cluster I am using. However, when I click on the "Compute" tab on the platform, I get automatically redirected to the "SQL Warehouses" page. I am not able to click and enter the "Compute" page. How can I solve this? Thank you
THIS IS NOT RELATED TO A FREE ACCOUNTI ran into the same issue—there was no option to create general compute, only SQL Warehouse or serverless.Solution:When creating a Databricks resource in the Azure portal, it defaults to a serverless workspace, wh...
I read somewhere that there's a max_concurrent_task_runs property, but can't find it anywhere in the docs. So, how to limit the maximum concurrent tasks run in a job?
Hi all,I’m using Databricks Free Edition with a serverless SQL warehouse. I’m the only user in this workspace.Warehouse config:- Type: Serverless SQL- Size: Large- Max clusters: 2Query:SELECTw.workspace_name,ROUND(SUM(u.usage_quantity * COALESCE(lp.p...
We are using DABs to deploy our jobs. DABs have source set to git branch or git tag depending on the environment. Repository is structured in mono repo fashion. We don't use wheels for our modules. Sometimes when the jobs run they "randomly" fail th...
Hello @pepco !I will share with you my personal experience about a very similar behaviour I got like you.If you check DBKS doc you will find that git_source and task source: GIT are not recommended for DAB because local relative paths may not point ...
Hi there, running into some trouble abstracting job_clusters configurations to improve reusability. At the moment, I have many job yaml files that require the following configuration:What would be the best approach(es) to remove this configuration fr...
Hello @ChristianRRL My doubt about your issue is happening in cluster_definitions.yml because it is not only defining a reusable cluster profile it is also redefining the same jobs that already exist in the individual fleet_*.yml files.Why ? because ...
In the January 2026 release notes, it was announced that: "Pipelines now support queued execution mode, where multiple update requests are automatically queued and executed sequentially instead of failing with conflicts. This simplifies operations fo...
Thank you very much for the detailed response! We unfortunately can't proceed with option 1, as we do require multiple places that can trigger the pipeline (an API call to the parent job, and a direct API call to the pipeline itself). This is due to ...
I am trying to stream my IoT data from azure event hub to databricks. Im running Databricks runtime 17.3 LTS with scala 2.13.
Hi @Areqio ,def getKafkaOptions( env: String, ehNameSpace: String, ehName: String, scopeName: String, kafkaOffset: String, ehConnKey: String, maxOffsetsPerTrigger: String = "50000" ): Map[String, String] = { val connStr = dbutils.sec...
[Sorry for a novice question.]I have multiple tables periodically updated from external sources (including insert, update, or delete). I need to update a target table, which is an outer join from multiple source tables without rewriting it each time....
EnvironmentCloud: AWSCompute: ServerlessTable: a_big_tableTable type: Streaming Table (SDP pipeline)Table size: 641 GB, 6,210 filesLiquid Clustering columns: [event_time, integer_userId]delta.dataSkippingStatsColumns:event_time, integer_userId, integ...
Hello @aonurdemir , I looked into your query and have compiled some helpful tips: I don't have direct access to your workspace internals, so I can't prove this definitively. But what you're seeing is consistent with how Delta's stats-based data skipp...
hello everyone, I'm trying use spotify's api to analyse my music data, but i'm receiving a error during authentication, specifically when I try get the token, above my code.Is it a databricks bug?pip install spotipyfrom spotipy.oauth2 import SpotifyO...
In this context, Elevate New is relevant as a digital content platform that covers technology trends, online platforms, software ecosystems, and modern internet-based solutions. As developers and tech communities continue discussing APIs, cloud ser...
Hi All,Lately I have had issues with my vector search index not syncing.The associated pipeline fails to create with error:failed to resolve flow: '__online_index_view'. com.databricks.sql.transaction.tahoe.DeltaAnalysisException: [DELTA_UNSUPPORTED_...
Been running notebooks on serverless compute and watching the indicator in the UI. After my last cell finishes, it goes from dark green to this fading green, sits there for maybe 5-10 minutes, then finally goes grey. Pretty sure I'm paying for that e...
I have the same concern and feedback as OP. I wish there's a way to set auto-terminate after the serverless cluster has been idle for X minutes and not be billed if our users left their notebooks attached to serverless compute and forgot to hit "term...
Hi there!We have different jobs/workflows configured in our Databricks workspace running on AWS and would like to know who actually started the job run? Are they started by a user or a service principle using curl?Currently one can only see, who is t...
Hi,I am not a Data Engineer, I want to connect to ssas. It looks like it can be connected through pyodbc. however looks like I need to install "ODBC Driver 17 for SQL Server" using the following command. How do i install the driver on the cluster an...
As SQL Server is included in the Lakehouse federation driver, it is built in databricks. Install only in case you need a different version - the built-in one is not working
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 565 | |
| 349 | |
| 287 |
