Databricks Community

jorperort

Hi,Is it possible to directly execute a Bash script or a binary executable from the operating system of a Databricks job compute node using a single node cluster?I’m using databricks asset bundels for job initialization and execution. When the job s...

jorperort · 02-25-2025

Hello everyone,Let me give you some context. I am trying to deploy a Delta Live Table pipeline using Databricks Asset Bundles, which requires a private library hosted in Azure DevOps.As far as I understand, this can be resolved in three ways:Installi...

jorperort · 12-30-2024

Good afternoon,I am looking for documentation to implement the WAP pattern using Unity Catalog, workflows, SQL notebooks, and any other services necessary to use this pattern. Could you share information on how to approach the problem with documentat...

jorperort · 12-02-2024

Hi everyone,I’m looking for the best way to implement integration tests for SQL notebooks in an environment that uses Unity Catalog and workflows to execute these notebooks.For unit tests on SQL functions, I’ve reviewed the https://docs.databricks.co...

jorperort · 05-02-2024

Good morning, I'm trying to run: databricks bundle run --debug -t dev integration_tests_job My bundle looks: bundle: name: x include: - ./resources/*.yml targets: dev: mode: development default: true workspace: host: x r...

jorperort

Good afternoon @eniwoke @ , and thank you very much for your commenYes, I had already considered using scripts, but I was hoping to find a different solution, since the idea is to pass parameters to these Bash scripts or compiled executables. I assum...

jorperort · 04-05-2025

Hello @116680,The problem I had was with the name of the startup script that I had uploaded to the volume and had the correct permissions. The name didn't match the one defined in the DAB resource that creates the DLT pipeline.

jorperort · 03-30-2025

Hi @ashraf1395 ,I'm working with Delta Live Tables (DLT) and the event_log table. I would like to know if it is possible to access the event handler that DLT uses to write custom logs and send them to this table when events are published.If this is n...

jorperort · 03-22-2025

Hi @susanneI’m sharing an option in case you find it useful. In my case, I was testing how to publish these tables using Databricks Asset Bundles, but I couldn’t get it to work. I'm not sure if I posted a question about this. Let’s see if we’re lucky...

jorperort · 02-25-2025

I detected the error; it was due to the path defined in the bundle where the init script was located.I'm closing the post.

Databricks Community

User Stats

User Activity

Executing Bash Scripts or Binaries Directly in Databricks Jobs on Single Node Cluster

Init Scripts Error When Deploying a Delta Live Table Pipeline with Databricks Asset Bundles

Wap pattern unity catalog

Help with Integration Testing for SQL Notebooks in Databricks

[Databricks Assets Bundles] no deployment state

Re: Executing Bash Scripts or Binaries Directly in Databricks Jobs on Single Node Cluster

Re: Init Scripts Error When Deploying a Delta Live Table Pipeline with Databricks Asset Bundles

Re: Not readable format of event_logs traceback in dlt pipeline

Re: How to write event_log destination into DLT Settings JSON via Asset Bundles

Re: Init Scripts Error When Deploying a Delta Live Table Pipeline with Databricks Asset Bundles