- 576 Views
- 2 replies
- 0 kudos
I created simplistic DLT pipeline that create one table. When I delete the pipeline the tables is dropped as well. That's not really desired behavior. Since I remember there was a strong distinction between data (stored in tables) and processing (spa...
- 576 Views
- 2 replies
- 0 kudos
Latest Reply
Hi @BobEng, Here are a few things that might help:
Pipeline Settings: Delta Live Tables provides a user interface for configuring and editing pipeline settings. You can configure most settings with either the UI or a JSON specification.
Table Mana...
1 More Replies
- 883 Views
- 4 replies
- 0 kudos
from pyspark.sql import functions as F
from pyspark.sql import types as T
from pyspark.sql import DataFrame, Column
from pyspark.sql.types import Row
import dlt
S3_PATH = 's3://datalake-lab/xxxx/'
S3_SCHEMA = 's3://datalake-lab/xxxx/schemas/'
@dl...
- 883 Views
- 4 replies
- 0 kudos
Latest Reply
Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...
3 More Replies
by
pyter
• New Contributor III
- 4046 Views
- 6 replies
- 2 kudos
Hello everyone,We use unity catalog, separating our dev, test and prod data into individual catalogs.We run weekly vacuums on our prod catalog using a service principal that only has (read+write) access to this production catalog, but no access to ou...
- 4046 Views
- 6 replies
- 2 kudos
Latest Reply
Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...
5 More Replies
- 361 Views
- 2 replies
- 0 kudos
I am implementing Structured Streaming using Delta Live Table. I want to delete the parquet files once they are used. What options should I set so that the files loaded in S3 are not deleted?
- 361 Views
- 2 replies
- 0 kudos
Latest Reply
Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...
1 More Replies
- 546 Views
- 4 replies
- 1 kudos
hello everyone, how to collect metrics provided by clusters metrics databricks runtime 13.1 using bash script
- 546 Views
- 4 replies
- 1 kudos
Latest Reply
Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...
3 More Replies
- 425 Views
- 1 replies
- 1 kudos
Hi Databricks community,Reverso is a language translation tool that converts texts from one language to another. I need to convert hundreds of text but its time consuming. Hence, I want to use its API to automate the process.How can I achieve this in...
- 425 Views
- 1 replies
- 1 kudos
Latest Reply
Hi @chari, To use the Reverso API inside Databricks notebooks, you can use the requests library in Python.
- 316 Views
- 2 replies
- 0 kudos
I have a number of functions in a schema in a catalog in Unity Catalog, is there a coding way to be able to change the owner of the functions created without having to do it manually via the gui?
- 316 Views
- 2 replies
- 0 kudos
Latest Reply
Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...
1 More Replies
- 1190 Views
- 7 replies
- 0 kudos
Hi Team,My requirement is to move build a solution to move zos(db2) CDC data to Delta table on Realtime bases(at least near realtime) , data volume and number of tables are little huge (100 tables) I have researched I dont find any inbuild options in...
- 1190 Views
- 7 replies
- 0 kudos
Latest Reply
Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...
6 More Replies
- 595 Views
- 2 replies
- 0 kudos
my project I want if job take longer time then it will terminate and again it will try even if there is timeout error and in databricks launched status should show retry by scheduler and it should follow min_retry_interval_millis before start retry...
- 595 Views
- 2 replies
- 0 kudos
Latest Reply
Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...
1 More Replies
- 444 Views
- 2 replies
- 0 kudos
I have a notebook that produces lots of excel files which I want downloading on my local machine.I can only currently download one by one which takes a long time when there are a lot of them.Is there a way without using Azure CLI to download all of t...
- 444 Views
- 2 replies
- 0 kudos
Latest Reply
Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...
1 More Replies
- 914 Views
- 5 replies
- 0 kudos
I tried to run this query and failing to load the data .What do I need to do load from federated data sources using DLT if this is not correct CREATE OR REPLACE LIVE TABLE bulkuploadhistory
COMMENT 'Table generated for bulkuploadhistory.'
TBLPROPERTI...
- 914 Views
- 5 replies
- 0 kudos
Latest Reply
Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...
4 More Replies
- 1652 Views
- 3 replies
- 1 kudos
Hi,I would like to create / update dashboard definition based on the json file. How can one do it? I tried the following:databricks api post /api/2.0/preview/sql/dashboards/$dashboard_id --json @file.json But it does not update the widgets...How can...
- 1652 Views
- 3 replies
- 1 kudos
Latest Reply
Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...
2 More Replies
- 386 Views
- 2 replies
- 0 kudos
Are there any in-built reports available in Databricks UI that list the users and roles with access to the workspace?
- 386 Views
- 2 replies
- 0 kudos
Latest Reply
Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...
1 More Replies
- 662 Views
- 3 replies
- 1 kudos
Hi Currently all data reauired resides in Az sql database. We have a project in which we need to query on demand this data in Salesforce data cloud to be further used for reporting in CRMA dashboard.do we need to move this data from az sql to delta l...
- 662 Views
- 3 replies
- 1 kudos
Latest Reply
Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...
2 More Replies
