Databricks Community

William_Scardua

Hi everyone,I’m currently looking for a data-quality solution for my environment. I don’t have DTL tables or a Unity Catalog in place.In your opinion, what is the best framework or package to implement reliable data-quality checks under these conditi...

William_Scardua · 04-23-2025

hi,I would like to request assistance on how to collect usage metrics and job execution data for my Databricks environment. We are currently not using Unity Catalog, but I would still like to monitor and analyze usageCould you please provide guidance...

William_Scardua · 04-02-2025

Hi guys,I'm adjusting a data upsert process from Databricks to CosmosDB using the .jar connector. As the load is very large, do you know if it's possible to change only the fields that have been modified?best regards

William_Scardua · 08-19-2024

Hi Guys,I tried to set the retention period in my Cluster, but it`s not workCluster:Notebook:Does not remove physical filesDo you have any ideas? thanks

William_Scardua · 06-28-2024

Hi guys,How can I get the block size ? have any idea ?Thank you

William_Scardua · 04-03-2024

Hi Sampath,In fact, I need to change this field in the table structure ... so, I need to apply an 'alter table' to remove the 'med' array from the 'equip' field.

William_Scardua · 05-22-2023

Thank you very much @Suteja Kanuri .. I will try to find the documentation for this

William_Scardua · 03-14-2023

Thank you @Vigneshraja Palaniraj ,For this I need to change the my cluster version, but thank you man

William_Scardua · 10-21-2022

Hi @Hubert Dudek in this case UUID are repeted

William_Scardua · 10-07-2022

Hi @Sivaprasad C S I tried to convert with uuid4().hex see the error it reported ).whenNotMatchedInsert(values = { "events.id" : uuid.uuid4().hex, "events.cod_operadora" : "updates.cod_operadora", "events.num_contrato" : "up...

Databricks Community

User Stats

User Activity

What the best Framework/Package for data quality

Collecting Job Usage Metrics Without Unity Catalog

Upsert from Databricks to CosmosDB

Deleted File Retention Duration in Cluster not work

Get Block Size

Re: Drop array in a struct field

Re: How to integrate pipeline with Dynatrace ?

Re: Upsert When the Origin NOT Exists, but you need to change status in the target

Re: uuid in Merge

Re: uuid in Merge