Databricks Community

eriodega

I've been using the system.access.table_lineage table, and I'm trying to understand when the source and target tables are defined.For example, picking a specific job run and looking at the lineage:selectsource_type, source_table_full_name, target_typ...

eriodega · 10-30-2024

What is the syntax in SQL for creating a widget in a notebook with a label?This documentation says "The last argument is label, an optional value for the label shown over the widget text box or dropdown."The one example provided on that page doesn't ...

eriodega · 08-29-2024

I am trying to do a regular expression replace in a Databricks notebook.The following query works fine in a regular query (i.e. not running it in a cell in a notebook): select regexp_replace('abcd', '^(.+)c(.+)$', '$1_$2') --normally outputs ab_d H...

eriodega

@Sidhant07thanks for the answer, I think it is good, but I am questioning scenario #5 (source=table,target=view).I'm looking at some examples in our table_lineage, and we aren't modifying the view or creating the view from within a job. I think scena...

eriodega · 10-28-2024

@vivilyhere's a repro: %scala val tableName = "delta.`/Volumes/<YOUR_PATH_HERE>/delete_me_test_table1`" spark.sql( "INSERT INTO IDENTIFIER(:table1)(id,name) VALUES(1,'alice');", Map("table1" -> tableName) ).show() //above succeeds. spark.sql( ...

eriodega · 10-03-2024

FYI, I just tested it again yesterday, and with the new runtime version 15.4 LTS, the bug is fixed.

eriodega · 09-18-2024

yes, that returns one row. The ordering of the resultant string is non-deterministic though (I just ran it and got "def,abc,ghi"), and that is likely fine for most people's use-cases (in fact Heman2 mentioned in in the first answer in this thread). H...

eriodega · 09-17-2024

Hmmm, when I try it, I get multiple rows back (I desire to only get one row back):;WITH Blah(col1, i) as ( SELECT 'abc',1 UNION SELECT 'def',2 UNION SELECT 'ghi',3 ) SELECT array_join(collect_set(col1) over (order by i), ',') FROM ...

Databricks Community

User Stats

User Activity

system.access.table_lineage - source and target table meanings

CREATED WIDGET - SQL syntax - how do I specify a label?

Escaping $ (dollar sign) in a regex backreference in notebook (so not seen as a parameter)

Re: system.access.table_lineage - source and target table meanings

Re: Misleading UNBOUND_SQL_PARAMETER even though parameter specified

Re: Misleading UNBOUND_SQL_PARAMETER even though parameter specified

Re: Databricks SQL string_agg

Re: Databricks SQL string_agg