SQL Server OUTPUT clause alternative

Nathant93 — Wed, 08 Nov 2023 11:50:15 GMT

I am looking at after a merge or insert has happened to get the records in that batch that had been inserted via either method, much like the OUTPUT clause in sql server.

Does anyone have any suggestions, the only thing I can think of is to add a timestamp to the records and then select them from the table, however that would require having a timestamp on all tables I have.

Thanks

Re: SQL Server OUTPUT clause alternative

Nathant93 — Wed, 29 Nov 2023 10:35:28 GMT

I've managed to do it like this

qry = spark.sql(f"DESCRIBE history <table_name> limit 1").collect()

current_version = int(qry[0][0])

prev_version = current_version - 1

Then do an except statement between the versions.

topic Re: SQL Server OUTPUT clause alternative in Data Engineering

SQL Server OUTPUT clause alternative

Re: SQL Server OUTPUT clause alternative