cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Databricks bug with show tblproperties - redacted - Azure databricks

SamDataWalk
New Contributor III

I am struggling to report what is a fairly fundamental bug. Can anyone help? Ideally someone from Databricks themselves. Or others who can confirm they can replicate it.

There is a bug where databricks seems to be hiding โ€œanyโ€ properties which have the characters โ€œuserโ€ in them.

This is not just in the display, but in the actual underlying result set.

 

To reproduce the bug.

 

create or replace view tmp_view

tblproperties (   notes='userid',   notes2='myuserid',   something='this will be fine')

as

select 1 as a;

 

 

Now in the next cell just run

 

show tblproperties tmp_view

 

and it doesnโ€™t just hide the data from the diusplay, but it changes the data inside the resultset. My application which connects to databricks and runs this command is actually getting the string *****(redacted)

 

 

1 ACCEPTED SOLUTION

Accepted Solutions

SamDataWalk
New Contributor III

I managed to get a response back from support at databricks.

Admittedly it is a bit nuclear, but there is a way of switching it off.

spark.conf.set("spark.databricks.behaviorChange.SC102534CommandRedactProperties.enabled", False)
So, I have managed to use this, to temporarily switch it off whilst this bit of my code is extracting the meta data.

View solution in original post

3 REPLIES 3

PatDBX
New Contributor II

I understand that yours is a View. For my case, it's a Table so I could use `desc detail <schema_name>.<table_name>` to get the table properties info that are not redacted in the `properties` column from the `desc detail` output.

SamDataWalk
New Contributor III

Yes, it is indeed a view which that command errors for.

thanks anyway, appreciate the suggestion 

SamDataWalk
New Contributor III

I managed to get a response back from support at databricks.

Admittedly it is a bit nuclear, but there is a way of switching it off.

spark.conf.set("spark.databricks.behaviorChange.SC102534CommandRedactProperties.enabled", False)
So, I have managed to use this, to temporarily switch it off whilst this bit of my code is extracting the meta data.
Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.