cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Hostname redaction in delta table

marvin1
New Contributor III

I am ingesting job-cluster failure notifications that we send to OpsGenie into a delta table to automate the creation and tracking of Jira tickets.  The alert notification includes the job run url, which we use to quickly respond to job failures.  However, when ingesting this data into a delta table, the hostname of the job run url is redacted.  This is not secret information to our data platform team.  How can I prevent this?  Thanks.    

3 REPLIES 3

Kaniz_Fatma
Community Manager
Community Manager

Hi @marvin1,The redaction of the hostname in the job run URL during ingestion into a Delta table might be due to some security or privacy settings in your data ingestion pipeline.

You might refer to the Delta Lake Documentation for best practices and optimization strategies.

marvin1
New Contributor III

Thanks for your response.  I read through best practices and did not see anything specific that might apply in my case.  What I have noticed is that it is related to the secret redaction changes on runtime > 11.3 as I only encounter the issue when running the notebook on later runtimes.  Consequently, we are currently migrating to Unity so it is something I will have to find a workaround for, either in the standard 13.3 cluster configuration that we are running, or in my automation process (will require more code).  I am getting the sense that the latter will be the more feasible approach.

marvin1
New Contributor III

I tracked this issue down to running the notebook on a shared Unity-enabled cluster.  When scheduling the notebook to run on a job cluster with single user mode, this issue does not occur.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group