cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Is writing custom function possible in transform(array,func) in databricks sql?

Jerry01
New Contributor III

This is the query I am trying to implement

Create function data_hide(data string)

Return if(is_member('groupName'),data,'****')

โ€‹

Table : my_table

Id Subject

โ€‹1. ['Eng','Bio']

2. ['Phy','Mat']

โ€‹

Select id, transform(Subject, x -> data_hide(x)) as new_data from my_table

โ€‹

โ€‹

The above select statement throwing me errorโ€‹

2 REPLIES 2

Anonymous
Not applicable

@Naveena Gโ€‹ :

The error in your SQL query may be caused by the fact that the transform function expects an array or map as its first argument, but you are passing a string instead. In this case, the Subject column appears to contain an array, but you are not specifying which element of the array to apply the data_hide function to. Assuming that you want to apply the data_hide function to each element of the Subject array separately, you can modify your query as follows:

SELECT id, transform(Subject, x -> data_hide(x)) AS new_data FROM my_table

However, if you only want to apply the data_hide function to the entire Subject array if the user is not a member of the groupName group, you can modify your query as follows:

SELECT id, IF(is_member('groupName'), Subject, transform(Subject, x -> data_hide(x))) AS new_data FROM my_table

This will return the original Subject array if the user is a member of the groupName group, otherwise it will apply the data_hide function to each element of the array.

Anonymous
Not applicable

Hi @Naveena Gโ€‹ 

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance! 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group