https://community.databricks.com/t5/machine-learning/mapping-control-data-maintained-by-business-user/m-p/18710#M1018 <P>We are ingesting our data from ADLS into databricks as delta table. After raw layer we need to refer to a control\mapping layer which defines certain logic\measure definition. This would be incorporated in the subsequent silver or gold layer. This control layer wil be maintained by business users and regularly updated by them. My questions based on this scenario:</P><P>a. Should this control layer reside in databricks or as a csv file? I do not think business users can directly manipulate a control (delta table) on regular basis without a script.</P><P>b. If this is going to be a csv file maintained by business users then should it reside in a Sharepoint and we pull this file into ADLS using ADF pipeline and then databricks can pick from ADLS location?</P><P>c. Do you propose a better way for the correct location and maintainance of the control table for the databricks to consume for transformation logic while giving the business users rights to modify it easily?</P><P>Thanks!</P> Sat, 04 Jun 2022 15:09:49 GMT MattM 2022-06-04T15:09:49Z https://community.databricks.com/t5/machine-learning/mapping-control-data-maintained-by-business-user/m-p/18710#M1018 <P>We are ingesting our data from ADLS into databricks as delta table. After raw layer we need to refer to a control\mapping layer which defines certain logic\measure definition. This would be incorporated in the subsequent silver or gold layer. This control layer wil be maintained by business users and regularly updated by them. My questions based on this scenario:</P><P>a. Should this control layer reside in databricks or as a csv file? I do not think business users can directly manipulate a control (delta table) on regular basis without a script.</P><P>b. If this is going to be a csv file maintained by business users then should it reside in a Sharepoint and we pull this file into ADLS using ADF pipeline and then databricks can pick from ADLS location?</P><P>c. Do you propose a better way for the correct location and maintainance of the control table for the databricks to consume for transformation logic while giving the business users rights to modify it easily?</P><P>Thanks!</P> Sat, 04 Jun 2022 15:09:49 GMT https://community.databricks.com/t5/machine-learning/mapping-control-data-maintained-by-business-user/m-p/18710#M1018 MattM 2022-06-04T15:09:49Z https://community.databricks.com/t5/machine-learning/mapping-control-data-maintained-by-business-user/m-p/18712#M1020 <P>Thanks for your response. Can business user without the help of any script modify any rows in the table after loading it onetime from CSV fiels? </P> Tue, 07 Jun 2022 19:52:50 GMT https://community.databricks.com/t5/machine-learning/mapping-control-data-maintained-by-business-user/m-p/18712#M1020 MattM 2022-06-07T19:52:50Z