https://community.databricks.com/t5/data-engineering/how-to-parameterize-key-of-spark-config-in-the-job-clusterlinked/m-p/11663#M6607 <P>how can we parameterize key of the spark-config in the job cluster linked service from Azure datafactory, we can parameterize the values but any idea how can we parameterize the key so that when deploying to further environment it takes the PROD/QA values. Thanks much in advance!!!</P> Tue, 17 Jan 2023 06:25:38 GMT KVNARK 2023-01-17T06:25:38Z https://community.databricks.com/t5/data-engineering/how-to-parameterize-key-of-spark-config-in-the-job-clusterlinked/m-p/11663#M6607 <P>how can we parameterize key of the spark-config in the job cluster linked service from Azure datafactory, we can parameterize the values but any idea how can we parameterize the key so that when deploying to further environment it takes the PROD/QA values. Thanks much in advance!!!</P> Tue, 17 Jan 2023 06:25:38 GMT https://community.databricks.com/t5/data-engineering/how-to-parameterize-key-of-spark-config-in-the-job-clusterlinked/m-p/11663#M6607 KVNARK 2023-01-17T06:25:38Z https://community.databricks.com/t5/data-engineering/how-to-parameterize-key-of-spark-config-in-the-job-clusterlinked/m-p/11664#M6608 <P>@KVNARK .​&nbsp;</P><P>You can use Databricks Secrets (create a Secret scope from AKV <A href="https://learn.microsoft.com/en-us/azure/databricks/security/secrets/secret-scopes" alt="https://learn.microsoft.com/en-us/azure/databricks/security/secrets/secret-scopes" target="_blank">https://learn.microsoft.com/en-us/azure/databricks/security/secrets/secret-scopes</A>) and then reference a secret in spark configuration (<A href="https://learn.microsoft.com/en-us/azure/databricks/security/secrets/secrets" alt="https://learn.microsoft.com/en-us/azure/databricks/security/secrets/secrets" target="_blank">https://learn.microsoft.com/en-us/azure/databricks/security/secrets/secrets</A>)</P> Tue, 17 Jan 2023 08:07:20 GMT https://community.databricks.com/t5/data-engineering/how-to-parameterize-key-of-spark-config-in-the-job-clusterlinked/m-p/11664#M6608 daniel_sahal 2023-01-17T08:07:20Z https://community.databricks.com/t5/data-engineering/how-to-parameterize-key-of-spark-config-in-the-job-clusterlinked/m-p/11665#M6609 <P>Thanks a lot. This helps @Daniel Sahal​&nbsp;</P> Tue, 17 Jan 2023 09:57:49 GMT https://community.databricks.com/t5/data-engineering/how-to-parameterize-key-of-spark-config-in-the-job-clusterlinked/m-p/11665#M6609 KVNARK 2023-01-17T09:57:49Z https://community.databricks.com/t5/data-engineering/how-to-parameterize-key-of-spark-config-in-the-job-clusterlinked/m-p/11666#M6610 <P>@Daniel Sahal​&nbsp;- what ever the links you share for any requirement is really very much helpful. Kudos...</P> Tue, 17 Jan 2023 10:10:28 GMT https://community.databricks.com/t5/data-engineering/how-to-parameterize-key-of-spark-config-in-the-job-clusterlinked/m-p/11666#M6610 KVNARK 2023-01-17T10:10:28Z https://community.databricks.com/t5/data-engineering/how-to-parameterize-key-of-spark-config-in-the-job-clusterlinked/m-p/11667#M6611 <P>@KVNARK .​&nbsp;Thanks! I'm always happy to help <span class="lia-unicode-emoji" title=":slightly_smiling_face:">🙂</span></P> Tue, 17 Jan 2023 10:11:54 GMT https://community.databricks.com/t5/data-engineering/how-to-parameterize-key-of-spark-config-in-the-job-clusterlinked/m-p/11667#M6611 daniel_sahal 2023-01-17T10:11:54Z