How worker nodes get the packages during scale-up?

[satycse06]

Hi,

We are working with one of the repository where we used to download the artifact/python package from that repository using index url in global init script but now the logic is going to be change we need give the cred to download the package and that will be valid for few hours.

So I have the question how the new worker node will get the same package/artifact during scale up it will download using the global init script or it will get it from the driver node so whether it will hit the configured index url or not?

I have second question If I consider that it(new worker node)  will execute the global init script and it will again download the package from the configured index url  then whether it will get the new credentials from the azure library where the new cred will update or it will execute with the old cred what it got it on driver node during the first execution?

Regards,

Satya

[Kaniz_Fatma]

Hi @satycse06, 

Great questions!

1. How will the new worker node get the same package/artifact during scale-up?

When a new worker node is added to the cluster, it executes the global init script. This means it downloads the package/artifact from the configured index URL, just like the driver node did initially. The new worker node does not get the package from the driver node; instead, it independently downloads it using the global init script.

2. Will the new worker node get the new credentials from the Azure library or use the old credentials?

If the global init script is designed to fetch credentials from the Azure library, the new worker node will execute this script and obtain the latest credentials available at that time. This means that if the credentials have been updated in the Azure library, the new worker node will use the updated credentials to download the package from the configured index URL.

Summary

• New worker nodes will execute the global init script and download the package/artifact from the configured index URL.
• Credentials: If the global init script fetches credentials from the Azure library, the new worker node will use the latest credentials available at the time of execution.

Would you like more details on how to configure the global init script or manage credentials in Azure?