Updating python version from 3.8 to 3.12 for s3 ingestion

[sandipkumar]

I went to AWS Cloudformation stack and edited the template from python 3.8 to 3.12 and updated. I did this for both the workspace stack and the s3 ingestion stack. Will it break anything? Do I need to make any changes in the python code in the template so that there are no problems? If so, what do I need to change?

What is the best way to update cloud formation template automatically created by Databricks while still not breaking it?

All the s3 connections I made were made in the last 3 months. Why is Databricks still using python 3.8 in its templates?

[sandipkumar]

Hi @Retired_mod ,

Thanks a lot! I will look at StackSets.

As I mentioned the code is not written by me but Databricks. Why does Databricks not use a new python version in its default stacks? We are low on resources and heavily rely on the default Databricks code and minimise devops interference. Moreover, even if we use StackSets to update the version, how do we know that the Databricks code will work and not break? Are we sure that code written by databricks is backward compatible?