migration Azure to AWS

[thiagoawstest]

Hello, I need to migrate from Databricks Azure to AWS, using tool-databricks-migration generates many errors, if I do it manually using databeicks-cli, what would be the best practice?

Any tips, for example:
-first migrate notebooks
-second jobs
-third unity catalog
-.........

Thanks.

[Kaniz_Fatma]

Hi @thiagoawstest, 

• One alternative is to use the databrickslabs/migrate library. It provides scripts for one-off ST-to-E2 migrations. You can find more details in the GitHub repository.
• Tips to migrate from Databricks Azure to AWS are as follows:-
  • Start by migrating your notebooks. Export them from Azure Databricks and import them into AWS Databricks.
  • Next, migrate your jobs. Recreate them in Databricks on AWS, ensuring that the configurations match.
  • If you’re using the Unity Catalog (Databricks Metastore), export the metadata from Azure and import it into AWS.
  • Set up your clusters in AWS Databricks, considering factors like instance types, autoscaling, and network settings.
  • Configure authentication and access controls in Databricks on AWS.
  • Set up monitoring and logging for your Databricks on AWS environment.
  • Test your migrated notebooks and jobs thoroughly.
  • Review your Databricks on AWS cost structure. Optimize instance types and storage options.
  • Leverage AWS Cost Explorer to track usage and cost.
  •  Additionally, consider reading Databricks’ blog series on deploying Databricks on AWS for more detai...guidance.If you encounter any specific issues during the migration, feel free to ask for further assistance! 😊

[thiagoawstest]

Hi, perfect, I even started using Databricks-Migration-Tool, but during the export it ended up generating a lot of errors in the way the objects were in Azure, so I decided to migrate separately.

I'm using databricks-cli, I've already migrated the notebook and jobs, for the Unity Catalog would it be possible to use databricks-cli?

Regarding authentication and access control, could you provide me with more direct documentation? Today in Azure it is used via AD (Enter ID), this would be the doc:
https://docs.databricks.com/en/admin/account-settings-e2/single-sign-on/azure-ad.html

Thanks.