Migrating to the Standalone or Automation Suite Environment

Note: A classic upgrade to AI Center 2021.10 is not available, instead you need to install AI Center standalone 2021.10 or as part of a new Automation Suite installation and you can migrate your work from the previous environment. This migration script works for both type of installations, AI Center standalone or Automation Suite environment.

Overview

Use this procedure to migrate from 2021.4 or 2020.10 to 2021.10. To do this, a new installation and a migration is needed instead of upgrade.

Use the following steps to migrate the AI Center tenant data to the AI Center standalone environment or Automation Suite environment. This procedure offers migration support for the following:

• Database
• Datasets
• ML Packages
  Note: The Document Understanding on-demand bundle must be executed before deploying and training the migrated DU ML packages.

  In case of airgapped installation, make sure that the models with the same major version are added to your out of the box models. For more information on installing models offline, see ML Packages Offline Installation.

Prerequisites

Make sure that AI Center is up and running in both environments.

Additionally, the following prerequisites must be met on the machine where the migration script runs:

• A Linux machine with the following dependencies installed: aws s3, s3cmd, jq, zip. The machine can be one of the AI Center machines or a new one. For more information, see the example below.

Installing aws s3, s3cmd, jq, zip dependencies

Note: In case of unmet dependencies, run the following commands with no packages:

sudo susudo apt install -y jq awscli s3cmd zip gitsudo apt --fix-broken installsudo apt install -y jq awscli s3cmd zip

• Install MS SQL utility from here. You can do so by running the following commands:

  sudo su
  curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list | sudo tee /etc/apt/sources.list.d/msprod.listsudo apt-get update 
  sudo apt-get install mssql-tools unixodbc-devsudo apt-get update 
  sudo apt-get install mssql-toolsecho 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bash_profileecho 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc
  source ~/.bashrc

• Connectivity with both AI Center instances.
• Connectivity with both Orchestrator instances (in case the instances are not the same).
• Connectivity with SQL servers (in case the instances are not the same).
• Possibility to access both AI Center instances.

Procedure

Step 1: Create a tenant with the same name in the Automation Suite or new AI Center environment

Create a corresponding tenant in the Automation Suite or new AI Center environment for every tenant that you want to migrate.

Step 2: Generate the Storage-creds.json file on the first machine (also referred as source machine)

This step needs to be run on the first machine, the one you want to migrate to the new environment.

1. Connect using SSH to the machine you want to migrate.
2. Generate the storage-creds.json file by executing the command below:

   wget https://raw.githubusercontent.com/UiPath/ai-customer-scripts/stable/platform/sfmigration/storagemigration/get-credentials.sh
   chmod 777 get-credentials.sh
   ./get-credentials.sh

Note: If running from a third machine make sure to run the following command :

./get-credentials.sh {PUBLIC_IP_REPLICATED_MACHINE}

Note: If the error message from below is displayed after running the above script, run the bash -l command and try running the above script again

The file generated at Step 2 (storage-creds.json) will be used in further steps. Make sure that you copy and save the content of the file locally.

Step 3: Generate the Storage-creds.json file on the second machine

This step needs to be run on the second machine, the one running on the Automation Suite environment.

1. Open port 22 on the Automation Suite machine.
2. Connect using SSH to the Automation Suite machine.
3. Run the command below:

   sudo su
   export KUBECONFIG=/etc/rancher/rke2/rke2.yaml PATH=$PATH:/var/lib/rancher/rke2/bin

4. Download the credential file by executing the command below:

   wget https://raw.githubusercontent.com/UiPath/ai-customer-scripts/stable/platform/sfmigration/storagemigration/get-credentials-sf.sh
   chmod 777 get-credentials-sf.sh
   ./get-credentials-sf.sh

The file generated at Step 4 (storage-creds-sf.json) will be used in further steps. Make sure that you copy and save the content of the file locally.

Step 4: Configure the machine executing the script

Before going through the steps below, create a tenant with the same name in the Automation Suite environment manually. Whichever tenant has to be migrated, a corresponding tenant must be created in Automation Suite as well.

This step needs to be run on the machine executing the script.

1. Run the command below to download the migration script (This will download the script from public git repository):

   sudo su 
   cd ~
   mkdir migration
   cd migration
   git clone https://github.com/UiPath/ai-customer-scripts
   cd ai-customer-scripts
   git checkout stable
   cd platform
   chmod 777 -R sfmigration
   cd ~/migration/ai-customer-scripts/platform/sfmigration/storagemigration/

2. Replace the content in the SOURCE_CREDENTIAL_FILE file with the content from storage-creds.json (see Step 2).
3. Replace the content in the TARGET_CREDENTIAL_FILE file with the content from storage-creds-sf.json (see Step 3).
4. Go to the base directory by running the following command:

   cd ~/migration/ai-customer-scripts/platform/sfmigration/

Replace the details in the input.json file (File already present in the directory). For easier reference, the credentials are marked as TO-BE-REPLACED in the input.json file.

The sample JSON file with description of each field is mentioned below.

{
"SRC_AIC_INSTALLATION_VERSION": "TO-BE-REPLACED", // Values can be : 20.10 OR 21.4
"SRC_SERVER": "TO-BE-REPLACED",   // Replicated SQL Server host
"SRC_PKGMANAGER_DB_NAME": "TO-BE-REPLACED", // Replicated SQL Server Pkgmanager DB name
"SRC_PKGMANAGER_DB_SCHEMA": "ai_pkgmanager", // Replicated SQL Server Pkgmanager DB schema, Note : Please check schema in case of multiple dbs in replicated
"SRC_PKGMANAGER_DB_USERNAME": "TO-BE-REPLACED", // Replicated SQL Server Pkgmanager DB Username
"SRC_PKGMANAGER_DB_PASSWORD": "TO-BE-REPLACED",  // Replicated SQL Server Pkgmanager DB Password
"SRC_TRAINER_DB_NAME": "TO-BE-REPLACED", // Replicated SQL Server AI-Trainer DB Name
"SRC_TRAINER_DB_SCHEMA": "ai_trainer", // Replicated SQL Server AI-Trainer DB Schema, Note : Please check schema in case of multiple dbs in replicated
"SRC_TRAINER_DB_USERNAME": "TO-BE-REPLACED", // Replicated SQL Server AI-Trainer DB Username
"SRC_TRAINER_DB_PASSWORD": "TO-BE-REPLACED",// Replicated SQL Server AI-Trainer DB Password
"DESTINATION_SERVER": "TO-BE-REPLACED", // Destination SQL Server host i.e ServiceFabric SQL Server host
"DESTINATION_DB_NAME": "TO-BE-REPLACED", // Destination SQL Server DB Name
"DESTINATION_PKGMANAGER_DB_SCHEMA": "ai_pkgmanager", 
"DESTINATION_TRAINER_DB_SCHEMA": "ai_trainer",
"DESTINATION_DB_USERNAME": "TO-BE-REPLACED", // Destination SQL Server Username
"DESTINATION_DB_PASSWORD": "TO-BE-REPLACED", // Destination SQL Server Password
"TENANT_MAP": [
{
"SRC_TENANT_ID": "TO-BE-REPLACED", // Source Tenant Id i.e tenant UUID in replicated environment
"DESTINATION_TENANT_ID": "TO-BE-REPLACED", // Destination Tenant Id i.e Tenant UUID in the destination environment
"DESTINATION_ACCOUNT_ID": "TO-BE-REPLACED" // Destination Account UUID Id , host if Migrating to ServiceFabric standalone environment otherwise provide the actual Account UUID
}
]

Sample input.json file with filled sample values for reference (Please note the below screen shot is for migration from Single Source DB to Single Destination DB)

{
        "SRC_AIC_INSTALLATION_VERSION": "20.10",
        "SRC_SERVER": "sankar-sf-migration1sqlserver.database.windows.net",
        "SRC_PKGMANAGER_DB_NAME": "aifabric",
        "SRC_PKGMANAGER_DB_SCHEMA": "ai_pkgmanager",
        "SRC_PKGMANAGER_DB_USERNAME": "test",
        "SRC_PKGMANAGER_DB_PASSWORD": "test",
        "SRC_TRAINER_DB_NAME": "aifabric",
        "SRC_TRAINER_DB_SCHEMA": "ai_trainer",
        "SRC_TRAINER_DB_USERNAME": "test",
        "SRC_TRAINER_DB_PASSWORD": "test",
        "DESTINATION_SERVER": "sfdev2290542-9a5254d2-sql.database.windows.net",
        "DESTINATION_DB_NAME": "AutomationSuite_AICenter",
        "DESTINATION_PKGMANAGER_DB_SCHEMA": "ai_pkgmanager",
        "DESTINATION_TRAINER_DB_SCHEMA": "ai_trainer",
        "DESTINATION_DB_USERNAME": "test",
        "DESTINATION_DB_PASSWORD": "test",
        "TENANT_MAP": [
                {
                        "SRC_TENANT_ID": "d1eb428c-e188-46bf-a1dd-8908f90b4084",
                        "DESTINATION_TENANT_ID": "03527165-b242-4b98-834f-2eb3e5957223",
                        "DESTINATION_ACCOUNT_ID": "8cd64e27-12f4-427c-a575-dd4f2ea82551"
                }
        ]
}

Sample input.json file with filled sample values for reference (Please note the below screen shot is for migration from Multi Source DB to Single Destination DB)

{
        "SRC_AIC_INSTALLATION_VERSION": "20.10",
        "SRC_SERVER": "sankar-sf-migration1sqlserver.database.windows.net",
        "SRC_PKGMANAGER_DB_NAME": "aifabric_pkgmanager",
        "SRC_PKGMANAGER_DB_SCHEMA": "ai_pkgmanager",
        "SRC_PKGMANAGER_DB_USERNAME": "test_1",
        "SRC_PKGMANAGER_DB_PASSWORD": "test_1",
        "SRC_TRAINER_DB_NAME": "aifabric_trainer",
        "SRC_TRAINER_DB_SCHEMA": "ai_trainer",
        "SRC_TRAINER_DB_USERNAME": "test_2",
        "SRC_TRAINER_DB_PASSWORD": "test_2",
        "DESTINATION_SERVER": "sfdev2290542-9a5254d2-sql.database.windows.net",
        "DESTINATION_DB_NAME": "AutomationSuite_AICenter",
        "DESTINATION_PKGMANAGER_DB_SCHEMA": "ai_pkgmanager",
        "DESTINATION_TRAINER_DB_SCHEMA": "ai_trainer",
        "DESTINATION_DB_USERNAME": "test",
        "DESTINATION_DB_PASSWORD": "test",
        "TENANT_MAP": [
                {
                        "SRC_TENANT_ID": "d1eb428c-e188-46bf-a1dd-8908f90b4084",
                        "DESTINATION_TENANT_ID": "03527165-b242-4b98-834f-2eb3e5957223",
                        "DESTINATION_ACCOUNT_ID": "8cd64e27-12f4-427c-a575-dd4f2ea82551"
                }
        ]
}

SRC_TENANT_ID can be obtained by following the steps below:

1. Click on the UiPath logo on the home page of AI Center application.
2. Right click on the browser and go to inspect .Once the inspect window is opened , check the network tab for projects API call. Tenant Id can be seen in the Preview Section as in the screenshot below.
   
   

   

   
   
3. Add the DNS of the object storage of the Automation Suite machine in the machine executing script. This can be found in the extension tab of the pipeline from where the Automation Suite environment was created.

   sudo bash -c "echo "LB_IP objectstore.DNS_NAME" >> /etc/hosts"

Step 5: Run the migration script

Go to the folder where the sfmigration is downloaded in the Usage directory: Run the following command:

cd ~/migration/ai-customer-scripts/platform/sfmigration/
 
 ./mastermigrationscript.sh input.json

Troubleshooting

Creating tenants

The tenant needs to be created in the Automation Suite environment before running the migration script.

Re-running the migration

The migration script is meant to migrate datasets and ML packages only once. On rare occasions, there are some extra datasets or ML packages added in the source environment post migration is complete then in this case the migration needs to be re-run. Run the following commands in the destination database before re-running the migration:

delete from [ai_pkgmanager].[ml_package_versions_temp];
delete from [ai_pkgmanager].[projects] where tenant_id in ('T1','T2',...);
delete from [ai_pkgmanager].[ml_packages] where tenant_id in ('T1','T2',...);
delete from [ai_pkgmanager].[ml_package_versions] where tenant_id in ('T1','T2',...);
delete from [ai_trainer].[datasets] where tenant_id in ('T1','T2',...);

NOTE : 'T1','T2',... in the query above are the tenant UUIDs in the destination environment. These are the DESTINATION_TENANT_ID as mentioned in the input.json file passed while running migration script ./mastermigrationscript.sh input.json

You can then re-run the migration using the following commands:

cd ~/migration/ai-customer-scripts/platform/sfmigration/
 
./mastermigrationscript.sh input.json

Cloned packages

After the migration, cloned ML Packages present in the source environment will be marked as VALIDATION_FAILED in the Automation Suite environment. These packages need to be cloned again in the destination environment once the migration is complete.

Note: In this case, cloned packages refers to the packages from UiPath and Open-Source Packages.