Migrating classic projects

Use the instructions from this page to migrate a classic project or a project based on AI Center. There are two main steps in migrating a project:

1. Export the dataset from the classic project or the project based on AI Center.
2. Import the dataset into the modern project.

Current limitations

• Currently, importing datasets larger than 3000 pages is not supported. Only the initial 3000 pages will be successfully imported, with any additional pages failing to do so. For example, if your dataset consists of 2999 pages and you try to import a document of 4 pages, the process will not succeed.
• Batch names and corresponding batch results are not currently available. If your data is organized into batches, this information is not displayed yet, but it is saved.

Exporting a dataset from a classic project

1. Navigate to the classic project you want to migrate and open it.
2. Go to the document type you want to export and select Open document type.
   Figure 1. Open document type
   

   

   
   
3. From the Filter documents drop-down list, select Training and validation set.
   Figure 2. Training and validation set
   

   

   
   
4. Select Export.
5. Leave Current search results selected and fill in a name for your export job.
6. Select Download.
   Figure 3. Download export
   

   

   
   

Export a dataset from a project based on AI Center

1. Open AI Center and navigate to the Data Labeling page.
2. Select the Data Labeling Session you want to migrate.
   
   

   

   
   
3. Once Document Manager is open, from the Filter documents drop-down list, select Training and validation set.
   Figure 4. Training and validation set
   

   

   
   
4. Select Export.
5. Leave Current search results selected and fill in a name for your export job.
6. Select Download.
   Figure 5. Download export
   

   

   
   

Importing a dataset

1. Navigate to and open the project into which you want to import data.
2. Select Add document type and create a new custom document type.
   Figure 6. Add document type
   

   

   
   
3. On the new custom document type, select Upload and choose the zip file of the classic project you exported. Wait for the upload to finish.
   Figure 7. Upload processing
   

   

   
   

Once the upload is finished, the documents are available for training.

Model training

Once the dataset is imported, the model training starts. After the training is complete, the model score is displayed. To check detailed model scores, select the score, and then Detailed model scores.

This action takes you to the Measure page where you can access detailed model metrics.

When the same dataset is used to train an ML twice, you can observe slightly different model metrics. This can occur due to a few reasons:

• Initialization: Machine learning uses optimization methods that need initial guesses to trigger the optimization algorithms. Different initial guesses during each training could lead to various outcomes due to the unpredictable nature of these algorithms.
• Random state: Some algorithms use randomness in their operations. For instance, when training a neural network, procedures like stochastic gradient descent and mini-batch gradient descent introduce randomness. Therefore, even with identical initial model parameters and datasets, the performance of models may vary in different runs.
• Regularization: Certain algorithms include a penalty term that encourages the model to maintain smaller weights. Due to the randomness involved, the model could operate with a different weight set each time.

However, it's vital to note that these minor differences don't necessarily imply that one model is superior or inferior to another. Even with slightly varying metrics, the models' ability to comprehend data essentially remains the same, provided the differences are not significantly large. Moreover, repeating this process numerous times and taking an average should yield similar performance metrics.

Change the base model in Document type manager

If there's a significant difference between the model results from your classic project and the modern one, it could be caused by a different base model. To change the base model, follow the steps below.

1. Select the three-dot menu from your custom document type and choose Document type manager.
   
   

   

   
   
2. Navigate to the Settings tab.
3. Select the desired model from the Base model drop-down list.
   
   

   

   
   
4. After making your selection, select Save. To exit, select Back.

Export types

For classic projects, there are various methods for exporting data. Not all types of exported data are compatible for importing into modern projects. To compare the model results across both project types,filter documents by Training and validation set and select Choose search results to export the dataset. For more information on each option, check the following table.

Table 1. Types of export

Type of export	Exported data	What happens to imported data
Current search results	Exports the current filtered dataset. Use it together with the Training and validation set filter.	Documents tagged as training are used to train the model. Documents tagged as validation are used to measure the model performance. Tip: To compare model results between two project types, always export and import the dataset as Train and validation.
All labeled	Exports all annotated documents from the dataset: Train set Validation set Evaluation set	Documents tagged as training are used to train the model. Documents tagged as validation are used to measure the model performance. Documents tagged as evaluation are ignored.
Schema	Exports the list of fields and their respective settings.	A schema is imported is imported if there is none. If a schema is already defined, importing fails.
All	Exports all annotated and unannotated documents.	Documents tagged as training are used to train the model. Documents tagged as validation are used to measure the model performance. Documents tagged as evaluation are ignored. Unannoted documents are pre-annotated and treated as unconfirmed.

Importing schemas

You can import schemas along with datasets into modern projects. Follow these steps to import a schema:

1. Create a custom document type in the Build section.
2. Import the zip file that holds the schema.

Note:

• Schema imports are limited to custom document types with no pre-existing schemas.
• If you import a schema into a document type that already contains a schema, the import will fail.