Migrating Elasticsearch Indices

By using the Snapshot and Restore features of Elasticsearch you can create a backup repository of all indices, which can then be used to restore your current server or to move to a new server, if needed. This is useful in providing a seamless transition in case of a hardware failure, or when needing to upgrade.

In this section we go through the migration procedure on a Windows server machine. For additional information about snapshots, and the full list of available settings and commands for all supported Elasticsearch versions, see the official Elastic documentation.

It is necessary to have a shared file system between your current server and the new one. In this example, we used a shared folder that was created on the new server.

Note:

Be sure the account being used on each server has Read and Write access to this shared directory.

Also, any Kibana visualizations previously set up are not saved and need to be recreated.

Creating a Snapshot

From your current server, follow these steps:

1. Stop the Elasticsearch service.
2. Add the shared directory in the elasticsearch.yml file: path.repo: [“\\\\server_name\\folder_name”].
3. Restart the Elasticsearch service.
4. In the Kibana Dashboard, in the Dev Tools section, click Console. The Console window opens.

5. Create the repository, using a meaningful name:

   PUT /_snapshot/backup_old/ { “type”: “fs”, “settings”:{ “location”: “\\server_name\\folder_name” } }

6. If successful, the console response is: “acknowledged”: true.
7. Create the snapshot, giving it a meaningful name: PUT /_snapshot/backup_old/snapshot_1?wait_for_completion=true.
   Note: The ?wait_for_completion parameter determines if the request returns when the snapshot is complete (true), or when it is initialized (false). For large repositories this command may take several minutes to return, even when this parameter is set to false.

When that command has completed, your snapshot is saved in the designated repository and is available for use. It is recommended to take frequent snapshots to keep the repository as current as possible. Snapshots are taken incrementally, saving only new data in any subsequent execution, so it is not resource intensive to do so.

Restoring From a Snapshot

From your new server, follow these steps:

1. Stop the Elasticsearch service.
2. Add the shared directory in the elasticsearch.yml file: path.repo: [“\\\\server_name\\folder_name”].
3. Restart the Elasticsearch service.
4. In the Kibana Dashboard, in the Dev Tools section, click Console. The Console window opens.

5. Create the repository, using the same name as before:

   PUT /_snapshot/backup_old/ { “type”: “fs”, “settings”:{ “location”: “\\server_name\\folder_name” } }

6. If successful, the console response is: “acknowledged”: true.
7. Load the snapshot: POST/_snapshot/backup_old/snapshot_1/_restore.

All previous indices are rebuilt, though depending on the size it can take some time before they all are synchronized. Restoration can be performed into an already running cluster, or a new machine.