Migrating from MongoDB to Percona Server

This article describes how to replace MongoDB Community Edition with Percona Server for MongoDB with disk encryption enabled.

The standard in-place upgrade method (replacing binary files on the existing server) is not suitable when encryption needs to be enabled, because existing data files cannot be encrypted in place. To solve this, a temporary MongoDB replication setup is used.

The procedure involves minimal platform downtime. Most of the migration is performed without stopping Altcraft Platform, with a brief downtime required only during the final switchover to the new server.

For more details, refer to the official Percona documentation.

caution

Before starting, make a backup of all MongoDB data.

Migration Overview

The migration follows this process:

1. On the existing MongoDB Community server, replication mode (replSet) is enabled, and the server becomes PRIMARY.
2. A second clean server is provisioned with Percona Server for MongoDB installed and encryption enabled.
3. The second server is added to the replication set and performs an initial sync. During synchronization, it receives data from the PRIMARY and writes it to disk in encrypted form.
4. After the initial sync completes and replication status is verified, the PRIMARY role is switched to the new server.
5. Altcraft Platform is reconfigured to point to the new server.
6. The old server is decommissioned.

Infrastructure Preparation

The second server must meet the following requirements:

• Same Ubuntu version as the primary server
• Disk space at least equal to the current MongoDB server
• Network connectivity between servers on port 27017 in both directions

The second server must be clean, with no MongoDB installed before starting.

It is also recommended to check the oplog size on the current server in advance. It must be large enough for the new server to complete the initial sync and catch up with changes written to the database during synchronization.

Check MongoDB Version

On the current server, ensure that the MongoDB Community version matches the Percona Server for MongoDB version to be installed:

/usr/local/mongodb/bin/mongod --version | head -1

The Percona Server for MongoDB version must be from the same major branch as MongoDB Community. For example: MongoDB 8.0.10 → Percona Server for MongoDB 8.0.x.

Configure Replication on the Current Server

Edit the configuration file /etc/mongod.conf:

replication:
  replSetName: rs0

net:
  port: 27017
  bindIp: 0.0.0.0   # required for the second server to connect

Restart MongoDB:

systemctl restart mongod

Initialize replication:

/usr/local/mongodb/bin/mongosh --eval "rs.initiate()"

Check the node status:

/usr/local/mongodb/bin/mongosh --eval "rs.status()"

Confirm that the current server has PRIMARY status.

Install and Configure Percona Server on the Second Server

On the second server, install Percona Server for MongoDB of the same version as the first server.

Add the Percona repository:

wget https://repo.percona.com/apt/percona-release_latest.generic_all.deb
dpkg -i percona-release_latest.generic_all.deb
percona-release setup psmdb80   # for version 8.0
apt update
apt install percona-server-mongodb

Installing a Specific Version of Percona Server for MongoDB

If you need to install a specific version rather than the latest, first list available versions:

apt-cache madison percona-server-mongodb

Example output:

percona-server-mongodb | 8.0.10-5.jammy | http://repo.percona.com/psmdb-80/apt jammy/main amd64 Packages
percona-server-mongodb | 8.0.9-4.jammy  | http://repo.percona.com/psmdb-80/apt jammy/main amd64 Packages

Then install the desired version by specifying the version number for the Percona Server for MongoDB packages:

apt install \
  percona-server-mongodb=<version> \
  percona-server-mongodb-server=<version> \
  percona-server-mongodb-shell=<version> \
  percona-server-mongodb-mongos=<version> \
  percona-server-mongodb-tools=<version>

Generate an encryption key:

openssl rand -base64 32 > /etc/mongodb-encryption-keyfile
chmod 600 /etc/mongodb-encryption-keyfile
chown mongod:mongod /etc/mongodb-encryption-keyfile

Edit /etc/mongod.conf on the second server:

systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

net:
  port: 27017
  bindIp: 0.0.0.0

storage:
  dbPath: /var/lib/mongodb
  wiredTiger:
    engineConfig:
      directoryForIndexes: true

security:
  enableEncryption: true
  encryptionKeyFile: /etc/mongodb-encryption-keyfile

replication:
  replSetName: rs0

info

The preferred method for storing the master key is HashiCorp Vault.

If HashiCorp Vault is not used, the mongodb-encryption-keyfile should be stored in a secure location, such as an encrypted partition or container, with permissions set to 600.

Disk encryption reduces the risk of data exposure in case of media compromise, such as disk theft or improper hardware disposal.

Using HashiCorp Vault Instead of a Local Key File

Instead of a local encryptionKeyFile, Percona Server for MongoDB can retrieve the master key from HashiCorp Vault.

Before starting the second server, ensure the following are prepared:

• A running HashiCorp Vault server
• KV v2 secrets engine
• Vault token with access to the required secret path
• Vault CA certificate if using a custom certificate authority

In this case, replace the encryptionKeyFile parameter with the following configuration:

security:
  enableEncryption: true
  vault:
    serverName: VAULT_HOST
    port: 8200
    tokenFile: /etc/mongodb-vault-token
    secret: secret/data/mongodb/node2
    serverCAFile: /etc/ssl/certs/vault-ca.crt

The tokenFile must contain the Vault token.

chown mongod:mongod /etc/mongodb-vault-token
chmod 600 /etc/mongodb-vault-token

info

It is recommended to store the tokenFile in a secure location, such as an encrypted partition or container.

Ensure the processManagement section is not present in the file.

Stop MongoDB and clear the data directory. It must be empty for initial sync:

systemctl stop mongod
rm -rf /var/lib/mongodb/*

Start MongoDB:

systemctl start mongod

Check the service status and MongoDB logs for any startup errors:

systemctl status mongod
tail -n 50 /var/log/mongodb/mongod.log

Add the Second Server to the Replica Set

On the first server (current PRIMARY), connect to MongoDB:

/usr/local/mongodb/bin/mongosh

Add the second server:

rs.add("SECOND_SERVER_IP:27017")

Monitor the synchronization process:

rs.status()

The new server typically goes through STARTUP and STARTUP2 states before transitioning to SECONDARY.

Wait until the second server reaches SECONDARY status.

Verify Replication Status

Before switching over, ensure that the initial sync is complete and the new server does not have significant lag behind the PRIMARY.

Check the replication status:

/usr/local/mongodb/bin/mongosh --eval "rs.status()"

Verify that the second server:

• is in SECONDARY state
• is not performing initial sync
• has no significant lag from the current PRIMARY

Only proceed to the platform switchover after these checks.

Verify Encryption

On the second server, perform the following checks.

Startup parameters should show enableEncryption: true:

mongosh --eval "db.serverCmdLineOpts()" | grep -A 5 encryption

If using a local key file, check permissions:

ls -la /etc/mongodb-encryption-keyfile

If using HashiCorp Vault, verify Vault parameters in the startup configuration:

mongosh --eval "db.serverCmdLineOpts()" | grep -A 10 vault

The log should contain a record indicating startup with encryption enabled:

grep -i encryption /var/log/mongodb/mongod.log

Data files on disk should not contain readable fragments of user data in plaintext:

strings /var/lib/mongodb/*.wt | head -20

Switch Altcraft Platform to the New Server

During the final switchover, Altcraft Platform must be stopped briefly to prevent new writes while changing the primary server.

Stop the platform services:

/opt/akd/akd stop -fk

Connect to the current PRIMARY and step it down:

/usr/local/mongodb/bin/mongosh

rs.stepDown()

After this, check the replication status:

rs.status()

Verify that the second server has become PRIMARY and the first server has transitioned to SECONDARY.

Update Altcraft Platform Configuration

On the Altcraft Platform server, edit /opt/akd/config/main.json:

{
  "CONTROLDB_IP": "SECOND_SERVER_IP",
  "FILEDB_IP": "SECOND_SERVER_IP"
}

Start the platform:

/opt/akd/akd start -f

Check Altcraft Platform logs for any MongoDB connection errors.

Decommission the First Server

After confirming that the platform is stable with the new server, the old MongoDB server can be decommissioned.

Stop and disable MongoDB on the first server:

systemctl stop mongod
systemctl disable mongod

If desired, remove MongoDB Community packages:

dpkg -l | grep mongo
apt remove mongodb-org mongodb-org-mongos mongodb-org-server mongodb-org-shell mongodb-org-tools

To free up disk space, remove data and log directories. Do this only after confirming data integrity on the new server:

rm -rf /var/lib/mongodb/*
rm -rf /var/log/mongodb/*