Skip to main content

Site Migration Tool

Overview​

The Site Migration Tool allows you to migrate rm_sites data between different database environments. It compares site data and only updates fields that have changed, ensuring safe and efficient data migration.

Location​

Middleware → Tools → Site Migration

Features​

  • Compare Environments - See exactly what data differs between source and target databases
  • Selective Migration - Choose which sites to migrate
  • Credential Control - Option to include or exclude API credentials and secrets
  • Change Detection - Only updates fields that have actually changed
  • Real-Time Progress - Live progress bar and status updates as sites are migrated in batches
  • Safe Operations - Validates environments and prevents accidental data loss
  • Detailed Reporting - Shows exactly what was migrated and any errors

Use Cases​

1. Production to Staging Migration​

Sync production site configurations to your staging environment for testing:

  • Source: Production environment
  • Target: Beta/staging environment
  • Include credentials: Optional (usually yes for full replication)

2. Staging to Production Promotion​

Push tested changes from staging to production:

  • Source: Beta/staging environment
  • Target: Production environment
  • Include credentials: Review carefully

3. Database Environment Updates​

After creating a new database environment, populate it with existing site data:

  • Source: Current active environment
  • Target: Newly created environment
  • Include credentials: Yes

4. Partial Data Sync​

Update only non-sensitive configuration data:

  • Source: Any environment
  • Target: Any environment
  • Include credentials: No (updates only names, URLs, sync settings, etc.)

How to Use​

Step 1: Select Environments​

  1. Navigate to Middleware → Tools → Site Migration
  2. Select your Source Environment (database to copy FROM)
  3. Select your Target Environment (database to copy TO)
  4. Choose whether to Include Credentials (API keys, tokens, secrets)

Step 2: Compare Data​

  1. Click Compare Data
  2. Review the comparison results:
    • New Sites - Sites that exist in source but not in target
    • Sites with Changes - Sites that have different values
    • Unchanged Sites - Sites that are identical (no action needed)

Step 3: Review Differences​

For each site with changes, you'll see:

  • Field name
  • Source value (what will be copied)
  • Target value (what will be replaced)

Step 4: Perform Migration​

Choose one of these options:

  • Migrate Selected Sites - Check the boxes next to specific sites and click this button
  • Migrate All Changes - Migrate all new sites and sites with changes

Progress Tracking: When migration starts, you'll see:

  • A progress bar showing percentage complete (0-100%)
  • Real-time counters: Total sites, Completed, Successful, Failed
  • Live updates as each batch of sites is processed
  • Detailed results table showing status for each site as they complete

Sites are migrated in batches of 5 for optimal performance and progress feedback.

Step 5: Review Results​

After migration:

  • ✓ Success indicators show which sites were updated
  • ✗ Error indicators show any failures with specific error messages
  • Number of fields updated for each site

Data Fields​

Always Compared & Migrated​

  • Site prefix, platform, name, URL, domain
  • Active status, staging flag
  • Sync settings (inventory, orders, tracking, Bazaarvoice, kits)
  • BigCommerce/WooCommerce configuration (store hash, channel ID, etc.)
  • Bazaarvoice configuration (name, host, URL, brand)
  • Health check URLs
  • Tracking settings (auto-complete, email, status filters)
  • Signifyd settings
  • Webhook status
  • Weight units
  • USPS Ground Advantage setting

Optionally Migrated (when "Include Credentials" is checked)​

  • BigCommerce client ID and auth token
  • WooCommerce consumer key and secret
  • Bazaarvoice username and password
  • NMI security key
  • Kit Builder API key
  • Webhook secret
  • Signifyd API key

Important Notes​

Security Considerations​

⚠️ Credentials are sensitive!

  • Credentials are encrypted in the database
  • When migrating credentials, they are decrypted from source and re-encrypted for target
  • Only users with manage_options capability can access this tool
  • Consider whether you really need to copy production credentials to staging

Database Connections​

  • Source and target must be different environments
  • Both database connections must be working
  • The tool creates temporary connections to each environment
  • Original environment connections are not affected

Data Integrity​

  • Existing records are updated, not replaced entirely
  • Only fields that have changed are updated
  • Site IDs may differ between environments (matched by prefix)
  • Timestamps (created_at, updated_at) are preserved
  • The tool does not delete data in the target
  • Protected Fields - Critical fields (domain, name, site_url, checkout_domain) will not be overwritten with empty values if the target already has a non-empty value. This prevents accidental data loss when migrating from incomplete source data.

Best Practices​

  1. Backup First - Always backup your target database before migrating
  2. Test in Safe Direction - Test migrations from production to staging first
  3. Review Carefully - Check the comparison results before migrating
  4. Credentials Last - Consider migrating without credentials first, then add them manually
  5. One Direction - Avoid bi-directional syncing to prevent conflicts
  6. Document Changes - Keep notes on what was migrated and when

Troubleshooting​

"Invalid environment selection"​

  • Ensure both source and target environments are configured in DB Environments
  • Check that database credentials are correct
  • Verify network connectivity to both databases

"Source and target environments must be different"​

  • You cannot migrate from an environment to itself
  • Choose different source and target environments

"No sites selected for migration"​

  • Check at least one checkbox before clicking "Migrate Selected Sites"
  • Or use "Migrate All Changes" to migrate everything at once

Migration fails for specific sites​

  • Check the error message in the results table
  • Common issues:
    • Missing required fields in source data
    • Database permissions problems
    • Network connectivity issues
    • Field validation failures

Credentials not migrating​

  • Ensure "Include Credentials" checkbox is checked before comparing
  • Verify that source environment has the credentials
  • Check encryption key is consistent

Technical Details​

Comparison Algorithm​

  1. Fetches all sites from both environments
  2. Indexes target sites by prefix for fast lookup
  3. For each source site:
    • If not in target → marked as "new site"
    • If in target → compares each field value
    • If values differ → marked as "site with changes"
    • If identical → marked as "unchanged"

Migration Process​

  1. Validates user permissions and environment selection
  2. For each selected site:
    • Fetches full source site data
    • Looks up matching site in target (by prefix)
    • Builds update/insert data with only changed fields
    • Performs database operation
    • Records success or failure

Database Operations​

  • Uses $wpdb->update() for existing sites
  • Uses $wpdb->insert() for new sites
  • Encrypts credentials during migration
  • Does not modify timestamps unless fields change

API Reference​

AJAX Endpoints​

Compare Data​

  • Action: gsm_compare_site_data
  • Method: POST
  • Nonce: gsm_site_migration
  • Parameters:
    • source_env (int) - Source environment ID
    • target_env (int) - Target environment ID
    • include_credentials (string) - '1' or '0'
  • Returns: Comparison results with new sites, sites to update, unchanged sites

Migrate Data​

  • Action: gsm_migrate_site_data
  • Method: POST
  • Nonce: gsm_site_migration
  • Parameters:
    • source_env (int) - Source environment ID
    • target_env (int) - Target environment ID
    • include_credentials (string) - '1' or '0'
    • site_ids (array) - Array of site IDs to migrate
  • Returns: Migration results with success count, error count, detailed status

Troubleshooting​

All Sites Showing as "Unchanged" When They Should Be Different​

Symptom: When comparing two different database environments (e.g., Beta to Local), the migration tool shows all sites as "Unchanged" even though the data in the databases is different (e.g., Beta has site names/URLs but Local doesn't).

Cause: This was a bug in versions prior to 1.14.1 where the Site_Repository class didn't check for custom database connections from the migration tool. Both source and target repositories ended up querying the same database (usually the active environment) instead of their respective configured databases.

Resolution: Fixed in version 1.14.1 by updating Site_Repository::get_tasks_db() to prioritize custom database connections from the Settings_Manager. The method now checks:

  1. Custom database connection (used by migration tool) - NEW
  2. wp-config.php constants
  3. Active DB environment
  4. WordPress database fallback

Verification: After upgrading, re-run the comparison. You should now see actual differences between environments. Check the PHP error log for debug messages showing the connection details for each database.

"Connection Failed" Errors​

Cause: Database credentials in the environment configuration may be incorrect or the database server may be unreachable.

Resolution:

  1. Go to Middleware → DB Environments
  2. Click "Edit" on the problematic environment
  3. Verify host, port, database name, username, and password
  4. Use the "Test Connection" button to verify
  5. Ensure the database server allows connections from your WordPress server

Selected Sites Not Migrating​

Cause: Database write permissions or constraint violations.

Resolution:

  1. Check PHP error log for detailed error messages
  2. Verify the target database user has INSERT and UPDATE permissions
  3. Ensure site prefixes don't violate unique constraints

Some Fields Not Updating (Domain, Name, Site URL)​

Symptom: After migration, certain fields like domain, name, site_url, or checkout_domain don't update in the target database even though they show differences in the comparison.

Cause: This is intentional protection behavior introduced in version 1.14.1. The migration tool will NOT overwrite non-empty values in the target database with empty values from the source database for critical fields.

Example:

  • Source (Beta): domain = '' (empty)
  • Target (Local): domain = 'example.com' (has value)
  • Result: domain in target remains 'example.com' (not overwritten with empty)

Rationale: This prevents accidental data loss when migrating from incomplete or partially configured source data. These fields are considered critical site identification data that should not be erased.

Protected Fields:

  • domain - Site domain name
  • name - Site display name
  • site_url - Full site URL
  • checkout_domain - Checkout domain for multi-domain setups

Workaround: If you intentionally want to clear these fields in the target:

  1. Manually set them to empty in the target database before migration, OR
  2. Update them directly in the target environment after migration

Version History​

  • 1.14.2 - Added real-time progress tracking with progress bar, live counters, and batch-by-batch status updates during migration
  • 1.14.1 - Fixed bug where both source and target repositories connected to the same database; Added protection to prevent empty values from overwriting non-empty critical fields
  • 1.14.0 - Initial release of Site Migration Tool