WP-CLI: Migrating Thousands of WooCommerce Products Without Timeouts

The Problem: 504 Gateway Timeout

You have 50,000 WooCommerce products and need to update a meta field on each one. You write a PHP script, hit it from the browser, and 30 seconds later: 504 Gateway Timeout.

This is a classic. The web server (Nginx, Apache) kills the connection before the script finishes. Increasing PHP’s max_execution_time only delays the problem — the real issue is that the browser isn’t designed for this.

Mike Davey explains it well in his article on Delicious Brains (source: deliciousbrains.com): the combination of server timeouts, PHP-FPM limits, and memory exhaustion makes the browser impractical for bulk operations.

The solution: WP-CLI. No web server, no timeout, no time limit. Just PHP on the command line, with direct access to the entire WordPress API.

The Memory Trap: get_posts() with -1

The first instinct when processing all products is to write:

$products = get_posts( [
    'post_type'      => 'product',
    'posts_per_page' => -1,
] );

This loads every object into memory at once. With 50,000 products, you’ll easily blow past PHP’s 256 MB RAM allocation. And even when processing in batches of 500, WordPress’s internal object cache grows with each iteration — memory never comes back down.

The Solution: PHP Generators + WP-CLI

The approach that scales combines three things: a PHP generator to load only one ID at a time into memory, cache clearing between each record, and SQL transactions to roll everything back if something goes wrong.

Registering the Command

if ( defined( 'WP_CLI' ) && WP_CLI ) {
    WP_CLI::add_command( 'db-migrate', 'DB_Migration_Command' );
}

The defined( 'WP_CLI' ) check prevents the code from loading outside the command line — it won’t slow down your front-end.

The ID Generator

Instead of loading all products into memory, we fetch the IDs and distribute them one at a time using yield:

private function get_all_product_ids() {
    global $wpdb;
    $ids = $wpdb->get_col(
        "SELECT ID FROM {$wpdb->posts}
         WHERE post_type = 'product' AND post_status = 'publish'"
    );

    foreach ( $ids as $id ) {
        yield (int) $id;
    }
}

The difference from a regular return $ids: the generator pauses execution after each yield. It uses the same amount of memory for 10 products as it does for 1,000,000.

The Complete Command

class DB_Migration_Command {

    /**
     * Bulk-updates product metadata.
     *
     * ## OPTIONS
     *
     * [--batch-size=<number>]
     * : Number of products before a global cache flush.
     * ---
     * default: 500
     * ---
     *
     * [--dry-run]
     * : Run without saving changes to the database.
     */
    public function products( $args, $assoc_args ) {
        global $wpdb;
        $dry_run    = isset( $assoc_args['dry-run'] );
        $batch_size = max( 1, (int) $assoc_args['batch-size'] );

        $total = $wpdb->get_var(
            "SELECT COUNT(ID) FROM {$wpdb->posts}
             WHERE post_type = 'product' AND post_status = 'publish'"
        );
        $progress = \WP_CLI\Utils\make_progress_bar( 'Updating Products', $total );

        $wpdb->query( 'START TRANSACTION' );

        try {
            foreach ( $this->get_all_product_ids() as $index => $post_id ) {

                if ( ! $dry_run ) {
                    update_post_meta( $post_id, '_new_meta_key', 'updated_value' );
                }

                $progress->tick();

                // Surgical cleanup: free memory for the post we just processed
                clean_post_cache( $post_id );

                // Global cleanup: flush the object cache every X posts
                if ( 0 === ( $index + 1 ) % $batch_size ) {
                    wp_cache_flush();
                }
            }

            $progress->finish();

            if ( ! $dry_run ) {
                $wpdb->query( 'COMMIT' );
                WP_CLI::success( 'Migration complete!' );
            } else {
                $wpdb->query( 'ROLLBACK' );
                WP_CLI::log( 'Dry run complete. No changes saved.' );
            }

        } catch ( Exception $e ) {
            $wpdb->query( 'ROLLBACK' );
            WP_CLI::error( 'Critical error: ' . $e->getMessage() );
        }
    }
}

Why Every Detail Matters

SQL Transactions

START TRANSACTION + COMMIT / ROLLBACK: either everything goes through, or nothing changes. Without this, a crash mid-process leaves you with a half-migrated database — the worst possible outcome.

Dual Cache Strategy

Two levels of cleanup working together:

clean_post_cache( $post_id ) frees the cache for the post we just processed — it’s the scalpel. wp_cache_flush() clears the entire global object cache every 500 posts (configurable via --batch-size) — it’s the sledgehammer.

If your server uses Redis or Memcached, the global flush is even more critical: without it, the persistent cache grows indefinitely and eventually saturates your cache server’s memory. This is also why a standard cache plugin is not enough when dealing with bulk WooCommerce operations.

Dry-Run

The --dry-run flag runs the entire migration inside a transaction, then issues a ROLLBACK at the end. You see the progress bar advance, warnings appear if there are any, but nothing gets written to the database. It’s the dress rehearsal before production.

Progress Bar

make_progress_bar() turns a silent, nerve-wracking script into a transparent process. You see exactly where the migration stands, how much is left, and whether something is stalling.

The Complete Production Workflow

Export the database before anything:

wp db export pre-migration-backup.sql

Test with dry-run:

wp db-migrate products --dry-run

Run the migration:

wp db-migrate products --batch-size=500

Verify the data manually on a sample of products.
If anything goes wrong, restore:

wp db import pre-migration-backup.sql

If you have a staging environment, run steps 1 through 4 there first. Only execute the migration on production once dry-run + staging are validated.

Common WooCommerce Use Cases

This approach applies to many scenarios beyond a simple update_post_meta, and is part of any deep WooCommerce optimization effort:

HPOS Migration — If you’re migrating orders from wp_posts to HPOS (High-Performance Order Storage) tables and the built-in process fails on large volumes, a custom WP-CLI command with the same structure (generator + transactions + cache flush) gives you full control.

Autoloaded Options Cleanup — Walking through the wp_options table to identify and disable autoload on large options. The same pattern works: fetch IDs, process one by one, flush cache regularly.

Bulk Price Updates — Updating prices across thousands of products by applying a percentage, a per-category rule, or an exchange rate. With SQL transactions, if the calculation is wrong on one product, you roll back the entire batch.

Thumbnail Regeneration — wp media regenerate already exists, but for custom processing (watermarks, specific formats), the generator + progress bar structure is exactly what you need.

The Takeaway

Once you’re past a few thousand records, the browser is no longer an option. WP-CLI gives you a timeout-free environment, and PHP generators guarantee constant memory usage. Add SQL transactions and dry-run, and you have a complete safety net.

The code in this article is adaptable to any WooCommerce migration. Change the SQL query, change the update_post_meta, and the rest of the structure — generator, cache, transactions, progress — stays identical.

Have a complex WooCommerce migration to handle? Request a free audit — I’ll help you plan and execute the migration without risking your production environment.