Fixing WordPress Multisite Cron for high-loaded networks

  1. How wp-cron works?
  2. CRON for WordPress Multisite: An Obvious Approach
  3. A Better Approach
  4. And my approach for high-loaded networks

As you remember, WordPress in-built CRON doesn’t work the same way as classical CRON does.

Instead of being triggered on a specified time, it fires only when someone visits the website. So if there are no visits, there would be no wp-cron triggering, and therefore no tasks would be executed.

And vice versa, if you’ve been lucky to get loads of visitors on your website WordPress CRON system will be triggered every single time when someone views any page and checks for scheduled tasks. This way WordPress CRON causes extra server load (CPU usage increases) which may significantly harm website performance. That’s speaking of single install.

When it comes to WordPress Multisite, your problems multiply on the number of websites on your install.

Fixing WordPress Multisite Cron for high-loaded networks

How wp-cron works?

WordPress CRON on Multisite works independently on each blog.

Say, you’ve got posts scheduled on website A and B.

When someone visits website A, the scheduled posts on blog A, though with delay, but will be published eventually. But posts on website B would never be published unless someone visits website B too.

Having missed schedule issue on every site of the network can be a rather annoying thing. And you may face another problem when you get some significant traffic on your Multisite install.

That CPU usage issue multiplies too, according to the number of websites in your network. That’s why using in-built CRON for Multisite is not a great idea at all. Instead, we should set a real CRON job.

How to set a CRON job for WordPress Multisite and fix missed schedule issue across the network?

An Obvious Approach

To reduce CPU usage and get tasks executed no matter, whether there are hits on the blog or not, we should replace WordPress Multisite CRON with real CRON.

Here I described in details how to do this for single install.

The steps for Multisite are pretty similar, except that we need to trigger CRON on every single site of the network (because they work independently, as you remember).

1. Disable WordPress CRON across the whole network in wp-config.php:

define('DISABLE_WP_CRON', true);

2. We will trigger publishing on every site with the help of foreach loop.

Create trigger.php in the same root directory, where you have your wp-config.php.
I have this awesome wrapper for doing bulk actions in WordPress Multisite, which has saved me loads of time. Let it work for you too:

So we just place this piece of code after the line //here's what we're gonna do...  

$site_url = get_site_url($blog_id); 
$command = $site_url.'/wp-cron.php?doing_wp_cron'; 
wp_remote_get($command); 

and it will trigger each site of our network automatically.

Entire trigger.php will look like this:

3. All that you need now, is just to add one single CRON job for your main blog on your server:

wget -O /dev/null https://network-main-blog.com/trigger.php

How to do it through CPanel see here in details.

Also, mind to set a reasonable time interval.

That’s all! According to the specified recurrence (e.g. once a day) Unix CRON will hit trigger.php -> which will hit wp-cron.php on every blog in the network-> which will check if there are any tasks scheduled, and execute them.

This is a sufficient and clear way to set scheduled tasks on WordPress Multisite install.

But what you’re gonna do if you’ve got rather a large network and you are tight on server resources? Let’s see a…

A Better Approach

All scheduled tasks in WordPress are stored in wp_options table.

When you schedule a task to publish 50 drafts daily, 50 tasks are saved in your wp_options table. Most part of this table including CRON jobs is loaded at every single page load. That’s okay for single install but it can significantly slow down your site speed on WordPress Multisite network.

Say, you’ve got a middle-sized network of 100 sites which leads us to 50*100=5000 additional lines in wp_options table (mind, this is the same database!). Furthermore, the more visits your network gets the more server recourses will be consumed for this operation only.

To sleep peacefully at night without fearing your MySQL server collapse any moment we would rather skip WordPress scheduling. Instead of CRON job for publishing scheduled posts we’ll set a CRON job to publish posts directly.

Farther I’m giving detailed instructions on how to arrange this, but before you proceed, please, keep in mind this approach may not be suitable for every single project. For example, if the specific time of publishing is extremely important for you this is not what you want to do.

But if you’ve got loaded network and you need to publish drafts daily, save server resources and don’t mind the publish time to be +- few hours, you should definitely try this out.

1. First, we’ll need the very function that will actually publish 50 drafts.

Create in the WordPress root directory publishing.php with the following content:

This simple piece of code will publish drafts directly, without scheduling.

2. Now we need to get this publishing.php triggered for execution across whole network. We are replacing WP CRON with server CRON the same we did in the previous option. 

Just mind place publishing.php instead of wp-cron.php?doing_wp_cron

$command = $site_url.'/publishing.php';
and it will trigger each site of our network automatically.

Voilà! We’ve managed to reduce CPU and memory usage by replacing WP Multisite CRON with real CRON job across the network and skipping scheduling at all. Cool, isn’t it?

But…

As I already said for small networks this approach is pretty good, but in case you got 100+ blogs it means publish function will be executed 50*100 = 5000 times at once. And you hardly want it, right?

My approach to set up CRON for high-loaded WordPress Multisite networks

How to publish 50 posts a day on more than 100 blogs without making your server collapse?

Overview: WordPress Multisite network that consists of 100+ blogs. 50 posts should be published daily on every blog. It leads us to 50*100=5000 posts that should be published by CRON every day.

Objective: This should be executed with minimal server load as this network is rather loaded apart of publish process.

Here I’m using another approach that reduces CPU usage dramatically. This way we’ve got:

  • no CRON event to schedule posts for publishing
  • no CRON events for publishing each of 50 posts
  • CRON is triggered once (!) a day on each website of the network
  • no overload by bulk publishing, just 50 posts are published at a time

This approach may be somewhat tricky for a developer but this is how we use as fewer server resources as possible.

We’ll add a separate CRON job for every website in the network and give it a command to execute  publishing.php directly.

We have already got this file prepared, now we need to add a separate CRON job for every site in the network.

Each cronjob should be scheduled with 1-minute interval. Imagine, you have 3 sites in your network (but you won’t be bothering with all these in that case, would you ;), so CRON jobs should be executed like this:

site A — at 00:01

site B — at 00:02

site C — at 00:03

Each executed once a day, at a precise time.

This way only one site got triggered in a minute, no overload, no problems.

So, there are 3 commands which you need to add to your crontab:

wget -O /dev/null http://siteA.com/publishing.php
wget -O /dev/null http://siteB.com/publishing.php
wget -O /dev/null http://siteC.com/publishing.php

Damn, Sabrina! I’ve got 100+ sites in the network! Are you telling me to add 100+ cronjobs manually? O_o

Sure, I’m not! Cron job bulk scheduling in Linux is already waiting for you.

Have you tried any of these approaches? How did it go? Share in comments!

6 Replies to “Fixing WordPress Multisite Cron for high-loaded networks”

  1. Sabrina, excellent post! Very detailed and understandable.

    One thing that wasn’t clear to me. . .
    Looking at “Multisite: better approach,” is that code publishing any post scheduled for today or is it publishing ALL drafts? Obviously, I would like it to publish only today’s scheduled posts.

    Thanks!

    1. Hello, Larry!
      This approach may suit your project if you don’t care which particular posts will be published today and which tomorrow but need just to be sure the certain amount (50 in the example above) will be published for sure. Check this line: “First of all we need the very function that will actually publish 50 drafts…” This function will take 50 random drafts from your site and publish them.

  2. Hey Sabrina,

    I appreciate this piece. Thank you.

    Is this a typo/mistake https://www.screencast.com/t/TecmVEBS ?

    I do not see why the final code would have /publishing.php I thought it should be /wp-cron.php?doing_wp_cron etc?

    If not, what should be in the publishing.php file?

    Let me know when you are able.

  3. Hi Sabrina.

    I noticed that in the “Entire trigger.php” you put this line of code:
    $command = $site_url.’/publishing.php’;
    instead that:
    $command = $site_url.’/wp-cron.php?doing_wp_cron’;

    And, telling about “Now we need this publishing.php” you put:
    $command = $site_url.’/wp-cron.php?doing_wp_cron’;
    instead:
    $command = $site_url.’/publishing.php’;

    In other words: you changed the order of the two alternatives, in your examples!

Leave a Reply

Your email address will not be published. Required fields are marked *