Site Speed WTF?!

Weekly Email

Please wait…


You have subscribed! A verification email will be sent to you.


SpeedGuard tracks Core Web Vitals from your WordPress. Check it out.

WordPress Multisite Cron for high-loaded networks

As you remember, WordPress in-built CRON doesn’t work the same way as classical CRON does.

Instead of being triggered on a specified time, it fires only when someone visits the website. So if there are no visits, there would be no wp-cron triggering, and therefore no tasks would be executed.

And vice versa, if you’ve been lucky to get loads of visitors on your website WordPress CRON system will be triggered every single time when someone views any page and checks for scheduled tasks. This way WordPress CRON causes extra server load (CPU usage increases) which may significantly harm website performance. That’s speaking of single install.

When it comes to WordPress Multisite, your problems multiply on the number of websites on your install.

WordPress Multisite Cron for high-loaded networks

How wp-cron works?

WordPress CRON on Multisite works independently on each blog.

Say, you’ve got posts scheduled on website A and B.

When someone visits website A, the scheduled posts on blog A, though with delay, but will be published eventually. But posts on website B would never be published unless someone visits website B too.

Having missed schedule issue on every site of the network can be a rather annoying thing. And you may face another problem when you get some significant traffic on your Multisite install.

That CPU usage issue multiplies too, according to the number of websites in your network. That’s why using in-built CRON for Multisite is not a great idea at all. Instead, we should set a real CRON job.

How to set a CRON job for WordPress Multisite and fix missed schedule issue across the network?

An Obvious Approach

To reduce CPU usage and get tasks executed no matter, whether there are hits on the blog or not, we should replace WordPress Multisite CRON with real CRON.

Here I described in details how to do this for single install.

The steps for Multisite are pretty similar, except that we need to trigger CRON on every single site of the network (because they work independently, as you remember).

1. Disable WordPress CRON across the whole network in wp-config.php:

define('DISABLE_WP_CRON', true);

2. We will trigger publishing on every site with the help of foreach loop.

Create trigger.php in the same root directory, where you have your wp-config.php.
I have this awesome wrapper for doing bulk actions in WordPress Multisite, which has saved me loads of time. Let it work for you too:

So we just place this piece of code after the line //here's what we're gonna do...  

$site_url = get_site_url($blog_id); 
$command = $site_url.'/wp-cron.php?doing_wp_cron'; 
wp_remote_get($command); 

and it will trigger each site of our network automatically.

The entire trigger.php will look like this:

3. All that you need now, is just to add one single CRON job for your main blog on your server:

wget -O /dev/null https://network-main-blog.com/trigger.php

How to do it through CPanel see here in details.

Also, mind to set a reasonable time interval.

This is it! According to the specified recurrence (e.g. once a day) Unix CRON will hit trigger.php -> which will hit wp-cron.php on every blog in the network-> which will check if there are any tasks scheduled, and execute them.

This is a sufficient and clear way to set scheduled tasks on WordPress Multisite install.

But what you’re gonna do if you’ve got rather a large network and you are tight on server resources? Let’s see a…

A Better Approach

All scheduled tasks in WordPress are stored in wp_options table.

When you schedule a task to publish 50 drafts daily, 50 tasks are saved in your wp_options table. Most part of this table including CRON jobs is loaded at every single page load. That’s okay for single install but it can significantly slow down your site speed on WordPress Multisite network.

Say, you’ve got a middle-sized network of 100 sites which leads us to 50*100=5000 additional lines in wp_options table (mind, this is the same database!). Furthermore, the more visits your network gets the more server recourses will be consumed for this operation only.

To sleep peacefully at night without fearing your MySQL server collapse any moment we would rather skip WordPress scheduling. Instead of CRON job for publishing scheduled posts we’ll set a CRON job to publish posts directly.

Further I’m giving detailed instructions on how to arrange this, but before you proceed, please, keep in mind this approach may not be suitable for every single project. For example, if the specific time of publishing is extremely important for you this is not what you want to do.

But if you’ve got loaded network and you need to publish drafts daily, save server resources and don’t mind the publish time to be +- few hours, you should definitely try this out.

1. First, we’ll need the very function that will actually publish 50 drafts.

Create in the WordPress root directory publishing.php with the following content:

This simple piece of code will publish drafts directly, without scheduling.

2. Now we need to get this publishing.php triggered for execution across whole network. We are replacing WP CRON with server CRON the same we did in the previous option. 

Just mind place publishing.php instead of wp-cron.php?doing_wp_cron

$command = $site_url.'/publishing.php';

Voilà! We’ve managed to reduce CPU and memory usage by replacing WP Multisite CRON with real CRON job across the network and skipping scheduling at all. Cool, isn’t it?

But…

As I already said for small networks this approach is pretty good, but in case you got 100+ blogs it means publish function will be executed 50*100 = 5000 times at once. And you hardly want it, right?

My approach to set up CRON for high-loaded WordPress Multisite networks

How to publish 50 posts a day on more than 100 blogs without making your server collapse?

Overview: WordPress Multisite network that consists of 100+ blogs. 50 posts should be published daily on every blog. It leads us to 50*100=5000 posts that should be published by CRON every day.

Objective: This should be executed with minimal server load as this network is rather loaded apart of publish process.

Here I’m using another approach that reduces CPU usage dramatically. This way we’ve got:

  • no CRON event to schedule posts for publishing
  • no CRON events for publishing each of 50 posts
  • CRON is triggered once (!) a day on each website of the network
  • no overload by bulk publishing, just 50 posts are published at a time

This approach may be somewhat tricky for a developer but this is how we use as fewer server resources as possible.

We’ll add a separate CRON job for every website in the network and give it a command to execute  publishing.php directly.

We have already got this file prepared, now we need to add a separate CRON job for every site in the network.

Each cronjob should be scheduled with 1-minute interval. Imagine, you have 3 sites in your network (but you won’t be bothering with all these in that case, would you ;), so CRON jobs should be executed like this:

site A — at 00:01

site B — at 00:02

site C — at 00:03

Each executed once a day, at a precise time.

This way only one site got triggered in a minute, no overload, no problems.

So, there are 3 commands which you need to add to your crontab:

wget -O /dev/null http://siteA.com/publishing.php
wget -O /dev/null http://siteB.com/publishing.php
wget -O /dev/null http://siteC.com/publishing.php

Damn, Sabrina! I’ve got 100+ sites in the network! Are you telling me to add 100+ cronjobs manually? O_o

Sure, I’m not! Cron job bulk scheduling in Linux is already waiting for you.

Have you tried any of these approaches? How did it go? Share in comments!

36 responses to “WordPress Multisite Cron for high-loaded networks”

  1. Larry Levenson Avatar

    Sabrina, excellent post! Very detailed and understandable.

    One thing that wasn’t clear to me. . .
    Looking at “Multisite: better approach,” is that code publishing any post scheduled for today or is it publishing ALL drafts? Obviously, I would like it to publish only today’s scheduled posts.

    Thanks!

    1. SZ Avatar
      SZ

      Hello, Larry!
      This approach may suit your project if you don’t care which particular posts will be published today and which tomorrow but need just to be sure the certain amount (50 in the example above) will be published for sure. Check this line: “First of all we need the very function that will actually publish 50 drafts…” This function will take 50 random drafts from your site and publish them.

  2. Larry Levenson Avatar

    Sabrina, excellent post! Very detailed and understandable.

    One thing that wasn’t clear to me. . .
    Looking at “Multisite: better approach,” is that code publishing any post scheduled for today or is it publishing ALL drafts? Obviously, I would like it to publish only today’s scheduled posts.

    Thanks!

    1. SZ Avatar
      SZ

      Hello, Larry!
      This approach may suit your project if you don’t care which particular posts will be published today and which tomorrow but need just to be sure the certain amount (50 in the example above) will be published for sure. Check this line: “First of all we need the very function that will actually publish 50 drafts…” This function will take 50 random drafts from your site and publish them.

  3. Greg Avatar
    Greg

    Hey Sabrina,

    I appreciate this piece. Thank you.

    Is this a typo/mistake https://www.screencast.com/t/TecmVEBS ?

    I do not see why the final code would have /publishing.php I thought it should be /wp-cron.php?doing_wp_cron etc?

    If not, what should be in the publishing.php file?

    Let me know when you are able.

    1. sz Avatar
      sz

      Hi Greg!
      You’re absolutely right. Just corrected this.
      Cheers!

  4. Greg Avatar
    Greg

    Hey Sabrina,

    I appreciate this piece. Thank you.

    Is this a typo/mistake https://www.screencast.com/t/TecmVEBS ?

    I do not see why the final code would have /publishing.php I thought it should be /wp-cron.php?doing_wp_cron etc?

    If not, what should be in the publishing.php file?

    Let me know when you are able.

    1. sz Avatar
      sz

      Hi Greg!
      You’re absolutely right. Just corrected this.
      Cheers!

  5. Aldemar Calazans Filho Avatar
    Aldemar Calazans Filho

    Hi Sabrina.

    I noticed that in the “Entire trigger.php” you put this line of code:
    $command = $site_url.’/publishing.php’;
    instead that:
    $command = $site_url.’/wp-cron.php?doing_wp_cron’;

    And, telling about “Now we need this publishing.php” you put:
    $command = $site_url.’/wp-cron.php?doing_wp_cron’;
    instead:
    $command = $site_url.’/publishing.php’;

    In other words: you changed the order of the two alternatives, in your examples!

    1. sz Avatar
      sz

      Hi Aldemar Calazans!
      Thank you for pointing this out, I finally corrected this.
      Cheers,
      Sabrina

  6. Aldemar Calazans Filho Avatar
    Aldemar Calazans Filho

    Hi Sabrina.

    I noticed that in the “Entire trigger.php” you put this line of code:
    $command = $site_url.’/publishing.php’;
    instead that:
    $command = $site_url.’/wp-cron.php?doing_wp_cron’;

    And, telling about “Now we need this publishing.php” you put:
    $command = $site_url.’/wp-cron.php?doing_wp_cron’;
    instead:
    $command = $site_url.’/publishing.php’;

    In other words: you changed the order of the two alternatives, in your examples!

    1. sz Avatar
      sz

      Hi Aldemar Calazans!
      Thank you for pointing this out, I finally corrected this.
      Cheers,
      Sabrina

  7. Jake P Avatar

    Thank you for this post, it’s been helpful. Is there a reason why you used wp_remote_get() to hit the php file, rather than just running the php file directly with include/require?

    1. sz Avatar
      sz

      Hi Jake,
      I can’t remember if it was any particular reason for the time of writing to be honest, but you’re right, I believe include directly would be more efficient if it can be used!

  8. Jake P Avatar

    Thank you for this post, it’s been helpful. Is there a reason why you used wp_remote_get() to hit the php file, rather than just running the php file directly with include/require?

    1. sz Avatar
      sz

      Hi Jake,
      I can’t remember if it was any particular reason for the time of writing to be honest, but you’re right, I believe include directly would be more efficient if it can be used!

  9. Stephen Sabatini Avatar

    I had to pass the path to get the correct paths: `get_site_url( $blog_id, $site->path )`. Hope this helps someone.

    1. sz Avatar
      sz

      Thanks for sharing, Stephen! Do you have any idea why you need to do so?

  10. Stephen Sabatini Avatar

    I had to pass the path to get the correct paths: `get_site_url( $blog_id, $site->path )`. Hope this helps someone.

    1. sz Avatar
      sz

      Thanks for sharing, Stephen! Do you have any idea why you need to do so?

  11. Ame. Avatar

    Hello Sabrina,
    Thank you very much for this post. I did all the steps above but I don’t have any cpanel on my webhosting.
    Should I add only one cronjob? My webhosting explanation is like this:
    https://www.combell.com/en/help/kb/programming-recurring-tasks-with-cronjobs/

    1. sz Avatar
      sz

      Hello! And thanks!
      Yeah, I believe if you put trigger.php content into script.php in their example this should work. Also, see another approach with publishing.php below that.
      Cheers!

  12. Ame. Avatar

    Hello Sabrina,
    Thank you very much for this post. I did all the steps above but I don’t have any cpanel on my webhosting.
    Should I add only one cronjob? My webhosting explanation is like this:
    https://www.combell.com/en/help/kb/programming-recurring-tasks-with-cronjobs/

    1. sz Avatar
      sz

      Hello! And thanks!
      Yeah, I believe if you put trigger.php content into script.php in their example this should work. Also, see another approach with publishing.php below that.
      Cheers!

  13. Ame Avatar

    Hello Sabrina,

    I already wrote you, but have an other question. I have a multisite and with two sites on it. I made multisite to have French translation of my main domain. So there is only one domain on the multisite. Example.be/ and example.be/fr
    In my google search console I see no issurs with speed. Do you think that in my situation a multisite could cause a problem for speed?
    With kind regards,
    Amelia

    1. sz Avatar
      sz

      Hi Amelia,
      Site speed in this case would depend on the same things as in any regular installation – which theme do you use, which plugins, how good they perform together.
      Also, make sure to run speed tests for both Example.be/ and example.be/fr separatedely.
      See my SpeedGuard plugin to have site speed monitored (you’ll need to activate on per-site basis)
      Hope this helps.
      Cheers,
      Sabrina

  14. Ame Avatar

    Hello Sabrina,

    I already wrote you, but have an other question. I have a multisite and with two sites on it. I made multisite to have French translation of my main domain. So there is only one domain on the multisite. Example.be/ and example.be/fr
    In my google search console I see no issurs with speed. Do you think that in my situation a multisite could cause a problem for speed?
    With kind regards,
    Amelia

    1. sz Avatar
      sz

      Hi Amelia,
      Site speed in this case would depend on the same things as in any regular installation – which theme do you use, which plugins, how good they perform together.
      Also, make sure to run speed tests for both Example.be/ and example.be/fr separatedely.
      See my SpeedGuard plugin to have site speed monitored (you’ll need to activate on per-site basis)
      Hope this helps.
      Cheers,
      Sabrina

  15. Dave Avatar
    Dave

    Great article – thanks for a detailed approach.

    I’m curious if you have ideas/thoughts on how I could do something like this for a site utilizing Single Sign-On (SSO). Basically, hitting any page on our web server enforces SSO authentication. So, simple http-based cron tasks will fail because the request never makes it to the web server in an authenticated state.

    My multisite instance, which generates ~30k pageviews/week, had wp-cron disabled long before I came along. Figuring out how to implement something manageable that can work around SSO and still trigger wp-cron (maybe server/CLI-based instead of http?) has stumped me.

    1. sz Avatar
      sz

      Thanks, Dave!
      I’m not quite sure I got it right. Do you want to trigger everything but SSO? What would be the perfect outcome?
      Cheers, Sabrina

  16. Dave Avatar
    Dave

    Great article – thanks for a detailed approach.

    I’m curious if you have ideas/thoughts on how I could do something like this for a site utilizing Single Sign-On (SSO). Basically, hitting any page on our web server enforces SSO authentication. So, simple http-based cron tasks will fail because the request never makes it to the web server in an authenticated state.

    My multisite instance, which generates ~30k pageviews/week, had wp-cron disabled long before I came along. Figuring out how to implement something manageable that can work around SSO and still trigger wp-cron (maybe server/CLI-based instead of http?) has stumped me.

    1. sz Avatar
      sz

      Thanks, Dave!
      I’m not quite sure I got it right. Do you want to trigger everything but SSO? What would be the perfect outcome?
      Cheers, Sabrina

  17. Pelton Avatar
    Pelton

    Thank you for this article. Really helpful. I’m using the first example. However when I visit my trigger.php I get the following error:

    “Fatal error: Uncaught Error: Call to undefined function is_multisite()”

    Any idea why?

  18. Laxman Prajapati Avatar
    Laxman Prajapati

    Hello Sabrina,

    Very good the blog is posted,

    I have integrated this code with my website which is running with multisite. But how the WordPress function will work with the trigger.php file? Without the file connect to WordPress? We don’t need to include the wp-load.php file in the trigger.php file? We getting errors so.

    Thanks,
    Laxman P

  19. Laxman Prajapati Avatar
    Laxman Prajapati

    Hello Sabrina,

    Very good the blog is posted,

    I have integrated this code with my website which is running with multisite. But how the WordPress function will work with the trigger.php file? Without the file connect to WordPress? We don’t need to include the wp-load.php file in the trigger.php file? We getting errors so.

    Thanks,
    Laxman P

  20. Ravi Avatar
    Ravi

    Thanks for the script!
    Although the script didn’t work until I had `require_once( dirname( __FILE__ ) . ‘/wp-load.php’ );` as the first line in the file to load all the WordPress functions etc.

Leave a Reply

Your email address will not be published. Required fields are marked *