Solve mysterious Drupal initial page load delay with cron

Sometimes you'll visit your Drupal site in the morning, or after logging in as admin for the first time in awhile, and the page will take forever to load, more than 30 seconds, even longer. (First, if you're using Cloudflare CDN, it could be a problem with their servers not sending the end of the page html when it's finished and thus causing the browser to hang for a long time.)

The main cause of this kind of page load lag is Drupal's cron task! There are two ways that Drupal will run cron. The first is when you trigger it externally (by pinging the cron.php url, or by running drush cron) or manually in the admin menu. The second way is for Drupal to implicitly run its cron tasks when a timer has been passed and cron hadn't run during that period. The second case can cause extreme lag for both you, the admin, or any anonymous users who happen to try to visit your site (and probably bounce due to the long loading times).

The solution to this is to never let cron run implicitly, triggered by an anonymous visit or your own visit. Cron was never supposed to work this way, but this is a hack to give Drupal a chance to run its cron when there's no other way. The reason there would be no other way is if Drupal was installed on a shared host without access to a real crontab. To disable automatic cron, go to Administration » Configuration » System » Cron and set "Run cron every..." to "Never". You can also find the link to run cron externally with the secret key here.

UNIX crontab vs Drupal cron

Drupal's cron is a set of tasks which Drupal is configured to run periodically, such as clearing caches, checking for updates, writing to the log, etc.

UNIX and Linux (and Mac OS X) has a background process called cron which will run commands any user configures in her crontab and these tasks are called cron jobs. The proper way to call Drupal's cron is from a crontab.

Configuring crontab to run Drupal cron via drush

You can run Drupal's cron by going to the 'run cron' page in admin. You can also call it externally, but then either anybody can call cron on your site (which can cause a denial-of-service (DoS)) or you need to set an access key and store that when you curl the cron url. A more consistent way to have cron run is via drush, assuming you have access to install/run drush on the Drupal server and configure crontabs on there.

The line looks like this:

0 */3 * * * cd /path/to/drupal && drush cron > /dev/null

This will run every 3 hours everyday, go into your Drupal's root directory so that drush has the context it needs to connect to your site's database and then run cron via drush (an important drush tip: you can have many Drupal sites on one server but drush doesn't know which one is stored where, you just need to cd to somewhere in the Drupal site including any subdirectory and the drush will find the settings.php file).

If you want to ping cron externally with the secret key from above and curl that url instead of using drush, then add something like this:

0 * * * * /usr/bin/curl

This will ping cron every hour.