Cronjobs are serious business

Sorry for the big nasty account suspension message today.  Our host suspended our account to “protect the server” since we were sending too many “cronjobs” through wordpress.  I won’t pretend to know what the heck that is all about, but I guess it has something to do with either the size of our site or how it is optimized.

If anyone has any experience with this topic, I welcome your input.  I also welcome all of you to let me know if you encounter errors, slow load times, or issues with the site.  I followed a few steps to add a custom cronjob schedule, and I disabled automated cronjobs.  Hopefully that begins to solve the issue so our host won’t suspend us again.

That’s all for now!  I’ll be writing up a blog post later on something gaming related, but I wanted to let you all know the status of our issue.

  • Cron jobs are just commands or scripts or programs that are executed on a schedule set for that cronjob. Could be anything form every minute to every 3rd monday of every odd month.

    My assumption is that a cronjob that was executing too frequently, or was executing something very loaded, ended up using too many resources and slowing down the server.

    Now, I don’t like using WordPress because I think its a bloated mess, but from what I remember, I think it has cronjobs for managing the cached data, cleaning things, compressing data, etc. And some of these can be quite taxing, especially for a site like yours that has a decently high load on a very regular basis.

    I think it might have been a fluke or you just hit critical mass finally.

    I can probably help more if you want to contact me, but that is my rough info session on it.

    Hope it helps.

  • cron job is a scheduled task/program, I am not familiar with wordpress so I do not know what is doing that unfortunately, shoudl not really have anything to do with the size of the site, most likely some plugin you are using?

  • @Yulian: Apparently it was bad enough for them to panic for the safety of the rest of their server. Unfortunately, WordPress is by far the best option for someone like myself without the technical skills to create my own CMS. WordPress is just too easy and convenient.

    @Thelg: I would have no idea how to pin point which plugin is the culprit, but that very well may be the issue. Apparently my site exceeded the cap for cronjobs, and according to the person I spoke with on the phone my site was sending them every time a page was loaded.

  • “every time a page was loaded” … Odd. cron (some lore says its a mash up of ‘chronos’, some lore says its Command Run ON) jobs are something that has to be scheduled to happen. I dont know WordPress, but statistics collection, ads reporting back, cache cleanup, anything else you might have there that happens every page load would be a foreground task. Ok, *should* be a foreground task. Also, if WordPress is something installed for you by the host, its their bloody job to configure it properly.

    Sounds like someone was talking out of their … mule … to be honest.

    Just out of curiosity how many hits per minute/second/hour/whatever your site gets on a busy day?

  • Researching a bit tells me that Wpress does offer the facility to run scheduled tasks. My next guess would be, you(?) are trying to do something on each page load, but instead of doing it, you are scheduling it back into the system to be run every minute for unknown number of minutes, getting a cascade.

    If that is not the case, my next guess would be, each page load runs some task that really can be left to run on a fixed schedule of once a minute or once every ten minutes or once an hour or whatever.

  • @Blargh: I installed it so that’s all on me. On a slow day I get about 2,000-2,500 unique visitors making upwards of 4-5k visits with some days spiking 10-12k or even in the 30-60k range. ‘Hits’ (not something I monitor since it is a worthless traffic indicator, but perhaps useful for server load) are over 100,000 on a slow day and millions on an amazing day.

    As for the scheduling stuff, I have no idea. I believe my WP was behaving in a way where every page load was doing stuff that should be scheduled.

  • A million jobs is not the kind of workload cron’s designed for, thats for sure 😛 Depends on how long they are scheduled to run for of course, but even if it is a single run per hit, that could still be … say 10 hits per second, 600 jobs scheduled per minute. Then at the top of the minute, 600 tasks will all fire up and run. Bear in mind that cron’s resolution is once per minute.

    All I can suggest not knowing anything about WP, is see if any of your per-page tasks can instead be run once per minute on a fixed schedule of some sort.

  • I disabled the Newsletter plugin I am using. I think it may have been the culprit for whatever reason. It relies heavily on cronjobs.