Turns out to be a combination of conflicts with other cron scheduling, server resources, the Cron Configuration and Google Sitemap cron configuration. Lots of variables. The solution is to get all those schedules, well . . . scheduled.
What we discovered was that the server’s cron was firing nicely, about every 15 minutes. To clarify, the server’s cron is not set within Magento but typically within the server’s control panel like Cpanel, Plesk . . . and the settings of the server’s cron schedule are these obscure symbols of confusion – that is until you understand how cron scheduling works (check out Wikipedia for some explaination).
Here is one example of cron scheduled to execute a command every 15 minutes.
*/15 * * * * – Some command
The problem we were having was that other cron dependent features of Magento would show up in the cron_schedule table but not that pesky and notoriously allusive “sitemap_generate”, Magento‘s Google Sitemap cron schedule code.
What is the “cron_schedule” table you ask? Well, it is where Magento queues all it’s cron schedule codes to be run. This is the best place to check on what is being run, what’s erroring, what’s completed and what’s pending to run next. It also includes date info for when the queued item was created, when it is scheduled to run, when it ran and when it finished. So, basically it is a great place to start troubleshooting all your cron problems.
What we ended up doing to check and see why the “sitemap_generate” code was not showing up in the cron_schedule table was editing the configuration setting for Google Sitemap within the core_config_data table. We updated the path value “crontab/jobs/sitemap_generate/schedule/cron_expr” to a new expression you can’t create in Magento Admin > Configuration > Catalog > Google Sitemap > Generation Settings and that new wonderful expression of freedom was */5 * * * *. After the change database change make sure you Flush Cache Storage at Magento Admin > System > Cache Management or your changes will not take effect. If for some reason it still does not work you can comment out the entire <section> element in system.xml found here: app/code/core/Mage/Sitemap/etc
So, what we told Magento to do is execute the generation of the Google Sitemap every 5-minutes. We wanted to see if it was some bug in the code. Turned out it was not a bug and low and behold the “sitemap_generate” code showed up in the cron_schedule table! It was a site for sore eyes, let me tell ya.
I’ll post our findings when we get all the scheduling line up correctly.