Kitt Cache Warmup Crawler for WooCommerce W3 Total Cache Plugin (W3TCWC) "plaid" Edition

Ask A Question LiteSpeed Authorized Support Partner

Kitt Cache Warmup Crawler for WooCommerce W3 Total Cache Plugin (W3TCWC) plaid Edition

HighAvailability Cache Crawler for even better Customer Experience

  • Up to 50-times faster than build-in Crawler
  • Dynamic Load Control
  • Post Type Selection (WooCommerce Support)
  • Available as Desktop Version and Version for Command Line Interface (Cron)
  • Tuneup Settings for Power Users
  • Cache Inventory
Note: Kitt Cache Crawler is not a WordPress Plugin. Kitt Cache Crawler is a standalone Application that runs parallel to WordPress, so don't try to install it as a Plugin in WordPress. This will fail! Download the .zip File, unzip it locally and read the install notes carefully.


Why to use Kitt Cache Crawler for W3 Total Cache Plugin

Cache plugins for WordPress abound, but few are as popular as the W3 Total Cache plugin. After all, in addition to the actual cache function, it offers numerous functions for optimization and thus for improving the loading time. Even if these optimization functions are not significantly different from those of other cache plugins, the W3 Total Cache plugin has the advantage that you can use this plugin with a conventional server configuration. It does not need a special web server, as is the case with the LiteSpeed ​​LScache plugin, although you can sometimes see a significant performance advantage with the LiteSpee LScache plugin. Despite these differences, all cache plugins share the same disadvantage. In order to be able to use the advantages of the page cache, this cache must be warmed up. Almost every cache plugin has a built-in crawler, but these are all far too slow, are not very flexible and cause a noticeably high server load. When using Wordpress and the WooCommerce plugin professionally, the built-in crawler can hardly keep up with the sometimes frequent purge processes of the cache. This significant disadvantage almost completely negates the benefit of the cache.

Kitt The Cache Crawler solves this significant problem. Kitt was not specifically designed to meet the needs of a page cache, but also to the needs of the W3 Total Cache Plugin for Wordpress. Kitt is therefore not only lightning fast and crawls up to 200,000 URLs per hour, but also adapts itself to the current load conditions of the server. If conventional crawlers are usually started at night because fewer visitors are expected at that time, this does not matter to Kitt. Kitt automatically adjusts the crawling speed to the current load, so that the server is never overloaded and the front end is never restricted.

Requirements to run Kitt Cache Crawler for W3 Total Cache with WooCommerce

Kitt Cache Crawler is a standalone Application that has to be installed in parallel to your WordPress Installation. Installation is simple and can be done by everyone. If you were able to install WordPress, then you can install Kitt as well.


  • WordPress
  • Apache web server (nginx and LiteSpeed (LSWS/OLS) are unsupported)
  • W3 Total Cache Plugin
  • WooCommerce Plugin
  • YOAST Seo Plugin for WordPress (optional)
    If YOAST Seo Plugin isn't installed URLs to be crawled cannot be filtered by Post Types.
  • PHP Version >= PHP 7.4 & < 8.2
  • cURL
  • ionCube Loader (Version >11)
  • /proc/stat must be readable
  • /proc/cpuinfo must be readable
  • PHP Function sys_getloadavg must exist
  • DirectAdmin is unsupported!!!

All requirements are mandatory! If any of the listed requirements are not met, Kitt will not run.
If you do not know whether all requirements are available on your server, ask your hosting provider. All listed Requirements are normally Standard, even on Shared Hosting Server.

The fastest and safest method to check the requirements is the Kitt Tester Script. Download the tester script, upload it to the root directory of the WordPress installation and request the tester script in the browser.

Raise W3 Total Cache to the next Level Cache Plugin

When developing Kitt Cache Crawler, we placed great value on high utility. However, this does not mean that Kitt should become a monster of attitude, so that it would take too much time to adapt Kitt to individual needs. Kitt is already pre-configured during installation, so in most cases it is sufficient to accept these default settings. However, experts have a wealth of settings available to make Kitt even faster. However, you should invest some time in the dynamic load control. Kitt will also support you, but the better you understand how your server reacts under load, the better you can support Kitt in his work.

Crawler Settings

Kitt Cache Crawler - Delay
Unlike almost every other crawler, Kitt is a so-called multi-thread crawler. This means that when requesting URLs, it does not request them one after the other, but several at the same time. However, for a smooth caching process, there needs to be a small delay between the requests. This delay time is specified in microseconds. 1 second = 1,000,000 microseconds.
Kitt Cache Crawler - Threads
A multi-thread crawler means that you can change the number of simultaneous threads. The larger the number of concurrent threads, the more URLs are requested at the same time. However, this also increases the server load. Kitt is designed in such a way that new requests are only made if the previous request was successful. If the number of simultaneous requests is set too high, a traffic jam will inevitably occur. This means that too many concurrent requests will be slower than with fewer. We therefore recommend leaving the default value even when using a dedicated server.

Max Server Load
Kitt Cache Crawler - Max Server Load
The outstanding features of Kitt Cache Crawler include the Dynamic Load Control. Not only control, but the automatic adaptation of crawling behavior is decisive. Conventional crawlers do not pay attention to this and, regardless of the most resources, claim whether this affects the front end. This also applies to the built -in crawler of the W3 Total Cache Plugin. Kitt adapts the crawling speed adaptive and analyzes an optimal default value for the maximum load setting. For the best possible setting, however, it is even better if you set this load limit individually.
Network Timeout
Kitt Cache Crawler - Network Timeout
It is not uncommon for network disruptions to occur when using the Internet. The reasons for this can be varied, but always means that the request for a URL is delayed. If the network disruptions accumulate, it will inevitably affect the crawling process time. A maximum waiting time can be set with the network timeout so that the delay times are not too long. Once this has been reached, Kitt aborts the request. In this case and if logging is activated, the cache inventory shows a failed cache process.

Kitt Cache Crawler - Limit
Limiting the URLs to be crawled allows for maximum flexibility.
Kitt Cache Crawler - Offset
Using the offset function goes hand in hand with using the limit setting and allows crawling from a specific row.

Kitt Cache Crawler - Logging
If logging is activated, the cache inventory provides an informative overview of the cache status of each URL and also of the setting-dependent cache varies.
cURL Debugging
Kitt Cache Crawler - cURL Debugging
If you want to know the exact details among the experts, cURL debugging provides a comprehensive report on every request.

Crawler Mode
Kitt Cache Crawler - Crawler Mode
With the setting for crawler mode, you have the option of deleting the cache before crawling.
Follow Redirection
Kitt Cache Crawler - Follow Redirection
Kitt can be configured to follow redirections so that the redirected URL will be cached. Forwarding should only be activated in exceptional cases, as this dramatically increases the crawl time.

Server IP Address
Kitt Cache Crawler - Follow Redirection
To warm up the cache, Kitt simulates conventional user requests. Therefore, Kitt is subject to the same network conditions as normal user requests. However, Kitt requires direct access to the host, bypassing a CDN node. It is therefore necessary to enter the public IP address of the origin host.
Crawl Output
Kitt Cache Crawler - Follow Redirection
If Kitt is run manually in the console, the output for it can be activated if necessary. This will display the currently requested URLs, the transfer time and the current server load for each URL. However, activating the output function should not generally be activated as this can have a negative impact on the crawl speed.

Post Type Selection
Kitt Cache Crawler - Post Type Selection
Kitt allows the choice of post types for maximum flexibility. This option can sometimes significantly reduce the number of URLs to be crawled. Kitt requires the mandatory installed YOAST Seo plugin.
URLs Most Wanted
Kitt Cache Crawler - Server IP Address
The URLs Most Wanted (UMW) feature is a unique feature that is only available with the Kitt Cache Crawler. The UMW function is an alternative to the sitemap, which usually provides the basis for the URLs to be crawled. As the name of the UMW function suggests, this function records the URLs requested by natural users. The unique advantage in this feature is that it also records the URLs that are not normally included in the sitemap and therefore cannot warm up the cache for these URLs. The further advantage is that up to 70% of the total existing URLs do not have to be crawled, because up to 70% of all URLs in the site size are either never or only very rarely requested. So why waste resources crawling URLs if no one gets any use out of it?

Kitt Cache Crawler - Post Type Selection
If the YOAST plugin is not installed, then at least 1 sitemap file must be specified. Kitt supports both traditional sitemaps and index sitemaps.

Unique Features for W3 Total Cache Plugin

  • Crawling up to 200,000 URLs within 1 hour Kitt Cache Crawler can crawl up to 200,000 URLs within 1 hour, even on Shared Hosting. This is up to 20x faster as built-in Crawler.
  • Dynamic Load Control Kitt Cache Crawler cares about the Load of your Server. A Dynamic Load Control checks the current Load while Crawling. When the load gets too high, Kitt automatically reduces the speed and the number of concurrent requests until a predefined limit is reached. This dynamic load control guarantees that frontend operation is not adversely affected by crawling.
  • Auto-Adjust Configuration Kitt Cache Crawler comes with pre-configured Settings, that fits in most cases. So you don't have to learn which settings are the best. Each setting is also provided with recommendations that give you even more security when making any changes.
  • Control Panel Version and Version for Command Line Interface (CLI) Each Kitt Cache Crawler version can be run in at least 2 ways. Kitt can be run from within the Control Panel, but also for use in the Command Line Interface. The CLI version is therefore also suitable for use as a cron job.
  • Tune-up Settings A wealth of tuning settings are available for experts to make Kitt even faster.
  • Cache Inventory The cache inventory optionally gives a detailed report on the cache status of each URL. This report also includes information about Cache Vary's cache status.
  • URLs Most Wanted URLs Most Wanted reduces cache warmup time and resources by up to 70% and dramatically offloads the cache warmup server.

Kitt Cache Crawler vs Built-in Crawler

Built-in Crawler
Kitt Cache Crawler
Request Method
Crawling Speed
Dynamic Load Control
Cache Inventory
Network Timeout
Curl Debug
Always Follow

Still Questions?

We try very hard to give you as much information as possible. If you can't find the information you need or if you don't understand something, don't hesitate to ask us.
Ask A Question

More Kitt Cache Crawler Versions