Kitt - The Cache Crawler
Fastest Cache Crawler for LiteSpeed LScache and other Caching Types
Cache Warmup Crawling with Warp Speed
Kitt is a Cache Crawler to Warmup the Fullpage HTTP Cache for almost any popular Application. Kitt is specialized for LiteSpeed's LScache Engine and almost any Extension or Plugin that makes Caching available. But Kitt can also be used for Applications that runs on Apache Web Server and if there is an Extension or Plugin for Caching available. Kitt is not a Universal Warmer. Kitt is custom build for the Individual Needs of each Applicationand Extension or Plugin. This Special Feature makes Kitt unique and allows Kitt to get the Max out of each Cache Type.
Kitt's Highlight Feature is its Crawling Speed. If Universal Warmers crawls URLs one by one Kitt does this Job simultaneously and as much as you configure Kitt. As a Result Kitt can crawl up to 200,000 URLs within 1 hour, but Kitt also takes care of the current Load of your Server, even possible with Shared Hosting. If the Load goes too high Kitt reduces the number of simultaneously requests, fully automatically. For Shared Hosting this is very important to prevent trouble with your host provider. This also prevents crawling from negatively affecting the Operation of your Application, so you can run Kitt at every time and not only at night. This feature also makes Kitt unique.
Kitt comes as a standalone Application and will be installed parallel to your Applications. Installation is easy and can be done by Everyone. Each Kitt version can be executed as GUI Version and as CLI version for the use with Cron Jobs. Features, Speed and Performance of each Kitt Version are the same. If you have LiteSpeed web server running you benefit from LiteSpeed's special Feature to overwrite typically PHP Limits like max_execution_time. That means Kitt has no Limits and Processes will not be canceled, even on Shared Hosting and no matter how long Kitt's Crawling Session takes.
Kitt's Highlight Feature is its Crawling Speed. If Universal Warmers crawls URLs one by one Kitt does this Job simultaneously and as much as you configure Kitt. As a Result Kitt can crawl up to 200,000 URLs within 1 hour, but Kitt also takes care of the current Load of your Server, even possible with Shared Hosting. If the Load goes too high Kitt reduces the number of simultaneously requests, fully automatically. For Shared Hosting this is very important to prevent trouble with your host provider. This also prevents crawling from negatively affecting the Operation of your Application, so you can run Kitt at every time and not only at night. This feature also makes Kitt unique.
Kitt comes as a standalone Application and will be installed parallel to your Applications. Installation is easy and can be done by Everyone. Each Kitt version can be executed as GUI Version and as CLI version for the use with Cron Jobs. Features, Speed and Performance of each Kitt Version are the same. If you have LiteSpeed web server running you benefit from LiteSpeed's special Feature to overwrite typically PHP Limits like max_execution_time. That means Kitt has no Limits and Processes will not be canceled, even on Shared Hosting and no matter how long Kitt's Crawling Session takes.
Login Protection
Login with Brute Force Protection
Access to Kitt Control Panel is not only Login protected. To prevent unauthorized Access Kitt has an intelligent Brute Force Protection that blocks unauthorized Access.News by Feed
News and Updates by Feed
To keep Users up to date a daily updated News Feed notifies about any News and of course about new Versions of Kit.Crawler Settings
Many Settings, but less to configure
Kitt Cache Crawler is Admin friendly. Kitt allows for many settings, but most of these settings are already preconfigured, so in most cases it is sufficient to either just start the crawling process manually or set up a cron job.For Experts Kitt offers many settings to customize and to tuneup Crawling Behaviour. All Settings are self-explanatory and require no Special Knowledge about how to configure Kitt. Even if you are not an Expert, simply accept the recommended Setting Values to get the best crawling Result.
Kitt takes Care of your Server
Warming up a HTTP Cache is not uncritical, so you have to be careful not to Overload the Server. Conventional Cache Crawlers do not pay attention to this, so that Overloading often occurs and the Operation in the Frontend is impaired as a Result. Therefore, Host Providers often disable the Execution of Cache Warmers.
To prevent this, Kitt offers a dynamic Server Load Monitoring. If a Preset Value is exceeded, Kitt automatically reduces the Speed and the Number of Requests executed at the same Time. In this Way it is always ensured that the Server is not overloaded and the frontend operation is not impaired by the Crawling. Your advantage, you can run Kitt at any time without affecting the frontend operation.
Your host Provider will not notice that Kitt is warming up the cache!
Kitt GUI Version
Inhouse Version of Kitt
Kitt comes as GUI and as Version for Command Line Interface (CLI). Handling of GUI Version is ease to use. Both Versions are almost equal and have the same Speed. For longer Crawl Sessions the CLI Version is suggested. It is important to know that the GUI Version has no Timeout Limit if LiteSpeed Web Server is used. Server Side Settings for max_execution to limit PHP Processes don't break the crawling Process.Speed and the Number of Requests are not fixed while crawling session. Kitt frequently checks the current Server Load. If the Load raises a defined Limit Kitt reduces the Frequency of Requests and decreases the Time during Requests. This dynamic Function ensures that a Warmup Session can be done at every Day Time. Your Server will never be overloaded!
Purge the Cache
Flush the Cache
Since each cache has an expiration time, there is usually no need to purge the cache manually. Nevertheless, there needs to be a way to purge the cache before its expiration time if necessary. In this case, Kitt provides numerous purge functions so that the cache can be specifically purged, from the entire cache to cache per URL to purge by tag.Most Wanted
Only requested URLs matter
The URLs Most Wanted function is a special highlight of Kitt, which can optionally be used as an alternative to the sitemap. Although the sitemap.xml has proven itself, there are several disadvantages to using the sitemap. On the one hand, the sitemap contains too many and often the wrong URLs. Anyone who analyzes their website's traffic knows that up to 70% of the URLs in the sitemap are either never or only very rarely requested. So why waste resources on cache warmup if no one benefits from warming up the cache but no one requests the URLs in question? A lot of URLs are missing from the sitemap.xml. This includes pagination, filters and a variety of other URLs, which results in the cache for these URLs not being warmed up.The URLs Most Wanted function solves this significant problem by recording the URLs requested by natural users and using them for cache warmup. This function enables a targeted and practical cache warmup and reduces time and resources for the cache warmup by up to 70% without any disadvantages.