Kitt Cache Crawler Knowledgebase

The Cache Mode feature is an undocumented LiteSpeed feature that not only speeds up crawling. It also allows to refresh the cache of a URL if the cache is not expired. If the cache is not expired the cache is refreshed by the set TTL. If the cache is expired it is normally warmed up. The cache...

For each cached URL, LiteSpeed LScache creates a static copy of the output of the URL in the /lscache directory, which is outside the public_html directory. The compression feature can be used todetermine whether the cache that has been preheated by Kitt should be compressed as...

With the Crawler Mode Feature you can specify whether the cache should be purged before the crawl starts. Except for Wordpress or when using the LiteSpeed LScache plugin for Wordpress,the entire cache created by the respective cache plugin is always purged. The LiteSpeed LScache Plugin...

cURL debugging enables a very detailed evaluation of each individual cURL request. When this feature is enabled, Kitt writes a debug_log to the /log directory.Please note that this will slow down crawling and result in a higher server load. Please activate this function only in...

The Kitt Cache Crawler is one of the few and is probably the only multithread cache warmup crawler that requests URLs in parallel rather than serially. The resulting differences are striking.If the serial method is used to request the URLs in sequence, i.e. one after the other, this...

The feature limit allows the quantity of URLs to be crawled to be limited and works in conjunction with the feature offset. The input parameterswork the same as when using MySql Queries. For example, if you want to crawl 1000 URLs, but start crawling at the thousandth URL instead of the...

If you want to use the cache inventory, activate the switch for the logging feature.

If you are using the Journal Theme, you can warm up the associated cache if the Product Quickview is activated.

See Description for Limit Feature.

The post type selection function is made possible by supporting the Yoast Seo plugin and allows the selection of specific (sub) post types.Depending on the selection, only the cache for the selected post type or types will be warmed up.

The Follow Redirection feature lets you set whether Kitt should follow a redirect. If this function is activated, the forwarding target is crawled and the cache is warmed up for it.Otherwise, Kitt stops the request and no cache is warmed up. The cache status "redirect" is then displayed...

In addition to the high crawl speed of the Kitt Cache crawler, the Max Server Load feature is the most important function. The automated crawling of a website,regardless of the purpose, is generally not uncritical, since without appropriate control the server can quickly become...

When a page is cached, it means that a static copy of the output is made from a requested URL. However, a static copy does not allow any changes to it. The dynamics are lost through anHTTP cache. If you now use several languages and maybe different currencies, the effects are fatal if...

Shopware classifies each page type into so-called routes. This route name classification allows you to limit the URLs to be crawled so that only the cache of the URLs of a selected route is warmed up.

The Network Timeout feature deals with the issue of waiting times. Since disruptions on the Internet are an integral part and can often occur uncontrolled, Kitt needs instructions onhow long to wait in the event of a disruption before aborting a request. The default value 10 should be...

The term "Most Wanded" already indicates what it is about. It's about the most requested URLs, but what do you need the most requested URLs for?Both the built-in crawler and the Kitt Cache Crawler use the sitemap as the basis for the URLs to be crawled, with Kitt also...

The webp Replacement feature is a special feature of the Kitt Cache Crawler that allows the use of .webp images. .webp images have a significantly smaller filesize with the same quality. However, .webp images are not supported by all browsers. However, the Apple Safari browser is not...