Kitt Cache Crawler Knowledgebase

The Kitt Cache Crawler is one of the few and is probably the only multithread cache warmup crawler that requests URLs in parallel rather than serially. The resulting differences are striking.If the serial method is used to request the URLs in sequence, i.e. one after the other, this...

In addition to the high crawl speed of the Kitt Cache crawler, the Max Server Load feature is the most important function. The automated crawling of a website,regardless of the purpose, is generally not uncritical, since without appropriate control the server can quickly become...

When a page is cached, it means that a static copy of the output is made from a requested URL. However, a static copy does not allow any changes to it. The dynamics are lost through anHTTP cache. If you now use several languages and maybe different currencies, the effects are fatal if...

Shopware classifies each page type into so-called routes. This route name classification allows you to limit the URLs to be crawled so that only the cache of the URLs of a selected route is warmed up.

The Network Timeout feature deals with the issue of waiting times. Since disruptions on the Internet are an integral part and can often occur uncontrolled, Kitt needs instructions onhow long to wait in the event of a disruption before aborting a request. The default value 10 should be...

The term "Most Wanded" already indicates what it is about. It's about the most requested URLs, but what do you need the most requested URLs for?Both the built-in crawler and the Kitt Cache Crawler use the sitemap as the basis for the URLs to be crawled, with Kitt also...

The webp Replacement feature is a special feature of the Kitt Cache Crawler that allows the use of .webp images. .webp images have a significantly smaller filesize with the same quality. However, .webp images are not supported by all browsers. However, the Apple Safari browser is not...

The threads setting determines the number of concurrent threads or URL requests. As can also be seen from the delay setting, the Kitt cache crawleris probably the only cache warmup crawler that works in multithread mode and therefore requests URLs simultaneously and not serially, like...

A cache warmup crawler is a tool used to warm up the cache of web applications before actual operation. This is an important process to ensure the application runs smoothly and minimize user loading times.The crawler visits and invokes all of the application's pages to ensure that the content...