HighPerformance Cache Warmer for LiteSpeed LScache Wordpress Plugin
Multi-Threading Process Mode
Crawls up to 200,000 URLs within 1 hour, even on Shared Hosting
300% faster as LiteSpeed built-in Crawler
Dynamic Load Control
Post Type Selection (no WooCommerce Support)
Auto-Adjust Configuration
Cache Vary Support for Guest Mode, Webp Replacement and Mobile View
Available as Desktop Version and Version for Command Line Interface (Cron)
Tune-up Settings for Power Users
Cache Inventory
Smart Re-cache
50% less Load while crawling
Free Updates and Support
High Quality "Made in Germany"
Note: Kitt Cache Crawler is not a WordPress Plugin. Kitt Cache Crawler is a standalone Application that runs parallel to WordPress, so don't try to install it as a Plugin in WordPress.This will fail! Download the .zip File, unzip it locally and read the install notes carefully.
Kitt is a Cache Crawler, also named Cache Warmer. If you are already using the LiteSpeed Cache Plugin LSCWP, then you already know the Cache Crawler Function it contains. But if this Plugin already uses a Cache Crawler, why should I use an additional one?
Well, Kitt basically does the same thing as the Plugin's built-in Crawler, but much more faster. Speed is an important argument, but not the most important. The built-in Crawler has quite a few Disadvantages and it also does not work correctly, so that the warmup Process is not always guaranteed depending on the Settings. Above all, speed is the primary Problem. Wordpress and the LiteSpeed Cache Plugin are mainly used on Shared Hosting. With Shared Hosting tariffs, the available Resources are typically always very limited. Because of this, Host Providers are very careful that the Load of a single User does not affect other Users.
Providers have the option of limiting the Load caused by the built-in Crawler so that crawling does not cause an Overload. In Case of Doubt, Settings made in the Plugin have no effect if the Load exceeds a preset Value. It is not uncommon for the Function of the built-in Crawler Server to be widely disabled.
Although it has already been suggested several Times, the vast Majority of Users cannot do anything with the Setting for Limiting the Load in the Crawler Settings. Especially since many users believe that the Load Limitation has something to do with the CPU load and/or Memory consumption. Due to an Error in the Programming of the built-in Crawler, the default value "1" for the maximum Load is mostly wrong, which slows down the Crawling Speed considerably.
Another and probably the biggest argument in favor of the Kitt Cache Crawler is that, unlike the built-in Crawler of the LiteSpeed Cache Plugin for Wordpress, Kitt warms up the cache correctly. Unfortunately, the built-in crawler has significant malfunctions that cause the wrong cache to be warmed up, making using the built-in crawler pointless. Anyone who has previously used the built-in crawler is wasting valuable resources because the built-in crawler does not do what it was programmed to do.
There are many more Reasons, but the Reasons already mentioned should convince you to use the Kitt Cache Crawler from now on.
This is what you need to run Kitt Cache Crawler
Kitt Cache Crawler is a standalone Application that has to be installed in parallel to your WordPress Installation. Installation is simple and can be done by everyone. If you were able to install Wordpress, then you can install Kitt as well.
Requirements:
Wordpress
LiteSpeed Enterprise (LSWS) or OpenLiteSpeed (OLS) (Notice: OLS has limited ionCube Support)
LiteSpeed LScache Wordpress Plugin (LSCWP)
YOAST Seo Plugin for Wordpress (optional) If YOAST Seo Plugin isn't installed URLs to be crawled cannot be filtered by Post Types.
PHP Version >= PHP 8.1 & <= 8.3
cURL
ionCube Loader (Version >11)
/proc/stat must be readable
/proc/cpuinfo must be readable
PHP Function sys_getloadavg must exist
DirectAdmin Path for document_root has to be modified in DA httpd configuration.
All requirements are mandatory! If any of the listed requirements are not met, Kitt will not run.
If you do not know whether all requirements are available on your server, ask your hosting provider. All listed Requirements are normally Standard, even on Shared Hosting Server.
The fastest and safest method to check the requirements is the Kitt Tester Script. Download the tester script, upload it to the root directory of the WordPress installation and request the tester script in the browser. If all requirements are met, you should see this test result: Test Result
This is why Kitt Cache Crawler is the best Cache Warmer
If you already use LiteSpeed Cache Plugin you should know Settings Kitt Cache Crawler also has. Some are the same, but there are some that are new. Many Settings have recommended values that you should confirm if you are on Shared Hosting. Follow these Recommendation and you will never get any trouble with your Provider. Some Settings like Mobile View, webp Replacements or Guest Mode can't be changed. They are fixed and they are based on current settings made in LSCWP Cache Plugin.
The most important Setting is the Load Feature where you should have a closer look on it. In Contrast to the built-in crawler of the LiteSpeed Cache Plugin, this Function not only gives you Information about the current Load on your Server, but you can also use it to set the maximum Load. If you are on Shared Hosting, please note that the Load shown refers to the entire Server. Also, keep in mind that Load doesn't represent how busy your CPU is or how much Memory you're using. The Information on the load is always relative to the entire Server and corresponds to the Sum of all relevant Factors that can affect the Load. This includes the currently running Processes in your Account as well as those of other Users on your Server.
HighSpeed Crawling Crawls up to 200.000 URLs within 1 hour, even on Shared Hosting and without causing too much load on your server.
But note that the crawling speed depends largely on the number and type of plugins installed.
Dynamic Load Control An adaptive dynamic Load Control checks the current Load on your server while crawling. If it goes too high Kitt automatically reduces the Crawling Speed until the Load goes under the predefined Limit. To determine the limit, Kitt examines the server configuration and compares the configuration with the current load. This means that your server is never overloaded by crawling and that frontend operation is not affected.
Post Type Selection Unlike the built-in crawler, you can configure Kitt to only crawl certain post types. If necessary, this selection allows only the cache of the selected post types to be purged before crawling. This results in significant time savings, because why crawl URLs that are very rarely changed.
Adaptive Configuration Kitt is pre-configured during installation, so in most cases it is sufficient to simply accept the suggested configuration. So you don't have to spend hours trying to figure out which setting is best for you.
Cache Vary Support Kitt Cache Crawler supports LiteSpeed LScache Cache Vary. This is unique among all 3rd party cache crawlers. Referring to the LiteSpeed LScache Plugin for Wordpress, this means that the different cache copies are created when using the Guestmode, Webp Replacement and Mobile View.
Multiple Crawling Methods Kitt can be run on 2 different ways. You can run Kitt from within Control Panel or as a System Cron. Each Method works as fast as each other.
Tune-up Settings Kitt's Cache Behaviour isn't fixed and allows to modify the Crawling Speed. If you have a powerful dedicated Server you can customize the settings for Crawling Speed.
Cache Inventory With Kitt's Cache Inventory you always know the current cache status of all of your URLs after a Crawling Session. This also includes all Cache Varies depending on your Plugin settings. Cache Inventory is an option that can be disabled/enabled.
URLs Most Wanted Like the built-in crawler of the LiteSpeed Cache plugin, the Kitt Cache Crawler uses the sitemap as the basis for which cache to warmup for which URLs. Alternatively, Kitt offers the Most Wanted feature. Most Wanted means that Kitt only crawls the URLs that are actually accessed by visitors. This saves a lot of time and resources for Cache Warmup and relieves the server for the cache warmup.
Crawler Settings
Delay
Kitt Cache Crawler requests URLs in parallel and not seriell (one by one). This Method is basically responsible for the Speed and how a long a Crawl Session takes. With Delay you can set the Time in Microseconds between each Request. For a proper Warmup result you should leave the recommended Value untouched, also on dedicated Server or VPS. More not always is better or faster.
Threads
Threads mean the Requests to URLs Kitt Cache Crawler does. With Threads you determine the Number of simultaneous Requests. With Threads you significantly influence the Load on your server. The entered Value are not always used, because cURL only starts new Requests if the currently started Requests are successful answered. Also in this Case let recommended Value be untouched. If the Value is too high, it is not uncommon for accessed URLs not to be cached.
Load
The Load Topic has already been described above. Take care on it and try to get a Feeling what the best Value for load is on your Server.
Network Timeout
To prevent Missunderstanding, if it is about Timeout Kitt Cache Crawler doesn't know any Timeout. On LiteSpeed Web Server it could run endless, but sometimes Network Timeouts may occur. A Network or Connection Timeout is not the same as Timeout that is caused by the Limit of max_execution Time. If you find that Network Problems are frequent, increase the recommended Value in Seconds to prevent the URL Request from being aborted and a Page not being cached as a Result.
Limit
If you are on Shared Hosting or for whatever reason you can limit the number of to be crawled URLs.
Offset
OFFSET works in combination with LIMIT and works the same as a MySQL query. OFFSET means you can start requests a specific row.
Logging
If Logging is enabled the result of each request will be written in a log file and with different information about the request the Cache Inventory is filled. Data that are written in the log file contains timing information about the request. Cache Inventory is for control if a requested URL has been successful cache.
cURL Debugging
cURL Debugging is for Experts only and gives verbose information about each cURL request.
Purge Mode
Crawler Mode can be a useful setting to always purge the Cache before Kitt starts crawling. With Crawler Mode ON all URLs get the same Time-to-Live value for caching. Crawler Mode always must be ON if LIMIT and OFFSET is used!
Cache Mode
Cache Mode provides another tuning option, but with different cache behavior. We call this the walker or runner mode. The recommended Walker Mode updates the cache of a page by the set TTL value if the cache of the respective page has not yet expired. The fast runner mode is only for cache warmup, but does not update the cache if a page's URL is still cached. The Cache Mode feature is also used to serve the different requirements when using a proxy or the LiteSpeed Web ADC.
Re-cache ModeNew Feature
Re-cache Mode is the Swiss Army Knife when it comes to keeping the Cache warm with little crawl Effort. When enabled, Re-cache Mode triggers Changes to Posts and Pages and warms up the Cache of these Post Types in near real Time. Re-cache Mode ensures that the Cache of URLs is always warmed up without having to crawl all URLs again.
AutocacheNew Feature
The Autocache function closes the gap left by the "Auto Purge Rules For Publish/Update" function of the LiteSpeed Cache plugin. Depending on the settings of this function, this function purges the cache, but does not provide a function to immediately warm up the relevant cache. You therefore have to either start the built-in crawler of the cache plugin manually or the cache will only be warmed up again at the next warmup interval. This is neither intelligent nor beneficial.
The Autocache function solves this problem without maintenance and time expenditure by checking the cache status at short intervals and starting the Kitt cache crawler automatically.
Cache Search ResultsNew Feature
The Cache Search Result Function closes the Gap of the LiteSpeed Cache Plugin and enables the caching of Search Results. Together with the URLs Most Wanted (UMW) Function, the Cache of Search Results is not only cached, but can even be warmed up with the UMW Function.
Compression
The Compression Setting compresses the static Cache Files, which significantly reduces Disk Space. However, the Benefit of this Setting goes further. The Compression Method corresponds to the same Method as is used in Content-Encoding. If a URL that has already been cached is called, the corresponding Cache File does not first have to be compressed for the Data Transfer, so that the Request is delivered much more faster.
Guest Mode
The settings for Guest Mode, Mobile View and webp Replacement cannot be changed and are only used to control the settings in the LiteSpeed Cache Plugin. Each of these settings and this also applies in combination with other settings changes the HTML output. However, since the URL remains unchanged regardless of the activated setting and each URL can only have a maximum of 1 cached file, a separate static cache file must be created for each activated setting. For crawling, this means that each URL must be requested up to 8 times.
Webp Replacement
See Guest Mode.
Cache Mobile
See Guest Mode.
Follow Redirection
Kitt Cache Crawler can follow redirections. Regardless of this feature, redirects are never good. Especially when it comes to search engine optimization. However, if redirects are still necessary, the function for following a redirect must be chosen with care, because this causes considerable delays in crawling. Ideally, you should leave the forwarding feature disabled.
Crawl Output
When Kitt is run manually in the CLI, Crawl Output can be enabled if necessary. This output shows the currently crawled URL, the current server load and the TTFB in real time.
Post Type Selection
To filter and to limit the number of to be crawled URLs you can select different Post Types. If WooCommerce Plugin is installed you can also select Products if you have downloaded the Kitt Cache Crawler Version that supports WooCommerce.
Most Wanted
The Most Wanted Title of this Feature stands for URLs Most Wanted and is a unique Feature unique to the Kitt Cache Crawler. If this Function is activated, Kitt does not use the Sitemap as the Basis for the URLs to be crawled, but rather the most requested URLs. This reduces the Number of URLs to be crawled by up to 70%, which inevitably leads to a significant Reduction in Server Load.
In addition, the Most Wanted Function also records the URLs that can never be included in the Sitemap. For the first time, the Pagination Pages, Search Result Pages and Filter URLs are also crawled and the cache is warmed up for them.
Cache Vary Selection
The LiteSpeed Cache plugin has a very important feature with the crawler simulation function. This crawler simulation is particularly important and almost absolutely necessary for all Wordpress installations that offer either a language selection and/or different currencies. If this is the case, it is imperative to enable the crawler simulation so that the built-in crawler warms up the cache for each language, currency or any other user-dependent change. At least that's the idea behind this feature. Unfortunately, this function has errors and can therefore only be used to a very limited extent. The Kitt Cache Crawler not only fixes this problem, but also makes this function usable in the first place with the Cache Vary Selection.
Sitemap Selection
If the YOAST plugin is not installed or disabled, Kitt uses the default sitemap file for crawling. Kitt supports normal sitemaps as well as index sitemaps.
Cache Inventory
The cache inventory provides you with detailed information about the cache status of each URL. Depending on the cache plugin settings, different cache vary are used for each URL. The cache inventory therefore also shows you the cache status for each URL and the cache vary used. If necessary, you can use the cache inventory to purge the cache per URL or exclude certain URLs from the warmup.
URLs Most Wanted Inventory
The URLs Most Wanted (UMW) function is one of the unique features of Kitt. In the UMW inventory you will find a real-time overview of all URLs currently visited by users, which are used for the cache warmup. The optional UMW function is a very efficient method to warm up the cache of only the URLs that are actually requested. This reduces the effort required for cache warmup by up to 70%. This not only dramatically reduces the time required for the cache warmup, but also requires significantly fewer server resources and therefore a significantly lower server load.
This is why Kitt Cache Crawler is unique
Crawls up to 200,000 URLs within 1 hourDepending on current Cache Status Kitt is able to warmup up to 200,000 URLs within 1 hour without heavy Load, even on Shared Hosting. This is up to 10x faster as built-in Crawler.
Dynamic Load ControlDon't care about the Load of your Server while Crawling. Kitt has a intelligent Load Control that dynamically reduces the Crawling Speed if the Load goes too high and encreases the Speed if the Load goes under the Limit. In Comparison to built-in Crawler this Load Control works and not only cancels Crawling if the Load goes too high.
Post Type SelectionLSCWP Cache Plugin Crawler can only nothing or all to crawl. Kitt Cache Crawler allows you to select Post Types. Less or only crawling what is needed costs less time. With Post Type Selection your Wordpress Blog has always the Maximum of Speed.
Auto-Adjust ConfigurationKitt Cache Crawler comes with pre-configured Settings, that fits in most cases. So you don't have to learn which settings are the best. Each setting is also provided with recommendations that give you even more security when making any changes.
Cache Vary SupportKitt Cache Crawler supports LSCWP Cache Varies. So if you have activated the Mobile View, webp Replacement or the Guest Mode, Kitt warms up the different Caches. However, Kitt does not use up to 8 different Crawlers for this, but does this Job at the same Time with only 1 Crawler.
Available as Desktop Version and Version for Command Line Interface (Cron)Kitt contains 2 Versions. You can run Kitt either via the Control Panel or scheduled and regularly via a Cron Job
Tune-up SettingsKitt Cache Crawler is extremely flexible, so you can make numerous Fine-Tuning and Tuning Settings depending on the System Resources.
Cache InventoryThe Cache Inventory gives you detailed Information about the current Status of the Cache, which is automatically generated after each Crawling Session.
Crawling without TimeoutKitt is able to remove the known Limitations on the maximum Execution Time of Scripts without having to make any Changes to the Server Configuration. This also applies to Shared Hosting! If you use Control Panel Version, this Feature ensures that Kitt can complete his Job successfully.
URLs Most WantedURLs Most Wanted reduces cache warmup time and resources by up to 70% and dramatically offloads the cache warmup server.
This is why Kitt Cache Crawler causes less load as the built-in Crawler
Kitt Cache Crawler is not only x-times faster as the built-in Crawler of LiteSpeed Cache Plugin for Wordpress. Kitt also causes much less load on your Server. If the built-in Crawler causes a Load of up to 100%, Kitt only takes max 60% Load! Perfectly not only for the use on Shared Hosting. This means you don't have to care about too much Load on your Server. Kitt always takes care on your Load and you can run Kitt whenever you want. Kitt will never stress your Frontpage!
LiteSpeed Built-In Crawler vs Kitt Cache Crawler
LiteSpeed Plugin Crawler
Kitt Cache Crawler
Feature Comparison
Feature
Kitt Cache Crawler
LSCWP built-in Crawler
Speed
<300%
100%
In terms of the speed of both crawlers, there is a discrepancy in which the Kitt Cache Crawler emerges as the clear winner. The fact that the built-in crawler cannotdemonstrate at least approximately the same performance despite MultiThread Mode is due to the fact that, on the one hand, the built-in crawler is insufficientlyoptimized. On the other hand, and this is the main reason, the built-in crawler is part of a WordPress plugin, which means that this crawler has to carry a largeamount of ballast with it. There are therefore crucial reasons why the Kitt Cache Crawler is a standalone application and was developed specifically for the cache warmupfunction. This special programming also results in up to 50% less server load when crawling through Kitt and is therefore particularly, but not only, suitable forshared hosting.
Delay
Both crawlers use MultiThreading mode. This means that URLs are not requested in series, but in parallel, i.e. at the same time, depending on the configuration. Thissignificantly shortens the required crawl time, but it also increases the server load significantly.
Despite the parallel requests, a short delay is necessary between the requests, which is set in microseconds using the delay setting. Shared hostings in particular should makeuse of this setting.
Threads
The number of simultaneous threads has a significant influence on the crawl time. However, this also applies to the server load during crawling. The recommended default valuesshould also be adopted for dedicated servers, as practice has shown that a smaller number of threads leads to a faster result.
Timeout
The timeout function of both crawlers determines the time how long the crawler should wait for the server's response before the request is processed.
Dynamic Server Load Management
Kitt Cache Crawler: Warming up the cache is a critical process because crawling significantly increases the server load and therefore shared hosting is particularly affected. To ensure thatcrawling does not lead to overloading, the Kitt Cache Crawler, in contrast to the built-in crawler of the cache plugin, has a Dynamic Server Load Management.This load management not only monitors the load in real time during crawling, but also controls the relevant settings depending on the performance of the server.This load management warns if necessary about settings that could overload the server and makes setting suggestions if the configuration is inadequate. The built-in crawlerdoes not have this load management, which regularly leads to incorrect settings being made and the crawling being aborted.
Built-in Crawler: Does not have this feature.
Extended Real-Time Load Monitoring
Kitt Cache Crawler: In contrast to the built-in crawler of the cache plugin, the Kitt Cache Crawler has a extended real-time load monitoring. This monitoring function not only monitors the currentload, but also shows in the Kitt tab if the set limit value is exceeded. If the limit value is exceeded, Kitt does not stop crawling, but rather overwrites the configurationvalues that may have been set too high until the limit value is exceeded again.
Built-in Crawler: Does not have this feature.
Abort Monitoring
Kitt Cache Crawler: It is not uncommon for a PHP process to terminate unexpectedly and prematurely for no apparent reason. Monitoring a process is extremely difficult and often requires functionsthat are not only security-critical when activated, but also require a special server configuration that is not available on many hostings.However, the Kitt Cache Crawler has a mechanism that can detect an unexpected termination of crawling without requiring special server functions or without affectingsecurity-relevant areas of the server.
Built-in Crawler: Does not have this feature.
Settings Verification
Kitt Cache Crawler: Both the built-in crawler of the cache plugin and the Kitt cache crawler have a large number of settings that are inevitably necessary. However, most users are overwhelmed byconfiguring the crawler because they lack the necessary background knowledge. As a result, misconfiguration occurs very often, especially with the built-in crawler, becausethis crawler does not check the settings made, let alone provide sufficient assistance to avoid misconfiguration.
The Kitt Cache Crawler has a control mechanism that not only warns of illogical settings or does not allow them, but also issues warnings if settings lead to overloading.In addition, detailed documentation is available for almost every setting, so that even inexperienced users can obtain the relevant knowledge.
Built-in Crawler: Does not have this feature.
Unblockable by Host Provider
Built-in Crawler: LiteSpeed is not only the publisher of the cache plugin, but also the LiteSpeed web server. Since the cache plugin is not cached, butrather the cache engine of the LiteSpeed web server, LiteSpeed is obliged to give the hosting providers the opportunity to deactivate the function of the built-in crawler ifnecessary. This blocking mostly affects shared hosting because crawling can result in a very high load. The cache plugin therefore checks in a very simple way whether avalue of a certain server variable exists and in turn deactivates the built-in crawler.
Kitt Cache Crawler: The Kitt Cache Crawler is not subject to this obligation and poses as a normal user during the crawl simulation, which allows it to be claimed that the Kitt Cache Crawlercannot be blocked. The Kitt Cache Crawler can therefore be used even if the hosting provider has disabled the crawl function.
Cache Vary Improvements
Built-in Crawler: Depending on whether Guestmode, webp Replacement or Cache Mobile is activated, so-called Cache Vary are used, which enable the creation ofmultiple cache copies of just 1 URL. Unfortunately, these cache copies, which are set in the .htaccess by the cache plugin, are not only faulty but also incorrect since thecache plugin was available. The consequences of this malfunction are made even more serious by incorrect values for the crawler simulation, which means that incorrectcache warmup regularly occur.
Kitt Cache Crawler:The Kitt Cache Crawler recognizes the settings of the cache plugin and adapts the type and scope of the URLs to be crawled to these settings. However, Kitt uses optimized and debugged simulation values, which allows for proper cache warmup.
Cache Inventory
Kitt Cache Crawler: The cache inventory is a central component of the Kitt Cache Crawler. The relevant cache status data for each request is displayed in a clear table. The crucial difference tothe similar overview in the cache plugin is that the cache status for each cache vary used is recorded in the cache inventory. This makes it possible to check, even withoutmanual debugging, whether any problems occurred during cache warmup for a specific cache vary for a URL. In addition, the cache status displayed is much more detailed andincludes all possible cache statuses. The cache inventory is supplemented by the option to purge the cache of individual URLs and also to blacklist individual URLs if certainURLs should be excluded from the cache warmup.
Built-in Crawler: The built-in crawler has a similar cache inventory function ("Map"). However, this map only provides very limited information about the cachestatus, especially since it is patchy and incomplete. This map is unsuitable for checking whether the cache warmup was successful for each URL and cache vary because essentialdata is missing.
Follow Redirection
Kitt Cache Crawler: If the sitemap is used as the source of the URLs to be crawled, then Kitt relies on all URLs existing. Especially since the sitemap is usually generated by a plugin and theplausibility of each URL must be guaranteed. If this is not the case, then the sitemap generation must be faulty. Since there can still be deviations regardless, the Kitt CacheCrawler enables the forwarded URL to be crawled and thus cached in case of doubt. However, activating this follow redirection function is not recommended, as it negatively affectsthe crawl behavior and drastically increases the crawl time. Ultimately it is a matter of discretion, so Kitt allows the user to choose whether Kitt should follow a redirect or not.
Built-in Crawler: The built-in crawler is subject to the same negative consequences of a redirect. However, this crawler does not allow deactivation, so thiscrawler follows every redirect, which leads to a significantly longer cache warmup time requirement.
Crawl Output
Kitt Cache Crawler: If the Kitt Cache Crawler is run manually in the console, the output of the currently crawled URLs can be observed by activating the Crawl Output function. The respectiveTTFB and the server load are displayed together with the URL.
Built-in Crawler: Does not have this feature.
Sub-Post Type Selection
Kitt Cache Crawler: If the Kitt Cache Crawler is used together with the YOAST Seo Plugin, then YOAST allows the selection of sub-post types instead of the sitemap. Sub-Post Types are a subdivisionof the rather rough WordPress differentiation Posts and Pages. The selection of sub-post types therefore allows, for example, the selection of shopping categories, so that,for example, only shopping categories are crawled, but not conventional blog categories. The same applies to all other (sub)posts and (sub)pages.
Built-in Crawler: Does not have this feature.
Sitemap Crawling
Both crawlers use the sitemap.xml as a source for the URLs to crawl. With the built-in crawler, the sitemap is absolutely necessary. With the Kitt Cache Crawler, the sitemap isonly an alternative source, as both the sub-post types and the URLs Most Wanted functions can serve as additional sources.
Cache Vary Selection
Built-in Crawler: The built-in crawler allows the crawler to be simulated if variable cookies are set, but this requires appropriate cache vaires definitions. However, these definitions arenot set by the built-in crawler, so this crawler simulation is pointless and useless. This means that the required cache vaires have to be set manually in .htaccess, whichwill be overwhelming for most users because there is no documentation about it.
Kitt Cache Crawler: If simulation settings are defined in the built-in crawler settings, the Kitt Cache Crawler takes over these settings and makes the selection available for each simulation andwarms up the cache accordingly. This eliminates the need to manually set cache varies. The Cache Vary Selection is therefore the completion of the inadequately programmedcrawler simulation function of the cache plugin.
Cache Mode
Kitt Cache Crawler: The Cache Mode function is based on an undocumented function of the LScache Engine of the LiteSpeed web server, although it should be mentioned that the LiteSpeed Cache Plugindoes not cache, but is simply a control panel to control the caching behavior of the web server based cache engine LScache. Depending on whether crawling is possible using thebuilt-in crawler, the cache mode function enables differentiated cache warmup behavior with a decisive impact. If the cache mode function is activated, the cache of alreadycached URLs will be renewed with the set cache TTL. If this function is disabled, then the cache status is only checked without affecting the cache lifespan.
Built-in Crawler: Does not have this feature.
Quic.cloud ready
Both crawlers are quic.cloud ready, which means that crawlers can warm up the cache of the origin server. However, this does not mean that the cache of every quic.cloud nodecan be warmed up. This is not possible with Kitt, nor with the built-in crawler, nor with any other cache crawler
GUI Crawler
Both crawlers provide a function to manually warm up the cache via the desktop.
CLI Crawler
Both crawlers provide a function to warm up the cache via a system cron job. This functionality enables a time-controlled cache warmup that is independent of WordPress,although controlling the CLI cron through the built-in crawler is only suitable for experts.
User friendly Error Management
Both crawlers have an error management, which generates a corresponding error log file if an error occurs during crawling. However, the error management of the built-in crawler is not user-friendly, so only LiteSpeed Support is able to identify a possible error.
URLs Most Wanted
Kitt Cache Crawler: The URLs Most Wanted (UMW) function is another fundamental component of the Kitt Cache Crawler, which can improve the reduction of crawling and thus the reduction of serverload by up to 70%. The UMW function records the URLs requested by natural users and uses this source for cache warmup. The basis of this function is the knowledge that a cachewarmup is only beneficial for the URLs that are actually requested by natural users. Any URL requested beyond this is a waste of time and performance.
Built-in Crawler: Does not have this feature.
Local VPI
Kitt Cache Crawler: The VPI or View Port Image function of the LiteSpeed Cache Plugin is not a direct crawler function, but the VPI function influences the crawler behavior. The VPI function offered by LiteSpeed, which is free, but only to a limited extent. If you exceed your monthly budget, you either have to pay for it or wait until the next month. The local VPI function of the Kitt Cache Crawler makes the VPI function independent of quic.cloud and therefore enables a free and therefore unlimited VPI function. Kitt only needs the free AWR plugin for LiteSpeed LScache.
Built-in Crawler: Does not have this feature.
webp Replacement
Built-in Crawler: The webp replacement function is not a direct function of the crawler, but it influences the crawl behavior. The cache varies provided by the LiteSpeed Cache Plugin are incorrect when the webp replacement function is activated, so the cache warmup is also incorrect. It is therefore not recommended to use this function!!!
Kitt Cache Crawler: If the AWR plugin for WordPress is installed, Kitt will correct the incorrect settings of the LiteSpeed Cache plugin. The AWR plugin enables an error-free cache warmup.
avif Replacement
Built-in Crawler: The replacement of webp or avif images is not a direct function of the crawler, but it does influence the crawl behavior. However, the cache plugin for LScache currently does not support avif replacement, so this function is not available.
Kitt Cache Crawler: The Kitt Cache Crawler supports avif replacement if the AWR plugin is installed.
Why Kitt can't be blocked?
LiteSpeed has added a Server Variable to LiteSpeed Web Server. Depending on Configuration of LSWS and OLS this Variable allows or prevents to run the built-in crawler of LiteSpeed cache plugin for Wordpress. So if you are on Shared Hosting your Provider can disallow to run the built-in crawler. (But can simply be bypassed....)
Kitt isn't affected from this possible Block, because Kitt doesn't care about this Variable and Providers have no way to detect if Kitt is running. Request Logging of Kitt is disabled as well. If you take care of the Load nothing will happen!
What do I have to consider for an installation?
There is nothing special to consider when installing the Kitt Cache Crawler. If you were able to install WordPress, then you can also install Kitt. Especially since the Kitt Cache Crawler is even easier and quicker to install. However, you should note that the Kitt Cache Crawler is not a WordPress plugin, but rather an independent application that runs parallel to Wordpress. So don't try to install the Kitt Download Archive using the WordPress plugin installation function. If this happens, you will always receive an error message.
Download the Kitt Cache Crawler and unzip the archive. There is an install_notes.txt in this archive. In this file you will find all information you need to install Kitt. The instructions are simple and understandable. If you still have problems, just ask us. You will get the answer immediately.
Still Questions?
We try very hard to give you as much information as possible. If you can't find the information you need or if you don't understand something, don't hesitate to ask us.
