Wordpress LScache Plugin: All Pages Added to Blacklist ? Crawler
Last Updated on: Wed, 15 Apr 2026 00:00:02 Hello, Sorry I start another thread, Im trying to optimize the plugin in the best way possible and its better to address each problem separately. Today I added a custom XML sitemap to the Crawler before manually starting it. After I started it, each crawler found 268 urls but were immediately blacklisted. I checked if my server IP is correct, according to your article this is usually the reason for the whole sitemap to be blacklisted but the server IP was correct. These are my current settings: https://prnt.sc/1t3fg6l https://prnt.sc/1t3fhwa Can you help me make it work? Crawler is an essential tool for me since I purge the cache one or sometimes twice a week. Hi, please provide the report number , you can get it in toolbox -> report -> click send to LiteSpeed and then please do me a screenshot on blacklist page Best regards, Hi, Report number: JXSELXEK Blacklist page: https://prnt.sc/1t79ob1 Kind regards if you hover mouse over these 1/2/3/4 for few seconds , it will return a response code , what do you see ? 404 for all of them https://prnt.sc/1tayv89 Hi, go to General -> set server IP to empty then clean blacklist and crawl again Best regards, Hello, This solved the problem! I ran the crawler and it started this time! But a little help in understanding it would be beneficial. I ran the 4 crawlers after I upgraded the plugin and the cache was purged. These are my results: https://prnt.sc/1tfy2gu The blue means that there was no cache and its generated now, correct? Why did the blue numbers went up in crawler 3 and 4, instead of the first two? Also ? After the crawling ended, I entered the website through mobile (Edges developer tools) and the pages were all cache miss. how many URIs you see in your [2] Map tab ? Sitemap Total: 180 https://prnt.sc/1tu3bd2 okay, please try this go to toolbox -> debug setting -> set debug to admin IP only , add your IP and 127.0.0.1 into the IP list , set log to advance then go to crawler page -> manually start to crawl , you may need to hit that button few times , wait for all 4 finishes , then please provide the file /wp-content/debug.crawler.log please do remove the sensitive info in the log if you want , like your IP and site domain if file is large , you can put on google drive or onedrive or dropbox ?etc and share the link. I think its better if I send it to you. Drop me a quick email ? anotherpo55 at gmail.com Hi, please send it to support at litespeedtech.com with reference link to this topic Best regards,
LiteCache Rush: Speed comes from using less, not from doing it faster
Reference