Crawler problems




Wordpress LScache Plugin: Crawler problems


Hi, Something is not okay with my crawler, I have a lot of pages (14 000) and he never managed to cache them all. When i setup multiple threads for example 3 and check crawler status i see all the time only threads: 1. 06 Jun 2022 20:54:31 Size: 14136 Crawler: #1 Position: 1700 Threads: 1 Status: crawling, updated position Crawler is also stopping without any reason.. Any idea what is wrong here? RAPORT: QNMSXFWD 06/06/2022 19:10:55 This topic was modified 5 months, 3 weeks ago by regipl. The page I need help with: https://klocki.fun please do me a full screenshot on crawler status page Print screen: https://klocki.fun/status.png As you can see for example now, it is not moving forward.. there was still half page above this one , thats the most important part. Sorry, one more try: https://klocki.fun/status1.png https://klocki.fun/status2.png Plus Debug Log if it helps: https://klocki.fun/debug.txt on your screenshot 1, you can see ended reason: stopped_reset this means there was a purge all happened during the crawler , so crawler reset its position and wait for the interval time passes try set the interval to very very short time for test, like 61 seconds and then make sure you have cron works/triggers properly and in time. When i restart the crawler it keeps going but only after 15 min stopping. I do not see anything that would explain this in the debug log, can you pelase check? https://klocki.fun/error.txt It stopps crawling at: 06/13/22 17:47:37.352 but i do not see any error messages. if you are checking on /wp-content/crawler.log , did you see /wp-content/debug.purge.log as well ? if so , whats content of it ?
Do you need help?
We are online almost any time and ready to answer any question. The answer to your question will come faster than it takes you to release the mouse button.

Ask A Question