Has anyone managed to get the crawler to work?

[johnysilvermon]

In 0.8.6 osx, the crawler is an option under 'network'. It states that it will "Listens for announces, prefers nearby low-hop nodes, fetches at most one page per node per day"

However, from what I can tell, the pages in my archive are ALL pages I manually clicked from the announce list - not pages the crawler archived itself.

In addition, the crawler seems to not be indexed or searchable, which I imagine implies it's because the devs intended the archive to be the index - however as indicated above that doesn't appear to work.

Am i missing the obvious?

[Sudo-Ivan]

Working on improving/fixing it v4.9.1 and onwards.

[johnysilvermon]

Thank you! I wasn't sure what was appropriate to use, so since you replied here, I'll put my thoughts below instead of opening an enhancement request:

The most important thing is that ATM i don't think it's triggering at all wether it's on OR off. However all the options in settings are brilliant and I look forward to seeing it in action.

In my humble opinion:

• The current behaviour of caching the page on open and having multiple archived copy is brilliant
• Therefore, it would be logical for the crawler to simply operate in the exact same fashion: BOTH the cache and the crawler using the same database, providing a cached copy if the page times out, AND be searchable like a search engine.
• This is because i might be offline when i browse a favorite I saved, or the nomadnet node I browse might be offline

One thing i think is very important is the logs should show all the crawler activity:

• discovery
• attempt to cache (meaning testing if it's up)
• caching activity
  ... otherwise it's going to be impossible to tell if it's working or not

mmm that about covers it :) I hope this is useful feedback!

Thank you again!