Track search result pages instead of individual product pages

[MRizkBV]

Description

A feature that can really cut the bandwidth usage, and also consume less resources is to scan search results on retailers like Newegg, or Amazon instead of crawling every single page.

In case of Newegg, the search results show an add to cart button if it is in stock, and in case of Amazon, the search results show a price if it is in stock.

I believe this can also extend to other retailers, most of them at least. This could really cut the resources used by the bot, and deliver faster results.

[jef]

This has been an interesting topic of a few reasons:

• Some result pages change frequently
  • Amazon does this a bit, but there are some queries that work OK
• Potentially more false positives due to text matching
• Even more pull requests to update these results

I'm not against the idea, but one thing I think should happen is easier configuration. That way if people want to use all the stores listed, they can. Or they can easily search a result set.

That being said, someone can create a "new store" and use whatever containers and text they want. Feel free to try it out. There are instructions in the wiki!

Thanks!