This update fundamentally changes the way a lot of scraping is done in order to add the ability to scrape for specific games individually. Please read the patch notes to understand the changes, and why things have changed.
In previous versions, there wasn't actually a way to scrape for individual games. The system actually "cheated" to get it done. What we did was do a simple full scrape, then tossed out streams not playing approved games. The problem with this was the full scrape only fetched the top 1000 streams, and with popular services like Twitch, there was a high chance that a lot of streams were being left off the list because they weren't in the top 1000.
So this new version adds in queries to fetch streams based on what game they are playing, instead of doing a full scrape and tossing. Unfortunately, not every service supports the ability to do a games scrape. To make things a bit easier to understand, I have made this chart:
This chart defines which service can scrape which kind of data. As you can see, every service supports channel scraping. With this new version, there is a new scrape restriction called "Channels OR Games". With this new addition, there are now 5 scrape restriction options:
With these new options, please review your settings and make sure they are accurate to your preferences. Also note that games scraping adds an extra cURL operation for each game in your approved games list. So as your games list increases, your cURL rate increases. If your cURL rate gets too high, you may end up getting rate limited.
- No Restrictions - this is the "full scrape" option. This option will scrape the top 1000 live streams on a supported service. Only 4 services currently support this option.
- Channels - this is the "channels" option. This option will scrape for only approved channels for each service.
- Games - this is the "games" option. This option will scrape for any live streams playing approved games on a supported service. Only 3 services currently support this option.
- Channels OR Games - this is a combination of "channels" and "games". This will scrape for approved channels regardless of what game they are playing... and then it will also scrape for any live streams playing approved games, even if they are not approved channels. This is basically the best of both worlds option as it will work with all services, while simply bypassing the second step for services that don't support the games scraping.
- Channels AND Games - this is a "channels" option. This will scrape for approved channels, and then if it finds that the channel isn't playing an approved game, it will throw the channel out of the listing. This option is independent of the games restriction and will work with every service, even those that don't have games scraping.
- Expanded the options for the bubble count on the navbar which shows the number of live streams. You can now set it to show the number of featured streams only.