Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Double encoding in search. #66

Open
silsebastian opened this issue Jan 27, 2020 · 1 comment
Open

Double encoding in search. #66

silsebastian opened this issue Jan 27, 2020 · 1 comment
Labels

Comments

@silsebastian
Copy link

Description:
If search contains special characters, it will be encoded twice: the first one due to the quote_plus library and the second one is done by Google's servers. It is easily solved by removing quote_plus from parameters. That is, having:

        self.params.update({
            'q': query,
            'c': 'apps',
        })

instead of:

        self.params.update({
            'q': quote_plus(query),
            'c': 'apps',
        })

To Reproduce
The input will be the developer web of Instagram: https://help.instagram.com/
When running res = play_scraper.search("https://help.instagram.com/", detailed=True)
Play_scraper does the following query:
/store/search?q=https%253A%252F%252Fhelp.instagram.com%252F&c=apps&gl=us&hl=en
Which is not right. If we look for that url in a browser, a query with encodings is written automatically in the searchbox:

Screenshot from 2020-01-27 10-37-02

Expected behavior
If we manually put https://help.instagram.com/ in the searchbox, the url will be:
https://play.google.com/store/search?q=https%3A%2F%2Fhelp.instagram.com%2F&c=apps&hl=en&gl=us

If we use the piece of code without quote_plus, the url that is searched for is exactly the same as the desired one.

Desktop (please complete the following information):

  • play_scraper Version 0.6.0
@milekey
Copy link

milekey commented Feb 5, 2020

Thank you silsg, your suggestion is quite exact.
I was troubled by seem-to-be-randomly-returned-results when I searched with multi-byte query word.
I found your comment then I downloaded this library and patched line 230 in scraper.py as you said, the error has gone!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants