Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

运行后,只得到一个meizi3的空文件,是怎么回事?大家来看看 #8

Open
actioncr opened this issue Jul 13, 2019 · 2 comments

Comments

@actioncr
Copy link

爬虫失败
0

@dingyuanhong2006
Copy link

dingyuanhong2006 commented Aug 27, 2019

同问: 调试发现。代码都没有跑到pipelines.py中去。连注册的def content(self, response):都跑不进去。是不是哪个配置不正确啊

@zklar
Copy link

zklar commented Sep 4, 2019

原网站反爬虫机制做了更改. 代码也需要相应更改(到2019年9月4日为止,更改两个地方即可绕过反爬机制)
①将AoiSolaSpider.py中的allowed_domains = ["www.mm131.com"]变为allowed_domains = ["www.mm131.com", "www.mm131.net"] (这里解决的是content被过滤的问题)
②将middlewares.py下AoisolasSpiderMiddleware类中process_request函数的内容整个换成: request.headers['referer'] = "http://www.mm131.com/?zzaqkey=4087969942"
(绕过反爬)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants