请稍等 ...
×

采纳答案成功!

向帮助你的同学说点啥吧!感谢那些助人为乐的人

网上查找的方法都解决不了这个fake_useragent.errors.FakeUserAgentError

现在报错

fake_useragent.errors.FakeUserAgentError: Maximum amount of retries reached

网上查来三种解决方法

# ua = UserAgent()
方法一
ua = UserAgent(cache=False)
方法二
ua = UserAgent(verify_ssl=False)
方法三
ua = UserAgent(use_cache_server=False)

# headers = {
#     "User-Agent": ua.random
# }
方法四
headers = {"User-Agent": UserAgent(use_cache_server=False).chrome}

以上四种方法,都还是报错。

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 814, in __init__
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1068, in do_handshake
    self._sslobj.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/utils.py", line 67, in get
    context=context,
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/django_manage.py", line 52, in <module>
    run_command()
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/django_manage.py", line 46, in run_command
    run_module(manage_file, None, '__main__', True)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/mg/Gamification/manage.py", line 15, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/management/__init__.py", line 371, in execute_from_command_line
    utility.execute()
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/management/__init__.py", line 365, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/management/base.py", line 332, in execute
    self.check()
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/management/base.py", line 364, in check
    include_deployment_checks=include_deployment_checks,
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/management/base.py", line 351, in _run_checks
    return checks.run_checks(**kwargs)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/checks/registry.py", line 73, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/checks/urls.py", line 40, in check_url_namespaces_unique
    all_namespaces = _load_all_namespaces(resolver)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/core/checks/urls.py", line 57, in _load_all_namespaces
    url_patterns = getattr(resolver, 'url_patterns', [])
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/utils/functional.py", line 36, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/urls/resolvers.py", line 536, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/utils/functional.py", line 36, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/django/urls/resolvers.py", line 529, in urlconf_module
    return import_module(self.urlconf_name)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/mg/Gamification/Gamification/urls.py", line 46, in <module>
    from english_books.views import WordsListView, WordsView, WordProficiencyView, SentenceListView
  File "/Users/mg/Gamification/Gamification/apps/english_books/views.py", line 15, in <module>
    from Gamification.media.books.audios.spider_words import get_icibe_word
  File "/Users/mg/Gamification/Gamification/media/books/audios/spider_words.py", line 41, in <module>
    headers = {"User-Agent": UserAgent(use_cache_server=False).chrome}
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/fake.py", line 69, in __init__
    self.load()
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/fake.py", line 78, in load
    verify_ssl=self.verify_ssl,
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/utils.py", line 250, in load_cached
    update(path, use_cache_server=use_cache_server, verify_ssl=verify_ssl)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/utils.py", line 245, in update
    write(path, load(use_cache_server=use_cache_server, verify_ssl=verify_ssl))
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/utils.py", line 178, in load
    raise exc
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/utils.py", line 154, in load
    for item in get_browsers(verify_ssl=verify_ssl):
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/utils.py", line 97, in get_browsers
    html = get(settings.BROWSERS_STATS_PAGE, verify_ssl=verify_ssl)
  File "/Users/mg/SynologyDrive/Python/virtualenv/Gamification/lib/python3.6/site-packages/fake_useragent/utils.py", line 84, in get
    raise FakeUserAgentError('Maximum amount of retries reached')
fake_useragent.errors.FakeUserAgentError: Maximum amount of retries reached

正在回答 回答被采纳积分+3

插入代码

1回答

bobby 2020-06-28 17:55:51

这是因为fake useragent访问的网站 在某些同学的网络下访问不了 可能是有些人使用了连通或者移动的网了, https://github.com/alecxe/scrapy-fake-useragent 可以使用其他的useragent库 github上很多的

0 回复 有任何疑惑可以回复我~
  • 提问者 两努 #1
    查了一下,都是配置
    DOWNLOADER_MIDDLEWARES = {
        'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware': None, # 关闭默认方法
        'scrapy_fake_useragent.middleware.RandomUserAgentMiddleware': 400, # 开启
    }
    但是,如果不是用 spider 框架。而是用基本爬虫方案,在页面中引入
    # 爬虫
    from fake_useragent import UserAgent。
    然后设置 headers
    headers = {
        "User-Agent": ua.random
    }
    普通爬虫方案能用scrapy-fake-useragent吗
    回复 有任何疑惑可以回复我~ 2020-07-04 15:30:58
  • bobby 回复 提问者 两努 #2
    当然能了啊
    回复 有任何疑惑可以回复我~ 2020-07-05 19:37:06
问题已解决,确定采纳
还有疑问,暂不采纳
意见反馈 帮助中心 APP下载
官方微信