scrapyd部署后，运行爬虫，出现FileNotFoundError

Question

本地运行没问题部署后运行爬虫很快就停了找不到文件FileNotFoundError不知道在哪里找原因

2017-12-04 18:43:29 [twisted] CRITICAL:
Traceback (most recent call last):
 File "c:\users\administrator\envs\p3spider\lib\site-packages\twisted\internet\defer.py", line 1386, in _inlineCallbacks
   result = g.send(result)
 File "c:\users\administrator\envs\p3spider\lib\site-packages\scrapy\crawler.py", line 77, in crawl
   self.engine = self._create_engine()
 File "c:\users\administrator\envs\p3spider\lib\site-packages\scrapy\crawler.py", line 102, in _create_engine
   return ExecutionEngine(self, lambda _: self.stop())
 File "c:\users\administrator\envs\p3spider\lib\site-packages\scrapy\core\engine.py", line 70, in __init__
   self.scraper = Scraper(crawler)
 File "c:\users\administrator\envs\p3spider\lib\site-packages\scrapy\core\scraper.py", line 71, in __init__
   self.itemproc = itemproc_cls.from_crawler(crawler)
 File "c:\users\administrator\envs\p3spider\lib\site-packages\scrapy\middleware.py", line 58, in from_crawler
   return cls.from_settings(crawler.settings, crawler)
 File "c:\users\administrator\envs\p3spider\lib\site-packages\scrapy\middleware.py", line 36, in from_settings
   mw = mwcls.from_crawler(crawler)
 File "c:\users\administrator\envs\p3spider\lib\site-packages\scrapy\pipelines\media.py", line 68, in from_crawler
   pipe = cls.from_settings(crawler.settings)
 File "c:\users\administrator\envs\p3spider\lib\site-packages\scrapy\pipelines\images.py", line 95, in from_settings
   return cls(store_uri, settings=settings)
 File "c:\users\administrator\envs\p3spider\lib\site-packages\scrapy\pipelines\images.py", line 52, in __init__
   download_func=download_func)
 File "c:\users\administrator\envs\p3spider\lib\site-packages\scrapy\pipelines\files.py", line 234, in __init__
   self.store = self._get_store(store_uri)
 File "c:\users\administrator\envs\p3spider\lib\site-packages\scrapy\pipelines\files.py", line 270, in _get_store
   return store_cls(uri)
 File "c:\users\administrator\envs\p3spider\lib\site-packages\scrapy\pipelines\files.py", line 48, in __init__
   self._mkdir(self.basedir)
 File "c:\users\administrator\envs\p3spider\lib\site-packages\scrapy\pipelines\files.py", line 77, in _mkdir
   os.makedirs(dirname)
 File "c:\users\administrator\envs\p3spider\lib\os.py", line 231, in makedirs
   makedirs(head, mode, exist_ok)
 File "c:\users\administrator\envs\p3spider\lib\os.py", line 241, in makedirs
   mkdir(name, mode)
FileNotFoundError: [WinError 3] 系统找ä¸åˆ°æŒ‡å®šçš„路径。: 'c:\\users\\admini~1\\appdata\\local\\temp\\articlespider-1512384126-2rpi5uvj.egg\\ArticleSpider'

Answer 1

SillyGGGG 2018-02-22 17:33:25

遇到同样的问题!请问怎么解决!

Answer 2

提问者 一一倾 2017-12-05 13:45:33

此问题已解决，要注意是在哪里运行的scrapyd命令

• bobby #1

  好的，

• SillyGGGG #2

  新年好!我想问一下!最后怎么解决的!

• 慕粉3691223 回复 SillyGGGG #3

  应该是使用scrapyd部署上去的时候，BASE_DIR会被其覆盖，导致我们很多的log都找错了地方，显示错误，,不要使用BASE_DIR在系统中拼装各种路径, 我们要新建一个 LOG_DIR代替BASE_DIR来拼装各路径 
  
  可以参考下:
  
  ​https://stackoverflow.com/questions/50453479/valueerror-while-deploying-scrapy

点开查看后面4条评论