windows 下pytorch dataloader num_workers 设置大于0的值 报错

Question

环境：windows10 cuda11.0

dataloader设置：

train_loader = DataLoader(dataset=train_dataset,
                               batch_size=512,
                               shuffle=True,
                               num_workers=0
                          )
test_loader = DataLoader(dataset=test_dataset,
                              batch_size=512,
                              shuffle=False,
                              num_workers=4
                         )

报错：

RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if name == ‘main’:
freeze_support()
…
The “freeze_support()” line can be omitted if the program
is not going to be frozen to produce an executable.

网上说要把num_workers的值改为0，但是num_wokers太小的话，训练过程很慢。

Answer 1

荼灬 2020-12-14 20:08:15

windows下就不能使用num_workers参数

Answer 2

一个很坏的好人 2020-08-23 12:45:08

https://github.com/pytorch/pytorch/pull/5585

官方解答