I'm not so sure that it works now. @yf225, Do you know when the CI can be put into work? If Windows CI can be setup quickly, then I'd rather have it tested on CI. Otherwise, I'll test it on my own PC.

@peterjc123 Windows CI will be enabled again after this fix #4469. We can test it on CI.

@pytorchbot retest this please

@apaszke It seems that the timeout options in dataloader.py is too small. The creation of processes takes longer time in Windows. Shall we modify them one by one or set a base timeout and use the multiplied timeout?

Let’s multiply the timeout for Windows only

@pytorchbot test this please

Thank you @peterjc123!

I'm curious. How does Windows build pass the test_segfault test? Wouldn't the main process get stuck waiting for data at data_queue.get (like the POSIX builds before SIGCHLD handler)?

@ssnl No, it won't. Because there's no fork in Windows. Although you initialized the CUDA context of one tensor, the one in the child process is a brand new one due to spawn, so the segfault won't be triggered.

@peterjc123 I'm still a bit confused. Are you saying that ctype.string(0) doesn't segfault on windows because child processes get new CUDA contexts due to spawn?

@ssnl Sorry, I messup that up with test_cuda_bad_call. As for test_segfault,the process should be test_segfault starting a new process to process function _test_segfault, the new process called the DataLoader and the processed was finally closed because there's an exception in the DataLoader. I guess you're asking about the timeout of the data_queue.get() right?
Here is full traceback:

Process Process-1:
Traceback (most recent call last):
  File "C:\Anaconda2\envs\test_new\lib\multiprocessing\process.py", line 258, in _bootstrap
    self.run()
  File "C:\Anaconda2\envs\test_new\lib\multiprocessing\process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "D:\pytorch\mp_dl_test.py", line 107, in _test_segfault
    _ = next(iter(dataloader))
  File "C:\Anaconda2\envs\test_new\lib\site-packages\torch\utils\data\dataloader.py", line 273, in __next__
    return self._process_next_batch(batch)
  File "C:\Anaconda2\envs\test_new\lib\site-packages\torch\utils\data\dataloader.py", line 293, in _process_next_batch
    raise batch.exc_type(batch.exc_msg)
OSError: Traceback (most recent call last):
  File "C:\Anaconda2\envs\test_new\lib\site-packages\torch\utils\data\dataloader.py", line 56, in _worker_loop
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "C:\Anaconda2\envs\test_new\lib\site-packages\torch\utils\data\dataloader.py", line 56, in <listcomp>
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "D:\pytorch\mp_dl_test.py", line 48, in __getitem__
    return ctypes.string_at(0)
  File "C:\Anaconda2\envs\test_new\lib\ctypes\__init__.py", line 492, in string_at
    return _string_at(ptr, size)
OSError: exception: access violation reading 0x0000000000000000

The data_queue in the child process is wrapped in the try..catch block, as commented below:

def _worker_loop(dataset, index_queue, data_queue, collate_fn, seed, init_fn, worker_id):
    global _use_shared_memory
    _use_shared_memory = True

    # Intialize C side signal handlers for SIGBUS and SIGSEGV. Python signal
    # module's handlers are executed after Python returns from C low-level
    # handlers, likely when the same fatal signal happened again already.
    # https://docs.python.org/3/library/signal.html Sec. 18.8.1.1
    _set_worker_signal_handlers()

    torch.set_num_threads(1)
    torch.manual_seed(seed)

    if init_fn is not None:
        init_fn(worker_id)

    while True:
        r = index_queue.get()
        if r is None:
            break
        idx, batch_indices = r
        try:
            samples = collate_fn([dataset[i] for i in batch_indices])   <- throw exception
        except Exception:
            data_queue.put((idx, ExceptionWrapper(sys.exc_info()))) <- exception returns here
        else:
            data_queue.put((idx, samples))

After the exception was thrown, the data_queue.get() receives this and after that, the function process_next_batch checks this and stops the new process. So it should be passing without problem.

@ssnl However, if the index_queue.get() throws exception in the previous example(worker_loop), then it will cause the deadlock.

@peterjc123 I see. Interesting that windows can throw an OSError upon invalid memory access. In posix, we had to do some signal handling to manually capture such issues. :)