-
Notifications
You must be signed in to change notification settings - Fork 26.3k
Description
I know from googling previous issues that the PyTorch team are not keen on implementing a View/Reshape module to be used in Sequential.
What's missing from those issues / comments is reasoning for or against.
I have 20+ years in technology, including machine learning and data mining, and more recently have focussed on teaching students of all ages. I hope this experience gives some weight to my case below.
.
1. Against
There doesn't appear to be a good case presented by pytorch developers against a View() module that can be used inside Sequential. The best I can find is indications of a preference for users to build their own forward() functions but again this is not explained - not that I could find despite searching.
.
2. For
There are several arguments for.
- The Sequential approach is easy, simple, sufficient and requires little code for many tasks. Even of these tasks are relatively simple, that is not a reason against. The majority of uses of PyTorch are simple cases not the most extremely large networks. Make the 80% use case easy.
- Permitting View in Sequential doesn't prevent the developers preference for users to build their own forward functions. Both are possible, and meet the needs of different users, or users at different stages of expertise.
- Every user having to implement their own View module is wasted time, and potentially significant sources of error - which is a burden on them and also support here. If its a common task, do it once and do it well.
- There is no machine learning reason not to encourage use of View in Sequential.
- Supporting an easy path for newcomers to PyTorch is good strategy. It increases the user community and that is a positive. Barriers to adoption are not good. Python itself has become the leading numerical computing language, because of easy adoption, not because it was designed for fast computation.
Those are my thoughts. I welcome challenge based on reasoned arguments for or against - which previously appeared lacking.
Here's my own implementation which I now find doesn't handle channels... so illustrating the problem of amateur implementations like mine..
# from https://github.com/pytorch/vision/issues/720
class View(nn.Module):
def __init__(self, shape):
super().__init__()
self.shape = shape
def forward(self, x):
return x.view(*self.shape)