The current instance norm implementation delegates to nn.functional.batch_norm for the actual computation, and if my understanding is not wrong, the latter requires the presence of running mean and variance. Instance norm, however, usually never needs these statistics even in testing mode, so it is waste of both time and space. It would be nice if batch_norm could be modified such that running mean and variance are optional. It won't be hard, will it?