-
Notifications
You must be signed in to change notification settings - Fork 75.2k
Description
The swish was originally coined the "SiLU" in https://arxiv.org/pdf/1606.08415.pdf and https://arxiv.org/abs/1702.03118 long before the swish paper. Renaming other peoples' exact same ideas is unacceptable and tensorflow's naming convention implicitly erases the research and work of people outside of Google.
This request inspired by a recent discussion, but this problem has been brought up every few months for the past few years. In light of recent efforts to make the ML community more equitable and fair, this is a no-brainer and long overdue.
Will this change the current api? How?
tf.nn.swish ought to be renamed tf.nn.silu and both of the aforementioned papers will be cited in the documentation. At the very least they should be documented in the swish documentation already.