Is it safe to assume that the password on key.pem was randomly generated with generate-csr.sh and isn't stored in this repo, and that the pfx_password is only used for the export password in make-pfx.sh?

The private key is encrypted in two places here:

In /windows-installer/key.pem it is encrypted with the key generated in /windows-installer/genreate-csr.sh.

In /windows-installer/codesign.pfx it is encrypted with the password generated by /windows-installer/make-pfx.sh. This password is encrypted with our AppVeyor account's public key and included in /.appveyor.yml. AppVeyor decrypts this key and injects it into the CI job where the private key is decrypted and used for signing releases.

We were talking past each other. The missing link was that the pfx_password in /.appveyor.yml is encrypted with a public key associated with our AppVeyor account. This value is decrypted by AppVeyor and injected into the CI build at run time.