Skip to content

Conversation

@piyueh
Copy link
Collaborator

@piyueh piyueh commented Aug 4, 2021

This PR:

  • in lesson 5:
    • runs the optimization with a fixed number of iterations, record the loss history, and use the parameters at the lowest validation loss as the solution.
    • uses $L$ instead of $\mathrm{loss}$
    • uses the means instead of the sums for both the loss and regularization term
    • decouples the calculations of the raw loss and the regularization
    • monitors the raw loss instead of the regularization loss
  • adds lesson 7
  • adds an appendix to explain terms

To see a cell by cell diff of lesson 5: Screenshot 2021-08-10 at 14-06-38 nbdime - diff and merge your Jupyter notebooks

piyueh added 4 commits August 3, 2021 22:56
- make lesson 5 and lesson 7 more comparabnle regarding the optimization
- use n1, n2, ... to replace nz1, nz2, ...
- monitor non-regularized loss
- change the loss to mean, instead of summation
- also use mean for the regularization term
- change random seed
@labarba labarba changed the base branch from master to develop August 6, 2021 16:28
@piyueh
Copy link
Collaborator Author

piyueh commented Aug 6, 2021

check the regularized and the original loss after the training.
Done. Turns out I was using different datasets for the raw and regularized losses.

@piyueh piyueh changed the title add lesson 7 and scripts update lesson 5 and add lesson 7 and appendix Aug 10, 2021
@piyueh piyueh mentioned this pull request Aug 24, 2021
@piyueh piyueh requested a review from labarba August 24, 2021 20:03
@labarba labarba merged commit dfdbf51 into engineersCode:develop Aug 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants