Ruleset updating mechanism to fetch, verify the authenticity of, and apply new ruleset databases#437
Ruleset updating mechanism to fetch, verify the authenticity of, and apply new ruleset databases#437chaoticsmol wants to merge 164 commits intoEFForg:masterfrom chaoticsmol:master
Conversation
…d json manifest for ruleset update information
…re field will include the hash of the db file
…eases appropriate to the extension are downloaded
…ormat version in one place before updating
…tInterval to periodically check for and apply updates
…tility development and a sample update.json file
…ow commenting and pull requests
There was a problem hiding this comment.
I think you need to use try_request here too.
There was a problem hiding this comment.
I decided not to use try_request itself here because this code uses a different structure (relying on xhr.response rather than xhr.responseText and xhr.onload rather than xhr.onreadystatechange) aside from setting xhr.responseType.
I'll go ahead and wrap this function in a recursive loop like I did with try_request to have the same effect.
There was a problem hiding this comment.
Yeah, otherwise if the ruleset db file request fails for some reason, it won't get retried.
|
Ignoring the update specification, I'm done reviewing. This looks good but should probably have some unit tests before merge. BTW, github is showing a bunch of my comments as outdated because of commit b27a090 but they apply just as well to the newly-renamed RulesetUpdater.js file. :) |
|
@diracdeltas I have made most or all of the changes you've suggested, now. You'll certainly want to review the new commits to make sure they are what you had in mind. I also merged in my features/tests branch to include the unit tests I had written a while ago. |
|
What can I do to get this merged? |
|
^ What he said |
|
I'd still be willing to put some time into making this fit back into the most recent versions of the extension if I can get a little instruction and a guarantee that the work won't go to waste. |
|
@pde @jsha @diracdeltas et al I know this is wanted functionality, what's needed to be able to get this moving (bar the obvious rebasing and any required updates to begin with), with the view to getting it actually merged soon? |
|
Sorry for dragging my feet on this for so long. I really appreciate all the work you've put into this branch, @redwire, and I don't want to see it go to waste. But, I think with all the recent changes we've made to make releases easier and do them on a faster cadence, the problem of shipping ruleset updates out of band is no longer as urgent. Ruleset updates require the same level of care in signing with an offline key as full releases, so we any time we would do a ruleset release we can just as well do a full extension release. Since we've consolidated the dev and stable release trains into a single stable release chain and added automated ruleset tests, we've been able to do a much faster release cadence, every couple of weeks or so. This has also meant that triaging ruleset bug reports has become much easier, with everyone typically on the same version. I think doing ruleset updates separately would introduce another variable that makes bug triage and managing releases harder again. There are two issues that still remain: (1) Sometimes people will go for weeks without restarting their browser, so they may not pick up the latest version of the extension right away. I think the best approach to fixing that is #1123, supporting restartless loading of the extension, which improves both the first-time install and subsequent upgrades. (2) Writing and testing user rules is time consuming because of the need to restart the browser each time: #1293. I expect to pull out some of the ruleset reloading code from this branch to use for that fix. I've archived a copy for future use on the EFF repo, on branch 'ruleset-auto-update'. I always hate to see code written and not merged, so I'm bummed to make this call, but I think it's the right one for HTTPS Everywhere right now. I'll leave this pull request open for another week or so for discussion. And again, thanks so much for the time and effort you put into making this branch, @redwire. Thanks, |
|
Closing after over a month!:/ |
I suggest that you should have 2 separate keys. The reasons for it are |
|
Duplicate of currently done work in |
5 participants
This pull request contains all of the commits I (Zack Mullaly) have made over the course of the Summer of 2014 to build a secure ruleset updating mechanism for my Google Summer of Code 2014 project.
There are still some details (such as a public key) that need to be filled in by whoever now is the primary maintainer of the project, but the feature should be functional once the TODOs are taken care of.