feat: unstable SHA256 support#1206
Conversation
dd35da4 to
a332c64
Compare
a332c64 to
fd975c7
Compare
c07b123 to
5c04d59
Compare
|
I believe to have found something that you missed in this PR. Note that Lines 271 to 275 in e6919b7 I believe that this means that the The function Line 231 in e6919b7 will hit the Line 29 in e6919b7 This is because I discovered this experimenting in code that is downstream of Then I started to think about how much #[inline]
pub(crate) fn zeroed_raw_oid(format: ObjectFormat) -> raw::git_oid {
raw::git_oid {
kind: format.raw(),
id: [0; raw::GIT_OID_MAX_SIZE],
}
}In fact one might even ask why not to have impl Oid {
pub const ZERO_SHA1: Self = Self {
raw: raw::git_oid {
kind: raw::GIT_OID_SHA1,
id: [0; raw::GIT_OID_MAX_SIZE],
}
};
pub const ZERO_SHA256: Self = Self {
raw: raw::git_oid {
kind: raw::GIT_OID_SHA256,
id: [0; raw::GIT_OID_MAX_SIZE],
}
};
// NOTE: Maybe call this `zero_with_object_format` to not break `zero`.
pub fn zero(format: ObjectFormat) -> Self {
// TODO: Match on `format` and return `Oid::ZERO_SHA1` or `Oid::ZERO_SHA256`.
}
}This way, users that know at compile time which kind of zero they want can just use the constant, while those that decide at run time would call If you do not want For |
|
Another comment regarding |
| #[cfg(not(feature = "unstable-sha256"))] | ||
| { | ||
| if bytes.len() != raw::GIT_OID_SHA1_SIZE { | ||
| return Err(Error::from_str("raw byte array must be 20 bytes")); |
There was a problem hiding this comment.
maybe a clearer error for trying to accidentally use 32 bytes when the feature isn't enabled?
Also, why have different ways of creating the SHA1 oid depending on if the SHA256 feature is available?
let oid_type = match bytes.len() {
raw::GIT_OID_SHA1_SIZE => raw::GIT_OID_SHA1,
#[cfg(feature = "unstable-sha256")]
raw::GIT_OID_SHA256_SIZE => raw::GIT_OID_SHA256,
#[cfg(not(feature = "unstable-sha256"))]
raw::GIT_OID_SHA256_SIZE => {
return Err(Error::from_str(
"raw byte array of 32 bytes requires unstable-sha256 feature to be enabled",
))
}
_ => {
return Err(Error::from_str(
"raw byte array must be 20 bytes (SHA1) or 32 bytes (SHA256, if enabled)",
))
}
};
unsafe {
try_call!(raw::git_oid_from_raw(&mut raw, bytes.as_ptr(), oid_type));
}
Ok(Oid { raw })There was a problem hiding this comment.
Good suggestion, though I am not sure if here we should talk about unstable-sha256 feature, downstream consumer of git2-rs may just propagate error message to end-users, and that might be weird to see that.
There was a problem hiding this comment.
If the error message is propagated to end users, presumably the end users had some kind of say in the input value
| } | ||
| #[cfg(not(feature = "unstable-sha256"))] | ||
| { | ||
| let _ = oid_type; |
There was a problem hiding this comment.
hmm, even if the git2-rs bindings don't have SHA256 enabled, if the oid_type corresponds to SHA256 we should say that and probably return an error, rather than just claiming that the repository is formatted with SHA1?
There was a problem hiding this comment.
IIUC, without unstable-sha256, the underlying libgit2-sys can't return a SHA256 oid or even open a SHA256 repo.
There was a problem hiding this comment.
What if the underlying library was built with sha256 enabled, but the git2 crate wasn't?
There was a problem hiding this comment.
AFAIK none of the mainstream packagers distribute libgit2.so with SHA256 support. The reason being it was broken before libgit2 1.9.3 (released a week ago). Also when sha256 is enabled, libgit2 built a shared libray with -experimental suffix. Unless packager intentionally patch that build script to remove the suffix, it is pretty distinguishable. However, it is unlikely someone does that, since the two are ABI incompatible and doing that breaks other programs dynamically linking to it.
I also raised this concern in rust-lang/cargo#16939 (comment) but from the other angle – Cargo may need to build from source always as nobody ships libgit2-experimental.so.
This comment has been minimized.
This comment has been minimized.
This is still ambiguous for those not 41-64 char hex strings. I'd still consider deprecating it and eventually remove. |
9b1f12d to
5778238
Compare
There was a problem hiding this comment.
New revision is up.
The major difference is that we stop follow libgit2@main but 1.9.3. The other is API design, which is stated in the PR description:
Introduce
ObjectFormataccessors and*_extAPI variants unconditionally,
The actual SHA256 wiring are gated behind theunstable-sha256Cargo feature
in order to keep public API additive.The
*_extsuffix follows libgit2's convention for extended function variants.
At next major version, the original methods may absorb the format parameter.
The actual change will leave for future.
|
@DanielEScherzer most of your review comments are addressed. Thanks! |
This comment has been minimized.
This comment has been minimized.
| } | ||
| } | ||
|
|
||
| // Sanity-check libgit2's OID size constants to keep the docs and the |
There was a problem hiding this comment.
can we use a bit more inclusive language here?
| // Sanity-check libgit2's OID size constants to keep the docs and the | |
| // Double-check libgit2's OID size constants to keep the docs and the |
|
|
||
| #[test] | ||
| fn conversions_ext_sha1() { | ||
| use crate::ObjectFormat; |
There was a problem hiding this comment.
instead of adding these uses to some tests, but using crate::ObjectFormat::* in others (object_format_always_sha1), maybe just add it to the overall mod tests?
|
This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed. Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers. |
This is a preparation of SHA256 support. * Added `Oid::object_format()` * Added `Oid::raw_bytes()` * Changed `Oid::from_bytes()` to use `GIT_OID_SHA1_SIZE` * Added `Repository::object_format()` * Added `Remote::object_format()` * Added `RepositoryInitOptions::object_format()`
Add `*_ext` method variants that accept an `ObjectFormat` param. The short-hand methods now delegate to `_ext` variants with Sha1. * Added `Oid::from_str_ext` * Added `Oid::hash_object_ext` * Added `Oid::hash_file_ext` * Added `Diff::from_buffer_ext` * Added `Index::new_ext` * Added `Index::open_ext` * Added `Indexer::new_ext` * Added `Odb::new_ext`
This makes these call sites automatically handle the correct object format once SHA256 support is wired up.
Declare the `unstable-sha256` Cargo feature,
reflecting upstream libgit2's `GIT_EXPERIMENTAL_SHA256`.
This is an ABI-breaking experimental feature.
Implement the SHA256 support gated behind this feature:
Things gated:
* `ObjectFormat::Sha256` and non_exhaustive when `unstable-sha256` is off
* all FFI call sites for the overloaded libgit2 function signatures
* git_oid_fromstrn
* git_oid_fromraw,
* git_odb_hash
* git_odb_hashfile
* git_diff_from_buffer,
* git_index_new
* git_index_open
* git_indexer_new
* git_odb_new
* `Repository::object_format` is format-aware
* `Remote::object_format` is format-aware
* `Oid::{as_bytes,object_format,from_bytes}` are format-aware
`Oid::zero().object_format()` panics with `unstable-sha256` because the raw `kind` field was left as zero. The zeroed struct is generally use for output param. Not really meaning to switch a `zero_ext` variant. Replace it with const values for both kinds.
Construct `git_oid` with a struct literal instead of `mem::zeroed`. Under `unstable-sha256`, default `kind` to `GIT_OID_SHA1` so the value is a valid `git_oid_t` even before libgit2 overwrites it as an output buffer.
So that we have a more thorough test coverage
This adds an
unstable-sha256Cargo feature, as a follow-up of #1201.Also adds smoke tests for affected operations/types.
Part of #1090
Design
Introduce
ObjectFormataccessors and*_extAPI variants unconditionally,The actual SHA256 wiring are gated behind the
unstable-sha256Cargo featurein order to keep public API additive.
The
*_extsuffix follows libgit2's convention for extended function variants.At next major version, the original methods may absorb the format parameter.
The actual change will leave for future.
Insta-stable
ObjectFormatenum (#[non_exhaustive]when unstable-sha256 is not enabled)Oid::ZERO_SHA1constOid::object_format()to access underlying OID typeOid::raw_bytes()to return the full underlying byte bufferRepository::object_format()to get hash algo on a repoRemote::object_format()to get hash algo on a remoteRepositoryInitOptions::object_format()setterOid::from_str_ext/Oid::hash_object_ext/Oid::hash_file_extDiff::from_buffer_extIndex::new_ext/Index::open_extIndexer::new_extOdb::new_extOid::zero()in favor ofOid::ZERO_SHA1Behind
unstable-sha256ObjectFormat::Sha256enum variantOid::ZERO_SHA256const_extmethods dispatch to SHA256-aware FFI pathsOid::as_bytesreturns logical bytes (SHA1 -> 20; SHA256 -> 32)Oid::from_bytesaccepts 20 or 32 bytes