Problem
In rust-lang/cargo we have a test for ensuring Cargo work well with a corrupted Git database. It used to be that libgit2 did both the fetch and the database reading. We recently are experimenting with Git CLI doing the fetch part inead. However, libgit2 got into infinite loop when
- Git CLI fetched loose objects
- the test deliberately truncated/corrupted those objects
- When libgit2 read from them, it got stuc
The observation is that when inflate() returned Z_BUF_ERROR, it was treated as non-fatal 1. libgit2 would continue trying, though
Z_STREAM_END was never reached.git_zstream_get_output_chunk would return 0 with no progress madegit_zstream_get_output continues the loop.
Reproduction
Here is a repro (sorry, LLM assisted) that would get you into infinite loop.
diff --git a/tests/libgit2/odb/loose.c b/tests/libgit2/odb/loose.c
index 7189a689e..f0933dbcb 100644
--- a/tests/libgit2/odb/loose.c
+++ b/tests/libgit2/odb/loose.c
@@ -23,6 +23,21 @@ static void write_object_files(object_data *d)
p_close(fd);
}
+static void write_truncated_object_file(object_data *d, size_t len)
+{
+ int fd;
+
+ cl_assert(len < d->blen);
+
+ if (p_mkdir(d->dir, GIT_OBJECT_DIR_MODE) < 0)
+ cl_assert(errno == EEXIST);
+
+ cl_assert((fd = p_creat(d->file, S_IREAD | S_IWRITE)) >= 0);
+ cl_must_pass(p_write(fd, d->bytes, len));
+
+ p_close(fd);
+}
+
static void cmp_objects(git_rawobj *o, object_data *d)
{
cl_assert(o->type == git_object_string2type(d->type));
@@ -240,6 +255,36 @@ void test_odb_loose__streaming_reads_sha256(void)
}
}
+static void test_read_truncated_object(object_data *data, size_t len)
+{
+ git_oid id;
+ git_odb_object *obj;
+ git_odb *odb;
+ git_odb_options opts = GIT_ODB_OPTIONS_INIT;
+
+ opts.oid_type = data->id_type;
+
+ write_truncated_object_file(data, len);
+
+ cl_git_pass(git_odb_open_ext(&odb, "test-objects", &opts));
+ cl_git_pass(git_oid_from_string(&id, data->id, data->id_type));
+ cl_git_fail(git_odb_read(&obj, odb, &id));
+
+ git_odb_free(odb);
+}
+
+void test_odb_loose__read_truncated_zlib_header_fails(void)
+{
+ /* only the first two bytes of the zlib stream are left on disk */
+ test_read_truncated_object(&commit, 2);
+}
+
+void test_odb_loose__read_truncated_object_body_fails(void)
+{
+ /* the stream is cut off in the middle of the object body */
+ test_read_truncated_object(&commit, commit.blen / 2);
+}
+
void test_odb_loose__read_header_sha1(void)
{
test_read_header(&commit);Solution
I have a LLM-assisted patch but not going to share that here, as I feel no confidence with it.
Problem
In rust-lang/cargo we have a test for ensuring Cargo work well with a corrupted Git database. It used to be that libgit2 did both the fetch and the database reading. We recently are experimenting with Git CLI doing the fetch part inead. However, libgit2 got into infinite loop when
The observation is that when
inflate()returnedZ_BUF_ERROR, it was treated as non-fatal 1. libgit2 would continue trying, thoughZ_STREAM_ENDwas never reached.git_zstream_get_output_chunkwould return 0 with no progress madegit_zstream_get_outputcontinues the loop.Reproduction
Here is a repro (sorry, LLM assisted) that would get you into infinite loop.
Solution
I have a LLM-assisted patch but not going to share that here, as I feel no confidence with it.
Footnotes
Which is documented in the stock Zlib code, though not sure if it is desired here:
https://github.com/libgit2/libgit2/blob/95ff8c2baceb270469657663ded9934d5faf5627/deps/zlib/zlib.h#L511-L514 ↩