Skip to content

Transfer progress notifications are prohibitively slow #919

Description

@bpeabody

Having even an empty transfer progress function for fetches adds a large amount of overhead. In the code below, for example, on the nodegit repo itself, specifying a transfer progress function adds about 38s for me.

The progress function is called once for each object fetched from the repository: about 25,000 times for the nodegit repo as of today. That would indicate an overhead of about 1.5ms for calling from C++ into JavaScript (running on a 3.1ghz i7). This amount of overhead feels a little high; I wonder if there might be some room for optimizing the JS/C++ binding/bridge code.

Even if significantly optimized (e.g., by a factor of 10), the overhead of calling from C++ into JS is probably high enough to be prohibitive; it might be good to be able to provide some conflation so that the progress function is called no more than a specified frequency (e.g., no more than once every 100ms).

If no one has an easy answer for this, I would be more than happy to lend a hand (I have a lot of experience with v8).

Sample code:

const NodeGit = require("nodegit");

console.time("no-progress");

NodeGit.Clone("https://github.com/nodegit/nodegit.git", "no-progress")
.then(function () {
    console.timeEnd("no-progress");
    console.time("progress");
    var options = new NodeGit.CloneOptions();
    var fetchOpts = new NodeGit.FetchOptions();
    options.fetchOpts = fetchOpts;
    var callbacks = new NodeGit.RemoteCallbacks();
    fetchOpts.callbacks = callbacks;
    callbacks.transferProgress = function () {
    };
    return NodeGit.Clone("https://github.com/nodegit/nodegit.git",
                         "progress",
                         options);
}).done(function () {
    console.timeEnd("progress");
    process.exit(0);
});

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions