Skip to content

Conversation

@mislav
Copy link
Contributor

@mislav mislav commented Apr 23, 2021

This introduces a concept of a Git object that is asked for git operations (only Push for now) from the context of a command. Instead of stubbing out the git push command, the call to Push() itself is stubbed via mock object generated by mockery.

Additionally, this introduces tests that exercise the logic of shelling out to git push, which is now rewritten to ensure synchronous I/O. This retires the approach of RegexpWriter from #2016 in favor of StderrPipe processed by a bufio.Scanner.

Features:

  • Instead of invoking methods from the git package directly, this proposes a mechanism by which commands could receive an object that implements git operations. It is thus easier to pass a mock object implementing the git interface rather than stubbing out the raw git commands that the operations would shell out to. In the long run, this allows us to tweak the underlying git commands without having to update individual tests of every command that relies on that git operation.
  • Now ensures that git push progress output is intact and forwarded to stderr as it comes in, respecting CR-terminated lines, but still filtering out unwanted lines.
  • Sets up go generate-powered mocks with mockery (brew install mockery).

Followup ideas:

  • Switch more git operations in pr create over to the Git interface? git.Commits(), git.CommitBody(), git.ReadBranchConfig(), git.ShowRefs(), git.UncommittedChangeCount(), git.AddRemote()
  • Extract pkg/cmd/pr/create/git.go (and associated mocks) to somewhere it could get used by other commands as well? Potentially back to the git package.

Fixes #3650

Browse(string) error
}

//go:generate mockery --name gitClient --structname GitClient
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running go generate ./... in our project will now update the associated mocks after we edit an interface.

This introduces a concept of a Git object that is asked for git
operations (only Push for now) from the context of a command. Instead of
stubbing out the `git push` command, the call to `Push()` itself is
stubbed via mock object generated by mockery.

Additionally, this introduces tests that exercise the logic of shelling
out to `git push`, which is now rewritten to ensure synchronous I/O.
Copy link
Contributor

@vilmibm vilmibm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of capturing git operations via an interface; it makes sense to me and is consistent with how we're exposing other surfaces (i/o, browser opening, repo resolution) to commands.

I am extremely suspicious of mockery, however; in general I am an opponent of code generation as I have found it to cause more trouble than it saves in the long run. I also consider it a smell that one is using a language in an inappropriate way: if it's not comfortable for a human to write, then that suggests we're just doing something we ought not do (or using a language that should not be used, which is not the case with Go). I also take it as a given that any code in a project -- third party or otherwise -- will need to be debugged at some point and debugging autogenerated code is never pleasant.

The .On().Return() syntax is certainly reminiscent of testing strategies in other languages (ruby/js) which, to me, is a big red flag: we're not writing those languages. We're writing a language with very different goals and design motivations.

I can be convinced that this is truly the only way to reliably test things like a git interaction layer, but I'd prefer to see some alternative proposals first that don't rely on code generation.

@mislav
Copy link
Contributor Author

mislav commented Apr 27, 2021

The .On().Return() syntax is certainly reminiscent of testing strategies in other languages (ruby/js) which, to me, is a big red flag: we're not writing those languages.

Agreed; that syntax doesn't sit really well with me because it doesn't feel like Go at all. Maybe we could continue shopping to find a more suitable mocking library.

The golang/mock syntax looks even worse to me:

  ctrl := gomock.NewController(t)
  defer ctrl.Finish()

  m := NewMockFoo(ctrl)

  // Does not make any assertions. Executes the anonymous functions and returns
  // its result when Bar is invoked with 99.
  m.
    EXPECT().
    Bar(gomock.Eq(99)).
    DoAndReturn(func(_ int) int {
      time.Sleep(1*time.Second)
      return 101
    }).
    AnyTimes()

I'll try to look into some others:

am extremely suspicious of mockery, however; in general I am an opponent of code generation as I have found it to cause more trouble than it saves in the long run.

I have also never even considered code generation coming from dynamic languages (except for if I would have to build a parser), but I can see the appeal in a statically typed language. For this particular purpose, the generated code is never used at runtime, and since the compiler enforces type safety anyway, I feel more confident checking generated code into version control.

Of course, nothing is forcing us to use code generators. However, it can be quite tedious and possibly error-prone to write stubbed implementations by hand.

@mislav
Copy link
Contributor Author

mislav commented Apr 28, 2021

I've looked into other mocking libraries for Go that I could find by searching:

They all seems to be primarily designed for generating code, i.e. to be used through go generate. Otherwise, setting any of these up per every mocked interface could be exhausting.

Copy link
Contributor

@vilmibm vilmibm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for this research!

@mislav seems like code generation is what's standard here and I'm up for trying it given that. Your point about the difference in culture between dynamic/static languages makes sense to me too.

In reviewing all your research I do actually like mockery the most, I think. I can make a feeble argument for golang/mock as I tend to prefer "official" things but given that we're already using testify going with mockery seems fine.

Sorry to hold this up, but I feel way better about this PR given the conversation and additional context.

@vilmibm vilmibm marked this pull request as draft June 2, 2021 16:23
@vilmibm
Copy link
Contributor

vilmibm commented Jun 2, 2021

mislav to redo with one or two other libraries so we can compare and contrast

@mislav
Copy link
Contributor Author

mislav commented Jun 18, 2021

My thoughts now that we have experimented with multiple of these libraries:

  • golang/mock - pros: full-featured. cons: verbose usage as it needs a controller
  • testify + mockery - pros: we already use the library, full-featured. cons: every stub call goes through the same Do() method (making it tricky to trace calls using gopls features), tricky to match incoming arguments when the method accepts an interface
  • counterfeiter - pros: every stub uniquely named by the method is stubs (e.g. DoThingsReturn() stubs DoThings()). cons: doesn't panic when a non-stubbed method is called, manual call count verification
  • moq - pros: define stubs as funcs inline, super simple. cons: manual call count verification

I found myself gravitating to moq because of its simplicity of defining funcs, meaning that the library entirely avoids having to invent an API for matching incoming arguments or to define return values. The caller simply defines a func for every stubbed method. I think I would like us to explore this approach more, even if it means manually having to verify the call count for each method that we care that it's called only N times. I've pushed an exploration here: git-push-mocks...git-push-mocks-moq

@darcyclarke
Copy link

Closing as this was a spike/investigation.

@samcoe samcoe deleted the git-push-mocks branch June 28, 2023 04:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gh pr create remote response is garbage/jibberish

4 participants