Skip to content

Add test for gh pr list#14

Merged
probablycorey merged 15 commits intomasterfrom
test
Oct 16, 2019
Merged

Add test for gh pr list#14
probablycorey merged 15 commits intomasterfrom
test

Conversation

@probablycorey
Copy link
Copy Markdown
Contributor

I had two goals for this PR:

  1. Write non-brittle test for gh pr list that helps us catch errors we accidentally introduce.
  2. Keep the individual test small and readable.

I had a lot of false starts with this code. Here is some Slack history of some ideas we tried). I think this is PR leaves us in a stat that is good enough to move forward. Things will need to be looked at again when a command issues multiple GraphQL queries, and it doesn't handle dealing with git repos, but for now it's ok ¯_(ツ)_/¯.

I'm going to leave some inline notes about things I'm still unsure about.

@probablycorey probablycorey self-assigned this Oct 11, 2019
api/queries.go Outdated
HeadRefName string
}

var OverriddenQueryFunction func(query string, variables map[string]string, v interface{}) error
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is moderately offensive. We need a way to override the call to the network in our code. Since you can't stub an exported function on a package I use the OverriddenQueryFunction variable to deal with this. Maybe there is a better way?

Copy link
Copy Markdown
Contributor

@vilmibm vilmibm Oct 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't stub an exported function on a package

i thought you could do exactly this? or do you mean the unexported graphQL function?

for that, I remember the book suggesting something along the lines of the answer in this SO post: https://stackoverflow.com/questions/24622388/how-to-test-a-unexported-private-function-in-go-golang

)

func TestPRList(t *testing.T) {
teardown := mockGraphQLResponse("fixtures/pr.json")
Copy link
Copy Markdown
Contributor Author

@probablycorey probablycorey Oct 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It kind of makes sense that the test and fixtures are in the command package. But I could also see this being in a top level test directory since it is a pretty high level integration test.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore the comment above! I moved the common test code into a test package. That is where test/helper and test/fixtures live now.

return output, nil
}

func captureOutput(f func()) string {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we use fmt.Printf to print to stdout, I had to capture that. An alternative would be to use fmt.Fprintf and override the file writer in our tests. I didn't go with this because this captureOutput method is pretty simple and gives us one less thing to worry about when we are writing production code.

@probablycorey probablycorey changed the base branch from graphql to master October 14, 2019 16:28
@probablycorey
Copy link
Copy Markdown
Contributor Author

This isn't ready for review yet. Locally the tests passed because they were running against my local git repo, but our test env doesn't have a local repo. My plan is to see if I can use Nate's context code to mock out the stuff I need instead of needed to make an entire temp git repo.

@probablycorey
Copy link
Copy Markdown
Contributor Author

This is ready for review now. I went ahead and finished up making this work with temporary git repos, it's not so bad.

@vilmibm vilmibm self-requested a review October 15, 2019 16:20
Copy link
Copy Markdown
Contributor

@vilmibm vilmibm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i like the direction of this but am curious to see the test package mocking approach incorporated (assuming it actually is the way to address the mocking problem)

@probablycorey
Copy link
Copy Markdown
Contributor Author

@vilmibm I switched up the mocking pattern to make api.GraphQL a package global. Now we can replace the var in tests. It seems like the standard way golang apps do mocking.

Copy link
Copy Markdown
Contributor

@vilmibm vilmibm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this is a good start, thank you!

fmt.Printf("%+v\n", resp)
*/
func graphQL(query string, variables map[string]string, data interface{}) error {
var GraphQL = func(query string, variables map[string]string, data interface{}) error {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i like this approach more 👍 given that a lot of different tests in presumably different packages will want to mock out graphql responses specifically, just having it be exported makes sense to me.

)

func TestPRList(t *testing.T) {
teardown := test.MockGraphQLResponse("pr.json")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is what i had in mind 👍

Copy link
Copy Markdown
Contributor

@mislav mislav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: accidentally checked in command.test?

This looks great! Some non-blocking comments/thoughts:

{
"node": {
"number": 8,
"title": "Strawberries are not actually berries",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙀

}

func UseTempGitRepo() *TempGitRepo {
github.CreateTestConfigs("mario", "i-love-peach")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💕

func RunCommand(root *cobra.Command, s string) (string, error) {
var err error
output := captureOutput(func() {
root.SetArgs(strings.Split(s, " "))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are testing the Cobra dispatcher this way by going through the root command, but I believe to test individual commands, we should pass that command directly.

I don't think we need to go through the e.g. pr list dispatcher since we already trust that Cobra works.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to do it this way (testing the prCmd instead of root) but ran into problems. You can't set args on a subcommand like this cmd.SetArgs(strings.Split(s, " ")). I dug into the cobra source and I think it is because when you execute a subcommand, it actually runs the root command https://github.com/spf13/cobra/blob/master/command.go#L873-L876. So I gave up and just used the root command.

One positive aspect of testing the root command is we ensure that the subcommand was added to the root command ¯_(ツ)_/¯

regexp.MustCompile(`#8.*\[strawberries\]`),
regexp.MustCompile(`#9.*\[apples\]`),
regexp.MustCompile(`#10.*\[blueberries\]`),
regexp.MustCompile(`#11.*\[figs\]`),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering where this data comes, and then later I saw the pr.json fixture. Do you think there might be benefit to inlining the mock JSON payload in the tests themselves instead of keeping them in a separate file? That way, somebody who reads or authors a test can see the API payload and the expected output in the same file.

Copy link
Copy Markdown
Contributor Author

@probablycorey probablycorey Oct 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory I like your idea! I'll put it inline and see if it's helpful or annoying.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I still like the core of your idea, but the problem I ran into was I lost the json syntax coloring when I made it a string. I really want this because I use syntax coloring as a linter when I edit json. If you have a way around this I'd be 😃!

So instead I made the MockGraphQLResponse function take the entire fixture path. At least this will make it more clear where the data is coming from.

fixturePath := filepath.Join(pwd, "..", "test", "fixtures", fixtureName)

originalGraphQL := api.GraphQL
api.GraphQL = func(query string, variables map[string]string, v interface{}) error {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something I was thinking about re: mocking HTTP responses:

What if, instead of replacing the whole GraphQL function, we make it so we can inject a custom RoundTripper into our HTTP client? https://golang.org/pkg/net/http/#RoundTripper

Then, the tests could set up that roundtripper to return pre-baked responses from memory (or from file) rather than activating the actual HTTP request. Just an idea! If we wanted to try that approach, it might make more sense to do it in a followup PR. 👍

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's interesting, this RoundTripper thing might work well. Are you thinking this is beneficial because we can test our response to HTTP status codes instead of just assuming we always get 200's back?

@probablycorey probablycorey merged commit 18d9467 into master Oct 16, 2019
@probablycorey probablycorey deleted the test branch October 16, 2019 18:28
vilmibm pushed a commit that referenced this pull request Jun 29, 2021
added combination of args, partial fix to args
mislav pushed a commit that referenced this pull request Sep 28, 2021
add workaround for ssh.channel.Close EOF
Stonre pushed a commit to Stonre/strands-fork that referenced this pull request Jul 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants