Skip to content

Consider using black as an autoformatter #18796

Closed

Description

@ianhi

Problem

I find flake8 errors to be confusing to fix at times. For example, I've expended more mental energy than I care to admit trying to figure out how to turn an 81 character line into two lines. This feels like it makes more difficult to contribute to matplotlib, especially for someone who hasn't gone to the trouble of setting up an ide to point out flake8 errors. More difficult both through the extra mental effort and the extra back and forth about formatting on PRs.

Proposed Solution

Autoformat all code using black. This would remove the mental burden of formatting by having contributors only need to run:

black .

Additional context and prior art

  1. Plunging the depths for flake8 commits: (those that never got squashed)

git log --all | grep "flake" -c -i
169

git log --all | grep "fix.*format" -c -i
259

  1. If done in a single commit then the commit can be saved in git-blame-ignore-revs to avoid messing up git blame (though I suppose it may be messed up on github :/ )
    https://black.readthedocs.io/en/stable/installation_and_usage.html#migrating-your-code-style-without-ruining-git-blame

  2. We did this on jupyterlab-git (Format code using black jupyterlab/jupyterlab-git#763) and it has sparked me trying to use this everywhere I can

Activity

  1. QuLogic commented on Oct 23, 2020

    @QuLogic
    Member

    Personally, I am not too happy with black's results, and I think there are others.

  2. anntzer commented on Oct 23, 2020

    @anntzer
    Contributor

    Please no. Formatting has semantic value too.

  3. jklymak commented on Oct 24, 2020

    @jklymak
    Member

    I guess if there is a groundswell of senior devs who think this is a good idea, we can reconsider, but there are three votes against..

    In terms of flake8 it can be run locally easily enough, though I admit to abusing CI and allowing it to run there quite often.

    I'll close this for now, but feel free to continue discussion or request a re-open if you are passionate about this.

  4. ianhi commented on Oct 24, 2020

    @ianhi
    ContributorAuthor

    I guess if there is a groundswell of senior devs who think this is a good idea, we can reconsider, but there are three votes against..

    I'll close this for now, but feel free to continue discussion or request a re-open if you are passionate about this.

    There was instead an immediate groundswell against so that's a pretty clear result 🙃. I'm happy to leave this closed, though I do have two closing thoughts/points:

    1. For senior mpl devs there's not a real gain an maybe even a small annoyance introduced by this. Which would be the price to pay for making it easier for new contributors. Though how much easier is likely to be extremely high-variance.
    2. https://github.com/django/deps/blob/master/accepted/0008-black.rst is a much more coherent and extensive version of what I wrote in this issue.
  5. tacaswell commented on Oct 24, 2020

    @tacaswell
    Member

    I am weakly 👍 for auto-applying black to new code bases, but very 👎 on applying it en-mass to existing code bases.

  6. jklymak commented on Oct 24, 2020

    @jklymak
    Member

    I'd be more favourable if we were spinning endless cycles arguing about code formatting, but it's usually pretty straightforward.

    None of this stops anyone from using black themselves before they submit a PR.

  7. timhoffm commented on Oct 25, 2020

    @timhoffm
    Member

    I'm on the edge of saying that it's unreasonable not to use a code formatter. And it seems that black is the only available formatter, that is really consistent.

    Black's format is mostly reasonable. But I really dislike putting every parameter on a single line for longer function signatures (and calls). That might start to be necessary when one uses type annotations. But we're not there and have quite a few functions with longer signatures. If only black wasn't that dogmatic and would allow to use a subset of it's rules (I'm not asking for different rules). There's --skip-numeric-underscore-normalization but I'm skeptical that they will go down that road further.

    So for now slightly 👎 on black.

  8. ianhi commented on Oct 26, 2020

    @ianhi
    ContributorAuthor

    but very -1 on applying it en-mass to existing code bases.

    There's also the opportunity to only apply autoformating to new diffs e.g. https://github.com/akaihola/darker and https://github.com/wbolster/black-macchiato

    I'm on the edge of saying that it's unreasonable not to use a code formatter. And it seems that black is the only available formatter, that is really consistent.

    I put black in the title because it seems to be the most popular, but the underlying title really was consider using an autoformatter

    How do you mean by consistent? Would coming up with a matploltib configuration for something like yapf be out? I like black because it required the minimal amount of thought on my part, but what I really want is not have to think much about formatting. So if I imagine myself as a new contributor then being handed a config for a formatter (no matter which) would be great, just to take the guesswork out.

    I'd be more favourable if we were spinning endless cycles arguing about code formatting, but it's usually pretty straightforward.

    My take is that there may not be arguments/discussion but there is still unnecessary mental energy being devoted by whoever submits a PR to figure out how to appease flake8.

    None of this stops anyone from using black themselves before they submit a PR.

    It's kinda tricky because you need to excise certain lines, I didn't even know about the above linked darker/black-machiatto until this issue. So I've been doing the shameful thing of copying sections into jupyterlab, running jupyterlab-code-formatter on it, and pasting it back in 😬. But regardless that all requires effort plus if I use black and a functions argument into a column that would conflict with #18796 (comment) so it's still a bit mysterious to me how to format well/consistently with the surrounding code.

  9. timhoffm commented on Oct 26, 2020

    @timhoffm
    Member

    Consistent: I was referring an old discussion, which states that yapf somtimes changes the format on a second run.

    I would not rule out yapf, but haven't used it, so can't really comment on it. I've read somewhere that it's significantly slower than black, which might be something to check since we have a sizable amount of code.

  10. timhoffm commented on Oct 26, 2020

    @timhoffm
    Member

    Given the rising popularity of black, we should discuss code formatters. Otherwise black-style will silently sneak into our code base via PRs anyway (cf. parameter wrapping in #18821). I'd rather have a conscious decision for or against it.

  11. jklymak commented on Oct 26, 2020

    @jklymak
    Member

    But I guess they are somewhat orthogonal: do we want an auto formatter to run on all code, and what should our format style be, beyond what is structured by PEP8?

  12. timhoffm commented on Oct 27, 2020

    @timhoffm
    Member

    Strictly speaking, they are orthogonal. However, if we decide on a certain style, we'll have to enforce it. And I don't want to make lots of style comments in reviews, in particular if these are project specific style choices.

  13. jklymak commented on Oct 27, 2020

    @jklymak
    Member

    My personal opinion is that if the code passes flake8 we should be relatively lenient plus or minus a reviewer complaining because something really seems terrible to them. Which is basically the status quo. I think there are lots of author-side tools to help ensure flake8 compatibility.

  14. tacaswell commented on Oct 27, 2020

    @tacaswell
    Member

    Otherwise black-style will silently sneak into our code base via PRs anyway

    Black style technically conforms to our style guide and as much as I am against mass-applying black, I am also against tweaking our rules to forbid the formatting black produces...

  15. anntzer commented on Oct 27, 2020

    @anntzer
    Contributor

    To take some concrete examples, it seems clear to me that

            if self._A.dtype != np.uint8 and not np.can_cast(
                self._A.dtype, float, "same_kind"
            ):
                raise TypeError(
                    "Image data of dtype {} cannot be converted to "
                    "float".format(self._A.dtype)
                )
            # <elided>
            if not (
                self._A.ndim == 2
                or self._A.ndim == 3
                and self._A.shape[-1] in [3, 4]
            ):
                raise TypeError(
                    "Invalid shape {} for image data".format(self._A.shape)
                )

    is less readable than

            if (self._A.dtype != np.uint8 and
                    not np.can_cast(self._A.dtype, float, "same_kind")):
                raise TypeError("Image data of dtype {} cannot be converted to "
                                "float".format(self._A.dtype))
    	    # <elided>
            if not (self._A.ndim == 2
                    or self._A.ndim == 3 and self._A.shape[-1] in [3, 4]):
                raise TypeError("Invalid shape {} for image data"
                                .format(self._A.shape))

    (exerpt from image.py, blackened with --line-length=79 (but similar problems occur with the default line-length of 88):

    • black fails to understand that the whole can_cast call should fit in a single line.
    • black fails to understand the implicit grouping of the (A or (B and C)) condition.

    Similarly, in cbook/__init__.py, black wants to transform

        conflicting = (get_aliased_and_aliases(preexisting_aliases)
                       & get_aliased_and_aliases(alias_d))

    into

        conflicting = get_aliased_and_aliases(
            preexisting_aliases
        ) & get_aliased_and_aliases(alias_d)

    which is just nonsensical.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions