-
Notifications
You must be signed in to change notification settings - Fork 11
prefer operator fncs to lambda fncs #57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
shopify_python/google_styleguide.py
Outdated
| # if shopify_python.ast.count_tree_size(node) == 1: | ||
| # print node.name | ||
| if shopify_python.ast.count_tree_size(node) == 3: | ||
| operator = OperatorUtils().get_bin_op(node.op) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You could instead write this as:
BINARY_OPERATORS = {
...
}
...
operator = BINARY_OPERATORS.get(node.op)
if operator:
...
And save yourself the getter functions and make those dicts constants.
cfournie
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.
shopify_python/google_styleguide.py
Outdated
| "-": "neg", | ||
| "+": "pos" | ||
| } | ||
| operator = unary_operators.get(node.body.op) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might as well check that this isn't None as well, e.g. if operator:
shopify_python/google_styleguide.py
Outdated
| left = node.left.name | ||
| right = node.right.name | ||
| lamfun = "lambda " + left + ', ' + right + ": " + " ".join([left, node.op, right]) | ||
| opfun = "Operator." + operator |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Operator should probably be lowercase since it's a module name.
| def __lambda_func(self, node): # type: (astroid.Lambda) -> None | ||
| """Prefer Operator Function to Lambda Functions""" | ||
|
|
||
| if isinstance(node.body, astroid.UnaryOp): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You probably want to make sure that the unary operation's operand is just a variable with the same id as the lambda's operand (e.g. you shouldn't recommend replacing lambda a: not (a + 3)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will ensuring that the unary operation's operand is not an instance of BinOp suffice?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably not, check its type and make sure that it's the lambda variable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you will false-fail on f = lambda x: x * 3.
Please add a test to ensure that you don't.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@erikwright just wondering, since we're preferring operator.mul(x, y) to f = lambda x, y: x * y, shouldn't we prefer operator.mul(x, 3) to f = lambda x: x * 3 ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zeedann Here is the guideline you are implementing:
use the functions from the operator module instead of lambda functions
Here's the case we are discussing:
f = lambda x: x * 3What is the example code that you are proposing?
f = lambda x: operator.mul(x, 3)Why is this better? It is not replacing a lambda with anything. It's replacing a bare operator with a function call.
Here is an alternative that replaces the lambda:
f = functools.partial(operator.mul, 3)But that's a bad idea :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, I had a misinterpretation of what operator.mul actually did. Thanks for clarifying!
shopify_python/google_styleguide.py
Outdated
| opfun = "Operator." + operator | ||
| self.add_message('lambda-func', node=node, args={'op': opfun, 'lamfun': lamfun}) | ||
| return | ||
| elif shopify_python.ast.count_tree_size(node) > 3: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We probably don't want to be recommending partial replacement of lambdas, e.g. lambda a, b: a * b + c shouldn't be replaced with lambda a, b: operator.mul(a, b + c)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should i be recommending lambda a, b, c; operator.mul(a, operator.add(b, c)) instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For common operations like multiplication, use the functions from the operator module instead of lambda functions. For example, prefer operator.mul to lambda x, y: x * y.
lambda a, b: a * b is definitely harder to read than operator.mul when used in reduce, e.g.
reduce(lambda a, b: a * b, integers)
reduce(operator.mul, integers)
But for larger expressions, I'd argue that lambda a, b, c: a * b + c is easier to read than lambda a, b, c: operator.mul(a, operator.add(b, c)), but it's debatable e.g.
c = 3
reduce(lambda a, b: a * b + c, integers)
reduce(lambda a, b: operator.mul(a, operator.add(b, c)), integers)
@erikwright @honkfestival @JasonMWhite thoughts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree. operator.sum over lambda a, b: a + b, but lambda a, b: (a * b) + b over operator.sum(operator.mul(a, b), b)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good point guys, i'll recommend replacements for when lambda is used only on 2 variables with one operator
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The point of using operator is to not need the lambda. Not to never see a primitive operator.
|
UPDATE: changes I've made from the first iteration
|
| with self.assertNoMessages(): | ||
| self.walk(root_not_msg) | ||
|
|
||
| def test_lambda_func(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you also test a unary operator?
Also add a test that you don't reject lambda a, b: a * b * c
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please split this test into a bunch of smaller tests, one for each case you are covering.
| unary_root = astroid.builder.parse(""" | ||
| def unaryfnc(): | ||
| unary_pass = lambda x: not (x+3) | ||
| unary_msg = lambda x: -x |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As a more realistic example, can you re-write these as some_var = map(operator.not_, iterable)
| def binaryfnc(): | ||
| binary_pass = lambda x, y, z: x * y + z | ||
| binary_pass2 = lambda x: x + 3 | ||
| binary_msg = lambda x, y: x * y |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As a more realistic example, can you re-write these as some_var = reduce(operator.mul, iterable)
|
@erikwright @JasonMWhite any final comments before I merge this? :D |
| opfun = "operator." + operator | ||
| self.add_message('lambda-func', node=node, args={'op': opfun, 'lamfun': lamfun}) | ||
| elif isinstance(node.body, astroid.BinOp): | ||
| if shopify_python.ast.count_tree_size(node.body) == 3 and len(node.args.args) == 2: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Couldn't/shouldn't you put an equivalent check in the unary op case?
| } | ||
| operator = unary_operators.get(node.body.op) | ||
| argname = node.args.args[0].name | ||
| if operator and not isinstance(node.body.operand, astroid.BinOp) and argname is node.body.operand.name: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The check for a BinOp operand seems fragile to me. Is that the only possible operand, besides a variable reference? Couldn't it be another unary operation? Or a function call?
IIUC the node type you actually expect is astroid.Name. Maybe look for that explicitly.
shopify_python/google_styleguide.py
Outdated
| "not": "not_", | ||
| "+": "pos" | ||
| } | ||
| operator = unary_operators.get(node.body.op) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please rename this to preferred_operator_function.
| def binaryfnc(): | ||
| binary_pass = reduce(lambda x, y, z: x * y + z, [1, 2, 3]) | ||
| binary_pass2 = map(lambda x: x + 3, [1, 2, 3, 4]) | ||
| binary_msg = reduce(lambda x, y: x * y, [1, 2, 3, 4]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add another test for:
binary_pass3 = reduce(lambda x, y: x + 1, [1, 2, 3])There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also for:
def binaryfnc():
SOMETHING = 2
...
binary_pass4 = reduce(lambda x, y: x * SOMETHING, [1,2,3,4])There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add a test for:
binary_msg = reduce(lambda y, x: x * y, [1, 2, 3, 4])| } | ||
| operator = binary_operators.get(node.op) | ||
| if operator: | ||
| left = str(node.left.value) if node.left.name == 'int' else node.left.name |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think if you verified that the operands were of the right node type you wouldn't have to worry about this case of literal operands, which I think you handle incorrectly anyway (see my requested test).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for catching that, i didn't handle it correctly. I've added a check to the if statement to check a match between the left & right arguments, and the left & right operands respectively
|
approval to merge? :D @erikwright @JasonMWhite |
shopify_python/google_styleguide.py
Outdated
|
|
||
| if isinstance(node.body, astroid.UnaryOp): | ||
| if shopify_python.ast.count_tree_size(node.body) == 2 and len(node.args.args) == 1: | ||
| unary_operators = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of creating this variable on every node visitation, it would be better to create this as a class constant, since you aren't modifying it anywhere that I can see.
shopify_python/google_styleguide.py
Outdated
| left_arg = node.args.args[0].name | ||
| right_arg = node.args.args[1].name | ||
| node = node.body | ||
| binary_operators = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here with the constants.
shopify_python/google_styleguide.py
Outdated
| if preferred_operator_function and left_arg is node.left.name and right_arg is node.right.name: | ||
| left = str(node.left.value) if node.left.name == 'int' else node.left.name | ||
| right = str(node.right.value) if node.right.name == 'int' else node.right.name | ||
| lamfun = "lambda " + left + ', ' + right + ": " + " ".join([left, node.op, right]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor point, but lamfun doesn't seem very Pythonic as a name. lambda_fun? lamba_fx?
| binary_msg = reduce(lambda x, y: x * y, [1, 2, 3, 4]) | ||
| """) | ||
|
|
||
| # Binary Op Test 1: This will not trigger a message |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO, these should be 4 separate tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alternatively, it could be one test, but you pass the whole thing in at once and assert that there is only one failure. Could be made a bit more clear by naming it binary_fail
| with self.assertNoMessages(): | ||
| self.walk(binary_pass2) | ||
|
|
||
| # Binary Op Test 3: This will trigger a message |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are 2 Binary Op Test 3s
| 'op': 'operator.neg', | ||
| 'lamfun': 'lambda x: - x' | ||
| }) | ||
| with self.assertAddsMessages(unary_message): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
However you decide to address the test comments for test_binary_lambda_func, do the same for test_unary_lambda_func
75e1dd1 to
6445c02
Compare
shopify_python/google_styleguide.py
Outdated
| "+": "pos" | ||
| } | ||
|
|
||
| binary_operators = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe make these allcaps? (since they're constants)
|
I'm failing some tests that are unrelated to my PR. |
|
Can you reproduce these errors locally? Try clearing out your venv, and reinstalling, maybe a dependency got bumped? |
|
can't reproduce these errors locally, everything looks fine locally |
|
Probably not your fault, I just re-ran master and it failed, looks like they removed that function at some point from pylint (by no longer inheriting from the unit test class). We can probably downgrade pylint temporarily to fix. |
|
Green has never looked so good 💚 Thanks @cfournie for the pylint update, UPDATES:
|
|
Any last comments/thoughts before I merge this in the morning? :D |
Part of a rule on #36
simple-lambdashas been implemented.This PR is for preference of Operator Fcns over Lambda Fcns (https://google.github.io/styleguide/pyguide.html?showone=Lambda_Functions#Lambda_Functions)