Repository navigation
Conversation
|
I can see the idea here. How do other linters deal with this? Have you looked up if this has been proposed on python-dev or python-ideas? |
|
@DimitrisJim Thanks for the comment!
I think other linters just analyze a merged string.
I haven't yet. |
|
Looks like pylint re-tokenizes code into tokens to detect implicit concatenation: |
|
pyright seems to have a node for an implicitly-concatenated string ( |
|
It's actually possible to break down an implicitly-concatenated string, but it requires re-tokenization and re-parsing, which can degrade the performance. a = "a" "b"
^^^^^^^
# 1. By tokenizing the ^^^ range, we can get two string tokens.
# 2. By parsing each string token, we can get two Constant nodes. |
yeah I thought that would be the issue. Not sure why the implicit joining is done before the AST phase, there could be some reason behind it which I'm unaware of. In general, I'd probably ask around other projects to see if something like this would be a feature many would find useful, then a proposal might be made to CPython. If that doesn't lead anywhere, sealing it behind a feature might be acceptable but @youknowone and @coolreader18 would have to give the ok for it. |
|
To change ast, as @DimitrisJim said, I think a proposal for CPython would be the best option. Now language developers understand tooling support is really important for the language ecosystem. It will be worth. In short term, I am open to add features to support toolings. But I also think we have to keep the update cost as small as possible. I'd like to accept it if the patch can make ast updating easy enough. |
|
@youknowone Thanks for the comment!
Yeah, this is what I'm concerned the most. I don't want this node to be a blocker for RustPython's development. |
I don't think this proposal is acceptable because the proposed node doesn't exist in Python. I just wanted to hear thoughts on it. This PR adds a new AST node
ImplicitConcatthat represents implicit concatenation (e.g."a" "b"). Currently, implicitly-concatenated strings are merged into a singleConstantorJoinedStrnode. This makes it difficult to analyze each string component. Here's an example:This code is parsed as:
This is equivalent to
f"a{x}", which is a valid f-string.If the python code above was parsed as:
that would allow a linter such as Ruff to analyze each token and provide more helpful errors.
The following video is a quick demo that utilizes
ImplicitConcat:Screen.Recording.2023-01-03.at.15.50.09.mov