Repository navigation
Literal raw string/bytes backslashes are highlighted as escapes #13702
Description
Environment data
- VS Code version:
Version: 1.48.2 (user setup)
Commit: a0479759d6e9ea56afa657e454193f72aef85bd0
Date: 2020-08-25T10:13:11.295Z
Electron: 7.3.2
Chrome: 78.0.3904.130
Node.js: 12.8.1
V8: 7.8.279.23-electron.0
OS: Windows_NT x64 10.0.19041
- Extension version (available under the Extensions sidebar):
v2020.8.105369 - OS and version:
Windows 10 - Python version (& distribution if applicable, e.g. Anaconda):
Python 3.8.5 - Type of virtual environment used:
virtualenvstdlib package - Relevant/affected Python packages and their versions:
Python 3.8.5 - Relevant/affected Python-related VS Code extensions and their versions:
v2020.8.105369 - Value of the
python.languageServersetting:v2020.8.105369Pylance language server 2020.8.3 (pyright 8c5415fd)(Sorry, unsure how to query this value, so previous values are just copy-pastes from extension page and Python Language Server output log.)
Expected behaviour
Backslashes (and the characters that follow them) in raw string literals and raw byte literals should not be highlighted. Highlighting in this case should be reserved for non-raw literals, where the backslashes create an escaped character.
Both string and bytes literals may optionally be prefixed with a letter 'r' or 'R'; such strings are called raw strings and treat backslashes as literal characters.
from https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals
Actual behaviour
Backslashes are highlighted in these cases as if they are escape characters, which they are not.
Steps to reproduce:
- Create a new python file and ensure the language server has started.
- Insert this text into the file:
RAW_STRING = r"C:\Users\tim\code\imagesearch\tests\fixtures"
RAW_BYTES = rb"C:\Users\tim\code\imagesearch\tests\fixtures"- Note the errant escape highlighting.
Activity
[-]Literal raw string/byte backslashes are highlighted as escapes[/-][+]Literal raw string/bytes backslashes are highlighted as escapes[/+]t-mart commentedon Sep 1, 2020
I was not sure if I posted this bug in the right place, so I dug around a little to try to find the token/grammar specification in this repo. I couldn't find anything. Maybe this comes from TextMate, the grammar provider for VSCode?
Also, to provide a little more context, I found a nifty command that brings up a tooltip about the tokenization that occurred at the current position in a document:
Developer: Inspect Editor Tokens and Scopes(which can be accessed from theCtrl + Shift + Pmenu). Here's what it shows for these escapes:Perhaps this file https://github.com/textmate/python.tmbundle/blob/master/Syntaxes/Python.tmLanguage. This file defines the scopes/tokens with the same names as those in the tooltip above, which adds confidence that this is the right file. Also, I did see some specific handling for raw literals in there. However, these files are complex and I'm not sure how they work.
brettcannon commentedon Sep 1, 2020
We don't control syntax highlighting for Python files. That comes from VS Code which gets it from MagicPython. It's best to report an issue there.
t-mart commentedon Sep 1, 2020
Thanks for pointing me in the right direction Brett Cannon (@brettcannon).
For posterity, one can workaround to the expected behavior by using capitalized "R"s in front of the literals (even though this distinction is not part of the Python grammar).
More info here: MagicStack/MagicPython#114