Summary

During the discussion about t-string support #17557, I thought that it might not be much effort to support nested f-strings with the current f-string parser. And it turned out relatively simple.

The way the MicroPython f-string parser works is:

1. it extracts the f-string arguments (things in curly braces) into a temporary buffer (a vstr)
2. once the f-string ends (reaches its closing quote) the lexer switches to tokenizing the temporary buffer
3. once the buffer is empty it switches back to the stream.

The temporary buffer can easily hold f-strings itself (ie nested f-strings) and they can be re-parsed by the lexer using the same algorithm. The only thing stopping that from working is that the temporary buffer can't be reused for the nested f-string because it's currently being parsed.

This PR fixes that by adding a second temporary buffer, which is the "injection" buffer. That allows arbitrary number of nestings with a simple modification to the original algorithm:

1. when an f-string is encountered the string is parsed and its arguments are extracted into fstring_args
2. when the f-string finishes, fstring_args is inserted into the current position in inject_chrs (which is the start of that buffer if no injection is ongoing)
3. fstring_args is now cleared and ready for any further f-strings (nested or not)
4. the lexer switches to inject_chrs if it's not already reading from it
5. if an f-string appeared inside the f-string then it is in inject_chrs and can be processed as before, extracting its arguments into fstring_args, which can then be inserted again into inject_chrs
6. once inject_chrs is exhausted (meaning that all levels of f-strings have been fully processed) the lexer switched back to tokenizing the stream

Amazingly, this scheme supports arbitrary numbers of nestings of f-strings using the same quote style.

Testing

A new test is added which will run under CI.

Trade-offs and Alternatives

This adds some code size and a bit more memory usage for the lexer. In particular for a single (non-nested) f-string it now makes an extra copy of the fstring_args data, when copying it across to inject_chrs. That could possibly be optimized to reuse the same buffer (inject_chrs would steal the memory from fstring_args).

Otherwise, memory use only goes up with the complexity of nested f-strings.