First build and install fe-language-server:

cargo install --git https://github.com/ethereum/fe.git fe-language-server

Then, open the Fe codebase root workspace in VSCode and press F5 to debug the extension using the "Launch Fe VSCode Extension" configuration.