Use tokens to extend CodeMirror editors #13639

fcollonval · 2022-12-21T13:07:23Z

References

Follow-up of #12861 (review)
Fixes #13362
Fixes #12886
Fixes #9124
Fixes #13206
Fixes #13162
Fixes #13199
Fixes #14011
Fixes #10486

Code changes

CodeMirror 6 can conform to the Token based approach of JupyterLab instead of using globals. The new tokens are:

@jupyterlab/codemirror:IEditorExtensionRegistry: Provide the CodeMirror editor extensions - default extensions are provided in a static method and loaded in the plugin providing the token.
@jupyterlab/codemirror:IEditorLanguageRegistry: Provide the languages for the editors - default languages are provided in a static method and loaded in the plugin providing the token. This allows to override the plugin with much less language for small application for example. A single default language for text/plain is provided.
@jupyterlab/codemirror:IEditorThemeRegistry: Provide the theme for the editors - default themes are provided in a static method and loaded in the plugin providing the token. This allows to override the plugin with less themes if required. A single default theme based on jupyter theme CSS variable is provided.

InputArea.defaultContentFactory and all its extensions (for Cell and Notebook) have been removed. The editor factory requires tokens. Hence we cannot provide a global default. This is similar to the evolution of the sanitizer.

TextModelFactory.preferredLanguage(path: string) is now a no-op (return '' for any path) as the languages are not available globally and as kernel language for text model makes no sense without more context.

Create a new Language for text/plain that parses nothing. This allows for a better retro-compatibility as well as a default display without syntax highlighting that is confusing for now on master (see for instance the console header).

Mode editor has been renamed to language to align with CodeMirror 6 naming and to clarify the meaning.

Drop configuration not supported in CodeMirror 6 - electricChars, coverGutterNextToScrollbar, scrollbarStyle, styleSelectedText and selectionPointer.
Add extensions to support cursorBlinkRate, rulers, showTrailingSpaces, scrollPastEnd.
lineWrap is now a simple boolean and wordWrapColumn is removed.

All configurable listed in #12812 are handled (= restored, changed or removed).

Improve code highlight style to be closer to 3.x.

The ipython CodeMirror language is for now identical to the Python language (as highlighting the ? is a minor feature compare to having code folding).

Upgrade all @codemirror and @lezer packages

Main point missing from #12812 is LaTeX highlighting in Markdown cell.

User-facing changes

None

Backwards-incompatible changes

Break API for CodeMirror Mode (compare to 3.x)

jupyterlab-probot · 2022-12-21T13:07:25Z

Thanks for making a pull request to jupyterlab!
To try out this branch on binder, follow this link:

JohanMabille · 2022-12-22T07:50:31Z

packages/codemirror/src/codemirror-ipythongfm.ts

-    );
-  }
-});
-


Why not removing the file?

I would say we should bring LaTeX highlighting back for ipythongfm mode, as this is feature parity with Notebook v6 and Lab < 4.

I agree, but I think it deserves a dedicated PR, syntax highlighting for CM6 is very different from CM5 and requires a complete rewriting.

I kept the file as it may be useful when working to restore syntax highlighting for CM6.

krassowski

This is amazing 🚀

Highlighting some integration-level issues from local testing:

I can no longer navigate cells with the cursor (using up and down keys), whereas it works on 4.0.0a34 (and tip of the master branch)

4.0.0a34 this PR
the CodeMirror UI settings shows validation error for me (I would love to double check this on binder but #14033)
- it is still an improvement compared to the state of things on master (#14011; I edited the description of the PR to link this PR to that issue)
smartIndent does not have a label
the CodeMirror JSON settings shows "missing schema description"; this is a regression compared to 4.0.0a34 but I am not sure if this works on tip of master branch (again would need a check in a clean environment, #14033):

4.0.0a34 this PR

If you believe that some of the above should be tackled as a follow-up I am happy to approve once these were extracted to separate issues (please ping me for review). Ideally we would add galata tests for the things flagged above.

packages/codemirror/src/extension.ts

fcollonval · 2023-02-22T08:08:05Z

bot please update snapshots

github-actions · 2023-02-22T08:49:55Z

Documentation snapshots updated.

github-actions · 2023-02-22T09:13:48Z

Galata snapshots updated.

krassowski · 2023-02-25T15:00:52Z

Is this ready to merge?

bot please update playwright snapshots

schmidi314 · 2023-02-25T22:39:52Z

I'm very much looking forward to this merge. But I just noticed that there are some strange defaults set in the code cell settings: Indentation unit is 2 spaces and line numbers are switched on. I'm fine with the line numbers (though it might unnecessarily break user experience) but two-spaces indentation for python is weird.

I tried this on binder. It might be related to notebook-extension/schema/tracker.json. It seems the settings have been deleted from there?

krassowski · 2023-02-25T23:13:47Z

Good catch, thank you @schmidi314. I would add two observations that can be moved to a separate issue and addressed in a follow up PR:

When switching between tab sizing options the "Default:" shown below appears random:
The "Default:" often gets lost for some reason but it does not happen for other groups of settings I tested:

Implementation of the new token interfaces First refactoring of codemirror pkg Don't get languagePreference in `TextModelFactory` Remove `Mode` usage Fix integrity default content factory is not possible Extract default extensions to use extensible API More refactoring Remove not applicable configuration in CM6 Fix build Fix tests Fix handling of language Update codemirror Use default python LanguageSupport for `ipython` Activate more features than using the legacy stream parser Fix codemirror tests Fix integrity Automatic application of license header Add note in migration guide Improve CodeMirror default theme Build dynamically the editor themes menu Simplify API for extension based on DynamicSetting See https://github.com/ChromeDevTools/devtools-frontend/blob/e813f07e6a47b39c04c64a409dd08be294432490/front_end/ui/components/text_editor/config.ts#L35 Add cursor blinking rate Restore custom paste event and D&D on md and raw cells Add rulers Rename handler to registry to align with other JLab API Automatic application of license header Fix registry rename Translate default elements CodeMirror settings Use the custom settings editor for all editors Bump minimal codemirror packages Fix integrity More fix following rebase Use displayName for syntax status Progress towards better configuration Improve setting editor Rename EditorLanguageRegistry.parse to highlight Set up configuration for console Refactor file editor feature in fileeditor-extension Automatic application of license header Fix following rebase Change default for markdown cell settings Restore keymap as extension Move translation to extension factory Clean binding API Introduce a extensions registry Editor binding as immutable extension Rename `Configuration` in `EditorExtensionRegistry` Drop themes from factory Language source of truth is model.mimeType not an option Restore custom styling Drop word wrap column Fix keymap Remove `CodeEditor.addKeydownHandler` Explicitly introduce CodeMirror in CodeEditor Allow providing configuration in `InputArea` Incidentally allow it in `CodeEditorWrapper` and `CodeEditorViewer` First self review Fix syntax status item Fix indentation status widget Fix CM unit tests One more self review Add test for EditorExtension Registry Add test for `ExtensionsHandler` Fix unit tests Update migration notes Fix unused Fix following rebase Fix the example Fix integrity check Fix console tests Fix examples Fix missing label Fix default settings for cells Fix python * operator [skip ci] Fix yarn.lock Update Playwright Snapshots Update Playwright Snapshots Fix overflow for raw settings editor Fix file editor and menus Fix galata tests Remove CM keymap from documentation Update Playwright Snapshots Update Playwright Snapshots Fix toggle editor theme Fix PR following upgrade to rjsf v5 Fix integrity More update of the migration guide Fix integrity Rename `Cell.setEditorConfig` to `Cell.updateEditorConfig` Improve typing Easier selector Fix settings bugs Fix edge requested Internalize Yjs binding Remove unused `documentType` Fix unit tests Update Playwright Snapshots Update snapshots Improve test robustness Lint code

fcollonval · 2023-02-27T08:02:27Z

bot please update snapshots

github-actions · 2023-02-27T09:02:55Z

Galata snapshots updated.

fcollonval · 2023-02-27T09:12:55Z

Kicking the CI

fcollonval · 2023-02-27T09:16:50Z

@schmidi314 and @krassowski thanks for the comments - as we are reaching API freezing day, I'll merge this as is. And I opened #14097 as follow-up from your comments

fcollonval added api-change A change that should be accompanied by a major version increase enhancement labels Dec 21, 2022

fcollonval added this to the 4.0.0 milestone Dec 21, 2022

github-actions bot added Design System CSS pkg:attachments pkg:cells pkg:codeeditor pkg:codemirror pkg:console pkg:csvviewer pkg:debugger pkg:docregistry pkg:documentsearch pkg:fileeditor pkg:notebook pkg:outputarea tag:CSS For general CSS related issues and pecadilloes labels Dec 21, 2022

github-actions bot assigned fcollonval Dec 21, 2022

fcollonval force-pushed the ft/extensible-codemirror-6 branch from afae538 to fd42b46 Compare December 21, 2022 14:19

github-actions bot added documentation pkg:themes labels Dec 21, 2022

JohanMabille reviewed Dec 23, 2022

View reviewed changes

krassowski self-requested a review December 23, 2022 16:45

fcollonval mentioned this pull request Jan 4, 2023

Weekly Team Meetings: Jan-Jun 2023 jupyterlab/frontends-team-compass#170

Closed

github-actions bot added tag:Testing pkg:completer pkg:lsp pkg:settingeditor pkg:settingregistry labels Jan 5, 2023

krassowski reviewed Feb 18, 2023

View reviewed changes

fcollonval commented Feb 20, 2023

View reviewed changes

packages/codemirror/src/extension.ts Show resolved Hide resolved

fcollonval force-pushed the ft/extensible-codemirror-6 branch 2 times, most recently from 3f7ad77 to 709ff0f Compare February 21, 2023 14:52

fcollonval force-pushed the ft/extensible-codemirror-6 branch from 6e90653 to 4c2b63d Compare February 22, 2023 11:17

fcollonval force-pushed the ft/extensible-codemirror-6 branch from 994be63 to 76c845f Compare February 27, 2023 08:02

Update Playwright Snapshots

f3d812f

fcollonval closed this Feb 27, 2023

fcollonval reopened this Feb 27, 2023

fcollonval mentioned this pull request Feb 27, 2023

CodeMirror indentation unit setting #14097

Closed

fcollonval merged commit 2142d76 into jupyterlab:master Feb 27, 2023

fcollonval deleted the ft/extensible-codemirror-6 branch February 27, 2023 11:12

This was referenced Feb 27, 2023

Move the file name searcher to the file browser toolbar #14064

Merged

Update to JupyterLab 4.0.0a35 jupyter/notebook#6757

Merged

mlucool mentioned this pull request Mar 7, 2023

Jupyterlab api change in 4.0.0 deshaw/jupyterlab-execute-time#82

Closed

jtpio mentioned this pull request Mar 22, 2023

Latest examples do not render outputs #14221

Closed

fcollonval mentioned this pull request Apr 12, 2023

Brainf... is listed as syntax highlight option #14373

Closed

krassowski mentioned this pull request Feb 4, 2024

File Editor does not scroll on undo/redo #15102

Closed

github-actions bot locked as resolved and limited conversation to collaborators Feb 28, 2024

Uh oh!

Use tokens to extend CodeMirror editors #13639

Use tokens to extend CodeMirror editors #13639

Uh oh!

Conversation

fcollonval commented Dec 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

References

Code changes

User-facing changes

Backwards-incompatible changes

Uh oh!

jupyterlab-probot bot commented Dec 21, 2022

Uh oh!

JohanMabille Dec 22, 2022

Choose a reason for hiding this comment

Uh oh!

krassowski Dec 23, 2022

Choose a reason for hiding this comment

Uh oh!

JohanMabille Dec 26, 2022

Choose a reason for hiding this comment

Uh oh!

fcollonval Jan 4, 2023

Choose a reason for hiding this comment

Uh oh!

krassowski left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

fcollonval commented Feb 22, 2023

Uh oh!

github-actions bot commented Feb 22, 2023

Uh oh!

github-actions bot commented Feb 22, 2023

Uh oh!

krassowski commented Feb 25, 2023

Uh oh!

schmidi314 commented Feb 25, 2023

Uh oh!

krassowski commented Feb 25, 2023

Uh oh!

fcollonval commented Feb 27, 2023

Uh oh!

github-actions bot commented Feb 27, 2023

Uh oh!

fcollonval commented Feb 27, 2023

Uh oh!

fcollonval commented Feb 27, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

fcollonval commented Dec 21, 2022 •

edited

Loading

krassowski left a comment •

edited

Loading