Jump to content

Parsoid

Issue tracker: #parsoid
From mediawiki.org
This page is a translated version of the page Parsoid and the translation is 25% complete.
Languages:
Projekty Fundacji Wikimedia
Parsoid
Dwukierunkowy parser wikikodu Konwertuje między wikikodem a HTML/XML DOM z RDFa.
Grupa: Content Transform Team
Początek: 2011-10
Członkowie zespołu: Arlo Breault, C.Scott Ananian, Isabelle Hurbain-Palatin, Mateus Santos, Otto Šleger, Subramanya Sastry, Yiannis Giannelos
Kolejka zadań: #parsoid
Lider: C.Scott Ananian
Zarząd: Sérgio Lopes
Aktualizacje: Parsoid/Releases
Postęp: Parsoid/Deployments
Diagram czasu uruchomienia wiki Parsoid HTML5 + RDFa
Aby znleźć wersję Parsoida napisaną w JavaScript (Node.js) do 2020, patrz Parsoid/JS .

Parsoid – biblioteka PHP dołączona do MediaWiki (1.35+), używana do konwersji wikikodu na HTML i z powrotem. Był pisany od 2012, początkowo w JavaScript i stworzony w celu wsparcia VisualEditor . Jest w MediaWiki od wersji 1.35. W MediaWiki 1.47, Parsoid zostanie domyślnym parserem(aktualny domyślny parser ), w projekcie znanym jako Parsoid/Parser Unification

Szczegóły techniczne

Parsoid jest aplikacją, która tłumaczy tam i z powrotem między składnią wikikodu MediaWiki i równoważnym modelem dokumentu HTML/RDFa z ulepszonym wsparciem dla zautomatyzowanego przetwarzania i edytowania tekstu sformatowanego.

Od 2012 roku jest rozwijany przez zespół Wikimedia Foundation. Jest obecnie używany w dużym stopniu przez VisualEditor , Tłumaczenie treści i inne aplikacje.

Parsoid ma zapewnić niezawodną konwersję dwukierunkową, tj. uniknąć utraty informacji i zapobiec "brzydkim różnicom".

Na wiki Wikimedia dla kilku aplikacji, Parsoid jest obecnie proxy za RESTBase , który przechowuje HTML przetłumaczony przez Parsoid. Oczekuje się, że RESTbase zostanie w przyszłości zastąpione przez cache bardziej zintegrowane z MediaWiki.

Więcej informacji na temat ogólnego projektu można znaleźć w tym poście na blogu z marca 2013 r. Aby przeczytać o używanym modelu HTML, zobacz Mediawiki DOM spec.

Parsoid został pierwotnie skonstruowany jako usługa internetowa i napisany w JavaScript, wykorzystując Node.js. Tech talk from February 2019 (slajdy) i post na blogu opisują portowanie tego do PHP. Rozszerzenie API Parsoid jest obecnie w fazie aktywnego rozwoju; prace te opisano w tech talk from August 2020.

Repozytorium w serwisie GitHub: https://github.com/wikimedia/parsoid

Użycie

Instalacja

This extension comes with MediaWiki 1.35 and later, so you do not need to download it. The remaining configuration instructions must still be followed.

Parsoid jest włączony w MediaWiki od wersji 1.35. Nie ma żadnej potrzeby na konfigurację, żeby ją włączyć.

W przypadku gdy w przypadku wprowadzania w życie w ramach programu RESTBase, w przypadku wprowadzenia w życie w celu uzyskania odpowiednich informacji, w przypadku gdy w ramach programu WMF wprowadzono w życie wzajemne wzajemne działania, w przypadku którego wprowadzono do obrotu w ramach programu, w przypadku których wprowadzono zwrot w ramach programu w ramach programu. Nie jest to już wymagane dla Visual Editor lub głównego przeglądu odczytu, a wewnętrzna API jest w trakcie przestarzaania i jest planowana usunięcie w MW 1.43.

Parsoid jest nominalizmowo biblioteką kompozytorów używaną przez mediawiki core. Jeśli nadal potrzebujesz wewnętrznego API z jakiegoś powodu, możesz wyraźnie załadować Parsoid "jako rozszerzenie" poprzez dodanie następujących elementów do LocalSettings.php: 

wfLoadExtension( 'Parsoid', "$IP/vendor/wikimedia/parsoid/extension.json" );

Pozostałe użytkownicy API wewnętrznego Parsoid są mocno zachęcani do migracji do punktu końcowego strony REST HTML, który zapewnia równoważną funkcjonalność.

Rozwój

Rozwój się dzieje w Parsoid Git repository. Recenzja kodu jest prowadzona w Gerricie. See Gerrit/Getting started to set up an account for yourself.

If you use the MediaWiki-Vagrant development environment using a virtual machine, you can simply add the role visualeditor to it and it will set up a working Parsoid along with Extension:VisualEditor .

The instructions below are for MediaWiki 1.35 or later. Check Parsoid/JS if you are running the old version of Parsoid written in JavaScript, and used for MW 1.34 and earlier.

Linking a developer checkout of Parsoid

In a standard MediaWiki installation, Parsoid is included from MediaWiki as a composer library, wikimedia/parsoid.

For development purposes you usually want to use a git checkout of Parsoid, and not the version bundled in MediaWiki core as a composer library. The following lines added to LocalSettings.php allow use of a git checkout of Parsoid (optionally), load the Parsoid REST API with wfLoadExtension (rather than using the version bundled in VisualEditor) and manually do the Parsoid configuration which is usually done by VisualEditor: 

$parsoidInstallDir = 'vendor/wikimedia/parsoid'; # bundled copy
#$parsoidInstallDir = '/my/path/to/git/checkout/of/Parsoid';

// For developers: ensure Parsoid is executed from $parsoidInstallDir,
// (not the version included in mediawiki-core by default)
// Must occur *before* wfLoadExtension()
if ( $parsoidInstallDir !== 'vendor/wikimedia/parsoid' ) {
    function wfInterceptParsoidLoading( $className ) {
        // Only intercept Parsoid namespace classes
        if ( preg_match( '/(MW|Wikimedia\\\\)Parsoid\\\\/', $className ) ) {
           $fileName = Autoloader::find( $className );
           if ( $fileName !== null ) {
               require $fileName;
           }
        }
    }
    spl_autoload_register( 'wfInterceptParsoidLoading', true, true );
    // AutoLoader::registerNamespaces was added in MW 1.39
    AutoLoader::registerNamespaces( [
        // Keep this in sync with the "autoload" clause in
        // $parsoidInstallDir/composer.json
        'Wikimedia\\Parsoid\\' => "$parsoidInstallDir/src/",
    ] );
}

wfLoadExtension( 'Parsoid', "$parsoidInstallDir/extension.json" );
unset( $parsoidInstallDir );
Unit tests in mediawiki-core do not use the standard autoloader (or LocalSettings.php) and so will not pick up your local Parsoid; set MEDIAWIKI_HAS_INTEGRATION_TESTS=1 in your environment to force the use of LocalSettings.php and the autoloader when running core unit tests against your local Parsoid checkout.

The LocalSettings.php lines are not necessary for most users of VisualEditor, who can use VisualEditor's auto-configuration and the bundled Parsoid code included in MediaWiki, but they will be required for most developers working on Parsoid.

If you're serving MediaWiki with Nginx, you'll need to also add something like this in your server block (Assuming your MediaWiki setup has its files residing in /w/): 

location /w/rest.php/ {
    try_files $uri $uri/ /w/rest.php?$query_string;
}

If you are running Mediawiki using Docker, and linking your local Parsoid repository to Mediawiki, you need to map additional volume to the docker container in docker-compose.override.yml file in media wiki project. To do so, the simplest way is to create docker-compose.override.yml in mediawiki project and put code bellow inside (with path modification). If you already have docker-compose.override.yml file, modify it accordingly. 

services:
  mediawiki:
    volumes:
      - ./:/var/www/html/w:cached
      - /my/path/to/git/checkout/of/Parsoid:/my/path/to/git/checkout/of/Parsoid

To test proper configuration, visit {$wgScriptPath}/rest.php/{$domain}/v3/page/html/Main%20Page where $domain is the hostname in your $wgCanonicalServer. (Note that production WMF servers do not expose the Parsoid REST API to the external network.)

Running the tests

To run all parser tests and mocha tests: 

$ composer test

The parser tests have quite a few options now which can be listed using php bin/parserTests.php --help.

If you have the environment variable MW_INSTALL_DIR pointing to a configured MediaWiki installation, you can run some additional tests with: 

$ composer phan-integrated

Converting simple wikitext

You can convert simple wikitext snippets from the command line using the parse.php script in the bin/ directory: 

$ echo '[[Foo]]' | php bin/parse.php

The parse script has a lot of options. php bin/parse.php --help gives you information about this.

Debugging Parsoid (for developers)

See Parsoid/Debugging for debugging tips.

Continuous Integration

Parsoid is always available as a library since it is a composer dependency of MediaWiki core. But two pieces are not enabled:

  • Parsoid ServiceWiring
  • Parsoid's external REST api

The test runner Quibble would enable it if it detects mediawiki/services/parsoid.git has been cloned as part of the build. In which case it:

  • points the autoloader for Wikimedia\Parsoid to the cloned code (effectively replacing the version installed by composer)
  • Load the extension wfLoadExtension( 'Parsoid', '/path/to/cloned/repo' );

The ServiceWiring should be enabled in MediaWiki starting with 1.38.

The REST API would theorically never get merged in MediaWiki: a) it has never been exposed to the public in production, it is an internal API used by RESTBase which is going away; b) it never has been security audited and c) it is redundant with the enterprise MediaWiki API. The solution will be for VisualEditor to invoke Parsoid directly via the VisualEditor Action API which would save a round trip through the REST API.

Loading the extension is thus a hack which enables using interfaces subject to change and which we don't really want people to use yet.

For most purposes, parsoid should thus not be added as a CI dependency, the only exception as of October 2021 is the Disambiguator MediaWiki extension.

Loading parsoid as an extension let us run MediaWiki integration test jobs against mediawiki/services/parsoid.git (such as Quibble, apitesting) and ensure Parsoid and MediaWiki work together.

An extension may be able to write tests with Parsoid even when the repository has not been cloned. Since it is a composer dependency of MediaWiki core the MediaWiki\Parsoid namespace is available, but the service wiring part is not (it is extension/src in the Parsoid repository and exposed as the \MWParsoid namespace). The ParsoidTestFileSuite.php code would only run the parser tests if Parsoid has been loaded (which should be the default with MediaWiki 1.38).

For CI, Parsoid is tested against the tip of MediaWiki, whereas MediaWiki is tested with the Composer dependency. In case of a breaking change, the Parsoid change get merged first (which breaks its CI but not MediaWiki one) and MediaWiki get adjusted when Parsoid is updated. It is thus a one way change.

Release build

For MediaWiki release builds, we have an integration of Parsoid ServiceWiring into VisualEditor in order to have VisualEditor work without further configuration (beside a wfLoadExtension( 'VisualEditor' )). The release build also enables the REST API and hook everything us so that parsoid works out of the box. This is done by copying a bit of Parsoid code into VisualEditor which is not in the master branch of VisualEditor since that would be obsolete as soon as Parsoid is updated. Instead the code is maintained in two places.

Technical documents

History

The original application was written in JavaScript (using Node.js) and started running on the Wikimedia cluster in December 2012. In 2019, Parsoid was ported to PHP, and the PHP version replaced the JS version on the Wikimedia cluster in December 2019. Parsoid is being integrated into core MediaWiki, with the goal of eventually replacing MediaWiki's current native parser. In early 2024, Parsoid began to be used on some production wikis of the Wikimedia Foundation as the default parser for read views. You can see them in the production config list.

Parsoid (the PHP version) has been natively bundled with MediaWiki since version 1.35, released in September 2020. For non-Wikimedia installations, Parsoid/JS was supported until the end-of-life of MediaWiki 1.31 (LTS) in September 2021.

FAQ

  • How do I see if a page was rendered with Parsoid?
    • The footer will say "Page was rendered with Parsoid".
  • How do I set a page to temporarily render with Parsoid?
  • How do I get an entire wiki to render with Parsoid?
    • Install Extension:ParserMigration
    • Set $wgParserMigrationEnableParsoidArticlePages = true;
    • Set $wgParserMigrationEnableParsoidDiscussionTools = true;
  • Which Wikimedia wikis render with Parsoid?
    • The list is at parsoidrendered.dblist. As of August 2025, it is approximately 250 of the 1000 wikis. Rollout will continue until all wikis are using Parsoid.

Zobacz też

Linki zewnętrzne

Kontakt

If you need help or have questions/feedback, you can contact us in #mediawiki-parsoid connect or the wikitext-l mailing list. If all that fails, you can also contact us by email at content-transform-team at the wikimedia.org domain.

Retrieved from "https://www.mediawiki.org/w/index.php?title=Parsoid/pl&oldid=8600993"