Jump to content

Parsoid

No rastreador: #parsoid
From mediawiki.org
This page is a translated version of the page Parsoid and the translation is 40% complete.
Outdated translations are marked like this.
Languages:
Projetos da Função Wikimédia
Parsoid
Um analisador sintático de texto wiki em tempo de execução bidirecional. Converte de ida e volta entre o wikitext e o DOM HTML/XML com RDFa.
Grupo: Content Transform Team
Início: 2011-10
Membros da equipa: Arlo Breault, C.Scott Ananian, Isabelle Hurbain-Palatin, Mateus Santos, Otto Šleger, Subramanya Sastry, Yiannis Giannelos
Trabalho em atraso: #parsoid
Líder: C.Scott Ananian
Gestão: Sérgio Lopes
Atualizações: Parsoid/Releases
Progresso: Parsoid/Deployments
Um diagrama do tempo de execução da wiki Parsoid HTML5 + RDFa
"Para a versão anterior de Parsoid escrita em JavaScript (Node.js), consulte Parsoid/JS ."

Parsoid is a PHP library that is used for converting back and forth between wikitext and HTML. It has been under development since 2012, originally written in JavaScript and built to support the VisualEditor . It has been bundled with MediaWiki since version 1.35. In MediaWiki 1.47, Parsoid is planned to become MediaWiki's default parser, replacing the current native parser , in a project known as parser unification .

Detalhes técnicos

Parsoid is an application which can translate back and forth between MediaWiki's wikitext syntax and an equivalent HTML/RDFa document model with enhanced support for automated processing and rich editing.

It has been under development by a team at the Wikimedia Foundation since 2012. It is currently used extensively by VisualEditor , Tradução de Conteúdo and other applications.

Parsoid is intended to provide flawless back-and-forth conversion, i.e. to avoid information loss and also prevent "dirty diffs".

On Wikimedia wikis, for several applications, Parsoid is currently proxied behind RESTBase , which stores the HTML translated by Parsoid. It is expected that RESTBase will eventually be replaced with a cache more tightly integrated with MediaWiki.

For more on the overall project, see this blog post from March 2013. To read about the HTML model being used, see MediaWiki DOM spec.

Parsoid was originally structured as a web service and written in JavaScript, making use of Node.js. A tech talk from February 2019 (slides) and blog post describes the process of porting it to PHP. The Parsoid extension API is currently under active development; a tech talk from August 2020 describes this work.

Repositório do GitHub: https://github.com/wikimedia/parsoid

Utilização

Instalação

This extension comes with MediaWiki 1.35 and later, so you do not need to download it. The remaining configuration instructions must still be followed.

O Parsoid está incluído no MediaWiki desde a versão 1.35. No configuration is necessary to enable it.

Parsoid exports an internal REST API which was historically used by RESTBase and not accessible outside the WMF internal cluster. This is no longer required for Visual Editor or core read views, and the internal API is being deprecated and is planned for removal in MW 1.43.

Parsoid is nominally a composer library used by mediawiki core. If you still require the internal API for some reason, you can explicitly load Parsoid "as an extension" by adding the following to LocalSettings.php: 

wfLoadExtension( 'Parsoid', "$IP/vendor/wikimedia/parsoid/extension.json" );

Any remaining third-party users of the internal Parsoid API are strongly encouraged to migrate to the core REST HTML page endpoint which provides equivalent functionality.

Desenvolvimento

O desenvolvimento ocorre no Parsoid Git repository. A revisão de código ocorre no Gerrit. Consulte Gerrit/Getting started para configurar a sua própria conta.

If you use the MediaWiki-Vagrant development environment using a virtual machine, you can simply add the role visualeditor to it and it will set up a working Parsoid along with Extension:VisualEditor .

The instructions below are for MediaWiki 1.35 or later. Check Parsoid/JS if you are running the old version of Parsoid written in JavaScript, and used for MW 1.34 and earlier.

Interligar um checkout de programador de Parsoid

Numa instalação do MediaWiki padrão, Parsoid está incluído no MediaWiki como uma biblioteca de composer, wikimedia/parsoid.

Para propósitos de desenvolvimento, geralmente quererá utilizar uma cópia de Parsoid obtida do Git, e não a versão empacotada no núcleo do MediaWiki como biblioteca de composer. As linhas seguintes adicionadas a LocalSettings.php permitem utilizar (opcionalmente) uma cópia de Parsoid proveniente do Git, carregar a API REST de Parsoid com wfLoadExtension (em vez de utilizar a versão empacotada no Editor Visual) e configurar Parsoid manualmente, o que geralmente se faz mediante o Editor Visual: 

$parsoidInstallDir = 'vendor/wikimedia/parsoid'; # cópia integrada
#$parsoidInstallDir = '/my/path/to/git/checkout/of/Parsoid';

// Para programadores: certifique-se que é executado de $parsoidInstallDir,
// (não a versão incluída por predefinição no núcleo do mediawiki)
// Deve ocorrer *antes* de wfLoadExtension()
if ( $parsoidInstallDir !== 'vendor/wikimedia/parsoid' ) {
    function wfInterceptParsoidLoading( $className ) {
        // Intercetar apenas classes de espaço nominal de Parsoid
        if ( preg_match( '/(MW|Wikimedia\\\\)Parsoid\\\\/', $className ) ) {
           $fileName = Autoloader::find( $className );
           if ( $fileName !== null ) {
               require $fileName;
           }
        }
    }
    spl_autoload_register( 'wfInterceptParsoidLoading', true, true );
    // AutoLoader::registerNamespaces foi adicionado no MW 1.39
    AutoLoader::registerNamespaces( [
        // Manter isto sincronizado com a cláusula "autoload" em
        // $parsoidInstallDir/composer.json
        'Wikimedia\\Parsoid\\' => "$parsoidInstallDir/src/",
    ] );
}

wfLoadExtension( 'Parsoid', "$parsoidInstallDir/extension.json" );
unset( $parsoidInstallDir );
Unit tests in mediawiki-core do not use the standard autoloader (or LocalSettings.php) and so will not pick up your local Parsoid; set MEDIAWIKI_HAS_INTEGRATION_TESTS=1 in your environment to force the use of LocalSettings.php and the autoloader when running core unit tests against your local Parsoid checkout.

The LocalSettings.php lines are not necessary for most users of VisualEditor, who can use VisualEditor's auto-configuration and the bundled Parsoid code included in MediaWiki, but they will be required for most developers working on Parsoid.

If you're serving MediaWiki with Nginx, you'll need to also add something like this in your server block (Assuming your MediaWiki setup has its files residing in /w/): 

location /w/rest.php/ {
    try_files $uri $uri/ /w/rest.php?$query_string;
}

If you are running Mediawiki using Docker, and linking your local Parsoid repository to Mediawiki, you need to map additional volume to the docker container in docker-compose.override.yml file in media wiki project. To do so, the simplest way is to create docker-compose.override.yml in mediawiki project and put code bellow inside (with path modification). If you already have docker-compose.override.yml file, modify it accordingly. 

services:
  mediawiki:
    volumes:
      - ./:/var/www/html/w:cached
      - /my/path/to/git/checkout/of/Parsoid:/my/path/to/git/checkout/of/Parsoid

To test proper configuration, visit {$wgScriptPath}/rest.php/{$domain}/v3/page/html/Main%20Page where $domain is the hostname in your $wgCanonicalServer. (Note that production WMF servers do not expose the Parsoid REST API to the external network.)

Executar os testes

Para executar todos os testes do analisador e de Mocha: 

$ composer test

The parser tests have quite a few options now which can be listed using php bin/parserTests.php --help.

If you have the environment variable MW_INSTALL_DIR pointing to a configured MediaWiki installation, you can run some additional tests with: 

$ composer phan-integrated

Converter texto wiki simples

You can convert simple wikitext snippets from the command line using the parse.php script in the bin/ directory: 

$ echo '[[Foo]]' | php bin/parse.php

The parse script has a lot of options. php bin/parse.php --help gives you information about this.

Depurar Parsoid (para programadores)

Consulte Parsoid/Debugging para dicas de depuração.

Integração Contínua

Parsoid is always available as a library since it is a composer dependency of MediaWiki core. But two pieces are not enabled:

  • Parsoid ServiceWiring
  • Parsoid's external REST api

The test runner Quibble would enable it if it detects mediawiki/services/parsoid.git has been cloned as part of the build. In which case it:

  • points the autoloader for Wikimedia\Parsoid to the cloned code (effectively replacing the version installed by composer)
  • Carregue a extensão wfLoadExtension( 'Parsoid', '/path/to/cloned/repo' );.

The ServiceWiring should be enabled in MediaWiki starting with 1.38.

The REST API would theorically never get merged in MediaWiki: a) it has never been exposed to the public in production, it is an internal API used by RESTBase which is going away; b) it never has been security audited and c) it is redundant with the enterprise MediaWiki API. The solution will be for VisualEditor to invoke Parsoid directly via the VisualEditor Action API which would save a round trip through the REST API.

Loading the extension is thus a hack which enables using interfaces subject to change and which we don't really want people to use yet.

For most purposes, parsoid should thus not be added as a CI dependency, the only exception as of October 2021 is the Disambiguator MediaWiki extension.

Loading parsoid as an extension let us run MediaWiki integration test jobs against mediawiki/services/parsoid.git (such as Quibble, apitesting) and ensure Parsoid and MediaWiki work together.

An extension may be able to write tests with Parsoid even when the repository has not been cloned. Since it is a composer dependency of MediaWiki core the MediaWiki\Parsoid namespace is available, but the service wiring part is not (it is extension/src in the Parsoid repository and exposed as the \MWParsoid namespace). The ParsoidTestFileSuite.php code would only run the parser tests if Parsoid has been loaded (which should be the default with MediaWiki 1.38).

For CI, Parsoid is tested against the tip of MediaWiki, whereas MediaWiki is tested with the Composer dependency. In case of a breaking change, the Parsoid change get merged first (which breaks its CI but not MediaWiki one) and MediaWiki get adjusted when Parsoid is updated. It is thus a one way change.

Compilação de lançamento

For MediaWiki release builds, we have an integration of Parsoid ServiceWiring into VisualEditor in order to have VisualEditor work without further configuration (beside a wfLoadExtension( 'VisualEditor' )). The release build also enables the REST API and hook everything us so that parsoid works out of the box. This is done by copying a bit of Parsoid code into VisualEditor which is not in the master branch of VisualEditor since that would be obsolete as soon as Parsoid is updated. Instead the code is maintained in two places.

Documentos técnicos

Hiperligações para os programadores de Parsoid

Hiperligações para os programadores do Parsoid (para o cluster da Wikimédia)

História

A aplicação original foi escrita em JavaScript (utilizando Node.js) e começou a ser executada no cluster da Wikimédia em dezembro de 2012. Em 2019, Parsoid foi transferido para PHP, e a versão PHP substituiu a versão em JS no cluster da Wikimédia em dezembro de 2019. Parsoid tem sido integrado no núcleo do MediaWiki, com o objetivo de eventualmente substituir o analisador nativo atual do MediaWiki. In early 2024, Parsoid began to be used on some production wikis of the Wikimedia Foundation as the default parser for read views. You can see them in the production config list.

Parsoid (a versão em PHP) tem sido integrada nativamente no MediaWiki desde a versão 1.35, lançada em setembro de 2020. Para instalações não Wikimédia, Parsoid/JS foi suportado até ao fim da vida útil do MediaWiki 1.31 (LTS) em setembro de 2021.

FAQ

  • How do I see if a page was rendered with Parsoid?
    • The footer will say "Page was rendered with Parsoid".
  • How do I set a page to temporarily render with Parsoid?
  • How do I get an entire wiki to render with Parsoid?
    • Install Extension:ParserMigration
    • Set $wgParserMigrationEnableParsoidArticlePages = true;
    • Set $wgParserMigrationEnableParsoidDiscussionTools = true;
  • Which Wikimedia wikis render with Parsoid?
    • The list is at parsoidrendered.dblist. Em agosto de 2025, é de aproximadamente 250 das 1000 wikis. Rollout will continue until all wikis are using Parsoid.

Consultar também

Hiperligações externas

Contacto

Se precisa de ajuda ou tem perguntas/comentários, pode contactar-nos em #mediawiki-parsoid connect ou na lista de discussão de wikitext-l. Se tudo isto falhar, também pode contactar-nos por correio eletrónico em content-transform-team no domínio wikimedia.org.

Retrieved from "https://www.mediawiki.org/w/index.php?title=Parsoid/pt&oldid=8601643"