Content translation/V2 - MediaWiki

Content translation version 2 (CX2) is a major refactoring and architectural update of Content translation (CX, or CX1 in this document). The goal is to provide a solid and reliable translation tool that is aligned with the Wikimedia standards in technology and design, and provides a great way to contribute for newcomers.

Version 2 uses the VisualEditor editing surface, an OOUI based front end, and follows the Wikimedia design guidelines.

In addition, learnings from existing and new research on the experience of new editors will be used to identify improvements to make translation a great way to start contributing to Wikipedia. The plan is to gradually replace version 1 with version 2 in several stages . A backwards compatibility plan will make sure that content created by users during the transition period won't be affected.

The new version is now the default, you can just access the tool from Special:ContentTranslation from Wikipedia in any language. When you start a new translation with Content translation you will be using the new version. Note that the previous version will be still used when opening old translations that you started with such version.

The new version is still in active development. Please, use it to translate articles and report feedback. We love to hear what works for you, and what still needs improvement.

Using version 2 on production will create content in real wikis when you publish your translation, so it is not suited for experiments that don't create a quality translation as a result. For more experimental testing, you can try the new feature in our testing servers. They are a separate wiki (you need to create a new user account since the log-in service is not integrated with Wikimedia projects). Although the content you translate in the test servers comes from real Wikipedias, the published content will be only created in the test server. This allows to experiment without interfering with the work done in those projects.

We are interested in hearing how well the new version works for both new and existing users of Content Translation.

In addition, articles published with the new version are marked with the #contenttranslation-v2 edit tag to facilitate finding them (e.g., using Recent Changes) and evaluate the quality of the content.

The new version will include a more powerful editing surface, which will bring new possibilities that were repeatedly requested by translators using the tool. However, other features from version 1 won't be initially available with the new version.

A general CX2 roadmap describes the planned interventions in more detail.

The main areas of intervention are:

• Align with the Wikimedia standards in technology and design
  • Visual Editor's editing surface with more editing tools to insert and edit templates, tables, multimedia, categories, etc.
  • Reliable undo/redo support.
  • UI revamp based on UI Standardisation initiative and OOUI components
• Quality control mechanisms. Control user modifications in more detail to encourage translators to create quality content.
• A great way to contribute for newcomers
  • Machine translation support for Template params, reference texts and practically all kind of elements in screen. In version 1, machine translation was limited to paragraphs alone.
  • Better support for References and Templates
  • Ability to add and remove categories
• Solid and reliable
  • Fixing lots of bugs that was too difficult to handle with previous version

The features listed above are possible with the new technology architecture. However, in order to be able to deliver those improvements soon, we have to limit the efforts of rewriting all the existing tools CX1 has. Thus, some of the existing tools won't be available in CX2 initially. We selected those based on our observations of current use, the value they provide in version 1 and their complexity, but we are looking for your feedback during this process.

These are the tools currently in CX1 that will be missing initially for CX2:

• Custom template translation editor. CX1 added support for a side-by-side editor of templates that allowed translators to map their parameters. The initial implementation allowed to evaluate a promising concept but it was far from being complete, and rewriting this for CX2 will require significant effort. Initially, the standard template editor dialog provided by Visual Editor will be available in CX2 instead. Although it is not optimized for transferring information across languages, it provides basic support for editing all parameters of a template in the translation.
• Dictionaries. CX1 had experimental support for dictionary information lookup for a few language pairs. Dictionaries are a very relevant tool for translators, and we'll keep track the progress of Wikimedia projects in this area that will enable their integration in the future. However, providing support for CX2 makes more sense when there is a clear plan to integrate more dictionaries.
• Progress indicator in the editor. A progress bar showed in CX1 how much of the article was translated and how much was missing. This information will be still visible from the dashboard, but not while editing the article. Based on our observations from users, having it on the editor was not providing much value.
• Announcements of new machine translation services. The automatic translation card became highlighted when a new machine translation service was made available for the current language. This was especially useful in the initial stages of the tool, where new services were added regularly. We can reconsider this feature once the migration to version 2 is completed, and we plan to integrate new machine translation services in the future.

Content Translation was developed iteratively for last 2+ years. During that time, the focus was to evaluate the core ideas on how to improve the translation experience for Wikipedia editors. The architecture was a flexible one where modules can be plugged and try these concepts. This allowed to move fast, but the approach and cut corners affected the code organisation, maintainability and reliability of the tool. The proposed refactoring and architectural update will contribute to provide a tool solid and reliable translation tool that is aligned with the Wikimedia standards in technology and design.

At the end of this intervention we want Content Translation to be a tool that:

• Is aligned with the Wikimedia standards in technology and design. Uses the editing surface technology of Visual Editor (VE), and follows the Design style guide principles.
• Is a great way to contribute for newcomers. The tool provides a quick and easy way for new editors to start contributing. Even if the tool does not support dealing with complex content or situations, it always provides a clear path forward for new editors.
• Is solid and reliable. The tool is reliable enough to go out of beta for at least one community.

The way to get there is detailed in different plans below.

Starting in February 2018, the CX2 roadmap defines the incremental stages to complete the development of the tool.

A rough plan is to enable version 2 in smaller wikis or subset of wikis to do QA and gradually rollout to more wikis. The list of representative wikis can be useful to identify candidates.

Versions 1 and 2 will coexist during a transition period. Given that the translations each version produce are not expected to be compatible, the following steps are considered to avoid issues related to breaking backwards compatibility:

1. Translations started with one version of the editor will be always opened with the same version, regardless of which is the current default editor. That is, when version 2 is the default, old translations started with the version1 will still be opened with version 1.
2. Once version 2 is considered the stable default, creating new translations with older versions will be prevented. That is, version 1 will not be available to create new translations, but it will be still available to edit the old ones.
3. With a process in place to automatically discard translations after one year, version 1 could be safely removed after such period pases since no new articles can be started with it.

For more recent activity related to Content translation (from July 2019 to nowadays), please check the Translation Boost Initiative.

Below are the archives prior to November 2019 (...and yes, both pages are overlapping between July 2019 and October 2019).

• The new version is the only version available for new translations, even when started through the URL. Old translations started with version 1 will still use it for backwards compatibility.
• Improved support for references by not translating reference contents automatically since information such as book titles is often kept untranslated, and avoid warnings insisting to translate the contents of the reference list.
• Support for standard visual components on the new translation dialog.
• Better guidance by communicating that publishing is only allowed to experienced editors for wikis configured in such way. Configuration has not been applied to any yet, and should make obsolete the use of abuse filters that some communities implemented.
• Better guidance by providing more detailed publish options in a dialog and more detailed information when users are not allowed to publish translations with too much unmodified contents.
• Improved persistency for templates by preventing user changes to get lost across sessions.
• Better support for exploring links in the source document.
• Performance improvements by reducing the number of code modules delivered to users, and regressions corrected for placeholder icons, and for event logging of suggestions.
• Better user guidance by communicating the English Wikipedia limitation to only allow publishing to extended confirmed users in a more detailed way and before the user makes the translation effort.
• Better control to start a translation by allowing to select pages from different namespaces, not just the main (article) namespace. Some namespaces such as file, gadget, template or talk are still excluded.
• More reliable process for correctly restoring and publishing translations that start with an empty paragraph.
• More reliable visual alignment of paragraphs between original and translated content by adapting to paragraphs becoming shorter and fixing regressions.
• More precise calculation of the percentage of unmodified contents for the whole document, and custom adjustments on a per wiki basis.
• Better connection between warnings and the affected content by using color icons in the issue messages that match the warning or error marks in content.
• More reliable editing support by recalculating the editing area size to elements that take time to load which prevent unnecessary scrollbars.
• Cleaner process for starting a new translation by reducing the terms of use text, and adjusting the edit area spacing for the translation title.

• Improved performance by prefetching requests to translation services in advance to deliver results immediately when users add a new paragraph, caching of API requests and an optimization of the caching system.
• More reliable saving process by retrying with an increased timeout after initial failure.
• More solid approach to divide the content into sections for the tool to handle, and preventing errors when adding sections to the translation.
• Improved support for inserting media, preventing the media dialog to block the article.
• Better support for references solving issues when editing references and displaying them.
• Improved control for excessive unmodified content by excluding types of content where the warning is not relevant in order to prevent false positives.
• Better support for red links, generating as red links only for the missing articles the user confirmed, and avoid rendering them as regular links in the source document.
• Better guidance by delaying content warnings only while the user is editing but showing previous warnings immediately when a translation is resumed.
• Improved alignment on RTL languages and a cleaner source article by avoiding noise characters showing from Visual Editor elements.
• More solid control of anonymous access by gracefully handling access errors.
• More reliable saving and recovering process to persist the contents when user switches between translation services, and avoid moving translations to in-progress when there are no further changes since they were published.
• Better guidance by communicating when templates could not be adapted.
• Better mapping references across languages by using Citoid metadata when mapping template parameters, avoiding issues with empty references and partially adapted ones.
• More effective integration with translation services by sending a more compact version of the content HTML to prevent exceeding the translation limits, improving efficiency and security. A regression in this area was also fixed.
• Facilitate the access to the new version by creating an outreach campaign link to be used in interactions with Wikipedia communities.
• Instrumentation improvements to better measure the issues translators experience.
• More reliable application of machine translation by avoiding quick manual modifications to be counted as machine translation, improving support for quote templates by avoiding blank text blocks to be sent for translation, avoid the adaptation of transclusion fragments, improved logging for Apertium, and better handling of templates and link attributes.
• More reliable visual alignment of paragraphs between original and translated content.

• Support for Captcha confirmation when target wikis request it.
• Layout adjustments to keep article titles visually aligned, make visual alignment reliable when resizing the browser window, and avoid menus to hide behind the main content.
• Showing a clean source article by removing irrelevant sections and hidden categories.
• Definition of success metrics: number of newcomers, percentage of successful completion, number of errors, and translation survival.
• Support for links with new link cards allowing to preview their content in both languages and manipulate links, support to search for links in the target wiki, and better control for text input cursor placement when editing links.
• Use of updated API for short descriptions which allows local wikis to override Wikidata default descriptions.
• Starting translations from a link is more fluent by automatically selecting the article to translate.
• More reliable saving by including source information, and language selection to avoid duplicates.
• Improved testability: editor version is persistent when navigating from dashboard, and captcha support added in the test environment.
• Automatic translation card with options to select the Machine Translation provider from those available, copying the source text or starting from scratch. Modifications on the original automatic translations are kept persistent with an option to reset them.
• Improving the persistence of categories and solving issues with missing categories.
• Improvements in code modularization, API parameter consistency, and regressions in the visual alignment of paragraphs.