==================================
Roundup Internationalization Notes
==================================

:Version: $Revision: 1.8 $

Overview
--------

How stuff works:

1. Strings that may require translation (messages in human language)
   are marked in the source code.  This step is discussed in
   `Marking Strings for Translation`_ section.

2. These strings are all extracted into Message Template File
   ``locale/roundup.pot`` (POT file).  See `Extracting Translatable
   Messages`_ below.

3. Language teams use POT file to make Message Files for national
   languages (PO files).  All PO files for Roundup are kept in
   the ``locale`` directory.  Names of these files are target
   locale names, usually just 2-letter language codes.  `Translating
   Messages`_ section of this document gives useful hints for
   message translators.

4. Translated Message Files are compiled into binary form (MO files)
   and stored in ``locale`` directory (but not kept in the `Roundup
   CVS`_ repository, as they may be easily made from PO files).
   See `Compiling Message Catalogs`_ section.

5. Roundup installer creates runtime locale structure on the file
   system, putting MO files in their appropriate places.

6. Runtime internationalization (I18N) services use these MO files
   to translate program messages into language selected by current
   Roundup user.  Roundup command line interface uses locale name
   set in OS environment variable ``LANGUAGE``, ``LC_ALL``,
   ``LC_MESSAGES``, or ``LANG`` (in that order).  Roundup Web User
   Interface uses language selected by currentluy authenticated user.

Additional details may be found in `GNU gettext`_ and Python `gettext
module`_ documentation.

`Roundup source distribution`_ includes POT and PO files for message
translators, and also pre-built MO files to facilitate installations
from source.  `Roundup binary distribution`_ includes MO files only.

.. _GNU gettext:

GNU gettext package
-------------------

This document is full of references to `GNU`_ `gettext package`_.
GNU gettext is a "must have" for nearly all steps of internationalizing
any program, and it's manual is definetely a recommended reading
for people involved in I18N.

There are GNU gettext ports to all major OS platforms.
Windows binaries are available from ``GNU mirror sites``.

Roundup does not use GNU gettext at runtime, but it's tools
are used for `extracting translatable messages`_, `compiling
message catalogs`_ and, optionally, for `translating messages`_.

Note that ``gettext`` package in some OS distributions (e.g.
`FreeBSD`_) means just runtime libraries.  Development tools
are usually distributed in separate package named ``gettext-devel``.

Marking Strings for Translation
-------------------------------

Strings that need translation must be marked in the source code.
Following sections of this chapter explain how this is done in
different cases.

Current status of the string marks in Roundup core files is shown
in `I18 Status`_ section at the end of this document.

If translatable string is used as a format string, it is recommended
to always use named format specifiers::

  _('Index of %(classname)s') % locals()

This helps translators to better understand the context of the
message and, in Python, remove format specifier altogether (which
is sometimes useful, especially in singular cases of `Plural Forms`_).
When there is more than one format specifier in the translatable
format string, named format specifiers *must* be used almost always,
because translation may require different order of items.

It is better to *not* mark for translation strings that are not
locale-dependent, as this makes it more difficult to keep track
of translation completeness.  For example: following string in
``index()`` method of the request handler in ``roundup_server``
script::

  </ol></body></html>

has no human readable parts at all, and needs no translations.
These strings are left untranslated in PO files, and are reported
as such by PO status checkers (e.g. ``msgfmt --statistics``).

Command Line Interfaces
~~~~~~~~~~~~~~~~~~~~~~~

Scripts and routines run from the command line use "static" language
defined by environment variables recognized by ``gettext`` module
from Python library (``LANGUAGE``, ``LC_ALL``, ``LC_MESSAGES``, and
``LANG``).  Primarilly, these are ``roundup-admin`` script and
``admin.py`` module, but also help texts and startup error messages
in other scripts and their supporting modules.

For these interfaces, Python ``gettext`` engine must be initialized
to use Roundup message catalogs.  This is normally done by including
the following line in the module imports::

  from i18n import _, ngettext

Simple translations are automatically marked by calls to builtin
message translation function ``_()``::

  print _("This message is translated")

Translations for messages whose grammatical depends on a number
must be done by ``ngettext()`` function::

  print ngettext("Nuked %i file", "Nuked %i files", number_of_files_nuked)

User Interfaces
~~~~~~~~~~~~~~~

*(not yet)*

This includes Mail Gateway and Web User Interfaces, where translation
depends on the language of current Roundup User.  These translations
will be done by the tracker configuration object.  Translatable strings
will be automatically marked by calls to the ``_()`` and ``ngettext()``
methods of that object::

  self.config._("This message is translated")
  self.config.ngettext("Nuked %i file", "Nuked %i files",
      number_of_files_nuked)

Deferred Translations
~~~~~~~~~~~~~~~~~~~~~

Sometimes translatable strings appear in the source code in untranslated
form [#note_admin.py]_ and must be translated elsewhere.
Example::

  for meal in ("spam", "egg", "beacon"):
      print _(meal)

In such cases, strings must be marked for translation without actual
call to the translating function.  To mark these strings, we use Python
feature of automatical concatenation of adjacent strings and different
types of string quotes::

  strings_to_translate = (
      ''"This string will be translated",
      ""'me too',
      ''r"\raw string",
      ''"""
      multiline string"""
  )

.. [#note_admin.py] In current Roundup sources, this feature is
   extensively used in the ``admin`` module using method docstrings
   as help messages.

Extracting Translatable Messages
--------------------------------

The most common tool for message extraction is ``xgettext`` utility
from `GNU gettext package`_.  Unfortunately, this utility has no means
of `Deferred Translations`_ in Python sources.
There is ``xpot`` tool from Francois Pinard free `PO utilities`_
that allows to mark strings for `deferred translations`_, but
it does not handle `plural forms`_.
Roundup overcomes these limitations by using both of these utilities.
This means that you need both `GNU gettext`_ tools and `PO utilities`_
to build the Message Template File yourself.

Latest Message Template File is kept in `Roundup CVS`_ and distributed
with `Roundup Source`_.
If you wish to rebuild the template yourself, make sure that you have
both ``xpot`` and ``xgettext`` installed and just run ``gmake`` (or
``make``, if you are on a `GNU`_ system like `linux`_ or `cygwin`_)
in the ``locale`` directory.

Translating Messages
--------------------

FIXME!

Compiling Message Catalogs
--------------------------

FIXME!

I18 Status
----------

This list has been generated using the MANIFEST file. We should be able to
write a simple script to compare the two and make sure that all MANIFEST
files appear in here.

This list was last updated Sat Jan 5 02:35:10 2002 UTC.

THESE FILES DO NOT USE _()
~~~~~~~~~~~~~~~~~~~~~~~~~~
roundup/hyperdb.py
roundup/i18n.py
roundup/init.py
roundup/install_util.py
roundup/instance.py
roundup/mailgw.py
roundup/password.py
roundup/roundupdb.py
roundup/templatebuilder.py
roundup/backends/__init__.py
roundup/backends/back_anydbm.py
roundup/backends/back_bsddb.py
roundup/backends/back_bsddb3.py
roundup/templates/__init__.py
roundup/templates/classic/__init__.py
roundup/templates/classic/dbinit.py
roundup/templates/classic/htmlbase.py
roundup/templates/classic/config.py
roundup/templates/classic/interfaces.py
roundup/templates/classic/detectors/__init__.py
roundup/templates/classic/detectors/nosyreaction.py


THESE FILES DO USE _()
~~~~~~~~~~~~~~~~~~~~~~
roundup-admin
roundup-mailgw
roundup-server
cgi-bin/roundup.cgi
roundup/__init__.py
roundup/admin.py
roundup/date.py
roundup/cgi/cgitb.py


WE DON'T CARE ABOUT THESE FILES
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
BUILD.txt
CHANGES.txt
INSTALL.txt
README.txt
setup.py
doc/implementation.txt
doc/index.html
doc/overview.html
doc/spec.html
doc/images/edit.png
doc/images/hyperdb.png
doc/images/logo-acl-medium.png
doc/images/logo-codesourcery-medium.png
doc/images/logo-software-carpentry-standard.png
doc/images/roundup-1.png
doc/images/roundup.png
roundup/templates/classic/html/file.index
roundup/templates/classic/html/file.newitem
roundup/templates/classic/html/issue.filter
roundup/templates/classic/html/issue.index
roundup/templates/classic/html/issue.item
roundup/templates/classic/html/msg.index
roundup/templates/classic/html/msg.item
roundup/templates/classic/html/style.css
roundup/templates/classic/html/user.index
roundup/templates/classic/html/user.item
test/README.txt
test/__init__.py
test/test_dates.py
test/test_db.py
test/test_init.py
test/test_mailsplit.py
test/test_multipart.py
test/test_schema.py
test/test_templating.py
test/unittest.py

.. External hyperlink targets

.. _cygwin: http://www.cygwin.com/
.. _emacs: http://www.gnu.org/software/emacs/
.. _gettext package: http://www.gnu.org/software/gettext/
.. _gettext module: http://docs.python.org/lib/module-gettext.html
.. _GNU: http://www.gnu.org/
.. _GNU mirror sites: http://www.gnu.org/prep/ftp.html
.. _FreeBSD: http://www.freebsd.org/
.. _linux: http://www.linux.org/
.. _Plural Forms:
    http://www.gnu.org/software/gettext/manual/html_node/gettext_150.html
.. _po filetype plugin:
    http://vim.sourceforge.net/scripts/script.php?script_id=695
.. _PO utilities: http://po-utils.progiciels-bpi.ca/
.. _poEdit: http://poedit.sourceforge.net/
.. _Roundup CVS: http://sourceforge.net/cvs/?group_id=31577
.. _Roundup Source:
.. _Roundup source distribution:
.. _Roundup binary distribution:
    http://sourceforge.net/project/showfiles.php?group_id=31577