From 71818067c5e7a9cbf4316ba5d7602cba1625f12f Mon Sep 17 00:00:00 2001 From: Jocelyn Delalande Date: Sat, 21 Oct 2017 14:26:01 +0200 Subject: [PATCH 1/2] Make the i18n README more clear MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This README.md is the only documentation for i18n setup, let's enhance it. Modifications include: - make the writing style less personal - fix markdown (syntax, headings…) - clearly separate the usage guide from the known bugs and future perspectives (as this guide is now in master branch, it is no longer relevant to say that it is an experimental thing) - rewordings --- notebook/i18n/README.md | 117 ++++++++++++++++++++-------------------- 1 file changed, 60 insertions(+), 57 deletions(-) diff --git a/notebook/i18n/README.md b/notebook/i18n/README.md index e439142574..20ef6737c6 100644 --- a/notebook/i18n/README.md +++ b/notebook/i18n/README.md @@ -1,88 +1,84 @@ # Implementation Notes for Internationalization of Jupyter Notebook -This is a prototype implementation of i18n features for Jupyter notebook, and should not -yet be considered ready for production use. I have tried to focus on the public user -interfaces in the notebook for the first cut, while leaving much of the console messages -behind, as their usefulness in a translated environment is questionable at best. +The implementation of i18n features for jupyter notebook is still a work-in-progress: -### Using a prototype translated version +- User interface strings are (mostly) handled +- Console messages are not handled (their usefulness in a translated environment is questionable) +- Tooling has to be refined -In order to use this preliminary version, you need to do things after installing the -notebook as normal: +However… -1. Set the LANG environment variable in your shell to "xx_XX" or just "xx". -where "xx" is the language code you're wanting to run in. If you're -running on Windows, I've found the easiest way to do this is to use Windows PowerShell, -and run the command: +## How the language is selected ? -`${Env:LANG} = "xx_XX"` +1. `jupyter notebook` command reads the `LANG` environment variable at startup, +(`xx_XX` or just `xx` form, where `xx` is the language code you're wanting to +run in). -2. Set the preferred language for web pages in your browser to YourLanguage (xx). At the moment, -it has to be first in the list. +Hint: if running Windows, you can set it in PowerShell with `${Env:LANG} = "xx_XX"`. -3. Run the `jupyter notebook` command to start the notebook. +2. The preferred language for web pages in your browser settings (`xx`) is + also used. At the moment, it has to be first in the list. -### Message extraction: +## Contributing and managing translations -I have split out the translatable material for the notebook into 3 POT, as follows: +### Requirements -notebook/i18n/notebook.pot - Console and startup messages, basically anything that is - produced by Python code. - -notebook/i18n/nbui.pot - User interface strings, as extracted from the Jinja2 templates - in notebook/templates/*.html - -noteook/i18n/nbjs.pot - JavaScript strings and dialogs, which contain much of the visible - user interface for Jupyter notebook. - -To extract the messages from the source code whenever new material is added, use the -`pybabel` command to extract messages from the source code as follows: -( assuming you are in the base directory for Jupyter notebook ) +- *pybabel* (could be installed `pip install pybabel`) +- *po2json* (could be installed with `npm install -g po2json`) -`pybabel extract -F notebook/i18n/babel_notebook.cfg -o notebook/i18n/notebook.pot --no-wrap --project Jupyter .` -`pybabel extract -F notebook/i18n/babel_nbui.cfg -o notebook/i18n/nbui.pot --no-wrap --project Jupyter .` -`pybabel extract -F notebook/i18n/babel_nbjs.cfg -o notebook/i18n/nbjs.pot --no-wrap --project Jupyter .` +### Message extraction -(Note: there is a '.' at the end of these commands, and it has to be there...) +The translatable material for notebook is split into 3 `.pot` files, as follows: -After this is complete you have 3 POT files that you can give to a translator for your favorite language. -Babel's documentation has instructions on how to integrate this into your setup.py so that eventually -we can just do: +- *notebook/i18n/notebook.pot* - Console and startup messages, basically anything that is + produced by Python code. +- *notebook/i18n/nbui.pot* - User interface strings, as extracted from the Jinja2 templates + in *notebook/templates/\*.html* +- *noteook/i18n/nbjs.pot* - JavaScript strings and dialogs, which contain much of the visible + user interface for Jupyter notebook. -`setup.py extract_messages` +To extract the messages from the source code whenever new material is added, use the +`pybabel` command (assuming you are in the base directory for Jupyter notebook): -I hope to get this working at some point in the near future. +```shell +pybabel extract -F notebook/i18n/babel_notebook.cfg -o notebook/i18n/notebook.pot --no-wrap --project Jupyter . +pybabel extract -F notebook/i18n/babel_nbui.cfg -o notebook/i18n/nbui.pot --no-wrap --project Jupyter . +pybabel extract -F notebook/i18n/babel_nbjs.cfg -o notebook/i18n/nbjs.pot --no-wrap --project Jupyter . +``` -### Post translation procedures +After this is complete you have 3 `.pot` files that you can give to a translator for your favorite language. -After the source material has been translated, you should have 3 PO files with the same base names -as the POT files above. Put them in `notebook/i18n/${LANG}/LC_MESSAGES`, where ${LANG} is the language -code for your desired language ( i.e. German = "de", Japanese = "ja", etc. ). The first 2 files then -need to be converted from PO to MO format for use at runtime. There are many different ways to do -this, but pybabel has an option to do this as follows: +### Messages compilation -`pybabel compile -D notebook -f -l ${LANG} -i notebook/i18n/${LANG}/LC_MESSAGES/notebook.po -o notebook/i18n/${LANG}/notebook.mo` +After the source material has been translated, you should have 3 `.po` files with the same base names +as the `.pot` files above. Put them in `notebook/i18n/${LANG}/LC_MESSAGES`, where `${LANG}` is the language +code for your desired language ( i.e. German = "de", Japanese = "ja", etc. ). -`pybabel compile -D nbui -f -l ${LANG} -i notebook/i18n/${LANG}/LC_MESSAGES/nbui.po -o notebook/i18n/${LANG}/nbui.mo` +*notebook.po* and *nbui.po* need to be converted from `.po` to `.mo` format for +use at runtime. -The nbjs.po needs to be converted to JSON for use within the JavaScript code. I'm using po2json for this, as follows: +```shell +pybabel compile -D notebook -f -l ${LANG} -i notebook/i18n/${LANG}/LC_MESSAGES/notebook.po -o notebook/i18n/${LANG}/notebook.mo +pybabel compile -D nbui -f -l ${LANG} -i notebook/i18n/${LANG}/LC_MESSAGES/nbui.po -o notebook/i18n/${LANG}/nbui.mo +``` -`po2json -p -F -f jed1.x -d nbjs notebook/i18n/${LANG}/LC_MESSAGES/nbjs.po notebook/i18n/${LANG}/LC_MESSAGES/nbjs.json` +*nbjs.po* needs to be converted to JSON for use within the JavaScript code, with *po2json*, as follows: -The conversions from PO to MO probably can and should be done during setup.py. + po2json -p -F -f jed1.x -d nbjs notebook/i18n/${LANG}/LC_MESSAGES/nbjs.po notebook/i18n/${LANG}/LC_MESSAGES/nbjs.json -When new languages get added, their language codes should be added to notebook/i18n/nbjs.json -under the "supported_languages" element. +When new languages get added, their language codes should be added to *notebook/i18n/nbjs.json* +under the `supported_languages` element. ### Tips for Jupyter developers The biggest "mistake" I found while doing i18n enablement was the habit of constructing UI messages from English "piece parts". For example, code like: +```javascript +var msg = "Enter a new " + type + "name:" +``` -`var msg = "Enter a new " + type + "name:"` - -where "type" is either "file", "directory", or "notebook".... +where `type` is either "file", "directory", or "notebook".... is problematic when doing translations, because the surrounding text may need to vary depending on the inserted word. In this case, you need to switch it and use complete phrases, @@ -104,9 +100,9 @@ create the plural form doesn't translate well. Some languages have as many as 5 plural forms for differing numbers, so using an API such as ngettext() is necessary in order to handle these cases properly. -### Known issues +### Known issues and future evolutions -1. Right now there are two different places where the desired language is set. At startup time, the Jupyter console's messages pay attention to the setting of the ${LANG} environment variable +1. Right now there are two different places where the desired language is set. At startup time, the Jupyter console's messages pay attention to the setting of the `${LANG}` environment variable as set in the shell at startup time. Unfortunately, this is also the time where the Jinja2 environment is set up, which means that the template stuff will always come from this setting. We really want to be paying attention to the browser's settings for the stuff that happens in the @@ -117,6 +113,13 @@ of languages in the UI ( never a good thing ). 2. We will need to decide if console messages should be translatable, and enable them if desired. 3. The keyboard shorcut editor was implemented after the i18n work was completed, so that portion does not have translation support at this time. +4. Babel's documentation has instructions on how to integrate messages extraction +into your *setup.py* so that eventually we can just do: -Any questions or comments please let me know @JCEmmons on github (emmo@us.ibm.com) + ./setup.py extract_messages +I hope to get this working at some point in the near future. +5. The conversions from `.po` to `.mo` probably can and should be done using `setup.py install`. + + +Any questions or comments please let me know @JCEmmons on github (emmo@us.ibm.com) From bd48b4235fc520a573842c897a5c343fa1d86ec1 Mon Sep 17 00:00:00 2001 From: Jocelyn Delalande Date: Sat, 21 Oct 2017 14:33:52 +0200 Subject: [PATCH 2/2] Use local paths for i18n-related commands Make the commands listings shorter and easier to focus on what matters. --- notebook/i18n/README.md | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/notebook/i18n/README.md b/notebook/i18n/README.md index 20ef6737c6..cbe00d8dab 100644 --- a/notebook/i18n/README.md +++ b/notebook/i18n/README.md @@ -26,6 +26,10 @@ Hint: if running Windows, you can set it in PowerShell with `${Env:LANG} = "xx_X - *pybabel* (could be installed `pip install pybabel`) - *po2json* (could be installed with `npm install -g po2json`) +**All i18n-related commands are done from the related directory :** + + cd notebook/i18n/ + ### Message extraction The translatable material for notebook is split into 3 `.pot` files, as follows: @@ -38,12 +42,12 @@ The translatable material for notebook is split into 3 `.pot` files, as follows: user interface for Jupyter notebook. To extract the messages from the source code whenever new material is added, use the -`pybabel` command (assuming you are in the base directory for Jupyter notebook): +`pybabel` command: ```shell -pybabel extract -F notebook/i18n/babel_notebook.cfg -o notebook/i18n/notebook.pot --no-wrap --project Jupyter . -pybabel extract -F notebook/i18n/babel_nbui.cfg -o notebook/i18n/nbui.pot --no-wrap --project Jupyter . -pybabel extract -F notebook/i18n/babel_nbjs.cfg -o notebook/i18n/nbjs.pot --no-wrap --project Jupyter . +pybabel extract -F babel_notebook.cfg -o notebook.pot --no-wrap --project Jupyter . +pybabel extract -F babel_nbui.cfg -o nbui.pot --no-wrap --project Jupyter . +pybabel extract -F babel_nbjs.cfg -o nbjs.pot --no-wrap --project Jupyter . ``` After this is complete you have 3 `.pot` files that you can give to a translator for your favorite language. @@ -58,13 +62,13 @@ code for your desired language ( i.e. German = "de", Japanese = "ja", etc. ). use at runtime. ```shell -pybabel compile -D notebook -f -l ${LANG} -i notebook/i18n/${LANG}/LC_MESSAGES/notebook.po -o notebook/i18n/${LANG}/notebook.mo -pybabel compile -D nbui -f -l ${LANG} -i notebook/i18n/${LANG}/LC_MESSAGES/nbui.po -o notebook/i18n/${LANG}/nbui.mo +pybabel compile -D notebook -f -l ${LANG} -i ${LANG}/LC_MESSAGES/notebook.po -o ${LANG}/notebook.mo +pybabel compile -D nbui -f -l ${LANG} -i ${LANG}/LC_MESSAGES/nbui.po -o ${LANG}/nbui.mo ``` *nbjs.po* needs to be converted to JSON for use within the JavaScript code, with *po2json*, as follows: - po2json -p -F -f jed1.x -d nbjs notebook/i18n/${LANG}/LC_MESSAGES/nbjs.po notebook/i18n/${LANG}/LC_MESSAGES/nbjs.json + po2json -p -F -f jed1.x -d nbjs ${LANG}/LC_MESSAGES/nbjs.po ${LANG}/LC_MESSAGES/nbjs.json When new languages get added, their language codes should be added to *notebook/i18n/nbjs.json* under the `supported_languages` element.