Add Localisation Support

Programming applications for making music on Linux.

Moderators: MattKingUSA, khz

Post Reply
tramp
Established Member
Posts: 2335
Joined: Mon Jul 01, 2013 8:13 am
Has thanked: 9 times
Been thanked: 454 times

Add Localisation Support

Post by tramp »

Adding Localisation support to your application isn't to hard on Linux systems. The tool of choice is gettext, which is installed on default on nearly all Linux distributions.
At first you need to include the following headers:

Code: Select all

#include <libintl.h>
#include <locale.h>
next step is to define a macro to reduce the needed work.

Code: Select all

#define _(S) gettext(S)
Now you need to init the gettext system in your main call like this:

Code: Select all

int main (int argc , char ** argv) {

    bind_textdomain_codeset(GETTEXT_PACKAGE, "UTF-8");
    textdomain(GETTEXT_PACKAGE);
were the macro GETTEXT_PACKAGE is your application name, you could define it in your config.h file, or put it in cleartext here.

Well, that's all, you've added localisation support in your app already. :)
But, that's just the support, you could build and run your app, but no magic translation will happen. A bit more work is needed. :(

You need to define in your source, which strings may be translated, to do so, we've defined the MACRO on top.
Now, put the strings you would translate in the clamps _() like this:

Code: Select all

set_title(_("This string will be to translate"));
Do that for all strings you would add translation support for.

When done, you could produce the .pot file for your app with the following command:

Code: Select all

xgettext --keyword=_ --language=C++ --add-comments --sort-output --package-name=$(NAME) --package-version=$(VER) -o po/$(NAME).pot sourcefilename.cpp sourcefilename1.cpp etc.cpp
here replace $(NAME) and $(VER) with your app name and version.
You may need to add the /po directory beforehand.

now you could generate the .po from the .pot file which then could be used to hold the translation.

Code: Select all

msginit --input=po/$(NAME).pot --locale=$(LANG) --output=po/$(LANG).po
This command will produce a po file for your locale settings, which you could open with, for example Gtranslator.

You properly wont to automate the po file generation so that supporters could easily generate the po file for there language, so here are some targets for your makefile, you could easily adapt them for your build system, if you use a other one:

Code: Select all

   #@localisation
MSGLANGS=$(notdir $(wildcard po/*po))
MSGOBJS=$(addprefix locale/,$(MSGLANGS:.po=/LC_MESSAGES/$(NAME).mo))

gettext: $(MSGOBJS)

locale/%/LC_MESSAGES/$(NAME).mo: po/%.po
	mkdir -p $(dir $@)
	msgfmt -c -o $@ po/$*.po

updatepot:
	xgettext --keyword=_ --language=C++ --add-comments --sort-output --package-name=$(NAME) --package-version=$(VER) -o po/$(NAME).pot $(NAME).cpp
	for POFILE in $(MSGLANGS) ; do msgmerge --update po/$$POFILE po/$(NAME).pot ; done

po:
	msginit --input=po/$(NAME).pot --locale=$(LANG) --output=po/$(basename $(LANG)).po
EDIT// remove utf8 suffix from po file name for better integration.
the po target must be declared as .PHONY
with this targets you could produce the po file by:

Code: Select all

make po 
or, produce it for a other language then your locale, use

Code: Select all

make po LANG=your_choice
check for available locales with

Code: Select all

locale -a
At least you need to add it to your installation target, you could do so by adding

Code: Select all

cp --parents $(MSGOBJS)  $(DESTDIR)$(SHARE_DIR)
where you clearly needs to define SHARE_DIR

That's it, now translation teams or supporters could easily add there language support to your application. :wink:
https://github.com/brummer10/jalv_selec ... ISATION.md
Last edited by tramp on Sat Mar 31, 2018 8:25 am, edited 1 time in total.
On the road again.
Lyberta
Established Member
Posts: 681
Joined: Sat Nov 01, 2014 8:15 pm
Location: The Internet
Been thanked: 1 time

Re: Add Localisation Support

Post by Lyberta »

So it uses raw english text as a key? Feels horrible. What about format strings?
tramp
Established Member
Posts: 2335
Joined: Mon Jul 01, 2013 8:13 am
Has thanked: 9 times
Been thanked: 454 times

Re: Add Localisation Support

Post by tramp »

Lyberta wrote:So it uses raw english text as a key? Feels horrible.
Sorry, I ain't understand what you mean, you could use any string for translation you like. If it is German or polish in the beginning doesn't matter. Just, it needs to be a string. The key is the clamp _("put here what ever you like to translate, in what ever language you prefer to wrote initially")
Lyberta wrote:What about format strings?
There is no problem with formatted strings. For example

Code: Select all

fprintf(stderr, _("X Error:  %s\n Global HotKey disabled\n"), buffer1);
could be translated easily, including keep the formatting of the string. If you would translate the var buffer1 as well, you need to translate it were it get filled with chars.
The verry same is true for boost formatted

Code: Select all

boost::format(_("couldn't rename %1% to %2%"))
				      % tmpfile % filename);
or Glib::ustring

Code: Select all

Glib::ustring::compose(_("parameter %1: value %2 out of range [%3, %4]"),
			       _id, value, lower, upper));
No problem.
On the road again.
Lyberta
Established Member
Posts: 681
Joined: Sat Nov 01, 2014 8:15 pm
Location: The Internet
Been thanked: 1 time

Re: Add Localisation Support

Post by Lyberta »

tramp wrote:Sorry, I ain't understand what you mean, you could use any string for translation you like.
If you change the string in original source code, the old translations files break. It is better to use some kind of ID.
tramp wrote:There is no problem with formatted strings.
So it is agnostic to format strings. Then the usefulness of gettext by itself gets diminished. I feel like other i18n solutions offer such thing out-of-the-box.
tramp
Established Member
Posts: 2335
Joined: Mon Jul 01, 2013 8:13 am
Has thanked: 9 times
Been thanked: 454 times

Re: Add Localisation Support

Post by tramp »

Lyberta wrote:If you change the string in original source code, the old translations files break. It is better to use some kind of ID.
Ah, okay, that is true, the translation for this string will fail then. Just, I'm not sure if that is a dis-advance, properly you would change the translation as well, when you change the original string. So properly you would use a new ID as well, wont you?

In the given gettext example, the

Code: Select all

updatepot 
macro will update the pot file and merge the changes in any existing po file. So it become easy to keep track for changes.
Lyberta wrote:So it is agnostic to format strings. Then the usefulness of gettext by itself gets diminished.
Not completely true. You'll be able to change the formatting in the translation, which, by the way may be needed sometimes, as text size may be differ a lot between different languages. Be able to change formatting in the translation could help to keep GUI formatting intact.
However, gettext throw a error when you try to build the mo file, for example, when you forgot to put a new line at the ending of a translated string, when the orig end with \n.
Lyberta wrote: I feel like other i18n solutions offer such thing out-of-the-box.
Why bother, feel free to name a few and show how easy they could integrated into existing projects, I'm not married with gettext, but, to be honest, I found it easy to use and maintain.
On the road again.
Lyberta
Established Member
Posts: 681
Joined: Sat Nov 01, 2014 8:15 pm
Location: The Internet
Been thanked: 1 time

Re: Add Localisation Support

Post by Lyberta »

tramp wrote:Ah, okay, that is true, the translation for this string will fail then. Just, I'm not sure if that is a dis-advance, properly you would change the translation as well, when you change the original string. So properly you would use a new ID as well, wont you?
If you use ID, you only change translation files, there is no user-visible text inside the source code. The change notification is up to some translation service. I haven't personally messed with such.

tramp wrote:In the given gettext example, the

Code: Select all

updatepot 
macro will update the pot file and merge the changes in any existing po file. So it become easy to keep track for changes.
This still means a lot of manual bookkeeping because it doesn't erase old phrases, right? So you end up with a lot of old stuff that you need to manually review.
tramp wrote:Not completely true. You'll be able to change the formatting in the translation, which, by the way may be needed sometimes, as text size may be differ a lot between different languages. Be able to change formatting in the translation could help to keep GUI formatting intact.
Gettext doesn't have this functionality built-in. You yourself said to use a different library such as Boost.Format.
tramp wrote:Why bother, feel free to name a few and show how easy they could integrated into existing projects, I'm not married with gettext, but, to be honest, I found it easy to use and maintain.
I haven't personally messed with translation yet. My ideal C++ API would be something like this:

Code: Select all

template <typename... Args>
std::string Translate(TranslationStringID id, const Args&... args);
This will do lookup of format string and formatting with reordering in a single function.

Btw I've noticed that getttext uses global settings to translate. Does it have local settings? What if you would want to use 2 locales at the same time in different parts of your program?
tramp
Established Member
Posts: 2335
Joined: Mon Jul 01, 2013 8:13 am
Has thanked: 9 times
Been thanked: 454 times

Re: Add Localisation Support

Post by tramp »

Lyberta wrote: If you use ID, you only change translation files, there is no user-visible text inside the source code. The change notification is up to some translation service. I haven't personally messed with such.
Well, I fail to see any advance in that. I've my strings in the source, on the places I need them. Other then that, maintenance will become horror very quick.
Lyberta wrote:This still means a lot of manual bookkeeping because it doesn't erase old phrases, right? So you end up with a lot of old stuff that you need to manually review.
You know what a merge is?
So, no, if you update the po files, new literals will be added, old, now longer used one will be deleted, changed ones will be replaced in the msgid, while the msgstr, the translation will be keep. Those changes will then be marked as fuzzy, and require your interaction, means mark then as OKAY, or change the translation.
Lyberta wrote:Gettext doesn't have this functionality built-in. You yourself said to use a different library such as Boost.Format.
Indeed, gettext just do the translation, nothing more, for text-formatting you could use the tools of your choice.
gettext only check if it is a plain literal, or a formatted text, if formatted, it tries to remind the translator for the formatting.

Lyberta wrote:Btw I've noticed that getttext uses global settings to translate. Does it have local settings? What if you would want to use 2 locales at the same time in different parts of your program?
You could set the locale (setlocale) in your source, if you wish. That, again has nothing to do with gettext, gettext only read your locale settings and act accordingly.
On the road again.
Lyberta
Established Member
Posts: 681
Joined: Sat Nov 01, 2014 8:15 pm
Location: The Internet
Been thanked: 1 time

Re: Add Localisation Support

Post by Lyberta »

tramp wrote:Well, I fail to see any advance in that. I've my strings in the source, on the places I need them. Other then that, maintenance will become horror very quick.
Using ID will make all languages equal and force uniform treatment of them.
tramp wrote:You know what a merge is?
Ah, so it is smarter than I thought, good. How does it determine changes though?
tramp wrote:Indeed, gettext just do the translation, nothing more, for text-formatting you could use the tools of your choice.
Which is very unfortunate and makes it almost useless by itself.
tramp wrote:gettext only check if it is a plain literal, or a formatted text, if formatted, it tries to remind the translator for the formatting.
It can't know the syntax of formatting library that the user is using. See, only half a solution and many problems.
tramp wrote:You could set the locale (setlocale) in your source, if you wish. That, again has nothing to do with gettext, gettext only read your locale settings and act accordingly.
setlocale modifies global state and therefore unusable in multithreaded programs. gettext could take a char* with locale name to its functions so it doesn't have to query the global state.
tramp
Established Member
Posts: 2335
Joined: Mon Jul 01, 2013 8:13 am
Has thanked: 9 times
Been thanked: 454 times

Re: Add Localisation Support

Post by tramp »

Lyberta wrote:Using ID will make all languages equal and force uniform treatment of them.
You need a fallback for the case translation fail, or for the case gettext isn't available on the system. So, you need to define your strings in source anyway. Define them in some out-of-source config files in what ever format you believe is nice, only, is stupid at all. At least, you need them backed in the binary to avoid segfaults.
If you prefer to make it complicated, you could use a literal class were you define your strings and then use your var's in place. :roll:
Lyberta wrote:Ah, so it is smarter than I thought, good. How does it determine changes though?
How does git do it? diff? patch?
Lyberta wrote:Which is very unfortunate and makes it almost useless by itself.
text formatting is a choice of the developer, not of the tool which do translation. All I suspect from a translation tool is, translate my strings, if needed, that's it, please, don't do more then that.
Lyberta wrote:setlocale modifies global state and therefore unusable in multithreaded programs. gettext could take a char* with locale name to its functions so it doesn't have to query the global state.
But you know what localisation is about? Translate a software to the locale settings of the user, so, the locale settings have to come from the users system. Why the hell one should use locale settings in a program to present different languages to the user? If you need that, there are several way's to accomplish that, but, that has less to do with localisation of a program.
On the road again.
Lyberta
Established Member
Posts: 681
Joined: Sat Nov 01, 2014 8:15 pm
Location: The Internet
Been thanked: 1 time

Re: Add Localisation Support

Post by Lyberta »

tramp wrote:You need a fallback for the case translation fail, or for the case gettext isn't available on the system. So, you need to define your strings in source anyway. Define them in some out-of-source config files in what ever format you believe is nice, only, is stupid at all. At least, you need them backed in the binary to avoid segfaults.
Segfaults? Why? A good system will print and English message during initialization and quit.
tramp wrote:How does git do it? diff? patch?
They do a lot of mistakes when determining what became what. This is important for a translation tool.
tramp wrote:text formatting is a choice of the developer, not of the tool which do translation. All I suspect from a translation tool is, translate my strings, if needed, that's it, please, don't do more then that.
gettext is a single wheel. To "Add Localisation Support" users need the rest of the car. I suspect that users don't like to build their own cars when they have other problems to solve like writing the business logic of their software.
tramp wrote:But you know what localisation is about? Translate a software to the locale settings of the user, so, the locale settings have to come from the users system. Why the hell one should use locale settings in a program to present different languages to the user? If you need that, there are several way's to accomplish that, but, that has less to do with localisation of a program.
Simple case - to test other languages without making your system inoperable. If I want to test a French version of my program, why should I switch the whole system to French? I won't be able to switch it back easily if I don't know French well.
tramp
Established Member
Posts: 2335
Joined: Mon Jul 01, 2013 8:13 am
Has thanked: 9 times
Been thanked: 454 times

Re: Add Localisation Support

Post by tramp »

Lyberta wrote:Segfaults? Why? A good system will print and English message during initialization and quit.
Ah, how you do that without a string defined in source?
Lyberta wrote:They do a lot of mistakes when determining what became what. This is important for a translation tool.
Anyone makes mistakes from time to time, happens to me as well. Only exception is, if you don't do anything.
Lyberta wrote:gettext is a single wheel. To "Add Localisation Support" users need the rest of the car. I suspect that users don't like to build their own cars when they have other problems to solve like writing the business logic of their software.
What, this becomes really annoying, in practise, gettext works very well, and brings all the tools you need.
Lyberta wrote:Simple case - to test other languages without making your system inoperable. If I want to test a French version of my program, why should I switch the whole system to French? I won't be able to switch it back easily if I don't know French well.
simple, check what locale you use, with

Code: Select all

locale
then run to test the French version, and set your locale back to the previous used.

Code: Select all

LANG=fr_FR.utf8 && your app && LANG=your_Locale.utf8
On the road again.
folderol
Established Member
Posts: 2069
Joined: Mon Sep 28, 2015 8:06 pm
Location: Here, of course!
Has thanked: 224 times
Been thanked: 400 times
Contact:

Re: Add Localisation Support

Post by folderol »

It might be worth taking a look at how Rosegarden does it - they've had localisation in place for years. Might be a qt thing though.
The Yoshimi guy {apparently now an 'elderly'}
tramp
Established Member
Posts: 2335
Joined: Mon Jul 01, 2013 8:13 am
Has thanked: 9 times
Been thanked: 454 times

Re: Add Localisation Support

Post by tramp »

folderol wrote:It might be worth taking a look at how Rosegarden does it - they've had localisation in place for years. Might be a qt thing though.
Yes, you are right, it's a qt thing, it's qtTranslator what they use now. (Before they use gettext as well)
qtTranslator have the advance of being nicely integrated in the toolkit in question, and, do, in opposite to gettext, automatic translation in several languages. So no supporters needed. It's in a same way then gettext easily to integrate with a few lines of code, and easily maintainable in a existing project.
But, you guess what comes now, the dis-advance. It's, the same as the advance. It's dependently integrated in the QT toolkit. So no support when your GUI isn't QT based.
On the road again.
Post Reply