Django session translation setting is sticky for all languages, except Chinese?

Question

I have a really weird problem I've never seen before. I'm using Django 1.10.

I have several dictionary files:

/locale/fr/LC_MESSAGES/django.po
/locale/de/LC_MESSAGES/django.po
/locale/zh/LC_MESSAGES/django.po

The application strings are written in English. The dictionary files are complete, and compiled to mo files.

I store each user's language preference in the language field of the UserProfile. When updating their profile, I apply the language translation to the session.

# 'up' is a UserProfile object pertaining to the user
up.update(language=form.cleaned_data['language'])
translation.activate(up.language)
self.request.session[translation.LANGUAGE_SESSION_KEY] = up.language
return super(self, UpdateUserProfile).form_valid(form)

This works fine for French and German. The return super renders the form template in French/German, and then I can navigate to other pages and see French/German text.

It doesn't work for Chinese (language code 'zh'). The return super page renders the form template in Chinese (and I've verified the language setting in the shell after saving), but then unlike French and German, all other pages revert to English when I navigate away.

I've restarted the dev server just in case it was due to old settings. I've tried it out on a clean pull of that branch on a fresh, isolated dev VM. I have the same problem for Chinese in both scenarios. What could cause this error?

Edit - additional information:

From settings.py

LANGUAGE_CODE = 'en-us'
LOCALE_PATHS = [ 
    BASE_DIR + '/locale/',
]
USE_I18N = True
USE_L10N = True

As mentioned, on POST for the UpdateLanguage FormView the request.session[translation.LANGUAGE_SESSION_KEY] code is set to zh, and the page displays in Chinese. On a GET request to the same view the page displays in English (however examining request.session[translation.LANGUAGE_SESSION_KEY] shows it is set to zh despite displaying English text).

I'm not using language prefixes in the urls.

Answer 1

You've correctly inferred that zh is not a code that is part of the default LANGUAGES setting and thus is not working right.

The reason you got Chinese when producing the first page is that you make a call equivalent to translation.activate("zh"). This call does not check LANGUAGES. It blindly activates the translation for the language zh. If the files are there for "zh" there's no problem. You could do translation.activate("turnip") and Django would be fine with it so long as the files for the language "turnip" exist..

The reason it did not work on subsequent page requests is that the locale middleware does check against LANGUAGES and rejects values that are not listed there: it switches back to the default language you configured for your site.

Here is code that illustrates the two cases:

import os

import django
from django.utils.translation import trans_real as translation, LANGUAGE_SESSION_KEY
from django.middleware.locale import LocaleMiddleware

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "btw.settings")
django.setup()

#
# Calling translation.activate directly.
#
translation.activate("zh")
value = translation._active.value
print "translation:", value.language()

#
# Middleware processing
#
class FakeRequest(object):

    path_info = ""
    session = {}
    COOKIES = {}
    META = {}

middleware = LocaleMiddleware()

for lang in ("fr", "zh", "zh-hans"):
    print "Trying:", lang
    request = FakeRequest()
    request.session[LANGUAGE_SESSION_KEY] = lang

    middleware.process_request(request)
    print request.LANGUAGE_CODE

You'll see in the 2nd part using the middleware that when you select "zh" as the language, request.LANGUAGE_CODE is set to the default language for your site instead of being "zh" or some variant of it. On my system, the output is:

translation: zh
Trying: fr
fr
Trying: zh
en-us
Trying: zh-hans
zh-hans

Answer 2

Wow, all the debugging output from django debug toolbar indicated that I was doing the right thing, so I looked at the source of global_settings.py

zh isn't actually a valid language code. You have to choose between simplified zh-hans and traditional zh-hant, and the fact that the page weirdly rendered in Chinese for the HttpResponse after the session language was set made me thing the problem was with the session, not the settings.