How to change letters in text to other letters | Python 2.7?

Question

In my Django project I have form with field ("title"). In this field user need to write in Russian language but user can by mistake write some Latin letters. I want to change them. I use PYTHON 2.7. Next code raise error. How to fix this error?

from string import maketrans

eng = 'ETOPAHKXCBMetopahkxcbm' # English letters
rus = 'ЕТОРАНКХСВМеторанкхсвм' # Russian letters

def form_valid(self, form):
    form.cleaned_data['title'].translate(maketrans(dict(zip(eng, rus))))
    form.save()

ERROR:

Traceback (most recent call last):
  File "/srv/envs/py27/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner
    response = get_response(request)
  File "/srv/envs/py27/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/srv/envs/py27/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/srv/envs/py27/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view
    return self.dispatch(request, *args, **kwargs)
  File "/srv/envs/py27/lib/python2.7/site-packages/django/views/generic/base.py", line 88, in dispatch
    return handler(request, *args, **kwargs)
  File "/srv/envs/py27/lib/python2.7/site-packages/django/views/generic/edit.py", line 217, in post
    return super(BaseCreateView, self).post(request, *args, **kwargs)
  File "/srv/envs/py27/lib/python2.7/site-packages/django/views/generic/edit.py", line 183, in post
    return self.form_valid(form)
  File "/home/nurzhan/dashboard.kase.kz/static_pages/views.py", line 54, in form_valid
    form.cleaned_data['title'].translate(maketrans(dict(zip(eng, rus))))
TypeError: maketrans() takes exactly 2 arguments (1 given)

When I use just form.cleaned_data['title'].translate(maketrans(eng, rus)) it raise error:

Internal Server Error: /static_page/create/
Traceback (most recent call last):
  File "/srv/envs/py27/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner
    response = get_response(request)
  File "/srv/envs/py27/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/srv/envs/py27/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/srv/envs/py27/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view
    return self.dispatch(request, *args, **kwargs)
  File "/srv/envs/py27/lib/python2.7/site-packages/django/views/generic/base.py", line 88, in dispatch
    return handler(request, *args, **kwargs)
  File "/srv/envs/py27/lib/python2.7/site-packages/django/views/generic/edit.py", line 217, in post
    return super(BaseCreateView, self).post(request, *args, **kwargs)
  File "/srv/envs/py27/lib/python2.7/site-packages/django/views/generic/edit.py", line 183, in post
    return self.form_valid(form)
  File "/home/nurzhan/dashboard.kase.kz/static_pages/views.py", line 54, in form_valid
    form.cleaned_data['title'].translate(maketrans(eng, rus))
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-21: ordinal not in range(128)

Answer 1

You can do it manually like this:

a_string = u"abcdeFghijklmn" # Let say it is cyrillic with one latin letter represented here by 'F' (uppercase)
# To change it to cyrillic i.e. here to lower latin you would do:
l2c = {u"F": u"f"} # Map all latin to cyrillic for real
l2c_get = l2c.get # Faster access but not necessary
a_string = u"".join(
    l2c_get(x, x) for x in a_string)

Of course, you can use unicode.translate() method which should do the same.

When your 'a_string' is processed then just push it back to django. All mapped latins to cyrillic will be substituted. Just pay attention that everything is unicode in the mapping. If putting the translated string to django raises the UnicodeDecode/Encode error, it means that you should before try:

a_string = a_string.encode("utf-8", "ignore")

UTF-8 in binary should be accepted.

Answer 2

It'll make things a bit easier if you use Unicode literals in your code as well, that way you have to worry less about str vs unicode type issues. This is the same as writing all your string literals as u'stuff' instead of just 'stuff'.

Next - the translate function of unicode and of string take different arguments - you need a dict for unicode mapping Unicode ordinals to ordinals or Unicode strings, not maketrans which is only for str translations.

You can use the ord built-in to get ordinals from your Unicode characters.

Try this:

from __future__ import unicode_literals  # Put this at the top of your source

eng = 'ETOPAHKXCBMetopahkxcbm' # English letters
rus = 'ЕТОРАНКХСВМеторанкхсвм' # Russian letters

table = dict(zip(map(ord, eng), map(ord, rus)))
assert eng.translate(table) == rus    # Verifying translation