Django image upload not uploading japanese named image

Question

The error i am getting when uploading japanese named image is : ascii' codec can't encode characters in position 45-48: ordinal not in range(128)

Images are uploading perfectly when named in english characters. Also, it's strange that the error i am encountering is only when i am uploading it to the server. Japanese named wouldn't upload on the deployment server but are working fine in development.

My model:

class Tenant_Post(models.Model):
    user = models.ForeignKey(User,on_delete=models.CASCADE,null=True)
    name = models.CharField(max_length=255,null=True)
    image = models.FileField(upload_to='image/tenant/',null=True)
    posted_on = models.DateTimeField(auto_now_add=True, auto_now=False)
    last_modified_on = models.DateTimeField(auto_now_add=False, 
    auto_now=True)

    def __unicode__(self):
        return self.name

My view:

@login_required(login_url='/')
def new(request):
    if request.method == 'POST':
        print request.POST
        form = TenantForm(request.POST or None, request.FILES or None)
        if form.is_valid():
            instance = form.save(commit=False)
            instance.user = request.user
            instance.save()
            print 'success'
            return HttpResponseRedirect(reverse('tenant:all'))
        else:
            print 'fail'
            return render(request,'tenant/new.html',{'form':form,})
    else:
        form = TenantForm()
        return render(request,'tenant/new.html',{'form':form,})

Full Trace back is here :

File "/opt/python/run/venv/lib/python2.7/site-packages/django/core/handlers/exception.py" in inner
  41.             response = get_response(request)

File "/opt/python/run/venv/lib/python2.7/site-packages/django/core/handlers/base.py" in _get_response
  187.                 response = self.process_exception_by_middleware(e, request)

File "/opt/python/run/venv/lib/python2.7/site-packages/django/core/handlers/base.py" in _get_response
  185.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/opt/python/run/venv/lib/python2.7/site-packages/django/contrib/auth/decorators.py" in _wrapped_view
  23.                 return view_func(request, *args, **kwargs)

File "/opt/python/current/app/tenant/views.py" in edit
  64.                 instance.save()

File "/opt/python/run/venv/lib/python2.7/site-packages/django/db/models/base.py" in save
  806.                        force_update=force_update, update_fields=update_fields)

File "/opt/python/run/venv/lib/python2.7/site-packages/django/db/models/base.py" in save_base
  836.             updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)

File "/opt/python/run/venv/lib/python2.7/site-packages/django/db/models/base.py" in _save_table
  900.                       for f in non_pks]

File "/opt/python/run/venv/lib/python2.7/site-packages/django/db/models/fields/files.py" in pre_save
  296.             file.save(file.name, file.file, save=False)

File "/opt/python/run/venv/lib/python2.7/site-packages/django/db/models/fields/files.py" in save
  94.         self.name = self.storage.save(name, content, max_length=self.field.max_length)

File "/opt/python/run/venv/lib/python2.7/site-packages/django/core/files/storage.py" in save
  53.         name = self.get_available_name(name, max_length=max_length)

File "/opt/python/run/venv/lib/python2.7/site-packages/django/core/files/storage.py" in get_available_name
  77.         while self.exists(name) or (max_length and len(name) > max_length):

File "/opt/python/run/venv/lib/python2.7/site-packages/django/core/files/storage.py" in exists
  392.         return os.path.exists(self.path(name))

File "/opt/python/run/baselinenv/lib64/python2.7/genericpath.py" in exists
  26.         os.stat(path)

Exception Type: UnicodeEncodeError at /tenant/edit/4/
Exception Value: 'ascii' codec can't encode characters in position 45-48: ordinal not in range(128)

Answer 1

I haven't test it with japanese language, but it works with some other languages like portuguese with special characters:

Add this on your settings.py

DEFAULT_FILE_STORAGE = 'app.models.ASCIIFileSystemStorage'

And your app.models.ASCIIFileSystemStorage

# This Python file uses the following encoding: utf-8
from django.db import models
from django.core.files.storage import FileSystemStorage
import unicodedata

class ASCIIFileSystemStorage(FileSystemStorage):
    """
    Convert unicode characters in name to ASCII characters.
    """
    def get_valid_name(self, name):
        name = unicodedata.normalize('NFKD', name).encode('ascii', 'ignore')
        return super(ASCIIFileSystemStorage, self).get_valid_name(name)

Answer 2

Your error is raised by the os.stat(path) call, which means your filesystem doesn't support japanese characters (actually it probably only support either ascii or some latin-xxx or windows-yyy encoding).

You have mainly two solutions here: either configure your system to use utf-8 everywhere (which is IMVHO a sane thing to do anyway), or make sure you only use your system encoding (or just plain ascii) for filesystem names etc (cf leeeandroo's answer).

Answer 3

add encode utf-8 then it can accept most languages also to the templates html use charset utf-8

# -*- coding: UTF-8 -*- in at the top the file

<meta charset="utf-8"/> in html templates

edit: please read official doc or add this line

 from __future__ import unicode_literals

https://docs.djangoproject.com/en/1.11/ref/unicode/