Django in-memory database model creation failure

Question

I'm using MariaDB MEMORY engine. I defined max_heap_table_size in my.cnf, restarted the database service. Now, I run the migration and get:

django.db.utils.ProgrammingError: Storage engine MEMORY doesn't support BLOB/TEXT columns

My erroneous model is:

class Department(models.Model):
    name = models.CharField(max_length=100)
    tag = models.CharField(max_length=10)
    dtype = models.PositiveSmallIntegerField()
    info = models.CharField(max_length=64000)

Though, max VARCHAR is 65535.What is the problem?

Answer 1

As the docs state here, 65.535 are actually bytes, which leads to a maximum of 21.844 characters if using UTF-8.

A variable-length string. M represents the maximum column length in characters. The range of M is 0 to 65,535. The effective maximum length of a VARCHAR is subject to the maximum row size (65,535 bytes, which is shared among all columns) and the character set used. For example, utf8 characters can require up to three bytes per character, so a VARCHAR column that uses the utf8 character set can be declared to be a maximum of 21,844 characters.

Since the max_length of a CharField in a Django model specifies the length in characters rather than in bytes, I assume this is what causes the error.

Answer 2

Make the table MyISAM or InnoDB. Either will quickly cache your string, thereby making respond as if it were "in-memory".

As for the confusing error message -- a VARCHAR that needs more than 64K bytes is silently turned into MEDIUMTEXT.

If your string can be broken into multiple pieces, you could have multiple rows. (Have a second column with a row number of some kind.)