Google App Engine memcache.Client.cas() keeps returning False for missing key

557 views Asked by At

The following is part of a Python Flask application running on Google App Engine:

@app.route('/blabla', methods=['GET'])
def blabla():
    # memcache.add('key', None)  # this "fixes" it!  
    memcache_client = memcache.Client()
    while True:
        value = memcache_client.gets('key')
        if value is None:  # First time
            updated_value = 'Bla'
        else:
            updated_value = value + ', bla'
        if memcache_client.cas('key', updated_value):
            return updated_value

Starting with an empty cache, if we make consecutive GET requests to /blabla, I would expect the requests to return:

Bla
Bla, bla
Bla, bla, bla
.
.

(If for some reason at some point between .gets() and cas() the cache gets flushed, then I would expect the sequence to restart, no problem.)

But we don’t get anything because memcache_client.cas() keeps returning False forever, so the program gets stuck in the while-loop. Apparently this happens because the key 'key' does not exist in the beginning.

I know this because if I uncomment the memcache.add('key', None), it sort-of works, because then the key exists and .cas() is happy and returns True. But if right between the .add() and the .gets() some other process were to flush the cache, we’d be back to where we started from, with a missing key, and .cas() would go back to returning False indefinitely. So it is not a good solution.

Why doesn’t .cas() work if the key is missing in the beginning? Or at least, why doesn’t .cas() accept an initial_value= parameter, like its sibling decr()? Is it a bug or a feature? I can’t find this documented properly anywhere, except that Guido van Rossum alludes to it in his single blog post on the matter—reffering to an assert he makes that the .gets() does not return None, he says:

Aside 2: The assert is kind of naive; in practice you'll have to somehow deal with counter initialization.

Dank je wel Guido—does anyone know how, please?

1

There are 1 answers

0
dwardu On

Ok, I figured it out.

@app.route('/blabla', methods=['GET'])
def blabla():
    memcache_client = memcache.Client()
    while True:

        if memcache.add('key', 'Bla'):
            # That's all folks!
            return 'Bla'

        # add() failed => the key must already exist, we have to compare-and-set.

        value = memcache_client.gets(key_name)

        if value is None:
            # At the time add() failed the key existed, but now gets() is returning None,
            # so somebody must have deleted the cache entry in between.
            # Let's start from scratch.
            continue

        updated_value = value + ', bla'

        if memcache_client.cas(key_name, updated_value):
            return updated_value
        else:
            continue

It’s more complicated than I would have liked it to be, but it works.

The last else: continue is redundant but I’m writing it to make it clear that we’re going to continue trying until we succeed.

Although in practice you'll have to somehow deal with giving up after a number of retries.