Flask/mongodb webapp inconsistent test

Question

I have a Flask integration test backed by a 1-node mongodb that randomly fails:

pytest/test_webapi.py:59: in test_register_test
>           assert res.status_code == 302
E           assert <Response streamed [404 NOT FOUND]>.status_code == 302

Fail rate is roughly 50%.

Test in test_webapi.py looking thus:

def test_register_user(self):
    res = self.client.get("/logout")

    class MySMTPServer(smtpd.SMTPServer):
        mails = []
        def process_message(self, peer, mailfrom, rcpttos, data):
            self.mails.append((rcpttos[0], data))

    server = MySMTPServer(('localhost', 12345), None)

    t = threading.Thread(target=asyncore.loop, args=(1,))
    t.start()
    time.sleep(.1)

    try:
        res = self.client.post("/register", data=self.registration)
        assert res.status_code == 200

        mail, hash = server.mails[0]

        self.conn.fsync()
        time.sleep(.1)

        res = self.client.get('/activate/' + hash)
        assert res.status_code == 302
    finally:
        server.close()

The relevant Flask methods from webapi.py:

@app.route("/register", methods=["POST"])
def register_user():
    mail = flask.request.form['mail']
    user = flask.request.form["user"]
    pw = flask.request.form["pass"]

    hash = users.register(user, pw, mail=mail)
    return flask.jsonify({'_id': None}) # XXX

@app.route('/activate/<hash>', methods=['GET'])
def activate_user(hash):
    key = users.activate(hash=hash)
    if not key:
        flask.abort(404)
    return flask.redirect("/")

... are backed by action methods:

make_key = lambda : base64.encodestring(os.urandom(32)).strip()

def register(self, user, pw, **kw):
    hash = self.make_key()
    user = self.new(user, pw, activation=hash, **kw)
    self._send_mail(**user)
    return hash

def activate(self, hash):
    user = self.users.find_one({'activation': hash})
    if not user:
        return None
    key = self.make_key()
    activation = {
        '$unset': {'activation': 1},
        '$set': {'status': 'active', 'key': key} }
    self.users.update({'_id': user['_id']}, activation)
    return user

... where self.users is a mongodb collection.

self.new() persists the entity using safe=True.

Interestingly, several other tests doing similar things never seem to encounter this problem.

I had thought that this would be enough to make sure that the persisted object would be visible to other threads in the pymongo connection pool. What part of the mongodb/pymongo documentation should I have read more carefully? Or is there some weird interaction with asyncore?

Answer 1

(ported from comments thread)

In the context of the tests, is the data value just the base64-encoded string of the activation key? base64 contains letters and digits, but also "+" and "/", both of which will be misinterpreted by URL parsers (particularly "/").