Why do my MongoDB simultaneous $push updates fail?

Question

I'm performing a number of updates of the form

update(
  { "uuid": someUuid, "revision.versionNumber": someVersionNumber},
  { "$set": { "meta.someId": someId }, "$push": { "meta.someMessages": someMessage } }
)

occasionally I see when this is called for the same uuid,versionNumber, & someId with a different someMessage the first update will succeed but the second will fail silently.

I see the following in the mongo logs so I know the updates are making it to the database, notice that the first update has the same query as the third one but the first has nupdated: 1 while the third has nupdated: 0

Wed Aug 28 14:50:24 [conn18] update some-db.some_collection query: { uuid: "b841f303-a054-4eb9-8885-9d3ebf9906a1", revision.versionNumber: 9 } update: { $set: { meta.someId: "521e6fe4036420f90371a922" }, $push: { meta.someMessages: { event: "instance.complete", timestamp: new Date(1377726624985) } } } nscanned:2507 nmoved:1 nupdated:1 keyUpdates:0 numYields: 19 locks(micros) w:6010 9ms
Wed Aug 28 14:50:24 [conn18] run command some-db.$cmd { getlasterror: 1, fsync: true }
Wed Aug 28 14:50:24 [conn14] update some-db.some_collection query: { uuid: "843f424d-8a62-4a8b-853f-dc2e9c42b309", revision.versionNumber: { $lt: 10 }, meta.deleted: true } update: { $set: { meta.deleted: false } } nscanned:3243 nupdated:0 keyUpdates:0 numYields: 23 locks(micros) w:8431 11ms
Wed Aug 28 14:50:24 [conn14] run command some-db.$cmd { getlasterror: 1, fsync: true }
Wed Aug 28 14:50:24 [conn5] update some-db.some_collection query: { uuid: "b841f303-a054-4eb9-8885-9d3ebf9906a1", revision.versionNumber: 9 } update: { $set: { meta.someId: "521e6fe4036420f90371a922" }, $push: { meta.someMessages: { event: "instance.complete.success", timestamp: new Date(1377726624985) } } } nscanned:3242 nupdated:0 keyUpdates:0 numYields: 20 locks(micros) w:5684 9ms

Also here is the output from mongosniff

update  flags:0 q:{ uuid: "85700d8c-8946-4b09-968b-968f76d31028", revision.versionNumber: 13 } o:{ $set: { meta.someId: "521e7b12036420f90371b515" }, $push: { meta.someMessages: { event: "instance.complete", timestamp: new Date(1377729439093) } } }
319 some-db.some_collection

    update  flags:0 q:{ uuid: "a460019d-443b-4b59-b23e-1eae19e26c31", revision.versionNumber: 14 } o:{ $set: { meta.someId: "521e7b2f036420f90371b579" }, $push: { meta.someMessages: { event: "task.start", timestamp: new Date(1377729439093) } } }
123 some-db.some_collection

    query: { uuid: "a2558f5c-d825-4ec4-bbc4-7e48b1cb3c60", isLatest: true }  ntoreturn: -1 ntoskip: 0
302 some-db.some_collection

    update  flags:0 q:{ uuid: "85700d8c-8946-4b09-968b-968f76d31028", revision.versionNumber: 13 } o:{ $set: { meta.someId: "521e7b12036420f90371b515" }, $push: { meta.someMessages: { event: "instance.complete.success", timestamp: new Date(1377729439093) } } }
173 some-db.some_collection

Answer 1

As a workaround for this bug, I recommend using findAndModify and the checking the results to make sure your update happened.

    dbCollection.findAndModify(
{ "uuid": someUuid, "revision.versionNumber": someVersionNumber},
[], { "$set": { "meta.someId": someId }, "$push": { "meta.someMessages": someMessage } }, {safe: true, 'new' : true}, function(err, updated){
     if(err){
     //handle the error
    }
    if(updated.meta.someMessages doesn't contain your message) {
     //try it again or report it to the client
    }     
    });