How can I craft a generic BulkUpsertAsync method for MongoDB in C#?

Question

I am trying to build an extension method for IMongoCollection<TDocument> in C# which will allow any List<TDocument> to be upserted into a MongoDB collection using an upsert. I have found other articles that suggest a List<WriteModel<TDocument>>, in combination with BulkWriteAsync to perform these operations in a batch.

In a non-generic manner, I can upsert a series of entries (in this case a List<Line>) using:

public static async Task<BulkWriteResult<Line>> BulkUpsertAsyncNonGeneric(this IMongoCollection<Line> collection, List<Line> entries)
        {
            var bulkOps = new List<WriteModel<Line>>();

            foreach (var entry in entries)
            {
                var filter = Builders<Line>.Filter.Eq(doc => doc.Id, entry.Id);

                var upsertOne = new ReplaceOneModel<Line>(filter, entry) { IsUpsert = true };

                bulkOps.Add(upsertOne);
            }

            return await collection.BulkWriteAsync(bulkOps);
        }

By changing <Line> for <TDocument> I have made this partly generic, but there is an assumption that every TDocument has an Id field and that every entry in entries also has an Id field. Of course, TDocument has no members. I want to make these field definitions fully generic, ideally using a lambda to match the format of the call to Filter.Eq(doc => doc.Id, entry.Id). However, I'm stuck. I really want to avoid simply passing a string literal with the field names, which I believe would work fine but isn't compile-time safe.

I've come up with the following, which unsurprisingly does not compile:

public static async Task<BulkWriteResult<TDocument>> BulkUpsertAsync<TDocument, TField>(this IMongoCollection<TDocument> collection, List<TDocument> entries, Expression<Func<TDocument, TField>> filterField, Expression<Func<TDocument, TField>> valueField)
    {
        var bulkOps = new List<WriteModel<TDocument>>();

        foreach (var entry in entries)
        {
            var filter = Builders<TDocument>.Filter.Eq(filterField, valueField);

            var upsertOne = new ReplaceOneModel<TDocument>(filter, entry) { IsUpsert = true };

            bulkOps.Add(upsertOne);
        }

        return await collection.BulkWriteAsync(bulkOps);
    }

I suspect the type of valueField is incorrect, but additionally the compiler complains that

Error CS1503: Argument 1: cannot convert from 'System.Linq.Expressions.Expression<System.Func<TDocument, TField>>' to 'MongoDB.Driver.FieldDefinition<TDocument, System.Linq.Expressions.Expression<System.Func<TDocument, TField>>>' (21, 48)

Answer 1

I managed to get this working using a compiled lambda.

public static async Task<BulkWriteResult<TDocument>> BulkUpsertAsync<TDocument, TField>(
        this IMongoCollection<TDocument> collection,
        List<TDocument> entries,
        Expression<Func<TDocument, TField>> filterField)
    {
        var bulkOps = new List<WriteModel<TDocument>>();
        foreach (var entry in entries)
        {
            var filterFieldValue = filterField.Compile();
            var filter = Builders<TDocument>.Filter.Eq(filterField, filterFieldValue(entry));
            var upsertOne = new ReplaceOneModel<TDocument>(filter, entry) { IsUpsert = true };

            bulkOps.Add(upsertOne);
        }

        return await collection.BulkWriteAsync(bulkOps);
    }

The Expression<Func<TDocument, TField>> in the method signature lets me point at a property or field on TDocument, which is used in both the creation of the filter for MongoDB, as well as the in the iterator that creates the ReplaceOneModel for each entry in entries.

It can be called as follows:

// Given some List<T> of entries to upsert...
List<SomePoco> SomePocos = GetListOfPocoFromSomewhere();
// This will match existing documents on a field called "Name".
await SomeMongoCollection.BulkUpsertAsync(SomePocos, filterField => filterField.Name);

Hope this helps someone!

Answer 2

I think you need to apply a generic constraint to your T. It still relies on having the item(s) of your interface (so id or whatever) but you can make that a very slim interface that you know most of your classes will be able to implement. And you get compile type safety.

We are not using extensions methods but have a wrapper nuget package that we use to access mongo via C#/Core 2.1

We have a specific interface IMongoDocument which we can then use to define our filters etc

public async Task UpdateAsync<T>(T entity) where T : IMongoDocument
{
    string originalDocumentVersion = entity.DocumentVersion;
    var filter = Builders<T>.Filter.Eq(x => x.Id, entity.Id);

    entity.DocumentVersion = ObjectId.GenerateNewId().ToString();

    ReplaceOneResult result = await this.Collection<T>().ReplaceOneAsync(this.session, filter, entity).ConfigureAwait(false);

    CheckResult(result.IsAcknowledged, result.IsAcknowledged ? result.ModifiedCount : 0);
}

public interface IMongoDocument : IDocumentIdentifier, IDocumentVersion
{     
}

public interface IDocumentIdentifier
{
    string Id { get; set; }
}

public interface IDocumentVersion
{
    string DocumentVersion { get; set; }
}

this.session is being used because we are using mongo transactions and can be ignore if you are not!

This example shows an update but passing in a list of IMongoDocuments (or whatever your interface might end up being) to a method with a generic constraint should allow you to be generic (to a point) for your bulk update. The key point seems to be building the filter correctly.

If you don't want to adhere to an interface the only other way I can think of is to dig into your class(es) via reflection to generate the filter but I would suggest that as a last resort!