Raven paging queries in a specific way

Question

I'm developing an ASP.NET MVC application using RavenDB 3. I don't have a lot of experience with raven.

In general, when executing queries to display data, the first 128 items are returned on the page. More records are added in an "infinite scroll"-manner by using paged queries.

Now, however, I have the requirement that items are loaded in 'groups'.

Assume the following class:

public class Item {
  public string Id { get; set; }
  public int Group { get; set; }
  public string text { get; set; }
}

Assume the database contains 40 items having group='1', 40 items having group='2' and 50 items having group='3'.

This is a total of 130 items. This would mean that the last 'group' fetched would not be complete. It would be missing 2 items.

I would like a mechanism that is aware of this, so that it would fetch at least 128 AND would fetch 'extra' if the last group is not completely included.

Afterwards, when I fetch the next page, I would like it to start with the next group.

Is there any way I can make this work without 'fabricating' a single page myself by doing more than one call?

EDIT: I cannot assume the groups are perfectly equal in size, but I can assume the sizes will be 'simular'

Also, I cannot change the design to store all items in a single 'group'-object for instance.

Answer 1

Okay so basically what you will need to do is calculate the number of results that were in the previous pages and the number of results that are in the current page. Below is a quick example app to give you an idea of how to do it. One caveat, if the number of results for the current group range exceeds the MaxNumberOfRequestsPerSession than an error will be thrown, so you might want to put some handling in there for that.

Note for running this example: Make sure your platform is set to x64 in visual studio if you are using the most recent versions of RavenDB. Otherwise, this example will thrown an error about Voron not being stable in 32 bit mode.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Raven.Client;
using Raven.Client.Embedded;
using Raven.Client.Listeners;

namespace ConsoleApplication
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            using (var gdm = new GroupDataManager())
            {
                Console.WriteLine("Started Seeding");
                gdm.Seed().Wait();
                Console.WriteLine("Finished Seeding");
                Console.WriteLine("===============================================================");
                Console.WriteLine("Get First Page");
                Console.WriteLine("===============================================================");
                var firstPage = gdm.GetPagedGroupResults(1, 3).Result;

                foreach (var item in firstPage)
                {
                    Console.WriteLine(item.Text);
                }
                Console.WriteLine("===============================================================");
                Console.WriteLine("Get Second Page");
                Console.WriteLine("===============================================================");
                var secondPage = gdm.GetPagedGroupResults(2, 3).Result;
                foreach (var item in secondPage)
                {
                    Console.WriteLine(item.Text);
                }
            }
            Console.ReadLine();
        }
    }

    public class GroupDataManager : IDisposable
    {
        private readonly IDocumentStore _documentStore = new EmbeddedRavenDatabase().Store;

        public void Dispose()
        {
            _documentStore.Dispose();
        }

        public async Task Seed()
        {
            var rnd = new Random();
            using (var session = _documentStore.OpenAsyncSession())
            {
                for (var groupNumber = 1; groupNumber < 15; groupNumber++)
                {
                    for (var i = 0; i < rnd.Next(5, 25); i++)
                    {
                        var item = new Item
                        {
                            Group = groupNumber,
                            Text = string.Format("Group: {0} Item:{1}", groupNumber, i)
                        };
                        await session.StoreAsync(item);
                    }
                }

                await session.SaveChangesAsync();
            }
        }

        public async Task<IList<Item>> GetPagedGroupResults(int page, int numberOfGroupsPerPage)
        {
            var startingGroup = ((page - 1) * numberOfGroupsPerPage) + 1;

            using (var session = _documentStore.OpenAsyncSession())
            {
                //  Calculate the number of items that were contained in the previous groups
                var numberToSkip = await session.Query<Item>().CountAsync(item => item.Group < startingGroup);

                var endGroup = startingGroup + numberOfGroupsPerPage;

                //  Calculate the number of items that are in the current range of groups
                var numberToTake = await session.Query<Item>().CountAsync(item => item.Group >= startingGroup && item.Group < endGroup);

                var results = await session.Query<Item>().Skip(numberToSkip).Take(numberToTake).ToListAsync();

                return results;
            }
        }
    }

    public class Item
    {
        public string Id { get; set; }
        public int Group { get; set; }
        public string Text { get; set; }
    }

    /// <summary>
    /// For Testing Only. Prevents stale queries
    /// </summary>
    public class NoStaleQueriesAllowed : IDocumentQueryListener
    {
        public void BeforeQueryExecuted(IDocumentQueryCustomization queryCustomization)
        {
            queryCustomization.WaitForNonStaleResults();
        }
    }

    public class EmbeddedRavenDatabase
    {
        private static bool _configured = false;

        private static readonly Lazy<IDocumentStore> _lazyDocStore = new Lazy<IDocumentStore>(() =>
        {
            var docStore = new EmbeddableDocumentStore
            {
                RunInMemory = true
            };

            docStore.RegisterListener(new NoStaleQueriesAllowed());
            docStore.Initialize();

            return docStore;
        });

        public IDocumentStore Store
        {
            get { return _lazyDocStore.Value; }
        }
    }
}