Firebase indexing on huge lists (100000+ items)

Question

I'm migrating my relational database to Firebase. In general, I have a planner for workers. They can add an item ('appointment') to their schedule. I've read the FireBase documentation, and found a section on indexing.

So I've created following structure (date = YYYYMMDD and time = HHMMSS):

{ 
     appointments : 
         'id1' : { 'date' : '20141207', 'time' : '170000', worker : 'worker1' },
         'id2' : { 'date' : '20141208', 'time' : '170000', worker : 'worker1' }
}

I've added an index for date, time and worker, to be able to query data like this (e.g. fetch all appointments for today):

curl -X GET 'https://myapp.firebaseio.com/appointments.json?orderBy="date"&equalsTo="20141207"'

This works as expected and does the job well. The problem is, the number of appointments can grow exponentially (about a year from now, there could be 100000+ appointments). Is it a good approach to use these indexes? Another option would be to store the date and time also separately, like this:

{
    '20141207' : 
        { '170000' : { 'id1' : true } },
    '20141208' : 
        { '170000' : { 'id2' : true } }
}

In order to ensure that appointments can be fetched per day very fast. Or is FireBase able to handle this just using indexes?

Answer 1

The number of records in the path won't be an issue; Firebase is a scalable, real-time back end that handles hundreds of thousands of concurrent connections and millions of nodes. Querying should be fast. This is the point of an index and, like all things Firebase, must meet our standards of speed and excellence.

Be sure to read about '.indexOn' and to implement this in your security rules:

{
  "rules": {
     "appointments": {
        ".indexOn": ["date", "time", "worker"]
     }
  }
}

Also, your real limitation here will be the bandwidth of transferring data over the tubes, so be sure to limit your results in some manner and paginate:

curl -X GET 'https://myapp.firebaseio.com/appointments.json?orderBy="date"&equalsTo="20141207"&limitToFirst=100'