Aerospike Memory Calculation for multiple Sets

Question

I need to store a large number of Keys in an shared memory Aerospike Primary Index. My specs are as follows: Namespaces : 2, Number of Records : 500 M, TTL : 1D, Replication: 2

According to Aerospike Capacity planning: By String Key should need 64 bytes × (replication factor) × (number of records)

= 64 * 2 * 0.5 * 10^9 = 64 GB of space per namespace.

Question is the the index separate for every set? For EG, if I have 20 sets per namespace, do I need 20 * 64 gigs in memory for that namespace?

Answer 1

The primary index is not separate for every set.

According to https://aerospike.com/developer/blog/of-queries-and-indexes, "In Aerospike, there is only one system-defined primary index for each namespace" which makes sense because you can change the way you store your primary index - shared memory (shmem), persistent memory (PMem) and flash (NVMe SSDs) at the namespace level.

Set is an optional unit to group records, its the equivalent of a table but without an explicit schema, for sets you have a dedicated optional Set Indexes which allows you to index the membership of records to their set.

Answer 2

Your calculation of 64GB per namespace is correct. And per previous answer, Primary Index storage is not separated by sets. Only by namespace.

Strictly speaking, assuming your number of records - million is powers of 10, you need to divide by 1024 instead of 1000. So, 10^9 / (1024 * 1024 * 1024) to get the GiB - gibibytes numbers - so more like 59.6 GiB. Further, you have to size the memory of each node. So, if you have a 4 node cluster, each node will need 15GiB of SHMEM for the PI alone. Memory is allocated in "arena stage chunks" - index-stage-size which is 1 GiB by default. The second namespace will use its own set of arena stages. But then, additionally, you must leave 10% headroom for stop-writes. So that pushes you back up to 66/number of nodes or about 16.6 GiB per node per namespace, sizing wise, in the 4 node example above.

(https://aerospike.com/docs/server/reference/configuration?context=all&version=7&search=stop#namespace__stop-writes-sys-memory-pct)

Think of the arena stage as two linked lists of 64B allocation elements - used list and free list. When your records expire, that 64B is marked free. Next allocation on record creation uses the a 64B element from the free list.