GemFire CacheLoader from Cassandra

Question

In my project using GemFire with Cassandra, my CacheLoader was able to pull data from Cassandra, however, the data was not available in GemFire. After getting the data from Cassandra, the data is not being updated in GemFire.

I have used code below:

public class TestLoader implements CacheLoader, Declarable {

    @Autowired
    private CassandraOperations cassandraOperations;

    @Override
    public void init(Properties props) {}

    @Override
    public Test load(LoaderHelper helper) {
        String key = (String) helper.getKey();
        key = key.replaceAll("\\s", "");
        Select select = QueryBuilder.select().from("Testing");
        select.where(QueryBuilder.eq(testkey, key));
        return cassandraOperations.selectOne(select, TEST.class);
    }

    @Override
    public void close() {}
}

Test - Domain class Name Testing - Cassandra Table name testkey - Cassandra primary key

Answer 1

So, your CacheLoader appears to be fine, other than the select.where(..) statement, which should read select.where(QueryBuilder.eq("testkey", key));. Clearly this code would not compile. Beyond this, though, I too am curious about your configuration and topology as Xiawei Zhang points out in the comment above. Clearly you are using a client/server topology, but the extent of which is not apparent, nor is your configuration.

You should be registering this CacheLoader on the server's cache PARTITION Region, not on the client Region (if that is the case). Regardless of whether the server cache Region is a PARTITION or a PARTITION_REDUNDANT should not really matter.

However, there is also a lot you need to understand in terms of your topology, which Xiawei Zhang also hints at.

In a typical setup, your (cache) client makes a request for a cache entry (by key) in a PROXY or CACHING_PROXY client Region. If the entry is not available locally or is invalid (in the CACHING_PROXY configuration), then the request is forwarded (NOTE: with PROXY, the request is always forwarded) to the server having a Region by the same name (e.g. "Test"). If the entry is not available in the server's Region, and a CacheLoader has been registered, then the CacheLoader with that key is invoked. In your case, the data for the entry will be loaded from Cassandra (as shown above) and "put" into the server Region using the configured CacheLoader (again, must be configured server-side), and then returned to the client.

Now, if you have multiple clients involved (which might possibly be your case), then you need to make sure you have "subscription" enabled, and you need to register interests in the specific key or keys needed by your clients. Alternatively, you can use CQs (recommended) to define a (GemFire OQL) query predicate to more succinctly express the data you are interested in receiving notifications for.

Subscription is configured/enabled on the Pool associated with the client PROXY|CACHING_PROXY Region and "interests" (for example, by key(s) or using regex, with overloaded variants) is expressed with the Region API.

You will find more information on Configuring Client/Server Event Messaging here. Additionally, you should consider Continuous Queries.

You should also make sure you really understand client/server messaging, especially with multiple clients in your topology, by reading this.

There are other configuration options available, server-side, such as consistency checking, especially when you have redundancy configured in your PRs and 1 of your clients may possibly be hitting a redundant copy, but that should be less of an issue here.

Technically, it is more likely to be an issue with your client queue since client queues are updated asynchronously and notifications sent from a server to a client are indeterministic as the documentation points out...

5. The event for entry X is sent out of the queue to Client2. When this happens is indeterminate.

Anyhow, start with this. Make sure that the request/load/update from a single client is consistent first, then involve other clients. Make sure you have expressed the right interests policies, etc. Read the documentation in the links I provided.

Hope this helps!

Regards, John