How to make xmemcached faster

Question

I have a project wherein we are trying to shift major portion of a php code in a java service. One of the critical portion is memcached transactions.

I started off with xmemcached. I am using binary protocol and connection pool of 5 for my test. I have 10000 items in my bucket two memcached instances on my machine itself. I hit get on all 10000 items in php-memcached and in java using xmemcached.

In both sides, I am using persistent connections. Same number of items are get. And PHP is doing some additional processing, by traversing array of 10K values. Still the result is that php finishes in 0.9 to 1.2 seconds and java takes 1.6 to 2 seconds.

This much delay is unacceptable for our project. What more can be done to speed up xmemcached? Please help

Here's php code:

$mem = new Memcached("p");
if(count($mem->getServerList()) <= 0) {
    $mem->addServer("10.90.15.104",11211);
    $mem->addServer("10.90.15.104",11311);
}
for($j=0; $j<1000; $j++) {
$startTime = microtime(true);

$memVals = array();
for($i=1; $i<=10000; $i++) {
    $memVals[$i] = $mem->get($i);
}
$count=0;
foreach($memVals as $key=>$val) {
    $val2 = $val;
    if($val2 != '') {
        $count++;
    }
}

$endTime = microtime(true);
file_put_contents("/tmp/time.log",($endTime-$startTime)."\n",FILE_APPEND);
//echo "count>>$count\n";
//echo "time taken: ".($endTime-$startTime)."\n";

And here's java code:

import AxMemcached.Adfeed;
import AxMemcached.AxMemcachedClientFactory;
import java.util.Calendar;
import net.rubyeye.xmemcached.MemcachedClient;

public class BigGet2 {
    public static void main(String[] args) {
        BigGet2 self = new BigGet2();
        for (int j = 0; j < 1000; j++) {
            Long timeStart = Calendar.getInstance().getTimeInMillis();
            String value = "";
            try {
                MemcachedClientBuilder builder = new XMemcachedClientBuilder(
                        net.rubyeye.xmemcached.utils.AddrUtil.getAddresses("10.90.15.104:11211 10.90.15.104:11311"));
                builder.setConnectionPoolSize(10);
                builder.setCommandFactory(new BinaryCommandFactory());
                builder.setSessionLocator(new ArrayMemcachedSessionLocator(net.rubyeye.xmemcached.HashAlgorithm.ONE_AT_A_TIME));
                MemcachedClient memcachedClient = builder.build();
                for (int i = 1; i <= 10000; i++) {
                    value = memcachedClient.get(Integer.toString(i));
                    memcachedClient.get(Integer.toString(i));
                }
                Long timeEnd = Calendar.getInstance().getTimeInMillis();
                Long timeTaken = timeEnd - timeStart;
                System.out.println(timeTaken);
            } catch (Exception e) {

            }
        }
    }
}

Answer 1

The following is a singleton design pattern for creating memcached connection. If your application client creates a new connection for every request, then you will see a ton of threads per connection inside the JVM, which will consume a lot of memory and thereby cause excessive garbage collection which will slow down your app.

@Singleton
public class MemcacheConnectionManager {


    public MemcachedClient clientBuilder() throws IOException {
        MemcachedClientBuilder builder = new XMemcachedClientBuilder(
                AddrUtil.getAddresses("localhost:11211"));
        builder.setConnectionPoolSize(5);
        MemcachedClient cacheClient = builder.build();
        return cacheClient;

    }
}

public class memcacheconsumer{
..
someCacheConsumerMethod(){
..
if(memcachedClient==null)
                memcachedClient = memcacheConnection.clientBuilder();
...
}
}