Hi I need to do multiple insertions of the form
SADD key value
I have the key value pair and needed to know how to perform mass insertions using JAVA . I have written a file in the Redis Protocol. How to proceed further
If you are doing the actual read/write operations through Spring CacheManager with RedisTemplate configured to use Redis as the cache, you can also use the executePipelined method of RedisTemplate which takes a callback as an argument. The callback needs to define the doInRedis method which does the work (read/write operations) in Redis that you want to do in a batch.
Following code shows inserting a List of objects wrapped in a CacheableObject interface that has a getKey() and getValue() by calling redisTemplate.opsForHash().put().
@Component
public class RedisClient {
@Autowired
RedisTemplate redisTemplate;
//batch-insert using Redis pipeline, a list of objects into the cache specified by cacheName
public void put(String cacheName, List<CacheableObject> objects) {
try {
this.redisTemplate.executePipelined(new RedisCallback<Object>() {
@Override
public Object doInRedis(RedisConnection connection) throws DataAccessException {
for(CacheableObject object: objects) {
redisTemplate.opsForHash().put(cacheName, object.getKey(), object.getValue());
}
return null;
}
});
}
catch(Exception e) {
log.error("Error inserting objects into Redis cache: {}", e.getMessage());
}
}
RedisTemplate itself is configured using a configuration class such as the following:
@Configuration
@EnableCaching
public class RedisCacheConfig extends CachingConfigurerSupport implements
CachingConfigurer {
@Value("${redis.hostname}")
private String redisHost;
@Value("${redis.port}")
private int redisPort;
@Value("${redis.timeout.secs:1}")
private int redisTimeoutInSecs;
@Value("${redis.socket.timeout.secs:1}")
private int redisSocketTimeoutInSecs;
@Value("${redis.ttl.hours:1}")
private int redisDataTTL;
@Bean
JedisConnectionFactory jedisConnectionFactory() {
RedisStandaloneConfiguration redisStandaloneConfiguration = new RedisStandaloneConfiguration(redisHost, redisPort);
return new JedisConnectionFactory(redisStandaloneConfiguration);
}
@Bean
public RedisTemplate<Object, Object> redisTemplate() {
RedisTemplate<Object, Object> redisTemplate = new RedisTemplate<Object, Object>();
redisTemplate.setConnectionFactory(jedisConnectionFactory());
return redisTemplate;
}
@Bean
public RedisCacheManager redisCacheManager (JedisConnectionFactory jedisConnectionFactory) {
RedisCacheConfiguration redisCacheConfiguration =
RedisCacheConfiguration.defaultCacheConfig().disableCachingNullValues()
.entryTtl(Duration.ofHours(redisDataTTL)) .serializeValuesWith(RedisSerializationContext.SerializationPair.fromSerializer(RedisSerializer.java()));
redisCacheConfiguration.usePrefix();
RedisCacheManager redisCacheManager =
RedisCacheManager.RedisCacheManagerBuilder.
fromConnectionFactory(jedisConnectionFactory)
.cacheDefaults(redisCacheConfiguration).build();
redisCacheManager.setTransactionAware(true);
return redisCacheManager;
}
@Bean
public JedisPoolConfig poolConfig() {
final JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
jedisPoolConfig.setTestOnBorrow(true);
jedisPoolConfig.setMaxTotal(100);
jedisPoolConfig.setMaxIdle(100);
jedisPoolConfig.setMinIdle(10);
jedisPoolConfig.setTestOnReturn(true);
jedisPoolConfig.setTestWhileIdle(true);
return jedisPoolConfig;
}
@Override
public CacheErrorHandler errorHandler() {
return new RedisCacheErrorHandler();
}
}
If you have inputs written to Redis protocol format then why don't just use pipe mode of redis-cli or nc? It's explained from http://redis.io/topics/mass-insert.
If you have mass (key, value) inputs then you can use Jedis to perform sadd with pipelining to get higher performance.
Below example assumes that iter (Iterator) has elements each item is key"\t"value form.