SpringBoot建立SSH通道整合S3Elasticache（Redis），并实现Redis多库切换

本文主要是介绍SpringBoot建立SSH通道整合S3Elasticache（Redis），并实现Redis多库切换，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

---

一、背景

使用Spring Boot自带的redis框架，访问S3的Elasticache（Redis），并从Redis的多个DB中同时取数据。

1、问题

• S3的Redis缓存服务，官方文档中指出Elasticache不能从外部访问（复杂、不成功）
• 但是可以通过同一个VPC下的AWS EC2来进行访问
• 本地开发调试的时候怎么去连redis呢？

2、解决

• 可以建立ssh通道，通过EC2作为跳板机进行端口转发，来访问AWS的Redis缓存服务

---

二、建立SSH通道

1、pom引入依赖

        <!-- ssh --><dependency><groupId>com.jcraft</groupId><artifactId>jsch</artifactId><version>0.1.55</version></dependency>

2、创建sshconfig

• ssh.yml

sshconfig:#监听的本地端口local-port: 10010#远程的redis地址remote-host: xxxxxxxxxxxx.cache.amazonaws.com.cn#远程redis端口号remote-port: 6379ssh:#EC2实例的地址host: xxxxxxxxxxxxxx.compute.amazonaws.com.cnport: 22user: ubuntupassword:#EC2的秘钥对pem_file_path: /root/.aws/xxxxxxx-devops.pem

3、SSHConnection 程序

• 这样当从程序启动的时候，可以将createSSH() 写入静态代码快，直接加载开启通道

import com.jcraft.jsch.JSch;
import com.jcraft.jsch.Session;
import lombok.extern.slf4j.Slf4j;import java.io.InputStream;
import java.util.Properties;/*** Through EC2 as a jumpServer, create SSH tunnel to access redis service.*/@Slf4j
public class SSHConnection {private static Integer localPort;private static String remoteHost;private static int remotePort;private static String user;private static String password;private static String path;private static String host;private static int port;private static Session session = null;static {try {// Get ss configuration file path.InputStream is = SSHConnection.class.getClassLoader().getResourceAsStream("ssh.yml");Properties prop = new Properties();prop.load(is);// Get each value.localPort = Integer.valueOf(prop.getProperty("local-port"));remoteHost = prop.getProperty("remote-host");remotePort = Integer.valueOf(prop.getProperty("remote-port"));user = prop.getProperty("user");password = prop.getProperty("password");path = prop.getProperty("pem_file_path");host = prop.getProperty("host");port = Integer.valueOf(prop.getProperty("port"));} catch (Exception e) {log.error("File not found exception: " + e);}}/*** Create ssh connection and set port forwarding.*/public static void createSSH() {JSch jsch = new JSch();try {if (path != null) {jsch.addIdentity(path);}session = jsch.getSession(user, host, port);if (path == null) {session.setPassword(password);}session.setConfig("StrictHostKeyChecking", "no");session.connect();int assinged_port = session.setPortForwardingL(localPort, remoteHost, remotePort);log.info("The ssh connection is OK.");} catch (Exception e) {if (null != session) {//close ssh connection.session.disconnect();}log.error("Create ssh connection exception: " + e);}}/*** Close ssh connection.*/public static void closeSSH() {log.info("The ssh connection is closed ! ");session.disconnect();}
}

---

三、Spring boot整合Redis

1、引入依赖

刚开始pool使用的spring boot默认的lettuce连接池，但是程序有时候出现错误，支持不太友好，又换回了Jedis连接池。

 <!-- redis 去除默认的lettuce连接池--><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-data-redis</artifactId><exclusions><exclusion><groupId>io.lettuce</groupId><artifactId>lettuce-core</artifactId></exclusion></exclusions></dependency><!-- commons-pool2  连接池工具包 --><dependency><groupId>org.apache.commons</groupId><artifactId>commons-pool2</artifactId><version>2.6.0</version></dependency><!-- redis.clients/jedis客户端--><dependency><groupId>redis.clients</groupId><artifactId>jedis</artifactId><version>2.9.3</version></dependency>

2、配置信息

• application.yml

spring:redis:host: localhost#要和SSH监听的本地端口一致port: 10010 #超时时间设置 这里要ms 因为后面用到的是Durationtimeout: 0msjedis:pool:max-active: 20 #最大连接数max-wait: 3000 #最大连接等待超时时间max-idle: 20 #最大连接空闲数# 0:Could not get a resource from the poolmin-idle: 10 #最小空闲数

#最大活动对象数     
redis.pool.maxTotal=1000    
#最大能够保持idel状态的对象数      
redis.pool.maxIdle=100  
#最小能够保持idel状态的对象数   
redis.pool.minIdle=50    
#当池内没有返回对象时，最大等待时间    
redis.pool.maxWaitMillis=10000    
#当调用borrow Object方法时，是否进行有效性检查    
redis.pool.testOnBorrow=true    
#当调用return Object方法时，是否进行有效性检查    
redis.pool.testOnReturn=true  
#“空闲链接”检测线程，检测的周期，毫秒数。如果为负值，表示不运行“检测线程”。默认为-1.  
redis.pool.timeBetweenEvictionRunsMillis=30000  
#向调用者输出“链接”对象时，是否检测它的空闲超时；  
redis.pool.testWhileIdle=true  
# 对于“空闲链接”检测线程而言，每次检测的链接资源的个数。默认为3.  
redis.pool.numTestsPerEvictionRun=50  
#redis服务器的IP    
redis.ip=xxxxxx  
#redis服务器的Port    
redis1.port=6379   详解：
maxActive：控制一个pool可分配多少个jedis实例，通过pool.getResource()来获取；如果赋值为-1，则表示不限制；如果pool已经分配了maxActive个jedis实例，则此时pool的状态就成exhausted了，在JedisPoolConfig
maxIdle：控制一个pool最多有多少个状态为idle的jedis实例；
whenExhaustedAction：表示当pool中的jedis实例都被allocated完时，pool要采取的操作；默认有三种WHEN_EXHAUSTED_FAIL（表示无jedis实例时，直接抛出NoSuchElementException）、WHEN_EXHAUSTED_BLOCK（则表示阻塞住，或者达到maxWait时抛出JedisConnectionException）、WHEN_EXHAUSTED_GROW（则表示新建一个jedis实例，也就说设置的maxActive无用）；
maxWait：表示当borrow一个jedis实例时，最大的等待时间，如果超过等待时间，则直接抛出JedisConnectionException；
testOnBorrow：在borrow一个jedis实例时，是否提前进行alidate操作；如果为true，则得到的jedis实例均是可用的；
testOnReturn：在return给pool时，是否提前进行validate操作；
testWhileIdle：如果为true，表示有一个idle object evitor线程对idle object进行扫描，如果validate失败，此object会被从pool中drop掉；这一项只有在timeBetweenEvictionRunsMillis大于0时才有意义；
timeBetweenEvictionRunsMillis：表示idle object evitor两次扫描之间要sleep的毫秒数；
numTestsPerEvictionRun：表示idle object evitor每次扫描的最多的对象数；
minEvictableIdleTimeMillis：表示一个对象至少停留在idle状态的最短时间，然后才能被idle object evitor扫描并驱逐；这一项只有在timeBetweenEvictionRunsMillis大于0时才有意义；
softMinEvictableIdleTimeMillis：在minEvictableIdleTimeMillis基础上，加入了至少minIdle个对象已经在pool里面了。如果为-1，evicted不会根据idle time驱逐任何对象。如果minEvictableIdleTimeMillis>0，则此项设置无意义，且只有在timeBetweenEvictionRunsMillis大于0时才有意义；
lifo：borrowObject返回对象时，是采用DEFAULT_LIFO（last in first out，即类似cache的最频繁使用队列），如果为False，则表示FIFO队列；其中JedisPoolConfig对一些参数的默认设置如下：
testWhileIdle=true
minEvictableIdleTimeMills=60000
timeBetweenEvictionRunsMillis=30000
numTestsPerEvictionRun=-1

3、RedisConfig的编写（切库处理配置）

之前使用lettuce设定切库，但是有多个请求时，取出的库中的数据是混乱的，是线程不安全的，加了锁还是不行，索性就写了多个redisTemplat，每一个redisTemplat都只负责一个库

• 同时操作Redis的三个库
• 多个connectionFactory设定不同的DB
• 建立多个 RedisTemplate<String, Object> redisTemplat

import com.bmw.fs.hp.service.hva.utils.SSHConnection;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.data.redis.RedisAutoConfiguration;
import org.springframework.boot.autoconfigure.data.redis.RedisReactiveAutoConfiguration;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.connection.RedisStandaloneConfiguration;
import org.springframework.data.redis.connection.jedis.JedisClientConfiguration;
import org.springframework.data.redis.connection.jedis.JedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.serializer.RedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;
import redis.clients.jedis.JedisPoolConfig;import java.time.Duration;@Configuration
@EnableAutoConfiguration(exclude = {RedisAutoConfiguration.class, RedisReactiveAutoConfiguration.class}) // 注意exclude
public class RedisConfig {//开启ssh通道static {SSHConnection.createSSH();}@Value("${spring.redis.host}")private String host;@Value("${spring.redis.port}")private int port;@Value("${spring.redis.timeout}")private Duration timeout;@Value("${spring.redis.jedis.pool.max-active}")private int maxActive;@Value("${spring.redis.jedis.pool.max-wait}")private long maxWait;@Value("${spring.redis.jedis.pool.max-idle}")private int maxIdle;@Value("${spring.redis.jedis.pool.min-idle}")private int minIdle;/** * DB11 * DB12 * DB13*/@Bean(name = "redisConnectionFactory11")public RedisConnectionFactory redisConnectionFactory11() {//Redis环境配置（单机、哨兵、集群）RedisStandaloneConfiguration standaloneConfiguration = new RedisStandaloneConfiguration();standaloneConfiguration.setHostName(host);standaloneConfiguration.setPort(port);//设定这个factory访问的DBstandaloneConfiguration.setDatabase(11);// Jedis客户端配置JedisClientConfiguration jedisClientConfiguration = getJedisClientConfiguration();return new JedisConnectionFactory(standaloneConfiguration, jedisClientConfiguration);}@Bean(name = "redisConnectionFactory12")public RedisConnectionFactory redisConnectionFactory12() {RedisStandaloneConfiguration standaloneConfiguration = new RedisStandaloneConfiguration();standaloneConfiguration.setHostName(host);standaloneConfiguration.setPort(port);standaloneConfiguration.setDatabase(12);JedisClientConfiguration jedisClientConfiguration = getJedisClientConfiguration();return new JedisConnectionFactory(standaloneConfiguration, jedisClientConfiguration);}@Bean(name = "redisConnectionFactory13")public RedisConnectionFactory redisConnectionFactory13() {RedisStandaloneConfiguration standaloneConfiguration = new RedisStandaloneConfiguration();standaloneConfiguration.setHostName(host);standaloneConfiguration.setPort(port);standaloneConfiguration.setDatabase(13);JedisClientConfiguration jedisClientConfiguration = getJedisClientConfiguration();return new JedisConnectionFactory(standaloneConfiguration, jedisClientConfiguration);}@Bean(name = "redisTemplate11")public RedisTemplate<String, Object> redisTemplate11() {RedisTemplate<String, Object> redisTemplateObject = new RedisTemplate<String, Object>();redisTemplateObject.setConnectionFactory(redisConnectionFactory11());//进行序列化setSerializer(redisTemplateObject);redisTemplateObject.afterPropertiesSet();return redisTemplateObject;}@Bean(name = "redisTemplate12")public RedisTemplate<String, Object> redisTemplat12() {RedisTemplate<String, Object> redisTemplateObject = new RedisTemplate<String, Object>();redisTemplateObject.setConnectionFactory(redisConnectionFactory12());setSerializer(redisTemplateObject);redisTemplateObject.afterPropertiesSet();return redisTemplateObject;}@Bean(name = "redisTemplate13")public RedisTemplate<String, Object> redisTemplate13() {RedisTemplate<String, Object> redisTemplateObject = new RedisTemplate<String, Object>();redisTemplateObject.setConnectionFactory(redisConnectionFactory13());setSerializer(redisTemplateObject);redisTemplateObject.afterPropertiesSet();return redisTemplateObject;}// 必须配置这个默认的，否则程序报错：匹配到多个Bean -> connectionFactory@Bean(name = "redisTemplate")public RedisTemplate<String, Object> redisTemplate() {RedisTemplate<String, Object> redisTemplateObject = new RedisTemplate<String, Object>();redisTemplateObject.setConnectionFactory(redisConnectionFactory11());setSerializer(redisTemplateObject);redisTemplateObject.afterPropertiesSet();return redisTemplateObject;}/*** Set configuration file for reids connection pool.** @return JedisPoolConfig*/private JedisPoolConfig jedisPoolConfig() {JedisPoolConfig poolConfig = new JedisPoolConfig();poolConfig.setMaxTotal(maxActive);poolConfig.setMaxIdle(maxIdle);poolConfig.setMinIdle(minIdle);//当池子内没用可用连接时，最大的等待时间poolConfig.setMaxWaitMillis(maxWait);//在获取连接时，检查有效性poolConfig.setTestOnBorrow(true);poolConfig.setTestOnReturn(true);//在空时检查连接有效性poolConfig.setTestWhileIdle(true);return poolConfig;}/*** Building Jedis client.** @return JedisClientConfiguration*/private JedisClientConfiguration getJedisClientConfiguration() {//通过构造器来构造客户端配置JedisClientConfiguration.JedisClientConfigurationBuilder builder = JedisClientConfiguration.builder();//  加入超时配置，不加的话，Jedis长时间不操作，连接会关闭，导致异常：//org.springframework.data.redis.RedisConnectionFailureException: Cannot get Jedis connection; //nested exception is redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketException: Connection resetif (timeout != null) {builder.readTimeout(timeout).connectTimeout(timeout);}//修改连接池配置builder.usePooling().poolConfig(jedisPoolConfig());return builder.build();}/*** Serializing values.** @param template*/private void setSerializer(RedisTemplate<String, Object> template) {RedisSerializer<String> stringSerializer = new StringRedisSerializer();template.setKeySerializer(stringSerializer);template.setValueSerializer(stringSerializer);}
}

4、Redis操作的工具类

• 设定不同库的RedisUtils
• 引入不同库的 redisTemplate
• 这里只写了DB9的工具类，其他的也是一样的就不再赘述

import org.springframework.data.redis.core.RedisConnectionUtils;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.SetOperations;
import org.springframework.stereotype.Component;import javax.annotation.Resource;
import java.util.Set;
import java.util.concurrent.TimeUnit;/*** Get values from redisDB_9*/
@Component
public class RedisSerDB9 {@Resource(name = "redisTemplate9")private RedisTemplate redisTemplate;/*** Take out all the keys of the prefix.** @param prefix* @return Set<String> : all keys*/public Set<String> getAllKey(String prefix) {Set keys = redisTemplate.keys(prefix + "*");RedisConnectionUtils.unbindConnection(redisTemplate.getConnectionFactory());return keys;}/*** Add value to set.** @param key* @param value* @param expireTime* @return boolean*/public boolean add(String key, Object value, Long expireTime) {boolean result = false;try {SetOperations<String, Object> set = redisTemplate.opsForSet();set.add(key, value);//设定过期时间，单位是秒redisTemplate.expire(key, expireTime, TimeUnit.SECONDS);result = true;} catch (Exception e) {e.printStackTrace();} finally {// 解除连接RedisConnectionUtils.unbindConnection(redisTemplate.getConnectionFactory());}return result;}/*** Get values from set.** @param key* @return Set<Object>: All values of the current key.*/public Set<Object> setMembers(String key) {SetOperations<String, Object> set = redisTemplate.opsForSet();Set<Object> members = set.members(key);RedisConnectionUtils.unbindConnection(redisTemplate.getConnectionFactory());return members;}
}

---

四、两个大坑

1、 长时间未操作，连接重置

• org.springframework.data.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketException: Connection reset

在RedisConfig刚开始写的connectionFactory中，没有配置timeout导致的

        if (timeout != null) {builder.readTimeout(timeout).connectTimeout(timeout);}

链接：这位博主的博客中有写到 ->

2、长时间未操作，无法获取resource

• Error: Could not get a resource from the pool

connectionFactory，配置timeout之后，长时间未操作，还是显示报错：Could not get a resource from the pool
我这里是因为pool中配置的参数有问题：

之前min-idle配置的是0，导致没有空闲的连接数。 改成非0就行了

    jedis:pool:max-active: 20max-wait: 3000max-idle: 20#0:Could not get a resource from the poolmin-idle: 10

---

五、总结

• 多想多做多尝试，不懂就问
• 别钻牛角尖，这个方法不行，就赶紧换下个方法，掌握方法论，在短时间内找到最有效的解决方法
• 技术基于业务场景，多思考应用场景，拓展思维

这篇关于SpringBoot建立SSH通道整合S3Elasticache（Redis），并实现Redis多库切换的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！

---