Mybatis Plus快速重构真批量sql入库操作

2024-09-08 02:52
文章标签 sql 操作 快速 批量 重构 mybatis plus database 入库

本文主要是介绍Mybatis Plus快速重构真批量sql入库操作，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

Mybatis快速重构真批量sql入库操作

基本思路

重构mybatis默认方法saveBatch和saveOrUpdateBatch的实现

基本步骤
  1. 真批量保存实现类InsertBatchMethod
  2. 真批量更新实现类MysqlInsertOrUpdateBath
  3. 注册InsertBatchMethod和MysqlInsertOrUpdateBath到EasySqlInjector
  4. 注册EasySqlInjector到Mybatis配置类
  5. 实现自定义RootMapper
  6. 要实现真批量操作的实体类的Mapper继承RootMapper
  7. 自定义MyServiceImpl实现类
  8. 实体类ServiceImpl继承MyServiceImpl
  9. 批量方法的使用
详细步骤

  1. 真批量保存实现类InsertBatchMethod

    import com.baomidou.mybatisplus.core.injector.AbstractMethod;
import com.baomidou.mybatisplus.core.metadata.TableInfo;
import lombok.extern.slf4j.Slf4j;
import org.apache.ibatis.executor.keygen.NoKeyGenerator;
import org.apache.ibatis.mapping.MappedStatement;
import org.apache.ibatis.mapping.SqlSource;/*** 批量插入方法实现*/
@Slf4j
public class InsertBatchMethod extends AbstractMethod {@Overridepublic MappedStatement injectMappedStatement(Class<?> mapperClass, Class<?> modelClass, TableInfo tableInfo) {final String sql = "<script>insert into %s %s values %s</script>";final String fieldSql = prepareFieldSql(tableInfo);final String valueSql = prepareValuesSql(tableInfo);final String sqlResult = String.format(sql, tableInfo.getTableName(), fieldSql, valueSql);log.debug("sqlResult----->{}", sqlResult);SqlSource sqlSource = languageDriver.createSqlSource(configuration, sqlResult, modelClass);// 第三个参数必须和RootMapper的自定义方法名一致return this.addInsertMappedStatement(mapperClass, modelClass, "insertBatch", sqlSource, new NoKeyGenerator(), null, null);}private String prepareFieldSql(TableInfo tableInfo) {StringBuilder fieldSql = new StringBuilder();fieldSql.append(tableInfo.getKeyColumn()).append(",");tableInfo.getFieldList().forEach(x -> {fieldSql.append(x.getColumn()).append(",");});fieldSql.delete(fieldSql.length() - 1, fieldSql.length());fieldSql.insert(0, "(");fieldSql.append(")");return fieldSql.toString();}private String prepareValuesSql(TableInfo tableInfo) {final StringBuilder valueSql = new StringBuilder();valueSql.append("<foreach collection=\"list\" item=\"item\" index=\"index\" open=\"(\" separator=\"),(\" close=\")\">");valueSql.append("#{item.").append(tableInfo.getKeyProperty()).append("},");tableInfo.getFieldList().forEach(x -> valueSql.append("#{item.").append(x.getProperty()).append("},"));valueSql.delete(valueSql.length() - 1, valueSql.length());valueSql.append("</foreach>");return valueSql.toString();}
}

  2. 真批量更新实现类MysqlInsertOrUpdateBath

    import com.baomidou.mybatisplus.core.injector.AbstractMethod;
import com.baomidou.mybatisplus.core.metadata.TableInfo;
import org.apache.ibatis.executor.keygen.NoKeyGenerator;
import org.apache.ibatis.mapping.MappedStatement;
import org.apache.ibatis.mapping.SqlSource;
import org.springframework.util.StringUtils;public class MysqlInsertOrUpdateBath extends AbstractMethod {@Overridepublic MappedStatement injectMappedStatement(Class<?> mapperClass, Class<?> modelClass, TableInfo tableInfo) {final String sql = "<script>insert into %s %s values %s ON DUPLICATE KEY UPDATE %s</script>";final String tableName = tableInfo.getTableName();final String filedSql = prepareFieldSql(tableInfo);final String modelValuesSql = prepareModelValuesSql(tableInfo);final String duplicateKeySql =prepareDuplicateKeySql(tableInfo);final String sqlResult = String.format(sql, tableName, filedSql, modelValuesSql,duplicateKeySql);//System.out.println("savaorupdatesqlsql="+sqlResult);SqlSource sqlSource = languageDriver.createSqlSource(configuration, sqlResult, modelClass);return this.addInsertMappedStatement(mapperClass, modelClass, "mysqlInsertOrUpdateBath", sqlSource, new NoKeyGenerator(), null, null);}/*** 准备ON DUPLICATE KEY UPDATE sql* @param tableInfo* @return*/private String prepareDuplicateKeySql(TableInfo tableInfo) {final StringBuilder duplicateKeySql = new StringBuilder();if(!StringUtils.isEmpty(tableInfo.getKeyColumn())) {duplicateKeySql.append(tableInfo.getKeyColumn()).append("=values(").append(tableInfo.getKeyColumn()).append("),");}tableInfo.getFieldList().forEach(x -> {duplicateKeySql.append(x.getColumn()).append("=values(").append(x.getColumn()).append("),");});duplicateKeySql.delete(duplicateKeySql.length() - 1, duplicateKeySql.length());return duplicateKeySql.toString();}/*** 准备属性名* @param tableInfo* @return*/private String prepareFieldSql(TableInfo tableInfo) {StringBuilder fieldSql = new StringBuilder();fieldSql.append(tableInfo.getKeyColumn()).append(",");tableInfo.getFieldList().forEach(x -> {fieldSql.append(x.getColumn()).append(",");});fieldSql.delete(fieldSql.length() - 1, fieldSql.length());fieldSql.insert(0, "(");fieldSql.append(")");return fieldSql.toString();}private String prepareModelValuesSql(TableInfo tableInfo){final StringBuilder valueSql = new StringBuilder();valueSql.append("<foreach collection=\"list\" item=\"item\" index=\"index\" open=\"(\" separator=\"),(\" close=\")\">");if(!StringUtils.isEmpty(tableInfo.getKeyProperty())) {valueSql.append("#{item.").append(tableInfo.getKeyProperty()).append("},");}tableInfo.getFieldList().forEach(x -> valueSql.append("#{item.").append(x.getProperty()).append("},"));valueSql.delete(valueSql.length() - 1, valueSql.length());valueSql.append("</foreach>");return valueSql.toString();}
}

  3. 注册InsertBatchMethod和MysqlInsertOrUpdateBath到EasySqlInjector

    import com.baomidou.mybatisplus.core.injector.AbstractMethod;
import com.baomidou.mybatisplus.core.injector.DefaultSqlInjector;
import com.baomidou.mybatisplus.extension.injector.methods.AlwaysUpdateSomeColumnById;
import com.baomidou.mybatisplus.extension.injector.methods.InsertBatchSomeColumn;
import org.springframework.stereotype.Component;
import java.util.List;@Component
public class EasySqlInjector extends DefaultSqlInjector {@Overridepublic List<AbstractMethod> getMethodList(Class<?> mapperClass) {List<AbstractMethod> methodList = super.getMethodList(mapperClass);// 真批量插入接口methodList.add(new InsertBatchSomeColumn());// 总是更新字段，不忽略nullmethodList.add(new AlwaysUpdateSomeColumnById(tableFieldInfo ->!tableFieldInfo.getColumn().equals("DELETED")&& !tableFieldInfo.getColumn().equals("ID")&& !tableFieldInfo.getColumn().equals("CREATE_TIME")&& !tableFieldInfo.getColumn().equals("CREATE_USER")&& !tableFieldInfo.getColumn().equals("CREATE_USER_NAME")));// 真批量更新methodList.add(new InsertBatchMethod());methodList.add(new MysqlInsertOrUpdateBath());return methodList;}
}

  4. 注册EasySqlInjector到Mybatis配置类

    @Slf4j
@Configuration
@EnableNacosConfig
@EnableConfigurationProperties
@MapperScan("com.**.dao")
public class OPAConfiguration implements EnvironmentAware,BeanFactoryAware{@Beanpublic xxxMybatisContext xxxMybatisContext(xxxProperties xxxProperties, MybatisPlusProperties properties) throws Exception {xxxMybatisContext xxxMybatisContext=new xxxMybatisContext();// TODO 使用 MybatisSqlSessionFactoryBean 而不是 SqlSessionFactoryBeanMybatisSqlSessionFactoryBean factory = new MybatisSqlSessionFactoryBean();factory.setDataSource(this.dataSource);factory.setVfs(SpringBootVFS.class);factory.setMapperLocations(new PathMatchingResourcePatternResolver().getResources("classpath*:com/xxx/xxx/xxx/mapping/*Mapper.xml"));// TODO 此处必为非 NULLGlobalConfig globalConfig = properties.getGlobalConfig();GlobalConfig newGlobalConfig=new GlobalConfig();BeanUtils.copyProperties(globalConfig, newGlobalConfig);newGlobalConfig.setMetaObjectHandler(new xxxMetaObjectHandler());newGlobalConfig.setSqlInjector(new EasySqlInjector());factory.setGlobalConfig(newGlobalConfig);SqlSessionFactory sqlSessionFactory=factory.getObject();newGlobalConfig.setSqlSessionFactory(sqlSessionFactory);xxxMybatisContext.setSessionFactory(sqlSessionFactory);xxxMybatisContext.setSqlSessionTemplate(new SqlSessionTemplate(sqlSessionFactory));ClassPathMapperScanner scanner = new ClassPathMapperScanner((BeanDefinitionRegistry) beanFactory);scanner.setSqlSessionTemplate(xxxMybatisContext.getSqlSessionTemplate());//   scanner.setBeanNameGenerator(new xxxBeanNameGenerator());scanner.registerFilters();scanner.scan(StringUtils.tokenizeToStringArray("com.xxx.xxx.xxx.mapping", ConfigurableApplicationContext.CONFIG_LOCATION_DELIMITERS));return xxxMybatisContext;}}

  5. 实现自定义RootMapper

    import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import org.apache.ibatis.annotations.Param;
import java.util.List;/*** 根Mapper，给表Mapper继承用的，可以自定义通用方法* {@link BaseMapper}* {@link com.baomidou.mybatisplus.extension.service.IService}* {@link com.baomidou.mybatisplus.extension.service.impl.ServiceImpl}*/
public interface RootMapper<T> extends BaseMapper<T> {/*** 自定义批量插入* 如果要自动填充，@Param(xx) xx参数名必须是 list/collection/array 3个的其中之一*/int insertBatch(@Param("list") List<T> list);/*** 自定义批量新增或更新* 如果要自动填充，@Param(xx) xx参数名必须是 list/collection/array 3个的其中之一*/int mysqlInsertOrUpdateBath(@Param("list") List<T> list);}

  6. 要实现真批量操作的实体类的Mapper继承RootMapper

    @Mapper
public interface BudgetVSActualMapper extends RootMapper<BudgetVSActual> {

  7. 自定义MyServiceImpl实现类

    import com.baomidou.mybatisplus.core.conditions.Wrapper;
import com.baomidou.mybatisplus.core.enums.SqlMethod;
import com.baomidou.mybatisplus.core.metadata.TableInfo;
import com.baomidou.mybatisplus.core.metadata.TableInfoHelper;
import com.baomidou.mybatisplus.core.toolkit.*;
import com.baomidou.mybatisplus.extension.service.IService;
import com.baomidou.mybatisplus.extension.toolkit.SqlHelper;
import com.pwc.sdc.OPA.mapping.RootMapper;
import org.apache.ibatis.binding.MapperMethod;
import org.apache.ibatis.logging.Log;
import org.apache.ibatis.logging.LogFactory;
import org.apache.ibatis.session.SqlSession;
import org.mybatis.spring.SqlSessionUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.ResolvableType;
import org.springframework.transaction.annotation.Transactional;
import java.io.Serializable;
import java.util.*;
import java.util.function.BiConsumer;
import java.util.function.Consumer;
import java.util.function.Function;
import java.util.stream.Collectors;
import java.util.stream.Stream;public  class MyServiceImpl<M extends RootMapper<T>, T> implements IService<T> {protected Log log = LogFactory.getLog(this.getClass());@Autowiredprotected M baseMapper;protected Class<T> entityClass = this.currentModelClass();protected Class<T> mapperClass = this.currentMapperClass();public MyServiceImpl() {}public M getBaseMapper() {return this.baseMapper;}public Class<T> getEntityClass() {return this.entityClass;}/** @deprecated */@Deprecatedprotected boolean retBool(Integer result) {return SqlHelper.retBool(result);}protected Class<T> currentMapperClass() {return (Class)this.getResolvableType().as(MyServiceImpl.class).getGeneric(new int[]{0}).getType();}protected Class<T> currentModelClass() {return (Class)this.getResolvableType().as(MyServiceImpl.class).getGeneric(new int[]{1}).getType();}protected ResolvableType getResolvableType() {return ResolvableType.forClass(ClassUtils.getUserClass(this.getClass()));}/** @deprecated */@Deprecatedprotected SqlSession sqlSessionBatch() {return SqlHelper.sqlSessionBatch(this.entityClass);}/** @deprecated */@Deprecatedprotected void closeSqlSession(SqlSession sqlSession) {SqlSessionUtils.closeSqlSession(sqlSession, GlobalConfigUtils.currentSessionFactory(this.entityClass));}/** @deprecated */@Deprecatedprotected String sqlStatement(SqlMethod sqlMethod) {return SqlHelper.table(this.entityClass).getSqlStatement(sqlMethod.getMethod());}@Transactional(rollbackFor = {Exception.class})public boolean saveBatch(Collection<T> entityList, int batchSize) {if(CollectionUtils.isEmpty(entityList)){return true;}int maxNum=2000;int limit = countStep(entityList.size(), maxNum);List<List<T>> subList = new ArrayList<>();Stream.iterate(0, n -> n + 1).limit(limit).forEach(i -> {subList.add(entityList.stream().skip(i * maxNum).limit(maxNum).collect(Collectors.toList()));});for (List<T> t : subList) {this.baseMapper.insertBatch(t);}return true;}protected String getSqlStatement(SqlMethod sqlMethod) {return SqlHelper.getSqlStatement(this.mapperClass, sqlMethod);}@Transactional(rollbackFor = {Exception.class})public boolean saveOrUpdate(T entity) {if (null == entity) {return false;} else {TableInfo tableInfo = TableInfoHelper.getTableInfo(this.entityClass);Assert.notNull(tableInfo, "error: can not execute. because can not find cache of TableInfo for entity!", new Object[0]);String keyProperty = tableInfo.getKeyProperty();Assert.notEmpty(keyProperty, "error: can not execute. because can not find column for id from entity!", new Object[0]);Object idVal = ReflectionKit.getFieldValue(entity, tableInfo.getKeyProperty());return !StringUtils.checkValNull(idVal) && !Objects.isNull(this.getById((Serializable)idVal)) ? this.updateById(entity) : this.save(entity);}}@Transactional(rollbackFor = {Exception.class})public boolean saveOrUpdateBatch(Collection<T> entityList, int batchSize) {if(CollectionUtils.isEmpty(entityList)){return true;}int maxNum=2000;int limit = countStep(entityList.size(), maxNum);List<List<T>> subList = new ArrayList<>();Stream.iterate(0, n -> n + 1).limit(limit).forEach(i -> {subList.add(entityList.stream().skip(i * maxNum).limit(maxNum).collect(Collectors.toList()));});for (List<T> t : subList) {this.baseMapper.mysqlInsertOrUpdateBath(t);}return true;}private static Integer countStep(Integer size, Integer maxNum) {return (size + maxNum - 1) / maxNum;}@Transactional(rollbackFor = {Exception.class})public boolean updateBatchById(Collection<T> entityList, int batchSize) {String sqlStatement = this.getSqlStatement(SqlMethod.UPDATE_BY_ID);return this.executeBatch(entityList, batchSize, (sqlSession, entity) -> {MapperMethod.ParamMap<T> param = new MapperMethod.ParamMap();param.put("et", entity);sqlSession.update(sqlStatement, param);});}public T getOne(Wrapper<T> queryWrapper, boolean throwEx) {return throwEx ? this.baseMapper.selectOne(queryWrapper) : SqlHelper.getObject(this.log, this.baseMapper.selectList(queryWrapper));}public Map<String, Object> getMap(Wrapper<T> queryWrapper) {return (Map)SqlHelper.getObject(this.log, this.baseMapper.selectMaps(queryWrapper));}public <V> V getObj(Wrapper<T> queryWrapper, Function<? super Object, V> mapper) {return SqlHelper.getObject(this.log, this.listObjs(queryWrapper, mapper));}/** @deprecated */@Deprecatedprotected boolean executeBatch(Consumer<SqlSession> consumer) {return SqlHelper.executeBatch(this.entityClass, this.log, consumer);}protected <E> boolean executeBatch(Collection<E> list, int batchSize, BiConsumer<SqlSession, E> consumer) {return SqlHelper.executeBatch(this.entityClass, this.log, list, batchSize, consumer);}protected <E> boolean executeBatch(Collection<E> list, BiConsumer<SqlSession, E> consumer) {return this.executeBatch(list, 1000, consumer);}

  8. 实体类ServiceImpl继承MyServiceImpl

    public class BudgetVSActualServiceImpl extends MyServiceImpl<BudgetVSActualMapper, BudgetVSActual> implements BudgetVSActualService {

  9. 批量方法的使用

    @Autowired
private IService<T> iService;//批量保存数据
iService.saveBatch(DataList);
//批量更新数据
iService.saveOrUpdateBatch(DataList);

这篇关于Mybatis Plus快速重构真批量sql入库操作的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！


http://www.chinasem.cn/article/1146908。

相关文章

MySQL 中的 CAST 函数详解及常见用法

MySQL 中的 CAST 函数详解及常见用法

《MySQL中的CAST函数详解及常见用法》CAST函数是MySQL中用于数据类型转换的重要函数,它允许你将一个值从一种数据类型转换为另一种数据类型,本文给大家介绍MySQL中的CAST... 目录mysql 中的 CAST 函数详解一、基本语法二、支持的数据类型三、常见用法示例1. 字符串转数字2. 数字
阅读更多...
Mysql实现范围分区表(新增、删除、重组、查看)

Mysql实现范围分区表(新增、删除、重组、查看)

《Mysql实现范围分区表(新增、删除、重组、查看)》MySQL分区表的四种类型（范围、哈希、列表、键值）,主要介绍了范围分区的创建、查询、添加、删除及重组织操作,具有一定的参考价值,感兴趣的可以了解... 目录一、mysql分区表分类二、范围分区（Range Partitioning1、新建分区表：2、分
阅读更多...
MySQL 定时新增分区的实现示例

MySQL 定时新增分区的实现示例

《MySQL定时新增分区的实现示例》本文主要介绍了通过存储过程和定时任务实现MySQL分区的自动创建,解决大数据量下手动维护的繁琐问题,具有一定的参考价值,感兴趣的可以了解一下... mysql创建好分区之后，有时候会需要自动创建分区。比如，一些表数据量非常大，有些数据是热点数据，按照日期分区MululbU
阅读更多...
SQL Server配置管理器无法打开的四种解决方法

SQL Server配置管理器无法打开的四种解决方法

《SQLServer配置管理器无法打开的四种解决方法》本文总结了SQLServer配置管理器无法打开的四种解决方法,文中通过图文示例介绍的非常详细,对大家的学习或者工作具有一定的参考学习价值,需要的... 目录方法一：桌面图标进入方法二：运行窗口进入检查版本号对照表php方法三：查找文件路径方法四：检查 S
阅读更多...
MyBatis-Plus 中 nested() 与 and() 方法详解(最佳实践场景)

MyBatis-Plus 中 nested() 与 and() 方法详解(最佳实践场景)

《MyBatis-Plus中nested()与and()方法详解(最佳实践场景)》在MyBatis-Plus的条件构造器中,nested()和and()都是用于构建复杂查询条件的关键方法,但... 目录MyBATis-Plus 中nested()与and()方法详解一、核心区别对比二、方法详解1.and()
阅读更多...
MySQL 删除数据详解(最新整理)

MySQL 删除数据详解(最新整理)

《MySQL删除数据详解(最新整理)》：本文主要介绍MySQL删除数据的相关知识,本文通过实例代码给大家介绍的非常详细,对大家的学习或工作具有一定的参考借鉴价值,需要的朋友参考下吧... 目录一、前言二、mysql 中的三种删除方式1.DELETE语句✅ 基本语法： 示例：2.TRUNCATE语句✅ 基本语
阅读更多...
MySQL中查找重复值的实现

MySQL中查找重复值的实现

《MySQL中查找重复值的实现》查找重复值是一项常见需求,比如在数据清理、数据分析、数据质量检查等场景下,我们常常需要找出表中某列或多列的重复值,具有一定的参考价值,感兴趣的可以了解一下... 目录技术背景实现步骤方法一：使用GROUP BY和HAVING子句方法二：仅返回重复值方法三：返回完整记录方法四：
阅读更多...
从入门到精通MySQL联合查询

从入门到精通MySQL联合查询

《从入门到精通MySQL联合查询》：本文主要介绍从入门到精通MySQL联合查询,本文通过实例代码给大家介绍的非常详细,需要的朋友可以参考下... 目录摘要1. 多表联合查询时mysql内部原理2. 内连接3. 外连接4. 自连接5. 子查询6. 合并查询7. 插入查询结果摘要前面我们学习了数据库设计时要满
阅读更多...
Python实现对阿里云OSS对象存储的操作详解

Python实现对阿里云OSS对象存储的操作详解

《Python实现对阿里云OSS对象存储的操作详解》这篇文章主要为大家详细介绍了Python实现对阿里云OSS对象存储的操作相关知识,包括连接,上传,下载,列举等功能,感兴趣的小伙伴可以了解下... 目录一、直接使用代码二、详细使用1. 环境准备2. 初始化配置3. bucket配置创建4. 文件上传到os
阅读更多...
MySQL查询JSON数组字段包含特定字符串的方法

MySQL查询JSON数组字段包含特定字符串的方法

《MySQL查询JSON数组字段包含特定字符串的方法》在MySQL数据库中,当某个字段存储的是JSON数组,需要查询数组中包含特定字符串的记录时传统的LIKE语句无法直接使用,下面小编就为大家介绍两种... 目录问题背景解决方案对比1. 精确匹配方案（推荐）2. 模糊匹配方案参数化查询示例使用场景建议性能优
阅读更多...

Copyright China编程 23002807@qq.com

沪ICP备2023033072号-2