EXPDP/IMPDP 中的并行度PARALLEL参数

2023-10-08 00:48

本文主要是介绍EXPDP/IMPDP 中的并行度PARALLEL参数,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

    如果设置 EXPDP parallel=4 必须要设置4个EXPDP文件,不然PARALLEL是有问题的,同时EXPDP会使用一个WORKER进程导出METADATA,其他WORKER进程会同时出数据。如果EXPDP作业低于250M,只会启动一个WORKER进程。如果是500M会启动2个,1000M及会启动4个WOKER进程。一般来说加上%U来设置多个文件。
    而IMPDP有所不同,会先启动一个WOKER进程METADATA导入,然后启动多个WORKER进程导入,所以再前期只会看到WOKER在导入METADATA,而且IMPDP如果PARALLE=4也需要不少于4个DMP文件,也可以使用%U来进行导入。
nohup expdp system/**** PARALLEL=2 JOB_NAME=full_bak_job full=y dumpfile=exptest:back_%U.dmp logfile=exptest:back.log &
impdp system/*** PARALLEL=2 EXCLUDE=STATISTICS JOB_NAME=full_imp cluster=no full=y dumpfile=test:back_%U.dmp logfile=test:back_imp.log;
    而在11GR2后EXPDP和IMDP的WORKER进程会在多个INSTANCE启动,所以DIRECTORY必须在共享磁盘上,如果没有设置共享磁盘还是指定cluster=no 来防止报错。
当观察EXPDP/IMPDP woker的时候如下:
Import> status
Job: FULL_IMP
  Operation: IMPORT                        
  Mode: FULL                          
  State: EXECUTING                     
  Bytes Processed: 150,300,713,536
  Percent Done: 80
  Current Parallelism: 6
  Job Error Count: 0
  Dump File: /expdp/back_%u.dmp
  Dump File: /expdp/back_01.dmp
  Dump File: /expdp/back_02.dmp
  Dump File: /expdp/back_03.dmp
  Dump File: /expdp/back_04.dmp
  Dump File: /expdp/back_05.dmp
  Dump File: /expdp/back_06.dmp
  Dump File: /expdp/back_07.dmp
  Dump File: /expdp/back_08.dmp
 
Worker 1 Status:
  Process Name: DW00
  State: EXECUTING                     
  Object Schema: ACRUN
  Object Name: T_PLY_UNDRMSG
  Object Type: DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA
  Completed Objects: 3
  Completed Rows: 3,856,891
  Completed Bytes: 1,134,168,200
  Percent Done: 83
  Worker Parallelism: 1
 
Worker 2 Status:
  Process Name: DW01
  State: EXECUTING                     
  Object Schema: ACRUN
  Object Name: T_FIN_PAYDUE
  Object Type: DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA
  Completed Objects: 5
  Completed Rows: 2,646,941
  Completed Bytes: 1,012,233,224
  Percent Done: 93
  Worker Parallelism: 1
 
Worker 3 Status:
  Process Name: DW02
  State: EXECUTING                     
  Object Schema: ACRUN
  Object Name: MLOG$_T_FIN_CLMDUE
  Object Type: DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA
  Completed Objects: 6
  Completed Bytes: 382,792,584
  Worker Parallelism: 1
 
Worker 4 Status:
  Process Name: DW03
  State: EXECUTING                     
  Object Schema: ACRUN
  Object Name: T_PAY_CONFIRM_INFO
  Object Type: DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA
  Completed Objects: 5
  Completed Rows: 2,443,790
  Completed Bytes: 943,310,104
  Percent Done: 83
  Worker Parallelism: 1
 
Worker 5 Status:
  Process Name: DW04
  State: EXECUTING                     
  Object Schema: ACRUN
  Object Name: T_PLY_TGT
  Object Type: DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA
  Completed Objects: 6
  Completed Rows: 2,285,353
  Completed Bytes: 822,501,496
  Percent Done: 64
  Worker Parallelism: 1
 
Worker 6 Status:
  Process Name: DW05
  State: EXECUTING                     
  Object Schema: ACRUN
  Object Name: T_FIN_PREINDRCT_CLMFEE
  Object Type: DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA
  Completed Objects: 5
  Completed Rows: 6,042,384
  Completed Bytes: 989,435,088
  Percent Done: 79
  Worker Parallelism: 1
 
英文如下:
    For Data Pump Export, the value that is specified for the parallel parameter should be less than or equal to the number of files in the dump file set. Each worker or Parallel Execution Process requires exclusive access to the dump file, so having fewer dump files than the degree of parallelism will mean that some workers or PX processes will be unable to write the information they are exporting. If this occurs, the worker processes go into an idle state and will not be doing any work until more files are added to the job. See the explanation of the DUMPFILE parameter in the Database Utilities guide for details on how to specify multiple dump files for a Data Pump export job.
    For Data Pump Import, the workers and PX processes can all read from the same files. However, if there are not enough dump files, the performance may not be optimal because multiple threads of execution will be trying to access the same dump file. The performance impact of multiple processes sharing the dump files depends on the I/O subsystem containing the dump files. For this reason, Data Pump Import should not have a value for the PARALLEL parameter that is significantly larger than the number of files in the dump file set.
 
     In a typical export that includes both data and metadata, the first worker process will unload the metadata: tablespaces, schemas, grants, roles, tables, indexes, and so on. This single worker unloads the metadata, and all the rest unload the data, all at the same time. If the metadata worker finishes and there are still data objects to unload, it will start unloading the data too. The examples in this document assume that there is always one worker busy unloading metadata while the rest of the workers are busy unloading table data objects.
 
    If the external tables method is chosen, Data Pump will determine the maximum number of PX processes that can work on a table data object. It does this by dividing the estimated size of the table data object by 250 MB and rounding the result down. If the result is zero or one, then PX processes are not used to unload the table
 
    The PARALLEL parameter works a bit differently in Import than Export. Because there are various dependencies that exist when creating objects during import, everything must be done in order. For Import, no data loading can occur until the tables are created because data cannot be loaded into tables that do not yet exist

这篇关于EXPDP/IMPDP 中的并行度PARALLEL参数的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/161563

相关文章

C#中通过Response.Headers设置自定义参数的代码示例

《C#中通过Response.Headers设置自定义参数的代码示例》:本文主要介绍C#中通过Response.Headers设置自定义响应头的方法,涵盖基础添加、安全校验、生产实践及调试技巧,强... 目录一、基础设置方法1. 直接添加自定义头2. 批量设置模式二、高级配置技巧1. 安全校验机制2. 类型

oracle 11g导入\导出(expdp impdp)之导入过程

《oracle11g导入导出(expdpimpdp)之导入过程》导出需使用SEC.DMP格式,无分号;建立expdir目录(E:/exp)并确保存在;导入在cmd下执行,需sys用户权限;若需修... 目录准备文件导入(impdp)1、建立directory2、导入语句 3、更改密码总结上一个环节,我们讲了

SpringBoot 获取请求参数的常用注解及用法

《SpringBoot获取请求参数的常用注解及用法》SpringBoot通过@RequestParam、@PathVariable等注解支持从HTTP请求中获取参数,涵盖查询、路径、请求体、头、C... 目录SpringBoot 提供了多种注解来方便地从 HTTP 请求中获取参数以下是主要的注解及其用法:1

HTTP 与 SpringBoot 参数提交与接收协议方式

《HTTP与SpringBoot参数提交与接收协议方式》HTTP参数提交方式包括URL查询、表单、JSON/XML、路径变量、头部、Cookie、GraphQL、WebSocket和SSE,依据... 目录HTTP 协议支持多种参数提交方式,主要取决于请求方法(Method)和内容类型(Content-Ty

python中的显式声明类型参数使用方式

《python中的显式声明类型参数使用方式》文章探讨了Python3.10+版本中类型注解的使用,指出FastAPI官方示例强调显式声明参数类型,通过|操作符替代Union/Optional,可提升代... 目录背景python函数显式声明的类型汇总基本类型集合类型Optional and Union(py

Go语言使用Gin处理路由参数和查询参数

《Go语言使用Gin处理路由参数和查询参数》在WebAPI开发中,处理路由参数(PathParameter)和查询参数(QueryParameter)是非常常见的需求,下面我们就来看看Go语言... 目录一、路由参数 vs 查询参数二、Gin 获取路由参数和查询参数三、示例代码四、运行与测试1. 测试编程路

Python lambda函数(匿名函数)、参数类型与递归全解析

《Pythonlambda函数(匿名函数)、参数类型与递归全解析》本文详解Python中lambda匿名函数、灵活参数类型和递归函数三大进阶特性,分别介绍其定义、应用场景及注意事项,助力编写简洁高效... 目录一、lambda 匿名函数:简洁的单行函数1. lambda 的定义与基本用法2. lambda

Spring Boot spring-boot-maven-plugin 参数配置详解(最新推荐)

《SpringBootspring-boot-maven-plugin参数配置详解(最新推荐)》文章介绍了SpringBootMaven插件的5个核心目标(repackage、run、start... 目录一 spring-boot-maven-plugin 插件的5个Goals二 应用场景1 重新打包应用

Java内存分配与JVM参数详解(推荐)

《Java内存分配与JVM参数详解(推荐)》本文详解JVM内存结构与参数调整,涵盖堆分代、元空间、GC选择及优化策略,帮助开发者提升性能、避免内存泄漏,本文给大家介绍Java内存分配与JVM参数详解,... 目录引言JVM内存结构JVM参数概述堆内存分配年轻代与老年代调整堆内存大小调整年轻代与老年代比例元空

一文详解PostgreSQL复制参数

《一文详解PostgreSQL复制参数》PostgreSQL作为一款功能强大的开源关系型数据库,其复制功能对于构建高可用性系统至关重要,本文给大家详细介绍了PostgreSQL的复制参数,需要的朋友可... 目录一、复制参数基础概念二、核心复制参数深度解析1. max_wal_seChina编程nders:WAL