本文主要是介绍Azkaban 简介及安装(3.70)，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！
Azkaban 简介及安装(3.70)
1.Azkaban 简介
官网：https://azkaban.github.io/
Azkaban 一个批量工作流任务调度器，使用Java语言开发。用于在一个工作流内以一个特定的顺序运行一组工作和流程。Azkaban使用job配置文件建立任务之间的依赖关系，并提供一个易于使用的web用户界面维护和跟踪你的工作流。
Azkaban 的优点
-
提供功能清晰，简单易用的Web UI界面
-
提供job配置文件快速建立任务和任务之间的依赖关系
-
提供模块化和可插拔的插件机制，原生支持command、Java、Hive、Pig、Hadoop
-
基于Java开发，代码结构清晰，易于二次开发
Azkaban 安装模式
Azkaban 有三种部署方式：单服务模式、2个服务模式、分布式多服务模式
-
solo server model（单服务模式）：该模式中 webServer 和 executorServer 运行在同一个进程中，进程名是AzkabanSingleServer。可以使用自带的H2数据库或者配置mysql数据。该模式适用于小规模的使用。此模式合适个人试用使用。它也可以用在小规模的使用案例中。
-
two server model（2个服务模式）：数据库为mysql，采用主从设置进行备份，管理服务器（webServer）和执行服务器（executorServer）在不同进程中运行，这种模式下，管理服务器和执行服务器互不影响。适用在较重的生成环境中。
-
multiple-executor（分布式多服务模式）：存放元数据的数据库为mysql，采用主从设置进行备份，管理服务器（webServer）和执行服务器（executorServer）在不同进程中运行。多个 Executor 模式为最重的生产环境。
2.Azkaban 3.x 安装(two-server 模式)
安装前准备
- 安装JDK1.8
- 安装MySQL
下载源码
git clone https://github.com/azkaban/azkaban.git
编译源码
cd azkaban; ./gradlew build installDist
编译报错：
> Task :az-crypto:test azkaban.crypto.DecryptionTest > testV1_1 FAILEDjava.lang.RuntimeException: java.lang.RuntimeException: org.jasypt.exceptions.EncryptionOperationNotPossibleException: Encryption raised an exception. A possible cause is you are using strong encryption algorithms and you have not installed the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files in this Java Virtual Machineat azkaban.crypto.Crypto.decrypt(Crypto.java:76)at azkaban.crypto.DecryptionTest.testV1_1(DecryptionTest.java:35)Caused by:java.lang.RuntimeException: org.jasypt.exceptions.EncryptionOperationNotPossibleException: Encryption raised an exception. A possible cause is you are using strong encryption algorithms and you have not installed the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files in this Java Virtual Machineat azkaban.crypto.CryptoV1_1.decrypt(CryptoV1_1.java:57)at azkaban.crypto.Crypto.decrypt(Crypto.java:74)... 1 moreCaused by:org.jasypt.exceptions.EncryptionOperationNotPossibleException: Encryption raised an exception. A possible cause is you are using strong encryption algorithms and you have not installed the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files in this Java Virtual Machineat org.jasypt.encryption.pbe.StandardPBEByteEncryptor.handleInvalidKeyException(StandardPBEByteEncryptor.java:1073)at org.jasypt.encryption.pbe.StandardPBEByteEncryptor.decrypt(StandardPBEByteEncryptor.java:1050)at org.jasypt.encryption.pbe.StandardPBEStringEncryptor.decrypt(StandardPBEStringEncryptor.java:725)at azkaban.crypto.CryptoV1_1.decrypt(CryptoV1_1.java:55)... 2 moreazkaban.crypto.EncryptionTest > testEncryption FAILEDorg.jasypt.exceptions.EncryptionOperationNotPossibleException: Encryption raised an exception. A possible cause is you are using strong encryption algorithms and you have not installed the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files in this Java Virtual Machineat org.jasypt.encryption.pbe.StandardPBEByteEncryptor.handleInvalidKeyException(StandardPBEByteEncryptor.java:1073)at org.jasypt.encryption.pbe.StandardPBEByteEncryptor.encrypt(StandardPBEByteEncryptor.java:924)at org.jasypt.encryption.pbe.StandardPBEStringEncryptor.encrypt(StandardPBEStringEncryptor.java:642)at azkaban.crypto.CryptoV1_1.encrypt(CryptoV1_1.java:42)at azkaban.crypto.Crypto.encrypt(Crypto.java:58)at azkaban.crypto.EncryptionTest.testEncryption(EncryptionTest.java:28)5 tests completed, 2 failed解决方法：
1.下载jre：https://www.oracle.com/technetwork/cn/java/javase/downloads/jce8-download-2133166-zhs.html
2.在/usr/java/jdk1.8.0_141-cloudera/jre/lib/security目录下替换原有的
3.重新编译 ./gradlew build installDist
启停单机版的服务
cd azkaban-solo-server/build/install/azkaban-solo-server
bin/start-solo.sh
bin/shutdown-solo.sh
数据库设置
# 建用户 建库 授权 刷新权限
CREATE DATABASE azkaban_zxl;
CREATE USER 'azkaban_zxl'@'%' IDENTIFIED BY 'azkaban_zxl';
GRANT SELECT,INSERT,UPDATE,DELETE ON azkaban_zxl.* to 'azkaban_zxl'@'%' WITH GRANT OPTION;
FLUSH PRIVILEGES;# 在/etc/my.cnf中配置Mysql Packet Size的大小
[mysqld]
...
max_allowed_packet=1024M# 重启mysql
/sbin/service mysqld restart# 创建Azkaban表
source /root/apps/create-all-sql.sql
配置web-server
vi /root/azkaban/azkaban-web-server/conf/azkaban.properties default.timezone.id=Asia/Shanghai
#database.type=h2
#h2.path=./h2
#h2.create.tables=true
database.type=mysql
mysql.port=3306
mysql.host=10.xxx.xxx.xx
mysql.database=azkaban
mysql.user=azkaban
mysql.password=azkaban
mysql.numconnections=100
executor.port=12321
azkaban.use.multiple.executors=true将azkaban-web-server/conf/azkaban.properties 同步到 /root/azkaban/azkaban-exec-server/conf下
启动分布式多服务模式
多服务模式是多个executorServer分布在 不同服务器上，只需要将/root/azkaban/azkaban-exec-server拷贝到不同机器上即可组成分布式。
先启动executorServer bin/start-exec.sh, 再启动webServer bin/start-web.sh。
**Note:**启动executorServer时会将executorServer的host,port同步到mysql的executors表中;
root@localhost|azkaban_zxl>select * from executors;
+----+---------------------------------------+-------+--------+
| id | host | port | active |
+----+---------------------------------------+-------+--------+
| 1 | 10.xxx.xx.xx | 12321 | 0 |
| 2 | 10.xxx.xx.xx | 12321 | 0 |
+----+---------------------------------------+-------+--------+启动webServer时会如下错误，是因为自动同步到executors表中的executor默认是未激活，需要在数据库中update：
2019/03/22 10:00:47.210 +0800 INFO [ExecutorManager] [Azkaban] Initializing executors from database.
2019/03/22 10:00:47.418 +0800 ERROR [ExecutorManager] [Azkaban] No active executors found
2019/03/22 10:00:47.418 +0800 ERROR [StdOutErrRedirect] [Azkaban] Exception in thread "main"
2019/03/22 10:00:47.419 +0800 ERROR [StdOutErrRedirect] [Azkaban] azkaban.executor.ExecutorManagerException: No active executors found
2019/03/22 10:00:47.419 +0800 ERROR [StdOutErrRedirect] [Azkaban] at azkaban.executor.ActiveExecutors.setupExecutors(ActiveExecutors.java:52)
2019/03/22 10:00:47.419 +0800 ERROR [StdOutErrRedirect] [Azkaban] at azkaban.executor.ExecutorManager.setupExecutors(ExecutorManager.java:231)
2019/03/22 10:00:47.419 +0800 ERROR [StdOutErrRedirect] [Azkaban] at azkaban.executor.ExecutorManager.initialize(ExecutorManager.java:155)
2019/03/22 10:00:47.420 +0800 ERROR [StdOutErrRedirect] [Azkaban] at azkaban.executor.ExecutorManager.start(ExecutorManager.java:169)
2019/03/22 10:00:47.420 +0800 ERROR [StdOutErrRedirect] [Azkaban] at azkaban.webapp.AzkabanWebServer.launch(AzkabanWebServer.java:235)
2019/03/22 10:00:47.420 +0800 ERROR [StdOutErrRedirect] [Azkaban] at azkaban.webapp.AzkabanWebServer.main(AzkabanWebServer.java:228)
root@localhost|azkaban_zxl>update executors set active=1 where id=1;
Query OK, 1 row affected (0.01 sec)
Rows matched: 1 Changed: 1 Warnings: 0root@localhost|azkaban_zxl>update executors set active=1 where id=2;
Query OK, 1 row affected (0.01 sec)
Rows matched: 1 Changed: 1 Warnings: 0
executor重启方式
1. 停止webServer bin/shutdown-web.sh
2. 停止executor bin/shutdown-exec.sh
3. 删除 xxx/azkaban-exec-server-3.65.0/currentpid 文件
4. 启动executor bin/start-exec.sh
5. 去数据库中确认 executor状态为1
6. 启动webServer bin/start-web.sh
shylin
这篇关于Azkaban 简介及安装(3.70)的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！
