dolphinscheduler单机版部署教程

2024-02-26 10:12
文章标签 部署 教程 dolphinscheduler 单机版

本文主要是介绍dolphinscheduler单机版部署教程，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

文章目录

  • 前言
  • 一、安装准备
    • 1. 安装条件
    • 2. 安装jdk
    • 3. 安装MySQL
  • 二、安装dolphinscheduler
    • 1. 下载并解压dolphinscheduler
    • 2. 修改配置文件
      • 2.1 修改 dolphinscheduler_env.sh 文件
      • 2.2 修改 application.yaml 文件
    • 3. 配置mysql数据源
      • 3.1 修改MySQL安全策略
      • 3.2 查看数据库
      • 3.3 创建数据库
      • 3.4 创建MySQL用户
      • 3.5 配置MySQL驱动
    • 4. 初始化数据库
    • 5. 启动dolphinscheduler单机版
    • 6. 登录 DolphinScheduler
  • 三、dolphinscheduler单机版命令
    • 1. 启停命令
      • 1.1 启动dolphinscheduler单机版
      • 1.2 停止dolphinscheduler单机版
      • 1.3 重启dolphinscheduler单机版
  • 总结

前言

本教程将介绍如何安装和配置 DolphinScheduler，一个分布式任务调度系统。您将学习如何准备安装环境，下载和解压 DolphinScheduler，修改配置文件，配置 MySQL 数据源，并进行数据库初始化。最后，您将学习如何启动 DolphinScheduler 单机版，并通过浏览器访问系统 UI。

一、安装准备

1. 安装条件

  • jdk版本：1.8+
  • MySQL版本：5.7+
  • MySQL驱动版本：8.0.16+

2. 安装jdk

下载地址：https://www.oracle.com/java/technologies/downloads/#java8

下载后上传到/tmp目录下。

然后执行下面命令，用于创建目录、解压，并设置系统级环境变量。

mkdir -p /opt/module
tar -zxvf /tmp/jdk-8u401-linux-x64.tar.gz -C /opt/module/
echo >> /etc/profile
echo '#JAVA_HOME' >> /etc/profile
echo "export JAVA_HOME=/opt/module/jdk1.8.0_401" >> /etc/profile
echo 'export PATH=$PATH:$JAVA_HOME/bin' >> /etc/profile
source /etc/profile

3. 安装MySQL

在hadoop101安装MySQL。

  • mysql5.7.44自动化安装教程
  • mysql5.7.37自动化安装教程

二、安装dolphinscheduler

1. 下载并解压dolphinscheduler

wget --no-check-certificate https://archive.apache.org/dist/dolphinscheduler/3.1.4/apache-dolphinscheduler-3.1.4-bin.tar.gz -P /tmp
mkdir -p /opt/soft/dolphinscheduler
tar -zxvf /tmp/apache-dolphinscheduler-3.1.4-bin.tar.gz -C /opt/soft/dolphinscheduler

2. 修改配置文件

2.1 修改 dolphinscheduler_env.sh 文件

修改 /opt/soft/dolphinscheduler/apache-dolphinscheduler-3.1.4-bin/bin/env/dolphinscheduler_env.sh 文件。内容如下所示。

# 修改为自己的jdk安装目录
export JAVA_HOME=${JAVA_HOME:-/opt/module/jdk1.8.0_391}# 修改MySQL配置
export DATABASE=${DATABASE:-mysql}
export SPRING_PROFILES_ACTIVE=${DATABASE}
export SPRING_DATASOURCE_URL="jdbc:mysql://localhost:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false"
export SPRING_DATASOURCE_USERNAME="dolphinscheduler"
export SPRING_DATASOURCE_PASSWORD="dolphinscheduler"# DolphinScheduler服务相关配置
export SPRING_CACHE_TYPE=${SPRING_CACHE_TYPE:-none}
export SPRING_JACKSON_TIME_ZONE=${SPRING_JACKSON_TIME_ZONE:-UTC}
export MASTER_FETCH_COMMAND_NUM=${MASTER_FETCH_COMMAND_NUM:-10}# 注册中心配置，修改为自己的zookeeper监听地址
export REGISTRY_TYPE=${REGISTRY_TYPE:-zookeeper}
export REGISTRY_ZOOKEEPER_CONNECT_STRING=${REGISTRY_ZOOKEEPER_CONNECT_STRING:-localhost:2181}# 这些环境变量根据自己的需要更改，没有保持默认即口
export HADOOP_HOME=${HADOOP_HOME:-/opt/soft/hadoop}
export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-/opt/soft/hadoop/etc/hadoop}
export SPARK_HOME1=${SPARK_HOME1:-/opt/soft/spark1}
export SPARK_HOME2=${SPARK_HOME2:-/opt/soft/spark2}
export PYTHON_HOME=${PYTHON_HOME:-/opt/soft/python}
export HIVE_HOME=${HIVE_HOME:-/opt/soft/hive}
export FLINK_HOME=${FLINK_HOME:-/opt/soft/flink}
export DATAX_HOME=${DATAX_HOME:-/opt/soft/datax}
export SEATUNNEL_HOME=${SEATUNNEL_HOME:-/opt/soft/seatunnel}
export CHUNJUN_HOME=${CHUNJUN_HOME:-/opt/soft/chunjun}export PATH=$HADOOP_HOME/bin:$SPARK_HOME1/bin:$SPARK_HOME2/bin:$PYTHON_HOME/bin:$JAVA_HOME/bin:$HIVE_HOME/bin:$FLINK_HOME/bin:$DATAX_HOME/bin:$SEATUNNEL_HOME/bin:$CHUNJUN_HOME/bin:$PATH

2.2 修改 application.yaml 文件

修改 /opt/soft/dolphinscheduler/apache-dolphinscheduler-3.1.4-bin/standalone-server/conf/application.yaml 文件。

修改了如下两处。

在这里插入图片描述

修改后的内容如下所示。

spring:jackson:time-zone: UTCdate-format: "yyyy-MM-dd HH:mm:ss"banner:charset: UTF-8cache:# default enable cache, you can disable by `type: none`type: nonecache-names:- tenant- user- processDefinition- processTaskRelation- taskDefinitioncaffeine:spec: maximumSize=100,expireAfterWrite=300s,recordStatssql:init:schema-locations: classpath:sql/dolphinscheduler_h2.sqldatasource:driver-class-name: com.mysql.cj.jdbc.Driverurl: jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8username: dolphinschedulerpassword: dolphinscheduler
#  datasource:
#    driver-class-name: org.h2.Driver
#    url: jdbc:h2:mem:dolphinscheduler;MODE=MySQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=true
#    username: sa
#    password: ""quartz:job-store-type: jdbcjdbc:initialize-schema: neverproperties:org.quartz.threadPool:threadPriority: 5org.quartz.jobStore.isClustered: trueorg.quartz.jobStore.class: org.springframework.scheduling.quartz.LocalDataSourceJobStoreorg.quartz.scheduler.instanceId: AUTOorg.quartz.jobStore.tablePrefix: QRTZ_org.quartz.jobStore.acquireTriggersWithinLock: trueorg.quartz.scheduler.instanceName: DolphinSchedulerorg.quartz.threadPool.class: org.quartz.simpl.SimpleThreadPoolorg.quartz.jobStore.useProperties: falseorg.quartz.threadPool.makeThreadsDaemons: trueorg.quartz.threadPool.threadCount: 25org.quartz.jobStore.misfireThreshold: 60000org.quartz.scheduler.makeSchedulerThreadDaemon: trueorg.quartz.jobStore.driverDelegateClass: org.quartz.impl.jdbcjobstore.StdJDBCDelegateorg.quartz.jobStore.clusterCheckinInterval: 5000servlet:multipart:max-file-size: 1024MBmax-request-size: 1024MBmessages:basename: i18n/messagesjpa:hibernate:ddl-auto: nonemvc:pathmatch:matching-strategy: ANT_PATH_MATCHERregistry:type: zookeeperzookeeper:namespace: dolphinschedulerconnect-string: localhost:2181retry-policy:base-sleep-time: 60msmax-sleep: 300msmax-retries: 5session-timeout: 30sconnection-timeout: 9sblock-until-connected: 600msdigest: ~security:authentication:# Authentication types (supported types: PASSWORD,LDAP)type: PASSWORD# IF you set type `LDAP`, below config will be effectiveldap:# ldap server configurls: ldap://ldap.forumsys.com:389/base-dn: dc=example,dc=comusername: cn=read-only-admin,dc=example,dc=compassword: passworduser:# admin userId when you use LDAP loginadmin: read-only-adminidentity-attribute: uidemail-attribute: mail# action when ldap user is not exist (supported types: CREATE,DENY)not-exist-action: CREATE# Traffic control, if you turn on this config, the maximum number of request/s will be limited.
# global max request number per second
# default tenant-level max request number
traffic:control:global-switch: falsemax-global-qps-rate: 300tenant-switch: falsedefault-tenant-qps-rate: 10#customize-tenant-qps-rate:# eg.#tenant1: 11#tenant2: 20master:listen-port: 5678# master fetch command numfetch-command-num: 10# master prepare execute thread number to limit handle commands in parallelpre-exec-threads: 10# master execute thread number to limit process instances in parallelexec-threads: 10# master dispatch task number per batchdispatch-task-number: 3# master host selector to select a suitable worker, default value: LowerWeight. Optional values include random, round_robin, lower_weighthost-selector: lower_weight# master heartbeat intervalheartbeat-interval: 10s# Master heart beat task error threshold, if the continuous error count exceed this count, the master will close.heartbeat-error-threshold: 5# master commit task retry timestask-commit-retry-times: 5# master commit task intervaltask-commit-interval: 1sstate-wheel-interval: 5s# master max cpuload avg, only higher than the system cpu load average, master server can schedule. default value -1: the number of cpu cores * 2max-cpu-load-avg: -1# master reserved memory, only lower than system available memory, master server can schedule. default value 0.3, the unit is Greserved-memory: 0.3# failover intervalfailover-interval: 10m# kill yarn jon when failover taskInstance, default truekill-yarn-job-when-task-failover: trueworker-group-refresh-interval: 10sworker:# worker listener portlisten-port: 1234# worker execute thread number to limit task instances in parallelexec-threads: 10# worker heartbeat intervalheartbeat-interval: 10s# Worker heart beat task error threshold, if the continuous error count exceed this count, the worker will close.heartbeat-error-threshold: 5# worker host weight to dispatch tasks, default value 100host-weight: 100# tenant corresponds to the user of the system, which is used by the worker to submit the job. If system does not have this user, it will be automatically created after the parameter worker.tenant.auto.create is true.tenant-auto-create: true#Scenes to be used for distributed users.For example,users created by FreeIpa are stored in LDAP.This parameter only applies to Linux, When this parameter is true, worker.tenant.auto.create has no effect and will not automatically create tenants.tenant-distributed-user: false# worker max cpuload avg, only higher than the system cpu load average, worker server can be dispatched tasks. default value -1: the number of cpu cores * 2max-cpu-load-avg: -1# worker reserved memory, only lower than system available memory, worker server can be dispatched tasks. default value 0.3, the unit is Greserved-memory: 0.3# default worker groups separated by comma, like 'worker.groups=default,test'groups:- default# alert server listen hostalert-listen-host: localhostalert-listen-port: 50052task-execute-threads-full-policy: REJECTalert:port: 50052# Mark each alert of alert server if late after x milliseconds as failed.# Define value is (0 = infinite), and alert server would be waiting alert result.wait-timeout: 0python-gateway:# Weather enable python gateway server or not. The default value is true.enabled: true# Authentication token for connection from python api to python gateway server. Should be changed the default value# when you deploy in public network.auth-token: jwUDzpLsNKEFER4*a8gruBH_GsAurNxU7A@Xc# The address of Python gateway server start. Set its value to `0.0.0.0` if your Python API run in different# between Python gateway server. It could be be specific to other address like `127.0.0.1` or `localhost`gateway-server-address: 0.0.0.0# The port of Python gateway server start. Define which port you could connect to Python gateway server from# Python API side.gateway-server-port: 25333# The address of Python callback client.python-address: 127.0.0.1# The port of Python callback client.python-port: 25334# Close connection of socket server if no other request accept after x milliseconds. Define value is (0 = infinite),# and socket server would never close even though no requests acceptconnect-timeout: 0# Close each active connection of socket server if python program not active after x milliseconds. Define value is# (0 = infinite), and socket server would never close even though no requests acceptread-timeout: 0server:port: 12345servlet:session:timeout: 120mcontext-path: /dolphinscheduler/compression:enabled: truemime-types: text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,application/json,application/xmljetty:max-http-form-post-size: 5000000management:endpoints:web:exposure:include: '*'endpoint:health:enabled: trueshow-details: alwayshealth:db:enabled: truedefaults:enabled: falsemetrics:tags:application: ${spring.application.name}audit:enabled: truemetrics:enabled: true# Override by profile
---
spring:config:activate:on-profile: postgresqlquartz:properties:org.quartz.jobStore.driverDelegateClass: org.quartz.impl.jdbcjobstore.PostgreSQLDelegatedatasource:driver-class-name: org.postgresql.Driverurl: jdbc:postgresql://127.0.0.1:5432/dolphinschedulerusername: rootpassword: root---
spring:config:activate:on-profile: mysqlsql:init:schema-locations: classpath:sql/dolphinscheduler_mysql.sqldatasource:driver-class-name: com.mysql.cj.jdbc.Driverurl: jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8username: dolphinschedulerpassword: dolphinscheduler

3. 配置mysql数据源

3.1 修改MySQL安全策略

修改安全策略为LOW，使其可以设置简单密码，根据自己的需要更改。

mysql -h localhost -P 3306 -uroot -proot -e "set global validate_password_length=4;"
mysql -h localhost -P 3306 -uroot -proot -e "set global validate_password_policy=LOW;"

3.2 查看数据库

查看dolphinscheduler数据库是否存在。

mysql -h localhost -P 3306 -uroot -proot -e "SHOW DATABASES;" | grep dolphinscheduler

若存在则删除。

mysql -h localhost -P 3306 -uroot -proot -e "DROP DATABASE dolphinscheduler;"

3.3 创建数据库

创建dolphinscheduler数据库

mysql -h localhost -P 3306 -uroot -proot -e "CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;"

3.4 创建MySQL用户

创建 dolphinscheduler 用户并授予操作 dolphinscheduler 数据库的权限。

此处创建dolphinscheduler用户并设置密码为dolphinscheduler。

mysql -h localhost -P 3306 -uroot -proot -e "CREATE USER 'dolphinscheduler'@'%' IDENTIFIED BY 'dolphinscheduler';"

授予操作 dolphinscheduler 数据库的权限并刷新权限。

mysql -h localhost -P 3306 -uroot -proot -e "GRANT ALL PRIVILEGES ON dolphinscheduler.* TO 'dolphinscheduler'@'%' IDENTIFIED BY 'dolphinscheduler';"
mysql -h localhost -P 3306 -uroot -proot -e "flush privileges;"

3.5 配置MySQL驱动

下载并配置MySQL驱动。

cd /opt/soft/dolphinscheduler
wget https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar -P /tmp
cp /tmp/mysql-connector-java-8.0.16.jar apache-dolphinscheduler-3.1.4-bin/worker-server/libs
cp /tmp/mysql-connector-java-8.0.16.jar apache-dolphinscheduler-3.1.4-bin/api-server/libs
cp /tmp/mysql-connector-java-8.0.16.jar apache-dolphinscheduler-3.1.4-bin/alert-server/libs
cp /tmp/mysql-connector-java-8.0.16.jar apache-dolphinscheduler-3.1.4-bin/master-server/libs
cp /tmp/mysql-connector-java-8.0.16.jar apache-dolphinscheduler-3.1.4-bin/tools/libs
cp /tmp/mysql-connector-java-8.0.16.jar apache-dolphinscheduler-3.1.4-bin/standalone-server/libs/standalone-server

4. 初始化数据库

初始化dolphinscheduler数据库。

cd /opt/soft/dolphinscheduler/apache-dolphinscheduler-3.1.4-bin
bash tools/bin/upgrade-schema.sh

初始化成功如下图所示。

在这里插入图片描述

5. 启动dolphinscheduler单机版

cd /opt/soft/dolphinscheduler/apache-dolphinscheduler-3.1.4-bin
./bin/dolphinscheduler-daemon.sh start standalone-server

6. 登录 DolphinScheduler

浏览器访问地址 http://localhost:12345/dolphinscheduler/ui 即可登录系统UI。默认的用户名和密码是 admin/dolphinscheduler123

如下图所示。
在这里插入图片描述
在这里插入图片描述

三、dolphinscheduler单机版命令

1. 启停命令

1.1 启动dolphinscheduler单机版

cd /opt/soft/dolphinscheduler/apache-dolphinscheduler-3.1.4-bin
./bin/dolphinscheduler-daemon.sh start standalone-server

1.2 停止dolphinscheduler单机版

cd /opt/soft/dolphinscheduler/apache-dolphinscheduler-3.1.4-bin
./bin/dolphinscheduler-daemon.sh stop standalone-server

1.3 重启dolphinscheduler单机版

cd /opt/soft/dolphinscheduler/apache-dolphinscheduler-3.1.4-bin
./bin/dolphinscheduler-daemon.sh stop standalone-server
./bin/dolphinscheduler-daemon.sh start standalone-server

总结

通过本教程，您已经学会了安装和配置 DolphinScheduler 的基本步骤。了解了如何准备安装环境，下载和解压 DolphinScheduler，修改配置文件，配置 MySQL 数据源，并进行数据库初始化。此外，还学会了如何启动和停止 DolphinScheduler 单机版，并通过浏览器访问系统 UI。

希望本教程对您有所帮助！如有任何疑问或问题，请随时在评论区留言。感谢阅读！

这篇关于dolphinscheduler单机版部署教程的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！


http://www.chinasem.cn/article/748512。

相关文章

Nginx分布式部署流程分析

Nginx分布式部署流程分析

《Nginx分布式部署流程分析》文章介绍Nginx在分布式部署中的反向代理和负载均衡作用,用于分发请求、减轻服务器压力及解决session共享问题,涵盖配置方法、策略及Java项目应用,并提及分布式事... 目录分布式部署NginxJava中的代理代理分为正向代理和反向代理正向代理反向代理Nginx应用场景
阅读更多...
全网最全Tomcat完全卸载重装教程小结

全网最全Tomcat完全卸载重装教程小结

《全网最全Tomcat完全卸载重装教程小结》windows系统卸载Tomcat重新通过ZIP方式安装Tomcat,优点是灵活可控,适合开发者自定义配置,手动配置环境变量后,可通过命令行快速启动和管理... 目录一、完全卸载Tomcat1. 停止Tomcat服务2. 通过控制面板卸载3. 手动删除残留文件4.
阅读更多...
Python的pandas库基础知识超详细教程

Python的pandas库基础知识超详细教程

《Python的pandas库基础知识超详细教程》Pandas是Python数据处理核心库,提供Series和DataFrame结构,支持CSV/Excel/SQL等数据源导入及清洗、合并、统计等功能... 目录一、配置环境二、序列和数据表2.1 初始化2.2  获取数值2.3 获取索引2.4 索引取内容2
阅读更多...
python依赖管理工具UV的安装和使用教程

python依赖管理工具UV的安装和使用教程

《python依赖管理工具UV的安装和使用教程》UV是一个用Rust编写的Python包安装和依赖管理工具,比传统工具（如pip）有着更快、更高效的体验,：本文主要介绍python依赖管理工具UV... 目录前言一、命令安装uv二、手动编译安装2.1在archlinux安装uv的依赖工具2.2从github
阅读更多...
C#实现SHP文件读取与地图显示的完整教程

C#实现SHP文件读取与地图显示的完整教程

《C#实现SHP文件读取与地图显示的完整教程》在地理信息系统(GIS)开发中,SHP文件是一种常见的矢量数据格式,本文将详细介绍如何使用C#读取SHP文件并实现地图显示功能,包括坐标转换、图形渲染、平... 目录概述功能特点核心代码解析1. 文件读取与初始化2. 坐标转换3. 图形绘制4. 地图交互功能缩放
阅读更多...
linux部署NFS和autofs自动挂载实现过程

linux部署NFS和autofs自动挂载实现过程

《linux部署NFS和autofs自动挂载实现过程》文章介绍了NFS（网络文件系统）和Autofs的原理与配置,NFS通过RPC实现跨系统文件共享,需配置/etc/exports和nfs.conf,... 目录（一）NFS1. 什么是NFS2.NFS守护进程3.RPC服务4. 原理5. 部署5.1安装NF
阅读更多...
SpringBoot集成redisson实现延时队列教程

SpringBoot集成redisson实现延时队列教程

《SpringBoot集成redisson实现延时队列教程》文章介绍了使用Redisson实现延迟队列的完整步骤,包括依赖导入、Redis配置、工具类封装、业务枚举定义、执行器实现、Bean创建、消费... 目录1、先给项目导入Redisson依赖2、配置redis3、创建 RedissonConfig 配
阅读更多...
通过Docker容器部署Python环境的全流程

通过Docker容器部署Python环境的全流程

《通过Docker容器部署Python环境的全流程》在现代化开发流程中,Docker因其轻量化、环境隔离和跨平台一致性的特性,已成为部署Python应用的标准工具,本文将详细演示如何通过Docker容... 目录引言一、docker与python的协同优势二、核心步骤详解三、进阶配置技巧四、生产环境最佳实践
阅读更多...
Nginx部署HTTP/3的实现步骤

Nginx部署HTTP/3的实现步骤

《Nginx部署HTTP/3的实现步骤》本文介绍了在Nginx中部署HTTP/3的详细步骤,文中通过示例代码介绍的非常详细,对大家的学习或者工作具有一定的参考学习价值,需要的朋友们下面随着小编来一起学... 目录前提条件第一步：安装必要的依赖库第二步：获取并构建 BoringSSL第三步：获取 Nginx
阅读更多...
redis-sentinel基础概念及部署流程

redis-sentinel基础概念及部署流程

《redis-sentinel基础概念及部署流程》RedisSentinel是Redis的高可用解决方案,通过监控主从节点、自动故障转移、通知机制及配置提供,实现集群故障恢复与服务持续可用,核心组件包... 目录一. 引言二. 核心功能三. 核心组件四. 故障转移流程五. 服务部署六. sentinel部署
阅读更多...

Copyright China编程 23002807@qq.com

沪ICP备2023033072号-2