CentOS7+Rsyslog+MySQL 搭建 Rsyslog 日志服务器

2024-05-13 04:08

本文主要是介绍CentOS7+Rsyslog+MySQL 搭建 Rsyslog 日志服务器,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

文章目录

    • 1、主机环境
    • 2、rsyslog搭建
      • 2.1、rsyslog-server搭建
      • 2.2、rsyslog-client
        • 2.2.1、测试
      • 2.3、rsyslog日志分类
        • 2.3.1、测试
    • 3、基于mysql存储日志信息
      • 3.1、安装mariadb
      • 3.2、配置mariadb数据库
      • 3.3、配置rsyslog-server
      • 3.4、配置rsyslog-client
      • 3.5、测试

1、主机环境

rsyslog-server   10.11.66.218
rsyslog-client   10.11.66.225
[root@localhost ~]# cat /etc/redhat-release
CentOS Linux release 7.7.1908 (Core)
[root@localhost ~]# uname -r
3.10.0-1062.el7.x86_64
[root@localhost ~]# free -htotal        used        free      shared  buff/cache   available
Mem:           2.9G        140M        2.7G        8.9M        103M        2.7G
Swap:          3.0G          0B        3.0G
[root@localhost ~]# hostnamectl --static set-hostname rsyslog-server

2、rsyslog搭建

2.1、rsyslog-server搭建

[root@rsyslog-server ~]# cp /etc/rsyslog.conf{,.bak}
[root@rsyslog-server ~]# vim /etc/rsyslog.conf
$ModLoad imudp        # 使用udp协议,也可以使用tcp协议
$UDPServerRun 514     # 开启514端口
[root@rsyslog-server ~]# systemctl restart rsyslog.service
[root@rsyslog-server ~]# systemctl enable rsyslog.service

2.2、rsyslog-client

[root@localhost ~]# hostnamectl --static set-hostname rsyslog-client
[root@rsyslog-client ~]# yum -y install nginx
[root@rsyslog-client ~]# cp /etc/rsyslog.conf{,.bak}   # 良好的习惯,从备份配置文件开始
[root@rsyslog-client ~]# vim /etc/rsyslog.conf
[root@rsyslog-client ~]# egrep -v "^$|#" /etc/rsyslog.conf
$ModLoad imudp        # 使用udp协议,也可以使用tcp协议
$UDPServerRun 514     # 开启514端口
$WorkDirectory /var/lib/rsyslog
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$IncludeConfig /etc/rsyslog.d/*.conf
$OmitLocalLogging on
$IMJournalStateFile imjournal.state
*.info;mail.none;authpriv.none;cron.none                @10.11.66.218  # 将日志存到远端rsyslog-server上
authpriv.*                                              /var/log/secure
mail.*                                                  -/var/log/maillog
cron.*                                                  /var/log/cron
*.emerg                                                 :omusrmsg:*
uucp,news.crit                                          /var/log/spooler
local7.*                                                /var/log/boot.log
[root@rsyslog-client ~]# systemctl restart rsyslog.service
[root@rsyslog-client ~]# systemctl enable rsyslog.service   # 以防万一
2.2.1、测试
[root@rsyslog-client ~]# systemctl restart nginx.service[root@rsyslog-server ~]# tail /var/log/messages
Jul 18 17:17:47 rsyslog-server systemd: Stopped System Logging Service.
Jul 18 17:17:47 rsyslog-server systemd: Starting System Logging Service...
Jul 18 17:17:47 rsyslog-server rsyslogd: [origin software="rsyslogd" swVersion="8.24.0-52.el7_8.2" x-pid="2419" x-info="http://www.rsyslog.com"] start
Jul 18 17:17:47 rsyslog-server systemd: Started System Logging Service.
Jul 18 17:17:52 rsyslog-server systemd: Reloading.
Jul 18 17:18:15 rsyslog-client systemd: Starting The nginx HTTP and reverse proxy server...
Jul 18 17:18:15 rsyslog-client nginx: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
Jul 18 17:18:15 rsyslog-client nginx: nginx: configuration file /etc/nginx/nginx.conf test is successful
Jul 18 17:18:15 rsyslog-client systemd: Failed to parse PID from file /run/nginx.pid: Success
Jul 18 17:18:15 rsyslog-client systemd: Started The nginx HTTP and reverse proxy server.
# rsyslog-server成功获取到rsyslog-client的日志

2.3、rsyslog日志分类

# 只需要在rsyslog-server上操作即可
[root@rsyslog-server ~]# vim /etc/rsyslog.d/default.conf
尽量避免修改主配置文件,我们在 '/etc/rsyslog.d/'中新建'default.conf',追加如下模板:
#### GLOBAL DIRECTIVES ####
# Use default timestamp format  # 使用自定义的格式$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat$template myFormat,"%timestamp% %fromhost-ip% %syslogtag% %msg%\n"$ActionFileDefaultTemplate myFormat# 根据客户端的IP单独存放主机日志在不同目录,rsyslog需要手动创建$template RemoteLogs,"/var/log/rsyslog/%fromhost-ip%/%syslogtag%_%$YEAR%-%$MONTH%-%$DAY%-%$hour%:%$minute%.log"
# 排除本地主机IP日志记录,只记录远程主机日志:fromhost-ip, !isequal, "127.0.0.1" ?RemoteLogs
# 忽略之前所有的日志,远程主机日志记录完之后不再继续往下记录& ~
[root@rsyslog-server ~]# egrep -v "^$|#" /etc/sysconfig/rsyslog
SYSLOGD_OPTIONS="-r -m 0 -c 2"
[root@rsyslog-server ~]# mkdir /var/log/rsyslog
[root@rsyslog-server ~]# chmod a+w /var/log/rsyslog/
[root@rsyslog-server ~]# systemctl restart rsyslog.service
2.3.1、测试
[root@rsyslog-client ~]# systemctl restart nginx.service[root@rsyslog-server ~]# cd /var/log/rsyslog/
[root@rsyslog-server rsyslog]# ls
10.11.66.225
[root@rsyslog-server rsyslog]# cd 10.11.66.225/
[root@rsyslog-server 10.11.66.225]# ls
nginx:_2020-07-18-17:24.log  systemd:_2020-07-18-17:24.log
[root@rsyslog-server 10.11.66.225]# cat nginx\:_2020-07-18-17\:24.log  # nginx没有操作,所以没有日志内容
[root@rsyslog-server 10.11.66.225]# cat systemd\:_2020-07-18-17\:24.log   # systemctl的操作日志,被记录在systemd日志下
Jul 18 17:24:54 10.11.66.225 systemd:  Starting The nginx HTTP and reverse proxy server...
Jul 18 17:24:54 10.11.66.225 systemd:  Failed to parse PID from file /run/nginx.pid: Success
Jul 18 17:24:54 10.11.66.225 systemd:  Started The nginx HTTP and reverse proxy server.

3、基于mysql存储日志信息

3.1、安装mariadb

# 注意主机名
[root@rsyslog-server ~]# yum -y install mariadb mariadb-server
[root@rsyslog-client ~]# yum -y install rsyslog-mysql mariadb-server

3.2、配置mariadb数据库

[root@rsyslog-server ~]# systemctl enable mariadb.service --now  # rsyslog-server和rsyslog-client都需要启动,方便测试
Created symlink from /etc/systemd/system/multi-user.target.wants/mariadb.service to /usr/lib/systemd/system/mariadb.service.

3.3、配置rsyslog-server

[root@rsyslog-server ~]# mysql_secure_installation   # 数据库初始化NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDBSERVERS IN PRODUCTION USE!  PLEASE READ EACH STEP CAREFULLY!In order to log into MariaDB to secure it, we'll need the current
password for the root user.  If you've just installed MariaDB, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.Enter current password for root (enter for none):
OK, successfully used password, moving on...Setting the root password ensures that nobody can log into the MariaDB
root user without the proper authorisation.Set root password? [Y/n] y
New password:
Re-enter new password:
Password updated successfully!
Reloading privilege tables..... Success!By default, a MariaDB installation has an anonymous user, allowing anyone
to log into MariaDB without having to have a user account created for
them.  This is intended only for testing, and to make the installation
go a bit smoother.  You should remove them before moving into a
production environment.Remove anonymous users? [Y/n] y... Success!Normally, root should only be allowed to connect from 'localhost'.  This
ensures that someone cannot guess at the root password from the network.Disallow root login remotely? [Y/n] n... skipping.By default, MariaDB comes with a database named 'test' that anyone can
access.  This is also intended only for testing, and should be removed
before moving into a production environment.Remove anonymous users? [Y/n] y... Success!Normally, root should only be allowed to connect from 'localhost'.  This
ensures that someone cannot guess at the root password from the network.Disallow root login remotely? [Y/n] n... skipping.By default, MariaDB comes with a database named 'test' that anyone can
access.  This is also intended only for testing, and should be removed
before moving into a production environment.Remove test database and access to it? [Y/n] y- Dropping test database...... Success!- Removing privileges on test database...... Success!Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.Reload privilege tables now? [Y/n] y... Success!Cleaning up...All done!  If you've completed all of the above steps, your MariaDB
installation should now be secure.Thanks for using MariaDB!
[root@rsyslog-server ~]# mysql -uroot -p  # 不要在终端明文输入密码
Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 11
Server version: 5.5.65-MariaDB MariaDB ServerCopyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.MariaDB [(none)]> create database rsyslog;   # 创一个rsyslog库
Query OK, 1 row affected (0.01 sec)MariaDB [(none)]> show databases;    # 查看是否创建成功
+--------------------+  
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| rsyslog            |
+--------------------+
4 rows in set (0.01 sec)
MariaDB [(none)]> grant all on rsyslog.* to "rsyslog"@"10.11.66.%" identified by "1234.com";   # 创建一个rsyslog的用户
Query OK, 0 rows affected (0.00 sec)MariaDB [(none)]> flush privileges;   # 刷新授权
Query OK, 0 rows affected (0.00 sec)
[root@rsyslog-server ~]# cp /etc/my.cnf{,.bak}
[root@rsyslog-server ~]# vim /etc/my.cnf
skip_name_resolve=on  			# 这个参数是禁止域名解析
innodb_file_per_table=on 		# 共享表空间转化为独立表空间
[root@rsyslog-server ~]# systemctl restart mariadb.service

3.4、配置rsyslog-client

[root@rsyslog-client ~]# cat /usr/share/doc/rsyslog-8.24.0/mysql-createDB.sql
USE rsyslog;
CREATE TABLE SystemEvents
(ID int unsigned not null auto_increment primary key,CustomerID bigint,ReceivedAt datetime NULL,DeviceReportedTime datetime NULL,Facility smallint NULL,Priority smallint NULL,FromHost varchar(60) NULL,Message text,NTSeverity int NULL,Importance int NULL,EventSource varchar(60),EventUser varchar(60) NULL,EventCategory int NULL,EventID int NULL,EventBinaryData text NULL,MaxAvailable int NULL,CurrUsage int NULL,MinUsage int NULL,MaxUsage int NULL,InfoUnitID int NULL ,SysLogTag varchar(60),EventLogType varchar(60),GenericFileName VarChar(60),SystemID int NULL
);CREATE TABLE SystemEventsProperties
(ID int unsigned not null auto_increment primary key,SystemEventID int NULL ,ParamName varchar(255) NULL ,ParamValue text NULL
);
[root@rsyslog-client ~]# mysql -ursyslog -h 10.11.66.218 -p  # 测试远程连接没有问题
Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 12
Server version: 5.5.65-MariaDB MariaDB ServerCopyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
[root@rsyslog-client ~]# mysql -ursyslog -h 10.11.66.218 -p < /usr/share/doc/rsyslog-8.24.0/mysql-createDB.sql
Enter password:

3.5、测试

[root@rsyslog-client ~]# vim /etc/rsyslog.conf
#### MODULES ####
$ModLoad ommysql
#### RULES ####
#*.info;mail.none;authpriv.none;cron.none                @10.11.66.218
*.info;mail.none;authpriv.none;cron.none               :ommysql:10.11.66.218,rsyslog,rsyslog,1234.com
[root@rsyslog-client ~]# systemctl restart rsyslog.service
[root@rsyslog-client ~]# systemctl restart nginx.service
[root@rsyslog-client ~]# mysql -ursyslog -h 10.11.66.218 -p
Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 7
Server version: 5.5.65-MariaDB MariaDB ServerCopyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| rsyslog            |
+--------------------+
2 rows in set (0.00 sec)MariaDB [(none)]> use rsyslog;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -ADatabase changed
MariaDB [rsyslog]> show tables;
+------------------------+
| Tables_in_rsyslog      |
+------------------------+
| SystemEvents           |
| SystemEventsProperties |
+------------------------+
2 rows in set (0.00 sec)
MariaDB [rsyslog]> select * from SystemEvents;
+----+------------+---------------------+---------------------+----------+----------+----------------+---------------------------------------------------------------------------------------------------------------------------------+------------+------------+-------------+-----------+---------------+---------+-----------------+--------------+-----------+----------+----------+------------+-----------+--------------+-----------------+----------+
| ID | CustomerID | ReceivedAt          | DeviceReportedTime  | Facility | Priority | FromHost       | Message                                                                                                                         | NTSeverity | Importance | EventSource | EventUser | EventCategory | EventID | EventBinaryData | MaxAvailable | CurrUsage | MinUsage | MaxUsage | InfoUnitID | SysLogTag | EventLogType | GenericFileName | SystemID |
+----+------------+---------------------+---------------------+----------+----------+----------------+---------------------------------------------------------------------------------------------------------------------------------+------------+------------+-------------+-----------+---------------+---------+-----------------+--------------+-----------+----------+----------+------------+-----------+--------------+-----------------+----------+
|  1 |       NULL | 2020-07-18 18:00:12 | 2020-07-18 18:00:12 |        3 |        6 | rsyslog-client | Stopping System Logging Service...                                                                                              |       NULL |       NULL | NULL        | NULL      |          NULL |    NULL | NULL            |         NULL |      NULL |     NULL |     NULL |          1 | systemd:  | NULL         | NULL            |     NULL |
|  2 |       NULL | 2020-07-18 18:00:12 | 2020-07-18 18:00:12 |        5 |        6 | rsyslog-client |  [origin software="rsyslogd" swVersion="8.24.0-52.el7_8.2" x-pid="17500" x-info="http://www.rsyslog.com"] exiting on signal 15. |       NULL |       NULL | NULL        | NULL      |          NULL |    NULL | NULL            |         NULL |      NULL |     NULL |     NULL |          1 | rsyslogd: | NULL         | NULL            |     NULL |
|  3 |       NULL | 2020-07-18 18:00:12 | 2020-07-18 18:00:12 |        3 |        6 | rsyslog-client | Stopped System Logging Service.                                                                                                 |       NULL |       NULL | NULL        | NULL      |          NULL |    NULL | NULL            |         NULL |      NULL |     NULL |     NULL |          1 | systemd:  | NULL         | NULL            |     NULL |
|  4 |       NULL | 2020-07-18 18:00:12 | 2020-07-18 18:00:12 |        3 |        6 | rsyslog-client | Starting System Logging Service...                                                                                              |       NULL |       NULL | NULL        | NULL      |          NULL |    NULL | NULL            |         NULL |      NULL |     NULL |     NULL |          1 | systemd:  | NULL         | NULL            |     NULL |
|  5 |       NULL | 2020-07-18 18:00:17 | 2020-07-18 18:00:17 |        5 |        6 | rsyslog-client |  [origin software="rsyslogd" swVersion="8.24.0-52.el7_8.2" x-pid="18007" x-info="http://www.rsyslog.com"] start                 |       NULL |       NULL | NULL        | NULL      |          NULL |    NULL | NULL            |         NULL |      NULL |     NULL |     NULL |          1 | rsyslogd: | NULL         | NULL            |     NULL |
|  6 |       NULL | 2020-07-18 18:00:17 | 2020-07-18 18:00:17 |        3 |        6 | rsyslog-client | Started System Logging Service.                                                                                                 |       NULL |       NULL | NULL        | NULL      |          NULL |    NULL | NULL            |         NULL |      NULL |     NULL |     NULL |          1 | systemd:  | NULL         | NULL            |     NULL |
|  7 |       NULL | 2020-07-18 18:01:01 | 2020-07-18 18:01:01 |        3 |        6 | rsyslog-client | Started Session 78 of user root.                                                                                                |       NULL |       NULL | NULL        | NULL      |          NULL |    NULL | NULL            |         NULL |      NULL |     NULL |     NULL |          1 | systemd:  | NULL         | NULL            |     NULL |
|  8 |       NULL | 2020-07-18 18:02:35 | 2020-07-18 18:02:35 |        3 |        6 | rsyslog-client | Starting The nginx HTTP and reverse proxy server...                                                                             |       NULL |       NULL | NULL        | NULL      |          NULL |    NULL | NULL            |         NULL |      NULL |     NULL |     NULL |          1 | systemd:  | NULL         | NULL            |     NULL |
|  9 |       NULL | 2020-07-18 18:02:35 | 2020-07-18 18:02:35 |        3 |        6 | rsyslog-client | nginx: the configuration file /etc/nginx/nginx.conf syntax is ok                                                                |       NULL |       NULL | NULL        | NULL      |          NULL |    NULL | NULL            |         NULL |      NULL |     NULL |     NULL |          1 | nginx:    | NULL         | NULL            |     NULL |
| 10 |       NULL | 2020-07-18 18:02:35 | 2020-07-18 18:02:35 |        3 |        6 | rsyslog-client | nginx: configuration file /etc/nginx/nginx.conf test is successful                                                              |       NULL |       NULL | NULL        | NULL      |          NULL |    NULL | NULL            |         NULL |      NULL |     NULL |     NULL |          1 | nginx:    | NULL         | NULL            |     NULL |
| 11 |       NULL | 2020-07-18 18:02:35 | 2020-07-18 18:02:35 |        3 |        6 | rsyslog-client | Failed to parse PID from file /run/nginx.pid: Success                                                                           |       NULL |       NULL | NULL        | NULL      |          NULL |    NULL | NULL            |         NULL |      NULL |     NULL |     NULL |          1 | systemd:  | NULL         | NULL            |     NULL |
| 12 |       NULL | 2020-07-18 18:02:35 | 2020-07-18 18:02:35 |        3 |        6 | rsyslog-client | Started The nginx HTTP and reverse proxy server.                                                                                |       NULL |       NULL | NULL        | NULL      |          NULL |    NULL | NULL            |         NULL |      NULL |     NULL |     NULL |          1 | systemd:  | NULL         | NULL            |     NULL |
+----+------------+---------------------+---------------------+----------+----------+----------------+---------------------------------------------------------------------------------------------------------------------------------+------------+------------+-------------+-----------+---------------+---------+-----------------+--------------+-----------+----------+----------+------------+-----------+--------------+-----------------+----------+
12 rows in set (0.00 sec)

这篇关于CentOS7+Rsyslog+MySQL 搭建 Rsyslog 日志服务器的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/984637

相关文章

MySQL 中的 CAST 函数详解及常见用法

《MySQL中的CAST函数详解及常见用法》CAST函数是MySQL中用于数据类型转换的重要函数,它允许你将一个值从一种数据类型转换为另一种数据类型,本文给大家介绍MySQL中的CAST... 目录mysql 中的 CAST 函数详解一、基本语法二、支持的数据类型三、常见用法示例1. 字符串转数字2. 数字

Mysql实现范围分区表(新增、删除、重组、查看)

《Mysql实现范围分区表(新增、删除、重组、查看)》MySQL分区表的四种类型(范围、哈希、列表、键值),主要介绍了范围分区的创建、查询、添加、删除及重组织操作,具有一定的参考价值,感兴趣的可以了解... 目录一、mysql分区表分类二、范围分区(Range Partitioning1、新建分区表:2、分

MySQL 定时新增分区的实现示例

《MySQL定时新增分区的实现示例》本文主要介绍了通过存储过程和定时任务实现MySQL分区的自动创建,解决大数据量下手动维护的繁琐问题,具有一定的参考价值,感兴趣的可以了解一下... mysql创建好分区之后,有时候会需要自动创建分区。比如,一些表数据量非常大,有些数据是热点数据,按照日期分区MululbU

SQL Server配置管理器无法打开的四种解决方法

《SQLServer配置管理器无法打开的四种解决方法》本文总结了SQLServer配置管理器无法打开的四种解决方法,文中通过图文示例介绍的非常详细,对大家的学习或者工作具有一定的参考学习价值,需要的... 目录方法一:桌面图标进入方法二:运行窗口进入检查版本号对照表php方法三:查找文件路径方法四:检查 S

MySQL 删除数据详解(最新整理)

《MySQL删除数据详解(最新整理)》:本文主要介绍MySQL删除数据的相关知识,本文通过实例代码给大家介绍的非常详细,对大家的学习或工作具有一定的参考借鉴价值,需要的朋友参考下吧... 目录一、前言二、mysql 中的三种删除方式1.DELETE语句✅ 基本语法: 示例:2.TRUNCATE语句✅ 基本语

MySQL中查找重复值的实现

《MySQL中查找重复值的实现》查找重复值是一项常见需求,比如在数据清理、数据分析、数据质量检查等场景下,我们常常需要找出表中某列或多列的重复值,具有一定的参考价值,感兴趣的可以了解一下... 目录技术背景实现步骤方法一:使用GROUP BY和HAVING子句方法二:仅返回重复值方法三:返回完整记录方法四:

从入门到精通MySQL联合查询

《从入门到精通MySQL联合查询》:本文主要介绍从入门到精通MySQL联合查询,本文通过实例代码给大家介绍的非常详细,需要的朋友可以参考下... 目录摘要1. 多表联合查询时mysql内部原理2. 内连接3. 外连接4. 自连接5. 子查询6. 合并查询7. 插入查询结果摘要前面我们学习了数据库设计时要满

怎样通过分析GC日志来定位Java进程的内存问题

《怎样通过分析GC日志来定位Java进程的内存问题》:本文主要介绍怎样通过分析GC日志来定位Java进程的内存问题,具有很好的参考价值,希望对大家有所帮助,如有错误或未考虑完全的地方,望不吝赐教... 目录一、GC 日志基础配置1. 启用详细 GC 日志2. 不同收集器的日志格式二、关键指标与分析维度1.

解读GC日志中的各项指标用法

《解读GC日志中的各项指标用法》:本文主要介绍GC日志中的各项指标用法,具有很好的参考价值,希望对大家有所帮助,如有错误或未考虑完全的地方,望不吝赐教... 目录一、基础 GC 日志格式(以 G1 为例)1. Minor GC 日志2. Full GC 日志二、关键指标解析1. GC 类型与触发原因2. 堆

MySQL查询JSON数组字段包含特定字符串的方法

《MySQL查询JSON数组字段包含特定字符串的方法》在MySQL数据库中,当某个字段存储的是JSON数组,需要查询数组中包含特定字符串的记录时传统的LIKE语句无法直接使用,下面小编就为大家介绍两种... 目录问题背景解决方案对比1. 精确匹配方案(推荐)2. 模糊匹配方案参数化查询示例使用场景建议性能优