Web监控笔记菜鸟运维

在CentOS中安装Awstats日志分析器 解析nginx日志并启用htpasswd认证访问

Posted 安迪克斯

AWStats是在Sourceforge上发展很快的一个基于Perl的WEB日志分析工具。

AWStats 是一个免费的强大而有个性的工具,带来先进的网络,流量，FTP或邮件服务器统计图. 本日志分析器作为CGI或从命令行在数个图形网页中显示你日志中包含的所有可能信息. 它利用一部分档案资料就能经常很快地处理大量日志档案, 它能分析日志文件来自从各大服务器工具 ,如 Apache日志档案 s (NCSA combined/XLF/ELF log format or common/CLF log format), WebStar, IIS (W3C的日志格式)及许多其他网站,Proxy(代理服务器)、Wap、流量服务器、邮件服务器和一些 FTP服务器 。

在CentOS中安装Awstats日志分析器 解析nginx日志并启用htpasswd认证访问插图

关于更多Nginx与日志分析文章，请参见：

 

首先确定系统已经安装了perl

perl -version

This is perl 5, version 16, subversion 3 (v5.16.3) built for x86_64-linux-thread-multi
(with 38 registered patches, see perl -V for more detail)

Copyright 1987-2012, Larry Wall

1.安装依赖，缺啥装啥

yum -y  install gcc gcc-c++ zlib-devel perl-devel perl-ExtUtils-MakeMaker httpd-tools

我们要分析的是nginx的日志，但认证访问使用的是httpd-tools这个工具，httpd-tools自带httpd，所以安装后需要确定httpd不是启动状态，停止服务，卸载开机启动

#centos6.x
service httpd stop
chkconfig httpd off
#centos7.x
systemctl stop httpd
systemctl disable httpd

2.开启nginx访问日志
开启nginx.conf日志格式段代码

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

假设要分析的网站是www.testawstats.xyz,编辑该站点的nginx配置,记录访问日志

    access_log  /var/log/nginx/www.testawstats.xyz.access.log  main;

3.安装GeoIP库

mkdir -p /usr/local/geoip
cd /usr/local/geoip
wget -c http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
wget -c http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
#解压
gzip -d GeoLiteCity.dat.gz
gzip -d GeoIP.dat.gz

4.安装GeoIP C库

#下载
wget -c https://www.maxmind.com/download/geoip/api/c/GeoIP-1.4.8.tar.gz
tar -zxvf GeoIP-1.4.8.tar.gz
cd GeoIP-1.4.8
#安装
./configure
make && make install

5.安装GeoIP perl库

#下载
wget -c https://www.maxmind.com/download/geoip/api/perl/Geo-IP-1.40.tar.gz
tar -zxvf Geo-IP-1.40.tar.gz
cd Geo-IP-1.40
#安装
perl Makefile.PL LIBS='-L/usr/local/lib'
make && make install

6.安装CPAN库，awstats支持中文要用到

#下载
wget -c https://cpan.metacpan.org/authors/id/G/GA/GAAS/URI-1.36.tar.gz
#如果提示证书问题则加--no-check-certificate参数
wget -c --no-check-certificate https://cpan.metacpan.org/authors/id/G/GA/GAAS/URI-1.36.tar.gz
tar -zxvf URI-1.36.tar.gz
cd  URI-1.36
#安装
perl Makefile.PL
make
make install

7.安装awstats
创建源码保存目录

mkdir -p /usr/local/src/awstats
cd /usr/local/src/awstats

下载，解压

wget -c https://prdownloads.sourceforge.net/awstats/awstats-7.7.tar.gz
tar -zxvf awstats-7.7.tar.gz

将源码拷贝到运行目录

cp -r awstats-7.7 /usr/local/awstats

8.配置awstats
进入awstats工具目录

cd /usr/local/awstats/tools/

创建配置文件

./awstats_configure.pl

----- AWStats awstats_configure 1.0 (build 20140126) (c) Laurent Destailleur -----
This tool will help you to configure AWStats to analyze statistics for
one web server. You can try to use it to let it do all that is possible
in AWStats setup, however following the step by step manual setup
documentation (docs/index.html) is often a better idea. Above all if:
- You are not an administrator user,
- You want to analyze downloaded log files without web server,
- You want to analyze mail or ftp log files instead of web log files,
- You need to analyze load balanced servers log files,
- You want to 'understand' all possible ways to use AWStats...
Read the AWStats documentation (docs/index.html).

-----> Running OS detected: Linux, BSD or Unix

-----> Check for web server install

Enter full config file path of your Web server.
Example: /etc/httpd/httpd.conf
Example: /usr/local/apache2/conf/httpd.conf
Example: c:\Program files\apache group\apache\conf\httpd.conf
Config file path ('none' to skip web server setup):
> none

我们要分析的是nginx日志，输入none继续

Your web server config file(s) could not be found.
You will need to setup your web server manually to declare AWStats
script as a CGI, if you want to build reports dynamically.
See AWStats setup documentation (file docs/index.html)

-----> Update model config file '/usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf'
  File awstats.model.conf updated.

-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
file (required if first install) [y/N] ? y

输入y继续

-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
> www.testawstats.xyz

输入www.testawstats.xyz继续，之后一路回车完毕
完毕之前会提示你创建的配置文件路径和分析日志命令

......
 Config file /etc/awstats/awstats.www.testawstats.xyz.conf created.

-----> Add update process inside a scheduler
Sorry, configure.pl does not support automatic add to cron yet.
You can do it manually by adding the following command to your cron:
/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.testawstats.xyz
......

修改配置

vim /etc/awstats/awstats.www.testawstats.xyz.conf

LogFile="/var/log/nginx/www.testawstats.xyz.access.log"
DirData="/var/lib/awstats"
LoadPlugin="decodeutfkeys"
LoadPlugin="geoip GEOIP_STANDARD /usr/local/geoip/GeoIP.dat"
LoadPlugin="geoip_city_maxmind GEOIP_STANDARD /usr/local/geoip/GeoLiteCity.dat"

创建数据目录

mkdir /var/lib/awstats

9.分析日志

/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.testawstats.xyz
Create/Update database for config "/etc/awstats/awstats.www.testawstats.xyz.conf" by AWStats version 7.7 (build 20180105)
From data in log file "/var/log/nginx/www.testawstats.xyz.access.log"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 12
 Found 0 dropped records,
 Found 0 comments,
 Found 0 blank records,
 Found 0 corrupted records,
 Found 0 old records,
 Found 12 new qualified records.

成功分析了日志文件
10.搭建web界面
创建站点目录,目录会是空的，主要是conf和站点目录一一对应,不但conf要在一起，站点目录也要在一起,管理上别乱

mkdir -p /www/awstats/wwwroot

复制php文件awstats的wwwroot目录

cp /usr/local/awstats/tools/nginx/awstats-fcgi.php /usr/local/awstats/wwwroot/cgi-bin/fcgi.php

生成密钥文件

htpasswd -c -m /usr/local/htpasswd.pass youname
New password: 
Re-type new password: 
Adding password for user youname

创建nginx配置

cp /usr/local/awstats/tools/nginx/awstats-nginx.conf /etc/nginx/conf.d/awstats-nginx.conf

修改nginx配置
假设我的awstats界面域名是awstats.testawstats.xyz,我的nginx配置是这样子的

server {
    listen 80;
    server_name awstats.testawstats.xyz;
    access_log /var/log/nginx/localhost.access_log main;
    error_log /var/log/nginx/localhost.error_log info;
    root /www/awstats/wwwroot;
    index index.html;

    # Restrict access
    auth_basic "Restricted";
    auth_basic_user_file /usr/local/htpasswd.pass;


    # Static awstats files: HTML files stored in DOCUMENT_ROOT/awstats/
    location /classes/ {
        alias /usr/local/awstats/wwwroot/classes/;
    }

    location /css/ {
        alias /usr/local/awstats/wwwroot/css/;
    }

    location /icon/ {
        alias /usr/local/awstats/wwwroot/icon/;
    }

    location /awstats-icon/ {
        alias /usr/local/awstats/wwwroot/icon/;
    }

    location /js/ {
        alias /usr/local/awstats/wwwroot/js/;
    }

    # Dynamic stats.
    location ~ ^/cgi-bin/(awredir|awstats)\.pl {
        gzip off;
        fastcgi_pass 127.0.0.1:9000;
        fastcgi_param SCRIPT_FILENAME /usr/local/awstats/wwwroot/cgi-bin/fcgi.php;
        fastcgi_param X_SCRIPT_FILENAME /usr/local/awstats/wwwroot/$fastcgi_script_name;
        fastcgi_param X_SCRIPT_NAME $fastcgi_script_name;
        include fastcgi_params;
    }
}

11.访问web界面
http://awstats.testawstats.xyz/cgi-bin/awstats.pl?config=www.testawstats.xyz
在CentOS中安装Awstats日志分析器 解析nginx日志并启用htpasswd认证访问

假如出现这个界面，说明htpasswd认证设置成功了，输入用户名密码，继续
在CentOS中安装Awstats日志分析器 解析nginx日志并启用htpasswd认证访问
ok，整个流程都通了

12.定时分析日志

#编辑定时任务
crontab -e
#设置时间和脚本，每天00：01分执行一次
1 0 * * * /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.testawstats.xyz
#esc然后:wq保存并退出

更多关于Awstats的文章请参见：《强大的日志分析系统AWStats+Apache详细图文安装教程》

（END）

原文地址：https://www.jmsite.cn/blog-351.html

浏览量: 280
打赏分享

相关文章:

  1. 强大的日志分析系统AWStats+Apache详细图文安装教程
  2. 超好用的Nginx日志分析工具GoAccess使用详解
  3. 如何使用Nginx+htpasswd实现web加密访问
  4. 菜鸟笔记：如何用Nginx实现 反向绑定/缓存加速/自动更新缓存/获取真实IP/CDN加速