Bash script to Get Top 25 error from Apache and PHP Logs Linux

Apach and PHP Logs
Apach and PHP Logs

Get Top 25 Errors From Apache & PHP Logs

Overview

In this article, I will share with you the bash script which helps you reduce the manual work of checking the logs file and provide you the top 25 Errors from Apache and Php logs.

Every Time when we face the issue on websites or any application. We need to go on the server and check for error logs. These error logs give us hint about what is the issue with the website? & accordingly, we start debugging the issue.

When I have faced such issues, I am doing the same thing like searching error logs, etc. This is a very time-consuming process to search for every log file.
So I have created the script which helps me to fetch the top 25 errors from Apache and PHP logs.

It saves a lot of my time and I am able to fix the issue immediately.

Below are the Conditions, I have used in the script file to fetch the Apache and PHP error logs,

  • Top 25 403 & 404 Error from Apache Access logs with No of counts & URL
  • Top 25 Errors from Apache error logs with No. of counts & complete error information
  • Top 25 Errors from PHP error logs with No. of counts & complete error information

Below is the complete script file which is well-tested and does not consume much CPU and memory on the server.

vim /opt/scripts/FetchErrors.sh

#!/bin/bash

YESTERDAY_DATE=`date --date="Yesterday" +"%Y%m%d"`
APACHE_ACCESS_LOG_FILE="access.log-${YESTERDAY_DATE}.gz"
APACHE_ERROR_LOG_FILE="error.log-${YESTERDAY_DATE}.gz"
PHP_ERROR_LOG_FILE="php-errors.log-${YESTERDAY_DATE}.gz"
MYSQL_SLOW_LOG_FILE=""

RSYNC=`which rsync`

#Add Webserver information

SSH_USERNAME='production-server-username'
SSH_HOSTNAME='production-server-hostname'

#Add Your WebServer IP address from where you want to fetch logs


WEB_SERVERS='webserver-ip webserver-ip2 webserver-ip3'

#Add Status code which wants to monitor or fetch info.

HTTP_STATUS_CODE='403 404'

#Provide path of log file on server


LOG_DIR='/var/log/sites/prod/logs'

#Create directory and touch file to save logs in tmp directory.


TMP_DIR='/tmp/prod_logs'
TMP_LOG_FILE='/tmp/prod_logs_mail.log'

echo "" > $TMP_LOG_FILE

#For loop to which used to fetch errors from all the servers one by one

for WEB_SERVER in ${WEB_SERVERS}; do
[[ ! -d ${TMP_DIR}/${WEB_SERVER} ]] && mkdir -p ${TMP_DIR}/${WEB_SERVER}
$RSYNC -avzP ${SSH_USERNAME}@${WEB_SERVER}${SSH_HOSTNAME}:${LOG_DIR}/${WEB_SERVER}/{$APACHE_ACCESS_LOG_FILE,$APACHE_ERROR_LOG_FILE,$PHP_ERROR_LOG_FILE} ${TMP_DIR}/${WEB_SERVER}
done

#For loop to search status code in each error log

for STATUS_CODE in $HTTP_STATUS_CODE; do
echo -e "Top 25 $STATUS_CODE errors:n" >> $TMP_LOG_FILE
zgrep "" $STATUS_CODE " ${TMP_DIR}/*/${APACHE_ACCESS_LOG_FILE} | awk -F ']' '{print $2}' | awk '{print $2}' | sort | uniq -c | sort -nr | head -n25 >> $TMP_LOG_FILE
echo -e "nn" >> $TMP_LOG_FILE
done

#Below code will fetch Top 25 Apache Errors, PHP Errors.

echo -e "Top 25 Apache errors:n" >> $TMP_LOG_FILE
zcat ${TMP_DIR}/*/${APACHE_ERROR_LOG_FILE} | awk -F ']' '{print $4}' | sort | uniq -c | sort -nr | head -n25 >> $TMP_LOG_FILE
echo -e "nn" >> $TMP_LOG_FILE

echo -e "Top 25 PHP errors:n" >> $TMP_LOG_FILE
zcat ${TMP_DIR}/*/${PHP_ERROR_LOG_FILE} | awk -F ']' '{print $2}' | awk -F 'request_id' '{print $1}' | sort | uniq -c | sort -nr | head -n25 >> $TMP_LOG_FILE
echo -e "nn" >> $TMP_LOG_FILE

Save & close the file.

Execute Permission

Give execute permission which is important to run the .sh file.

$ chmod +x /opt/scripts/FetchErrors.sh

Also, you can set Cron to run the script on a daily or weekly basis which will provide you data or logs.

That’s it. You can modify the above script according to your requirement. If you still face any issues to setup this script then comments down below in comment box.

Stay connected & subscribe with Linuxgrow.com for more such articles.

Prashant

Welcome to Linuxgrow. I'm Prashant, a tech-blogger from Mumbai, India. I started Linuxgrow as a passion and to share my knowledge about technologies. Here at Linuxgrow, I write about Linux technologies, Aws Cloud, Wordpress blogging and scripting knowledge. You can read more about me at About us page. Thank You :)

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *

Facebook