Sometimes there is a need to generate fake Apache Server Logs. Like you want to create a big data project, so you will need a large amount of data. You can perform analysis on that data and then you can produce useful results. We will be using a Python library to generate fake Apache server logs and use it for our project
- Prerequisite
- What is the Apache server log
- Install Fake log generator
- Generate logs
- Write Apache logs to an output file
- Generate Apache logs in a certain interval
- Generate an infinite amount of logs with a certain interval
- References
Prerequisite
- Python must be installed on your machine
- virtualenv must be installed
- pip must be installed
What is the Apache server log
A sample Apache log looks something like this
167.164.149.210 - - [31/Aug/2020:00:26:01 +0530] "GET /apps/cart.jsp?appID=8844 HTTP/1.0" 200 5002 "http://www.romero-johnson.com/homepage/" "Mozilla/5.0 (Windows NT 6.2; it-IT; rv:1.9.1.20) Gecko/2014-10-16 05:22:05 Firefox/3.8" Each log contains very useful information about the request. If any component of the log is absent then it is written as a hyphen(-)
- IP Address of the client
- The client’s identity is determined by identifying on the client’s machine. Will return a hyphen (-) if this information is not available. In the above log, that information is missing.
- The user ID of the client if the request was authenticated. In the above log, that information is missing.
- The time that the request was received.
- The request line includes the HTTP method used, the requested resource path, and the HTTP protocol that the client used.
- The status code that the server sends back to the client.
- The size of the object requested.
- Referrer information
- UserAgent Information e.g. from which browser this request has been received.
Install Fake log generator
Clone the codebase to your local machine from the below repository
git clone https://github.com/codewithrajranjan/Fake-Apache-Log-Generator
cd Fake-Apache-Log-Generatorcreate the virtual environment and then install the dependencies
pip install -r requirements.txtGenerate logs
- Once the installation is complete, You can generate fake Apache server logs by running the apache-fake-log-gen.py file. This file accepts many options.
- suppose you want to generate 5 logs then you can use the -n option with the apache-fake-log-generator program
python apache-fake-log-gen.py -n 5This will return you five fake Apache logs
209.37.80.163 - - [31/Aug/2020:00:58:20 +0530] "GET /wp-content HTTP/1.0" 200 4986 "http://davis-davis.com/main/" "Opera/8.77.(Windows 95; en-US) Presto/2.9.189 Version/11.00"
33.64.233.136 - - [31/Aug/2020:01:00:00 +0530] "GET /app/main/posts HTTP/1.0" 200 4963 "http://shepherd.org/" "Mozilla/5.0 (Windows NT 4.0) AppleWebKit/5310 (KHTML, like Gecko) Chrome/15.0.810.0 Safari/5310"
47.226.87.119 - - [31/Aug/2020:01:04:14 +0530] "GET /wp-admin HTTP/1.0" 200 4993 "http://burnett.info/homepage.php" "Mozilla/5.0 (Windows NT 4.0; sl-SI; rv:1.9.1.20) Gecko/2011-04-30 03:09:38 Firefox/15.0"
246.244.182.65 - - [31/Aug/2020:01:06:33 +0530] "POST /list HTTP/1.0" 200 5061 "http://www.fowler.com/home.html" "Mozilla/5.0 (Windows NT 5.01; sl-SI; rv:1.9.1.20) Gecko/2013-04-02 20:43:31 Firefox/3.8"
252.46.76.238 - - [31/Aug/2020:01:08:25 +0530] "GET /apps/cart.jsp?appID=4003 HTTP/1.0" 200 4901 "http://martinez-moon.com/main/" "Mozilla/5.0 (Windows 98; Win 9x 4.90; sl-SI; rv:1.9.1.20) Gecko/2015-06-20 12:17:42 Firefox/3.8"Write Apache logs to an output file
- You can use the -o option and then give the format as LOG.
- You will see a random access_log file in the current directory where logs will be written
python apache-fake-log-gen.py -n 5 -o LOGGenerate Apache logs in a certain interval
- You can also mimic a scenario where Apache server logs are generated in regular time intervals.
- You need to use the -s option and give the value in seconds.
- The below command will generate logs in interval of 5 seconds. It will exit when five logs are generated
python apache-fake-log-gen.py -n 5 -s 1Generate an infinite amount of logs with a certain interval
- use the -n flag and give the value as 0
- You will see that after every 1-second logs will be generated and it will never stop until you close the program
python apache-fake-log-gen.py -n 0 -s 1