flm

fake log maker

This script will generate a list of random urls, a list of random ip address and time stamps.

The goal is to demonstrate that such data don't take much space to store.

You might need to install pip install humanfriendly

The output will look like that:

log lines: 2000000
original size: 109.12 MB
bz2 size: 16.05 MB
gzip size: 28.84 MB
zlib size: 29.7 MB

As you can see (this above is a real output, you might get a different result as it uses random number(actually, this reminded me to add a seed...)) so get ready to generate a large file. Also, depending on your computer and number of log lines, the file might crash or at least take a long time to load in your editor. Check the size first.

The script will also generate a log.txt file

You can try differents settings by simply updating the code:

population = 10000

loglines = 2000000

lines_per_minute = 5

domain_count = 120
page_per_domain_min = 3
page_per_domain_max = 13

If you want to get an idea of the range of results with the same population and url settings but with different data, consider changing the seed upon each run or simply comment the line random.seed(12345) at the beginning of the script