Skip to content

Home

Jump to bottom
Andrei Belov edited this page Apr 11, 2017 · 29 revisions

modsecurity-performance wiki

Sample results

2017-02-28 (1)

Summary for /modsec-off, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10      39445.56      46228.25      44283.87     43476.616     2184.7312
 latency (ms)
x  10           1.1          1.37          1.21         1.222   0.091627264

Summary for /modsec-light, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10       8071.25      11953.68      10634.85     10374.126     1200.2281
 latency (ms)
x  10          4.23          7.82             5         5.358     1.2109941

Summary for /modsec-full, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10         238.8        259.02        246.88       247.418     5.9640549
 latency (ms)
x  10        208.11        230.52        217.76       218.968     6.9131273

2017-02-28 (2)

Summary for /modsec-off, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10       38136.4      47561.79      44300.42     43351.954     2743.1755
 latency (ms)
x  10          1.07           1.4          1.19         1.206   0.099911072

Summary for /modsec-light, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10      10120.76       12979.9      12727.41     12290.594     891.67524
 latency (ms)
x  10          3.88          5.02          3.98         4.128    0.34726871

Summary for /modsec-full, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10        334.64        370.86        363.77       356.733     13.667706
 latency (ms)
x  10        142.49        158.27        147.04       148.176     5.8598297

2017-03-05

  • nginx: 1.11.10
  • ModSecurity: v3/dev/parser (8b8fd84)
  • ModSecurity-nginx: v3/dev/parser (9f6d3a7)
  • environment: 12-core KVM/libvirt VM on bare-metal server (Intel Xeon E5645 2.4GHz, 24 cores total)
  • configuration details:
  • nginx: worker_processes 6; worker_cpu_affinity 111111000000;
  • wrk: taskset -c 0-5 wrk -t6 -c600 -d30s (10 iterations)
Summary for /modsec-off, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10      87538.41      97062.44      93506.49      92832.83     2717.0839
 latency (ms)
x  10          6.22          6.88          6.49         6.508    0.21054427

Summary for /modsec-light, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10      27984.07      31588.76      31013.95     30484.634     1151.9494
 latency (ms)
x  10         18.98         21.57         20.06         20.01    0.81266366

Summary for /modsec-full, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10         164.5        251.93        181.79       191.226     28.760167
 latency (ms)
x  10        197.41        429.26        330.67       308.639     73.228166

The numbers for /modsec-off and /modsec-light with multi-worker nginx setup are significantly better than in single-worker mode, but /modsec-full does not show any difference. In the process of investigation it turned out that disabling audit log (by setting SecAuditEngine Off) greatly improves overall performance with OWASP CRS v3.0.0 loaded:

Summary for /modsec-full, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10        577.99        864.63        854.69       823.902      87.17359
 latency (ms)
x  10        633.42        730.81        646.95       654.834     28.487218

Further investigation showed that in case of SecAuditEngine RelevantOnly only one CPU core (among the set of cores nginx is using according to worker_cpu_affinity) is 100% busy:

With SecAuditEngine Off, all cores that are being used by nginx are constantly busy:

2017-03-07

Summary for /modsec-off, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10      44040.08       56882.7      53699.65     51600.859     4057.3947
 latency (ms)
x  10          0.87          1.14          0.99         0.977   0.084859361

Summary for /modsec-light, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10      10568.21      13532.37      12921.28      12336.21     1207.9385
 latency (ms)
x  10          3.71          4.74           3.9         4.104    0.42893149

Summary for /modsec-full, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10        238.79        259.41        253.79        250.74     7.4913906
 latency (ms)
x  10        207.47        221.61        213.96       213.635     5.7969556

2017-03-10

Summary for /modsec-off, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10      48151.25      54508.49      52135.16     51337.515     2114.2494
 latency (ms)
x  10          0.92          1.14          0.99         0.996   0.069633964

Summary for /modsec-light, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10      10456.84       13941.7      13011.89      12441.49     1375.0929
 latency (ms)
x  10          3.59           4.8          3.97         4.078    0.48276518

Summary for /modsec-full, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10        270.44        284.01        278.54       278.824     4.1302763
 latency (ms)
x  10        187.65        198.52        191.41       191.247     3.5242337

2017-04-11

Summary for /modsec-off, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10      46879.99      58925.23      53077.84     53435.713     3596.6361
 latency (ms)
x  10          0.85        843.78          0.96        85.236      266.5252

Summary for /modsec-light, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10      10060.67      13613.14      12955.81     12213.674     1312.7484
 latency (ms)
x  10          3.67          5.16          4.02         4.164     0.5084661

Summary for /modsec-full, RPS (count):
    N           Min           Max        Median           Avg        Stddev
x  10        271.63        295.32        290.55       287.362     7.2825313
 latency (ms)
x  10        179.23        192.48        183.71       185.278     4.2224369
Clone this wiki locally