posted 17 Nov 2023

Hello this is another set of benchmarks of the hinsightd webserver. Again hinsightd v0.10 is a multithreaded server and the current branch under active development while hinsightd v0.9 is a singlethreaded server with more features but also more bugs.

What we test here is how fast the server responds to increases in concurrent connections. While this is not a perfect metric for servers, a webserver that can respond to more requests in a given time period is not only more likely to resist a DDoS attack but you can also add more processing per request before the delays become noticeable. Please consult a full list of features before deciding which webserver to use for your own site.


index.html concurrency graph

Testing a small 102 bytes file. This tests how fast a server can open a connection, parse headers, and close the connection, nothing that impressive. But it should be noted the multithreaded branch is the fastest server tested.

test hinsightd/0.9.19 hinsightd/0.10.1 nginx/1.25.3 lighttpd/1.4.72 Apache/2.4.58 caddy/2.7.5
index.html -c 50 93850.01 127296.1 117020.65 56887.03 40069.56 37654.71
index.html -c 100 95571.23 121204.78 114876.51 56396.17 41065.91 36727.71
index.html -c 250 90076.29 106621.17 106724.72 53473.93 38698.64 35235.71
index.html -c 500 78003.12 91496.33 91003.4 50046.54 35879.2 33626.33
index.html -c 750 71083.31 83508.7 82136.87 50651.63 35209.66 32187.05
index.html -c 1000 66276.52 77152.16 76078.6 48633.87 33906.01 31376.68


jquery.js concurrency graph

For a more realistic test, the current jquery library, a 89.7kB minified js file. Unfortunately my server doesn't support sendfile so to make the test 'fair' I've disabled it across the board. I actually believe without sendfile this is a more realistic benchmark because in the web development world of today you rarely have unencrypted or uncompressed connections. This should test how fast a server is able to get data from the disk and push it to the http client with minimal changes.

In previous benchmarks (and in the 0.9 branch) the lack of TCP CORK flag really slowed down everything, adding support for it vastly improved performance but it's still under other servers. Further research and optimization needed.

test hinsightd/0.9.19 hinsightd/0.10.1 nginx/1.25.3 lighttpd/1.4.72 Apache/2.4.58 caddy/2.7.5
jquery.js -c 50 2103.32 28276.54 36029.15 18746.64 26397.06 28822.85
jquery.js -c 100 3288.06 27483.85 32395.48 18618.75 27594.5 28424.11
jquery.js -c 250 5974.85 25550.88 31045.68 18116.5 24566.16 27583.62
jquery.js -c 500 10033.39 23537.39 30233.58 17595.81 23997.39 26233.43
jquery.js -c 750 10200.92 21585.31 29622.17 17745.63 23632.95 25614.69
jquery.js -c 1000 9777.69 21598.88 28680.33 17556.98 23047.9 24684.84

index.html via ssl

While I would like to benchmark SSL connections, previous tests proved that without a secondary machine this is impossible, testing SSL connections is very CPU intensive so without another machine it just tests how fast the benchmark software can create connections on a CPU bottlenecked system, not very useful data.

If you find these tests useful a more comprehensive set of tests and benchmarks is include with the source code in the external/tests directory.


  1. tests were ran on a 4 core machine with no hyperthreading. hinsightd 0.10.1 and nginx have 4 workers/threads enabled the rest are left at the default.
  2. tests are run 6 times for each concurrency level and only the highest result is taken into account.
  3. both hinsightd branches are using the current HEAD.
  4. nginx config
  5. lighttd config
  6. the apache2 config is too complicated to post, but it's the distribution default.
  7. caddy config
Related topics: benchmark, hinsightd
posted 17 Nov 2023 📝 by tiotags


posted 11 Sep 2023

This post is about concurrency tests for the hinsightd server. Hinsightd v0.9 is a purely single threaded/single process webserver while the new v0.10 rewrite is a multithreaded webserver using mostly the same code as hinsightd v0.9. Currently the rewrite is still a work in progress so it doesn't have all the features of the previous version.

Edit: a flaw was detected in the testing methodology so the results are fairly useless.


index.html concurrency graph

Testing against a small file. This test basically tests how fast a server can open a connection, parse headers, and close the connection. I have to say I'm surprised of the results in my tests my development branch is always faster than nginx but for these tests I had to disable access logging in nginx, I did not expect such a large different. In any case time for optimizations.


jquery.js concurrency graph

This tests against a medium sized file. My own server seems to have some persistent bandwidth issues that I haven't pinpointed yet. The apache2 line is so bad because apache keeps throwing errors, usually it gets fixed by increasing the number of iterations but for medium sized file this takes too long for me to test.

index.html via ssl

ssl index.html concurrency graph

This test is the one that surprises me the most. I wanted to implement threads in the server because it should logically increase ssl throughput but this test looks exactly the opposite of that. I have to investigate further, I don't understand why threading would decrease performance for ssl processing unless openssl is single threaded but as far as I know this is not the case.

Edit: This benchmark is basically useless because I tested on a single machine so both the benchmark software and the server were fighting for the same resources, this is true for the tests above too but if the cpu is not bottlenecked the effect is slightly less. I will retest some day when I get access to another linux machine.

test hinsightd/0.10.1 hinsightd/0.9.19 nginx/1.25.2 lighttpd/1.4.71 Apache/2.4.57 caddy/2.7.4
index.html -c 50 88235.55 89009.15 111221.1 57542.03 41061.69 37059.68
index.html -c 100 113072.29 90904.96 109440.32 57628.58 41711.33 36194.77
index.html -c 250 93982.31 86229.94 103420.1 54405.19 38868.61 35264.54
index.html -c 500 82594.8 75005.81 90039.8 50410.34 38918.99 33320.12
index.html -c 750 77411.36 68771.53 81939.0 50894.73 38114.99 32053.03
index.html -c 1000 72974.06 64592.39 75759.87 49168.32 37654.15 31623.95
jquery.js -c 50 2534.11 2205.48 30930.45 17948.91 26170.89 28317.06
jquery.js -c 100 10683.82 3621.08 28285.42 17870.61 25567.6 28060.88
jquery.js -c 250 13657.51 6018.14 27123.35 17457.67 24235.43 27269.8
jquery.js -c 500 13657.47 8494.7 26590.51 17128.92 23576.01 25962.63
jquery.js -c 750 13221.6 8376.68 25870.81 17267.22 6619.22 25533.33
jquery.js -c 1000 12419.48 8177.4 24944.75 16978.42 648.32 25040.0
ssl_index.html -c 50 29414.62 33393.0 33263.59 1725.03 14528.04 16864.54
ssl_index.html -c 100 21528.99 26042.68 23523.49 1726.26 5700.76 12044.7
ssl_index.html -c 250 11090.64 14406.29 12238.76 1395.36 5073.35 5912.12
ssl_index.html -c 500 6121.92 9036.61 6503.25 1055.53 3271.95 2937.05
ssl_index.html -c 750 4135.2 6644.76 4363.63 1274.92 3496.11 1950.87
ssl_index.html -c 1000 3081.8 5260.24 3370.99 987.59 5125.96 1460.06


  1. tests were ran on a 4 core machine with no hyperthreading with 4 threads enabled in: hinsightd 0.10.1 and nginx.
  2. tests are run 5 times for each concurrency level and only the highest result is taken into account
  3. nginx config
  4. lighttd config
  5. the apache2 config is complicated and has many files so I will not post it here, it's also mostly the distribution default
  6. caddy config
  7. due to testing on a single machine the ssl benchmark is basically useless
Related topics: benchmark, hinsightd
posted 11 Sep 2023 📝 by tiotags


posted 22 Nov 2022

New benchmarks are here, after several optimization patches and some unexplained performance improvements, here we are.

Optimization takes a lot of time and depends on trial and error plus modern magic, old code is rarely in a good shape, frequently breaks at the lightest change, insufficient documentation also hampers optimization efforts, and that dreaded feeling of "what does this do again ?" is encountered way too often. But the biggest problem by far is motivation, it's very hard to justify spending 90% time to increase performance by 10%; despite all of this sometimes magic prevails and the stars align to improve performance. Though I have to be honest I wish I knew why moving 3 lines of code to another function increases performance by 15k req/s.

So on to the tests, spoiler alert we managed to beat Lighttpd.

for 250 concurrency ab -k -c 250 -n 10000 http://localhost:<port>/

server and version failed requests requests/sec (mean)
hinsightd/0.9.17 0 63035.01
nginx/1.23.2 0 26673.64
lighttpd/1.4.67 0 53693.29
Apache/2.4.54 414 37474.10
Caddy/2.6.2 0 35412.02

for 500 concurrency ab -k -c 500 -n 10000 http://localhost:<port>/

server and version failed requests requests/sec (mean)
hinsightd/0.9.17 0 54984.63
nginx/1.23.2 0 26172.73
lighttpd/1.4.67 215 1613.59
Apache/2.4.54 1221 34305.55
Caddy/2.6.2 0 33995.57


Please note I am a server developer not a server administrator and might not know of all the ways to increase performance of other servers while I know how to get every last bit of performance out of my own. So please take these kind of tests with a drop of salt.

Due to rapid io uring development and possible variance, it's needs to be noted we are using linux kernel version 6.0.9.

I wish I had the time to investigate why Lighttpd starts going very slowly when overloaded.

Related topics: benchmark, hinsightd
posted 22 Nov 2022 📝 by tiotags


posted 06 Jul 2022

Performance usually comes with tradeoffs. These can take many forms, from skipping security checks to only optimizing a small part of the program that is being benchmarked. If you let performance guide a design other aspects of a program can suffer. Or in other words it doesn't matter how fast a broken program runs.

Despite optimization not being a priority some benchmarks are so interesting that are worth sharing. The condensing patch referenced below condenses multiple network write request into a single one.

for 500 concurrency ab -k -c 500 -n 10000 http://localhost:<port>/

server and version requests/sec (mean)
before patches 22790.46
-O2 alone 23167.02
After the condensing patch 26393.86
After patch + -O2 30257.64
nginx/1.21.6 26683.67
lighttpd/1.4.64 fails at 500 concurrency
Apache/2.4.54 fails at 500 concurrency

bonus for 100 concurrency due to other server giving errors ab -k -c 100 -n 10000 http://localhost:<port>/

server and version requests/sec (mean)
after patch + -O2 32136.88
nginx/1.21.6 26788.39
lighttpd/1.4.64 56971.62
Apache/2.4.54 fails at 100 concurrency

Switching from -Os to -O3 has a small performance impact, but reducing write requests adds around 4k req/s (about 20% faster). Looking at the other servers defaults I have to say I'm confused, I thought nginx would be faster, it does handle concurrency better than either apache or lighttpd though. Also it seems lighttpd is really fast at static requests.

While testing consumes time and I can't say I'm a big fan of optimization, free performance with minimal tradeoffs is best performance.

Now on to figuring out why file transfers are so slow.

Related topics: benchmark, hinsightd
posted 06 Jul 2022 📝 by tiotags


posted 09 Jun 2022

They say brevity is the soul of wit. This is expecially true for programming where the more you write the farther from efficiency you get.

Even knowing that I always struggle to make managable git commits. I have no problem writing small code, but whenever version control is involved the desire to fix x and y and z gets the better of me.

Sure smaller commits are easier to review, but what about the cognitive load of fighting your urge to write long commits ? It might sound like a joke but is it really ?

posted 09 Jun 2022 by tiotags


posted 05 May 2022

read more ...

posted 05 May 2022 📝 by tiotags

Better writing

posted 02 Feb 2022

read more ...

posted 02 Feb 2022 by tiotags


posted 28 Jan 2022

read more ...

posted 28 Jan 2022 by tiotags
Archived posts