Blog / Benchmark3

This post is about concurrency tests for the hinsightd server. Hinsightd v0.9 is a purely single threaded/single process webserver while the new v0.10 rewrite is a multithreaded webserver using mostly the same code as hinsightd v0.9. Currently the rewrite is still a work in progress so it doesn't have all the features of the previous version.

Edit: a flaw was detected in the testing methodology so the results are fairly useless.

index.html

Testing against a small file. This test basically tests how fast a server can open a connection, parse headers, and close the connection. I have to say I'm surprised of the results in my tests my development branch is always faster than nginx but for these tests I had to disable access logging in nginx, I did not expect such a large different. In any case time for optimizations.

jquery.js

This tests against a medium sized file. My own server seems to have some persistent bandwidth issues that I haven't pinpointed yet. The apache2 line is so bad because apache keeps throwing errors, usually it gets fixed by increasing the number of iterations but for medium sized file this takes too long for me to test.

index.html via ssl

This test is the one that surprises me the most. I wanted to implement threads in the server because it should logically increase ssl throughput but this test looks exactly the opposite of that. I have to investigate further, I don't understand why threading would decrease performance for ssl processing unless openssl is single threaded but as far as I know this is not the case.

Edit: This benchmark is basically useless because I tested on a single machine so both the benchmark software and the server were fighting for the same resources, this is true for the tests above too but if the cpu is not bottlenecked the effect is slightly less. I will retest some day when I get access to another linux machine.

test	hinsightd/0.10.1	hinsightd/0.9.19	nginx/1.25.2	lighttpd/1.4.71	Apache/2.4.57	caddy/2.7.4
index.html -c 50	88235.55	89009.15	111221.1	57542.03	41061.69	37059.68
index.html -c 100	113072.29	90904.96	109440.32	57628.58	41711.33	36194.77
index.html -c 250	93982.31	86229.94	103420.1	54405.19	38868.61	35264.54
index.html -c 500	82594.8	75005.81	90039.8	50410.34	38918.99	33320.12
index.html -c 750	77411.36	68771.53	81939.0	50894.73	38114.99	32053.03
index.html -c 1000	72974.06	64592.39	75759.87	49168.32	37654.15	31623.95
jquery.js -c 50	2534.11	2205.48	30930.45	17948.91	26170.89	28317.06
jquery.js -c 100	10683.82	3621.08	28285.42	17870.61	25567.6	28060.88
jquery.js -c 250	13657.51	6018.14	27123.35	17457.67	24235.43	27269.8
jquery.js -c 500	13657.47	8494.7	26590.51	17128.92	23576.01	25962.63
jquery.js -c 750	13221.6	8376.68	25870.81	17267.22	6619.22	25533.33
jquery.js -c 1000	12419.48	8177.4	24944.75	16978.42	648.32	25040.0
ssl_index.html -c 50	29414.62	33393.0	33263.59	1725.03	14528.04	16864.54
ssl_index.html -c 100	21528.99	26042.68	23523.49	1726.26	5700.76	12044.7
ssl_index.html -c 250	11090.64	14406.29	12238.76	1395.36	5073.35	5912.12
ssl_index.html -c 500	6121.92	9036.61	6503.25	1055.53	3271.95	2937.05
ssl_index.html -c 750	4135.2	6644.76	4363.63	1274.92	3496.11	1950.87
ssl_index.html -c 1000	3081.8	5260.24	3370.99	987.59	5125.96	1460.06

Notes:

1. tests were ran on a 4 core machine with no hyperthreading with 4 threads enabled in: hinsightd 0.10.1 and nginx.
2. tests are run 5 times for each concurrency level and only the highest result is taken into account
3. nginx config
4. lighttd config
5. the apache2 config is complicated and has many files so I will not post it here, it's also mostly the distribution default
6. caddy config
7. due to testing on a single machine the ssl benchmark is basically useless