Differences between revisions 3 and 4
Revision 3 as of 2010-03-16 22:11:04
Size: 8182
Comment:
Revision 4 as of 2013-04-04 14:46:24
Size: 5128
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= HttpCore performance benchmarks = = HttpCore performance micro-benchmark =
Line 5: Line 5:
In order to execute the benchmark locally install the latest snapshot of HttpCore from the Apache SVN repository and two optional modules
{{{httpcore-ab}}} and {{{httpcore-benchmark}}}
Line 9: Line 6:
http://svn.apache.org/repos/asf/httpcomponents/httpcore/trunk/httpcore-ab/
http://svn.apa
che.org/repos/asf/httpcomponents/httpcore/trunk/httpcore-benchmark/
https://svn.apache.org/repos/asf/httpcomponents/benchmark/httpcore/trunk/
Line 13: Line 9:
and then execute {{{mvn test}}} command of the {{{httpcore-benchmark}}} module and then execute {{{mvn test}}}
Line 18: Line 14:
CPU: Intel(R) Core(TM)2 Quad CPU    Q6600 @ 2.40GHz Intel(R) Core(TM) i7-3517U CPU @ 1.90GHz
Line 20: Line 16:
RAM MemTotal: 3353824 kB RAM MemTotal: 3933112 kB
Line 23: Line 19:
=== Software versions === === Components versions ===
Line 25: Line 21:
 * !HttpCore: {{{4.1-ALPHA2 (Revision: 923417)}}}  * Beanchmark revision {{{r1464575}}}
Line 27: Line 23:
 * Jetty: {{{7.0.1.v20091125}}}  * !HttpCore: {{{4.3-BETA1}}}

 * Jetty: {{{8.1.7.v20120910}}}

 * Netty: {{{3.6.2}}}
Line 31: Line 31:

=== Linux Ubuntu 9.10 / JRE 1.6 ===
Line 35: Line 32:
Linux ubuntu 2.6.31-19-generic #56-Ubuntu SMP Thu Jan 28 01:26:53 UTC 2010 i686 GNU/Linux Linux ubuntu 3.5.0-26-generic #42-Ubuntu SMP Fri Mar 8 23:18:20 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
Line 39: Line 36:
java version "1.6.0_18"
Java(TM) SE Runtime Environment (build 1.6.0_18-b07)
Java HotSpot(TM) Server VM (build 16.0-b13, mixed mode)
java version "1.7.0_13"
Java(TM) SE Runtime Environment (build 1.7.0_13-b20)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)
Line 46: Line 43:
Jetty (blocking I/O); version: 7.0.1.v20091125 Jetty (blocking I/O); version: 8.1.7.v20120910
Line 49: Line 46:
Server Software: Jetty(7.0.1.v20091125) Server Software: Jetty(8.1.7.v20120910)
Line 52: Line 49:
Line 56: Line 52:
Concurrency Level: 25
Time taken for tests: 14.147477 seconds
Complete requests: 500000
Failed requests: 0
Concurrency Level: 50
Time taken for tests: 101.296000 seconds
Complete requests: 4999999
Failed requests: 1
Line 61: Line 57:
Kept alive: 500000
Total transferred: 1024000000
bytes
Requests per second: 35,341.99 [#/sec] (mean)
Time per request: 0.707 [ms] (mean)
Time per request: 0.028 [ms] (mean, across all concurrent requests)
Transfer rate: 72,3
80.39 [Kbytes/sec] received
Kept alive: 4999999
Total transferred: 102399979
52 bytes
Requests per second: 49,360.28 [#/sec] (mean)
Time per request: 1.013 [ms] (mean)
Time per request: 0.020 [ms] (mean, across all concurrent requests)
Transfer rate: 101,0
89.85 [Kbytes/sec] received
Line 68: Line 64:
    72,380.39 kb/s total     101,089.85 kb/s total
Line 71: Line 67:
HttpCore (blocking I/O); version: 4.1-alpha2-SNAPSHOT HttpCore (blocking I/O); version: 4.3-beta1
Line 77: Line 73:
Line 81: Line 76:
Concurrency Level: 25
Time taken for tests: 8.899113 seconds
Complete requests: 500000
Concurrency Level: 50
Time taken for tests: 69.299000 seconds
Complete requests: 5000000
Line 86: Line 81:
Kept alive: 500000
Total transferred: 1024000000 bytes
Requests per second: 56,185.37 [#/sec] (mean)
Time per request: 0.445 [ms] (mean)
Time per request: 0.018 [ms] (mean, across all concurrent requests)
Transfer rate: 115,067.65 [Kbytes/sec] received
Kept alive: 5000000
Total transferred: 10240000000 bytes
Requests per second: 72,151.11 [#/sec] (mean)
Time per request: 0.693 [ms] (mean)
Time per request: 0.014 [ms] (mean, across all concurrent requests)
Transfer rate: 147,765.48 [Kbytes/sec] received
Line 93: Line 88:
    115,067.65 kb/s total     147,765.48 kb/s total
Line 95: Line 90:
Error: Socket closed
Line 97: Line 91:
Jetty (NIO); version: 7.0.1.v20091125 Jetty (NIO); version: 8.1.7.v20120910
Line 100: Line 94:
Server Software: Jetty(7.0.1.v20091125) Server Software: Jetty(8.1.7.v20120910)
Line 103: Line 97:
Line 107: Line 100:
Concurrency Level: 25
Time taken for tests: 266.941399 seconds
Complete requests: 500000
Concurrency Level: 50
Time taken for tests: 87.334000 seconds
Complete requests: 5000000
Line 112: Line 105:
Kept alive: 500000
Total transferred: 1024000000 bytes
Requests per second: 1,873.07 [#/sec] (mean)
Time per request: 13.347 [ms] (mean)
Time per request: 0.534 [ms] (mean, across all concurrent requests)
Transfer rate: 3,836.05 [Kbytes/sec] received
Kept alive: 5000000
Total transferred: 10240000000 bytes
Requests per second: 57,251.47 [#/sec] (mean)
Time per request: 0.873 [ms] (mean)
Time per request: 0.017 [ms] (mean, across all concurrent requests)
Transfer rate: 117,251.01 [Kbytes/sec] received
Line 119: Line 112:
    3,836.05 kb/s total     117,251.01 kb/s total
Line 122: Line 115:
HttpCore (NIO); version: 4.1-alpha2-SNAPSHOT HttpCore (NIO); version: 4.3-beta1
Line 128: Line 121:
Line 132: Line 124:
Concurrency Level: 25
Time taken for tests: 19.253212 seconds
Complete requests: 500000
Concurrency Level: 50
Time taken for tests: 86.098000 seconds
Complete requests: 5000000
Line 137: Line 129:
Kept alive: 500000
Total transferred: 1024000000 bytes
Requests per second: 25,969.69 [#/sec] (mean)
Time per request: 0.963 [ms] (mean)
Time per request: 0.039 [ms] (mean, across all concurrent requests)
Transfer rate: 53,185.93 [Kbytes/sec] received
Kept alive: 5000000
Total transferred: 10240000000 bytes
Requests per second: 58,073.36 [#/sec] (mean)
Time per request: 0.861 [ms] (mean)
Time per request: 0.017 [ms] (mean, across all concurrent requests)
Transfer rate: 118,934.24 [Kbytes/sec] received
Line 144: Line 136:
    53,185.93 kb/s total
}}}

=== Windows Vista / JRE 1.6 ===

{{{
java version "1.6.0_18"
Java(TM) SE Runtime Environment (build 1.6.0_18-b07)
Java HotSpot(TM) Client VM (build 16.0-b13, mixed mode, sharing)
}}}

{{{
    118,934.24 kb/s total
Line 157: Line 138:
Jetty (blocking I/O); version: 7.0.1.v20091125 ---------------------------------------------------------------
Netty; version: 3.6.2
Line 160: Line 142:
Server Software: Jetty(7.0.1.v20091125) Server Software: null
Line 163: Line 145:
Line 167: Line 148:
Concurrency Level: 25
Time taken for tests: 15.801919 seconds
Complete requests: 500000
Concurrency Level: 50
Time taken for tests: 79.597000 seconds
Complete requests: 5000000
Line 172: Line 153:
Kept alive: 500000
Total transferred: 1024000000 bytes
Requests per second: 31,641.73 [#/sec] (mean)
Time per request: 0.790 [ms] (mean)
Time per request: 0.032 [ms] (mean, across all concurrent requests)
Transfer rate: 64,
802.26 [Kbytes/sec] received
Kept alive: 5000000
Total transferred: 10240000000 bytes
Requests per second: 62,816.44 [#/sec] (mean)
Time per request: 0.796 [ms] (mean)
Time per request: 0.016 [ms] (mean, across all concurrent requests)
Transfer rate: 1
28,648.06 [Kbytes/sec] received
Line 179: Line 160:
    64,802.26 kb/s total     128,648.06 kb/s total
Line 181: Line 162:
---------------------------------------------------------------
HttpCore (blocking I/O); version: 4.1-alpha2-SNAPSHOT
---------------------------------------------------------------

Server Software: HttpCore-Test/1.1
Server Hostname: localhost
Server Port: 8989

Document Path: http://localhost:8989/rnd?c=2048
Document Length: 2048 bytes

Concurrency Level: 25
Time taken for tests: 16.984985 seconds
Complete requests: 500000
Failed requests: 0
Write errors: 0
Kept alive: 500000
Total transferred: 1024000000 bytes
Requests per second: 29,437.76 [#/sec] (mean)
Time per request: 0.849 [ms] (mean)
Time per request: 0.034 [ms] (mean, across all concurrent requests)
Transfer rate: 60,288.54 [Kbytes/sec] received
    -1 kb/s sent
    60,288.54 kb/s total
---------------------------------------------------------------
Error: socket closed
---------------------------------------------------------------
Jetty (NIO); version: 7.0.1.v20091125
---------------------------------------------------------------

Server Software: Jetty(7.0.1.v20091125)
Server Hostname: localhost
Server Port: 8989

Document Path: http://localhost:8989/rnd?c=2048
Document Length: 2048 bytes

Concurrency Level: 25
Time taken for tests: 82.707270 seconds
Complete requests: 500000
Failed requests: 0
Write errors: 0
Kept alive: 500000
Total transferred: 1024000000 bytes
Requests per second: 6,045.42 [#/sec] (mean)
Time per request: 4.135 [ms] (mean)
Time per request: 0.165 [ms] (mean, across all concurrent requests)
Transfer rate: 12,381.02 [Kbytes/sec] received
    -1 kb/s sent
    12,381.02 kb/s total
---------------------------------------------------------------
---------------------------------------------------------------
HttpCore (NIO); version: 4.1-alpha2-SNAPSHOT
---------------------------------------------------------------

Server Software: HttpCore-NIO-Test/1.1
Server Hostname: localhost
Server Port: 8989

Document Path: http://localhost:8989/rnd?c=2048
Document Length: 2048 bytes

Concurrency Level: 25
Time taken for tests: 38.239003 seconds
Complete requests: 500000
Failed requests: 0
Write errors: 0
Kept alive: 500000
Total transferred: 1024000000 bytes
Requests per second: 13,075.65 [#/sec] (mean)
Time per request: 1.912 [ms] (mean)
Time per request: 0.076 [ms] (mean, across all concurrent requests)
Transfer rate: 26,778.94 [Kbytes/sec] received
    -1 kb/s sent
    26,778.94 kb/s total

HttpCore performance micro-benchmark

Benchmark source code

https://svn.apache.org/repos/asf/httpcomponents/benchmark/httpcore/trunk/

and then execute mvn test

Hardware specs of the system used to execute the benchmark

Intel(R) Core(TM) i7-3517U CPU @ 1.90GHz
Processor cache size: 4096 KB
RAM MemTotal:  3933112 kB

Components versions

  • Beanchmark revision r1464575

  • HttpCore: 4.3-BETA1

  • Jetty: 8.1.7.v20120910

  • Netty: 3.6.2

Benchmark results

Linux ubuntu 3.5.0-26-generic #42-Ubuntu SMP Fri Mar 8 23:18:20 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

java version "1.7.0_13"
Java(TM) SE Runtime Environment (build 1.7.0_13-b20)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)

---------------------------------------------------------------
Jetty (blocking I/O); version: 8.1.7.v20120910
---------------------------------------------------------------

Server Software:                Jetty(8.1.7.v20120910)
Server Hostname:                localhost
Server Port:                    8989
Document Path:                  http://localhost:8989/rnd?c=2048
Document Length:                2048 bytes

Concurrency Level:              50
Time taken for tests:           101.296000 seconds
Complete requests:              4999999
Failed requests:                1
Write errors:                   0
Kept alive:                     4999999
Total transferred:              10239997952 bytes
Requests per second:            49,360.28 [#/sec] (mean)
Time per request:               1.013 [ms] (mean)
Time per request:               0.020 [ms] (mean, across all concurrent requests)
Transfer rate:                  101,089.85 [Kbytes/sec] received
                                -1 kb/s sent
                                101,089.85 kb/s total
---------------------------------------------------------------
---------------------------------------------------------------
HttpCore (blocking I/O); version: 4.3-beta1
---------------------------------------------------------------

Server Software:                HttpCore-Test/1.1
Server Hostname:                localhost
Server Port:                    8989
Document Path:                  http://localhost:8989/rnd?c=2048
Document Length:                2048 bytes

Concurrency Level:              50
Time taken for tests:           69.299000 seconds
Complete requests:              5000000
Failed requests:                0
Write errors:                   0
Kept alive:                     5000000
Total transferred:              10240000000 bytes
Requests per second:            72,151.11 [#/sec] (mean)
Time per request:               0.693 [ms] (mean)
Time per request:               0.014 [ms] (mean, across all concurrent requests)
Transfer rate:                  147,765.48 [Kbytes/sec] received
                                -1 kb/s sent
                                147,765.48 kb/s total
---------------------------------------------------------------
---------------------------------------------------------------
Jetty (NIO); version: 8.1.7.v20120910
---------------------------------------------------------------

Server Software:                Jetty(8.1.7.v20120910)
Server Hostname:                localhost
Server Port:                    8989
Document Path:                  http://localhost:8989/rnd?c=2048
Document Length:                2048 bytes

Concurrency Level:              50
Time taken for tests:           87.334000 seconds
Complete requests:              5000000
Failed requests:                0
Write errors:                   0
Kept alive:                     5000000
Total transferred:              10240000000 bytes
Requests per second:            57,251.47 [#/sec] (mean)
Time per request:               0.873 [ms] (mean)
Time per request:               0.017 [ms] (mean, across all concurrent requests)
Transfer rate:                  117,251.01 [Kbytes/sec] received
                                -1 kb/s sent
                                117,251.01 kb/s total
---------------------------------------------------------------
---------------------------------------------------------------
HttpCore (NIO); version: 4.3-beta1
---------------------------------------------------------------

Server Software:                HttpCore-NIO-Test/1.1
Server Hostname:                localhost
Server Port:                    8989
Document Path:                  http://localhost:8989/rnd?c=2048
Document Length:                2048 bytes

Concurrency Level:              50
Time taken for tests:           86.098000 seconds
Complete requests:              5000000
Failed requests:                0
Write errors:                   0
Kept alive:                     5000000
Total transferred:              10240000000 bytes
Requests per second:            58,073.36 [#/sec] (mean)
Time per request:               0.861 [ms] (mean)
Time per request:               0.017 [ms] (mean, across all concurrent requests)
Transfer rate:                  118,934.24 [Kbytes/sec] received
                                -1 kb/s sent
                                118,934.24 kb/s total
---------------------------------------------------------------
---------------------------------------------------------------
Netty; version: 3.6.2
---------------------------------------------------------------

Server Software:                null
Server Hostname:                localhost
Server Port:                    8989
Document Path:                  http://localhost:8989/rnd?c=2048
Document Length:                2048 bytes

Concurrency Level:              50
Time taken for tests:           79.597000 seconds
Complete requests:              5000000
Failed requests:                0
Write errors:                   0
Kept alive:                     5000000
Total transferred:              10240000000 bytes
Requests per second:            62,816.44 [#/sec] (mean)
Time per request:               0.796 [ms] (mean)
Time per request:               0.016 [ms] (mean, across all concurrent requests)
Transfer rate:                  128,648.06 [Kbytes/sec] received
                                -1 kb/s sent
                                128,648.06 kb/s total
---------------------------------------------------------------

HttpCoreBenchmark (last edited 2013-04-04 14:46:24 by OlegKalnichevski)