Benchmarking
hugusmaximus opened this issue · comments
I'm trying to make a simple benchmark to a very simple code using "tiny_http" like this:
fn main(){
let server = Server::http("0.0.0.0:80");
match server {
Ok(_) => println!("Success"),
Err(_) => println!("Error")
}
for request in server.expect("REASON").incoming_requests() {
let response = Response::from_string("Hello!");
let _ = request.respond(response);
}
}
I'm doing an extremely soft "benchmark" and getting those results:
hugo@anakin:~/hermit-rs-template$ ab -n 10 -c 3 -r "http://10.0.5.3/pruebastress"
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 10.0.5.3 (be patient).....done
Server Software: tiny-http
Server Hostname: 10.0.5.3
Server Port: 80
Document Path: /pruebastress
Document Length: 0 bytes
Concurrency Level: 3
Time taken for tests: 0.286 seconds
Complete requests: 10
Failed requests: 20
(Connect: 0, Receive: 10, Length: 0, Exceptions: 10)
Total transferred: 156 bytes
HTML transferred: 13 bytes
Requests per second: 34.97 [#/sec] (mean)
Time per request: 85.794 [ms] (mean)
Time per request: 28.598 [ms] (mean, across all concurrent requests)
Transfer rate: 0.53 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 12 14 4.5 12 27
Waiting: 0 0 0.0 0 0
Total: 12 14 4.5 12 27
Percentage of the requests served within a certain time (ms)
50% 12
66% 12
75% 13
80% 14
90% 27
95% 27
98% 27
99% 27
100% 27 (longest request)
Looks like concurrent connections are not handled also with no concurrency at all I get this:
hugo@anakin:~/hermit-rs-template$ ab -n 1000 -r "http://10.0.5.3/pruebastress"
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 10.0.5.3 (be patient)
Completed 100 requests
Completed 200 requests
(...)
Finished 1000 requests
(...)
Document Path: /pruebastress
Document Length: 13 bytes
Concurrency Level: 1
Time taken for tests: 213.315 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 156000 bytes
HTML transferred: 13000 bytes
Requests per second: 4.69 [#/sec] (mean)
Time per request: 213.315 [ms] (mean)
Time per request: 213.315 [ms] (mean, across all concurrent requests)
Transfer rate: 0.71 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 20 30 4.3 32 56
Processing: 146 183 6.2 185 208
Waiting: 132 166 5.9 167 189
Total: 170 213 5.9 213 256
Percentage of the requests served within a certain time (ms)
50% 213
66% 214
75% 215
80% 215
90% 217
95% 219
98% 222
99% 224
100% 256 (longest request)
My console output (qemu-system-aarch64):
[LOADER][INFO] Loader: [0x40200000 - 0x4021f000]
[LOADER][INFO] Found ELF file with size 30194072
[LOADER][INFO] Parsing kernel from ELF at 0x48000000..0x49ccb998 (len = 0x1ccb998 B / 30194072 B)
[LOADER][INFO] Loading kernel to 0x40400000..0x40a2d228 (len = 0x62d228 B / 6476328 B)
[LOADER][INFO] TLS is at 0x408d90a8..0x408d9148 (len = 0xa0 B / 160 B)
[LOADER][INFO] Detect 1 CPU(s)
[LOADER][INFO] Detect UART at 0x9000000
[LOADER][INFO] Jumping to HermitCore Application Entry Point at 0x406b2794
[0][INFO] Welcome to Hermit 0.6.7
[0][INFO] Kernel starts at 40400000
[0][INFO] BSS starts at 0x40a2b7b0
[0][INFO] tls_info = Some(
TlsInfo {
start: 0x408d90a8,
filesz: 0x20,
memsz: 0xa0,
align: 0x8,
},
)
[0][INFO] RAM starts at physical address 40000000
[0][INFO] Physical address range: 16384GB
[0][INFO] Support of 4KB pages: true
[0][INFO] Support of 16KB pages: false
[0][INFO] Support of 64KB pages: true
[0][INFO] Total memory size: 500 MB
[0][INFO] Kernel region: [40400000 - 40c00000]
[0][INFO] A pure Rust application is running on top of Hermit!
[0][INFO] Heap: size 432 MB, start address 40c00000
[0][INFO] Heap is located at 0x40c00000..0x5bc00000 (0 Bytes unmapped)
[0][INFO]
[0][INFO] ===================== PHYSICAL MEMORY FREE LIST ======================
[0][INFO] 0x0000005BCD4000 - 0x00000060000000
[0][INFO] ======================================================================
[0][INFO]
[0][INFO]
[0][INFO] ================== KERNEL VIRTUAL MEMORY FREE LIST ===================
[0][INFO] 0x0000005BC00000 - 0x00000100000000
[0][INFO] ======================================================================
[0][INFO]
[0][INFO] Intialize generic interrupt controller
[0][INFO] Found GIC Distributor interface at 8000000 (size 0x10000)
[0][INFO] Found generic interrupt controller at 80a0000 (size 0xF60000)
[0][INFO]
[0][INFO] ========================== CPU INFORMATION ===========================
[0][INFO] Processor compatiblity: arm,cortex-a72
[0][INFO] Counter frequency: 62500000 Hz (from CNTFRQ_EL0)
[0][INFO] Run on hypervisor
[0][INFO] ======================================================================
[0][INFO]
[0][INFO] Hermit booted on 2023-12-29 11:39:58.0 +00:00:00
[0][INFO] Mapping PCI Enhanced Configuration Space interface to virtual address 60000000 (size 0x10000000)
[0][INFO] Scanning PCI Busses 0 to 255
[0][INFO] Compiled with PCI support
[0][INFO] Compiled with ACPI support
[0][INFO] Compiled with FSGSBASE support
[0][INFO] Compiled with SMP support
[0][INFO]
[0][INFO] ======================== PCI BUS INFORMATION =========================
[0][INFO] 00:00 Host bridge [0600]: Red Hat, Inc. QEMU PCIe Host bridge [1B36:0008]
[0][INFO] 00:01 Ethernet controller [0200]: Red Hat, Inc. Virtio network device [1AF4:1041], IRQ 4, BAR1 Memory32 { address: 0x0, size: 0x1000, prefetchable: false }, BAR4 Memory64 { address: 0x8000000000, size: 0x4000, prefetchable: true }
[0][INFO] ======================================================================
[0][INFO]
[0][INFO] Hermit is running on common system!
[0][INFO] Found virtio network device with device id 0x1041
[0][WARN] Currently only mapping of 64 bit BAR's is supported!
[0][WARN] Currently only mapping of 64 bit BAR's is supported!
[0][INFO] Non Virtio PCI capability with id 11 found. And NOT used.
[0][ERROR] Found virtio capability whose BAR is not mapped or non existing. Capability of type 5 and id 0 for device 1041, can not be used!
[0][INFO] Feature set wanted by network driver are in conformance with specification.
[0][INFO] Feature set wanted by network driver are in conformance with specification.
[0][INFO] Driver found a subset of features for virtio device 1041. Features are: [VIRTIO_NET_F_CSUM, VIRTIO_NET_F_GUEST_CSUM, VIRTIO_NET_F_MAC, VIRTIO_NET_F_MRG_RXBUF, VIRTIO_NET_F_STATUS, VIRTIO_F_RING_INDIRECT_DESC, VIRTIO_F_VERSION_1]
[0][INFO] Features have been negotiated between virtio network device 1041 and driver.
[0][INFO] Created SplitVq: idx=0, size=256
[0][INFO] Created SplitVq: idx=1, size=256
[0][INFO] Network driver successfully initialized virtqueues.
[0][INFO] Device specific initialization for Virtio network device 1041 finished
[0][INFO] Network device with id 1041, has been initialized by driver!
[0][INFO] Virtio-net link is up after initialization.
[0][INFO] Virtio network driver initialized.
[0][INFO] Install virtio interrupt handler at line 4
[0][INFO] Trying to initialize network!
[0][INFO] MAC address 52-54-00-12-34-56
[0][INFO] Configure network interface with address 10.0.5.3/24
[0][INFO] Configure gateway with address 10.0.5.1
[0][INFO] MTU: 1514 bytes
[0][WARN] Unable to read entropy! Fallback to a naive implementation!
Success
I'm using Hermit 0.8.0. I'm missing something for sure as this low performance is not normal.... any hint?
I have to check. In the past, we had some performance issue. But I thought that we have solved it. Maybe it come back with an PR. Currently, our checks do not evaluate, if a PR creates some power losses.
On my system I got better performance. Naive question. Do you build the Hermit application with --release?
No. But I got 4/5 requests/second which is several orders of magnitude
below what I would expect... how much performance are you getting? (just in
case I'll try with --release...).
El jue., 4 ene. 2024 19:34, Stefan Lankes ***@***.***>
escribió:
… On my system I got better performance. Naive question. Do you build the
Hermit application with --release?
—
Reply to this email directly, view it on GitHub
<#516 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQVGKU5TPEAOWFSD6SIMEVLYM3Y4DAVCNFSM6AAAAABBGQ7ZBKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZXGU3TMNZYGA>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
Yeah, it's expected to be slow without --release. We should add a hint to the README in https://github.com/hermit-os/hermit-rs-template.
Yes, with --release I got 40 req/sec. Not a rocket but it works for me. Thanks!
Network performance is something that we are actively looking into, so it should get better in the future. :)
My apologies. I'm getting much, much more. This 40 req/sec is because an experimental KASLR I was implementing and also plus SSL offloading proxy before. Just tried your vanilla code with plain HTTP and I get 10.000 req/sec, which looks pretty good!!!
hugo@anakin:~/home/hugo$ wrk -c100 -d10000 -t 10 http://10.137.0.3:9975
Running 167m test @ http://10.137.0.3:9975
10 threads and 100 connections
^C Thread Stats Avg Stdev Max +/- Stdev
Latency 8.97ms 2.27ms 25.83ms 82.25%
Req/Sec 1.08k 323.60 10.69k 86.06%
164378 requests in 15.32s, 38.22MB read
Requests/sec: 10731.98
Transfer/sec: 2.50MB
Ah, I am relieved. Thanks for clarifying!