Registrar performance tests

Testing SER in the role of registrar on sequential and random usernames in different database modes:

mode = 0 ... no database
mode = 1 ... write through (written immediately into DB)
mode = 2 ... write back (delayed write)
mode = 3 ... readonly (?)

There was generated 100 000 sequential/random usernames used by sipp. Sequential means that they were in the form +14290xxxxxx, x are numbers; random usernames were at the same length but randomly generated. (Note: when using 100k usernames but doing 1M iterations there are 10 contacts registered per user.)

SER used 1240 MB of shared memory.

db_mode_0_sequential: Registrar with db_mode 0 with sequential usernames.
db_mode_0_random: Registrar with db_mode 0 with random usernames.
db_mode_1_sequential: Registrar with db_mode 1 with sequential usernames.
db_mode_1_random: Registrar with db_mode 1 with random usernames.
db_mode_2_sequential: Registrar with db_mode 2 with sequential usernames.
db_mode_2_random: Registrar with db_mode 2 with random usernames.
db_mode_3_sequential: Registrar with db_mode 3 with sequential usernames.
db_mode_3_random: Registrar with db_mode 3 with random usernames.

db_mode_0_sequential

Registrar with db_mode 0 with sequential usernames.

additional SIPP args: -inf data/sequential_usernames.sipp

Used config file was registrar.cfg (log here), sipp scenario register_users.xml, sipp output screens are in db_mode_0_sequential.screen.

SAR statistics: platform dependent binary data, average values, all values.

SIPP command

sipp -nd -sf scenarios/register_users.xml -trace_screen -trace_rtt -domain "example.com" -r 20000 -l 32 -m 1000000 213.192.30.219:5060 -inf data/sequential_usernames.sipp

Test stopped after 1000000 iterations, maximum allowed simultaneous calls were 32; reached average call throughput was 8057 calls per second.

Iterations	1000000
Max concurent calls	32
Max allowed Rate	20000	requests per second
Average Call Rate	8057	requests per second
Failures	0
Retransmissions	0
Timeout	0
Elapsed Time	00:02:04:120	hour:min:sec:msec

db_mode_0_random

Registrar with db_mode 0 with random usernames.

additional SIPP args: -inf data/random_usernames.sipp

Used config file was registrar.cfg (log here), sipp scenario register_users.xml, sipp output screens are in db_mode_0_random.screen.

SAR statistics: platform dependent binary data, average values, all values.

SIPP command

sipp -nd -sf scenarios/register_users.xml -trace_screen -trace_rtt -domain "example.com" -r 20000 -l 32 -m 1000000 213.192.30.219:5060 -inf data/random_usernames.sipp

Test stopped after 1000000 iterations, maximum allowed simultaneous calls were 32; reached average call throughput was 8022 calls per second.

Iterations	1000000
Max concurent calls	32
Max allowed Rate	20000	requests per second
Average Call Rate	8022	requests per second
Failures	0
Retransmissions	0
Timeout	0
Elapsed Time	00:02:04:656	hour:min:sec:msec

db_mode_1_sequential

Registrar with db_mode 1 with sequential usernames.

additional SIPP args: -inf data/sequential_usernames.sipp

Used config file was registrar.cfg (log here), sipp scenario register_users.xml, sipp output screens are in db_mode_1_sequential.screen.

SAR statistics: platform dependent binary data, average values, all values.

SIPP command

sipp -nd -sf scenarios/register_users.xml -trace_screen -trace_rtt -domain "example.com" -r 20000 -l 32 -m 1000000 213.192.30.219:5060 -inf data/sequential_usernames.sipp

Test stopped after 1000000 iterations, maximum allowed simultaneous calls were 32; reached average call throughput was 2348 calls per second.

Iterations	1000000
Max concurent calls	32
Max allowed Rate	20000	requests per second
Average Call Rate	2348	requests per second
Failures	0
Retransmissions	0
Timeout	0
Elapsed Time	00:07:05:927	hour:min:sec:msec

db_mode_1_random

Registrar with db_mode 1 with random usernames.

additional SIPP args: -inf data/random_usernames.sipp

Used config file was registrar.cfg (log here), sipp scenario register_users.xml, sipp output screens are in db_mode_1_random.screen.

SAR statistics: platform dependent binary data, average values, all values.

SIPP command

sipp -nd -sf scenarios/register_users.xml -trace_screen -trace_rtt -domain "example.com" -r 20000 -l 32 -m 1000000 213.192.30.219:5060 -inf data/random_usernames.sipp

Test stopped after 1000000 iterations, maximum allowed simultaneous calls were 32; reached average call throughput was 1037 calls per second.

Iterations	1000000
Max concurent calls	32
Max allowed Rate	20000	requests per second
Average Call Rate	1037	requests per second
Failures	0
Retransmissions	13511
Timeout	0
Elapsed Time	00:16:04:204	hour:min:sec:msec

db_mode_2_sequential

Registrar with db_mode 2 with sequential usernames.

additional SIPP args: -inf data/sequential_usernames.sipp

Used config file was registrar.cfg (log here), sipp scenario register_users.xml, sipp output screens are in db_mode_2_sequential.screen.

SAR statistics: platform dependent binary data, average values, all values.

SIPP command

sipp -nd -sf scenarios/register_users.xml -trace_screen -trace_rtt -domain "example.com" -r 20000 -l 32 -m 1000000 213.192.30.219:5060 -inf data/sequential_usernames.sipp

Test stopped after 1000000 iterations, maximum allowed simultaneous calls were 32; reached average call throughput was 2788 calls per second.

Iterations	1000000
Max concurent calls	32
Max allowed Rate	20000	requests per second
Average Call Rate	2788	requests per second
Failures	224
Retransmissions	2176
Timeout	224
Elapsed Time	00:05:58:726	hour:min:sec:msec

db_mode_2_random

Registrar with db_mode 2 with random usernames.

additional SIPP args: -inf data/random_usernames.sipp

Used config file was registrar.cfg (log here), sipp scenario register_users.xml, sipp output screens are in db_mode_2_random.screen.

SAR statistics: platform dependent binary data, average values, all values.

SIPP command

sipp -nd -sf scenarios/register_users.xml -trace_screen -trace_rtt -domain "example.com" -r 20000 -l 32 -m 1000000 213.192.30.219:5060 -inf data/random_usernames.sipp

Test stopped after 1000000 iterations, maximum allowed simultaneous calls were 32; reached average call throughput was 2099 calls per second.

Iterations	1000000
Max concurent calls	32
Max allowed Rate	20000	requests per second
Average Call Rate	2099	requests per second
Failures	320
Retransmissions	3264
Timeout	320
Elapsed Time	00:07:56:512	hour:min:sec:msec

db_mode_3_sequential

Registrar with db_mode 3 with sequential usernames.

additional SIPP args: -inf data/sequential_usernames.sipp

Used config file was registrar.cfg (log here), sipp scenario register_users.xml, sipp output screens are in db_mode_3_sequential.screen.

SAR statistics: platform dependent binary data, average values, all values.

SIPP command

sipp -nd -sf scenarios/register_users.xml -trace_screen -trace_rtt -domain "example.com" -r 20000 -l 32 -m 1000000 213.192.30.219:5060 -inf data/sequential_usernames.sipp

Test stopped after 1000000 iterations, maximum allowed simultaneous calls were 32; reached average call throughput was 8170 calls per second.

Iterations	1000000
Max concurent calls	32
Max allowed Rate	20000	requests per second
Average Call Rate	8170	requests per second
Failures	0
Retransmissions	0
Timeout	0
Elapsed Time	00:02:02:399	hour:min:sec:msec

db_mode_3_random

Registrar with db_mode 3 with random usernames.

additional SIPP args: -inf data/random_usernames.sipp

Used config file was registrar.cfg (log here), sipp scenario register_users.xml, sipp output screens are in db_mode_3_random.screen.

SAR statistics: platform dependent binary data, average values, all values.

SIPP command

sipp -nd -sf scenarios/register_users.xml -trace_screen -trace_rtt -domain "example.com" -r 20000 -l 32 -m 1000000 213.192.30.219:5060 -inf data/random_usernames.sipp

Test stopped after 1000000 iterations, maximum allowed simultaneous calls were 32; reached average call throughput was 8176 calls per second.

Iterations	1000000
Max concurent calls	32
Max allowed Rate	20000	requests per second
Average Call Rate	8176	requests per second
Failures	0
Retransmissions	0
Timeout	0
Elapsed Time	00:02:02:315	hour:min:sec:msec

Test information

Test done by kubartv from jin on Wed Mar 28 13:10:21 CEST 2007.

SIPP version:
Sipp v1.1, version 20060829, built Mar 28 2007, 11:22:45. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA Author: see source files.

Tested machine

Test done under user kubartv on 213.192.30.219, helping machine (if used) is 213.192.30.237, user kubartv.

Linux pres 2.6.17-2-686-bigmem #1 SMP Sat Sep 23 21:15:25 UTC 2006 i686 GNU/Linux

CPU

processor : 0 vendor_id : GenuineIntel cpu family : 15 model : 4 model name : Intel(R) Xeon(TM) CPU 3.00GHz stepping : 3 cpu MHz : 3000.621 cache size : 2048 KB physical id : 0 siblings : 2 core id : 0 cpu cores : 1 fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 5 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc pni monitor ds_cpl cid cx16 xtpr bogomips : 6006.61 processor : 1 vendor_id : GenuineIntel cpu family : 15 model : 4 model name : Intel(R) Xeon(TM) CPU 3.00GHz stepping : 3 cpu MHz : 3000.621 cache size : 2048 KB physical id : 0 siblings : 2 core id : 0 cpu cores : 1 fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 5 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc pni monitor ds_cpl cid cx16 xtpr bogomips : 6000.68

Memory

total used free shared buffers cached Mem: 8312532 2348984 5963548 0 156740 2016080 -/+ buffers/cache: 176164 8136368 Swap: 7807352 0 7807352

SER version

version: ser 2.0.0-rc1 (i386/linux) flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_DST_BLACKLIST ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535 poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. @(#) $Id: main.c,v 1.228.2.2 2007/03/01 13:53:37 andrei Exp $ main.c compiled on 10:34:32 Mar 26 2007 with gcc 3.3.5

Graph descriptions

There are these graphs for each scenario:

response delay: Time for each response; on X-axis is request time, i.e. when was the request sent, on Y-axis is how long time was between sending request and receiving the response. This value is not the same as values measured by watching network traffic probably due to sipp processing delay.
response delay histogram: Histogram made from time differences between sending request and receiving response. It shows numbers of response times grouped by 10 ms.
CPU usage: Shows sumary for all monitored processes for given time. Monitored processors can be set within variables script when generating the output (not needed in runtime). This output is taken from top which collects this information for one second. It can be more than 100% in the case of more processors (for example tc4 has two hyperthreaded processors thus it can show values up to 400%).
sent request histogram: Shows numbers of sent requests grouped by one second. Sometimes it shows strange jumps (for example in t_relay scenario after shmem is gone)...