aiven'blog

zaqar性能测试

背景

本文主要是针对社区P版本的zaqar进行性能测试,选取后端从出为mongo和redis的情况进行。测试的作者是社区的开发者,这里只做简单的分析说明。

环境说明

zaqar运行在物理环境:32 cores 2.4G; mem:125G
zaqar运行在虚拟环境: 8cores 2.3G; mem:32G
zaqar测试程序运行的环境: 32 cores 2.4G; mem:125G
使用zaqar-bench进行测试,zaqar-bench是zaqar自带的测试工具。

  • pp:Number of Producer Processes
  • pw:Number of Producer Workers
  • cp:Number of Consumer Processes
  • cw:Number of Consumer Workers
  • op:Number of Observer Processes
  • ow:Number of Observer Workers
  • t:测试时间

后端使用mongo测试

1)zaqar server + M version keystone + physical environment: (storage back to mongodb)

使用M版本的keystone,基于物理环境,后端使用mongo

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
zaqar-bench -pp 1 -pw 1 -cp 1 -cw 1 -ow 0 -t 30
Using 'envvars' credentials
Using 'keystone' authentication method
Resetting queues...
Producer
========
duration_sec: 30.9
ms_per_req: 291.4
reqs_per_sec: 3.3
successful_reqs: 103.0
total_reqs: 103.0
Consumer
========
claim_total_requests: 25.0
duration_sec: 32.1
messages_processed: 72.0
ms_per_claim: 323.3
ms_per_delete: 313.5
reqs_per_sec: 3.0
successful_reqs: 97.0
total_reqs: 97.0

2) zaqar server + no keystone authentication + physical environment: (storage back to mongodb)

不使用keystone,基于物理环境,后端使用mongo

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
zaqar-bench -pp 1 -pw 1 -cp 1 -cw 1 -op 0 -ow 0 -t 120
{
"consumer": {
"claim_total_requests": 1560,
"ms_per_claim": 16.619511598195785,
"total_reqs": 8908,
"reqs_per_sec": 74.20981874717658,
"successful_reqs": 8908,
"duration_sec": 120.03802394866943,
"ms_per_delete": 12.768336625999904,
"messages_processed": 7348
},
"producer": {
"duration_sec": 120.01488304138183,
"ms_per_req": 15.17950070666844,
"total_reqs": 7895,
"successful_reqs": 7895,
"reqs_per_sec": 65.78350784442091
},
}

3) zaqar server + M version keystone + virtual machine environment: (storage back to mongodb)

使用M版本的keystone,基于虚拟机环境,后端使用mongo

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
zaqar-bench -pp 1 -pw 1 -cp 1 -cw 1 -ow 0 -t 30
Using 'envvars' credentials
Using 'keystone' authentication method
Resetting queues...
Benchmarking Zaqar API v2...
Starting producer (pp=1, pw=1)...
Starting consumers (cp=1, cw=1)...
Producer
========
duration_sec: 30.3
ms_per_req: 107.2
reqs_per_sec: 9.3
successful_reqs: 280.0
total_reqs: 280.0
Consumer
========
claim_total_requests: 66.0
duration_sec: 30.6
messages_processed: 224.0
ms_per_claim: 104.8
ms_per_delete: 104.7
reqs_per_sec: 9.5
successful_reqs: 290.0
total_reqs: 290.0

4) zaqar server + no keystone authentication + virtual machine environment: (storage back to mongodb)

不使用keystone,基于虚拟机环境,后端使用mongo

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
zaqar-bench -pp 1 -pw 1 -cp 1 -cw 1 -op 0 -ow 0 -t 120
Using 'noauth' authentication method
Resetting queues...
Benchmarking Zaqar API v2...
Starting producer (pp=1, pw=1)...
Starting consumers (cp=1, cw=1)...
Producer
========
duration_sec: 120.0
ms_per_req: 25.3
reqs_per_sec: 39.5
successful_reqs: 4740.0
total_reqs: 4740.0
Consumer
========
claim_total_requests: 1206.0
duration_sec: 120.0
messages_processed: 4707.0
ms_per_claim: 25.1
ms_per_delete: 19.0
reqs_per_sec: 49.3
successful_reqs: 5913.0
total_reqs: 5913.0

后端使用redis测试

1
2
3
4
5
The best performance is based on redis , followed mongodb, and performance worst based on swift(very poor).
The most stable based on mongodb. In the physical machine, there is no exception in high stress test based on mongodb.
There will be abnormal in high stress test based on redis:[Could not claim messages : Error response from Zaqar. Code: 503. Title: Service temporarily unavailable. Description: Claim could not be created. Please try again in a few seconds..]
It is most unstable based on swift.

1) zaqar + redis (message) + mysql (management):

不使用keystone,基于物理环境,消息后端使用redis,管理后端使用mysql

1
2
3
4
uwsgi :
threads = 1
processes = 8
noauth

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
zaqar-bench -pp 1 -pw 1 -cp 1 -cw 1 -op 0 -ow 0 -t 120
Producer
========
duration_sec: 120.1
ms_per_req: 12.1
reqs_per_sec: 82.3
successful_reqs: 9883.0
total_reqs: 9883.0
Consumer
========
claim_total_requests: 1656.0
duration_sec: 120.1
messages_processed: 7981.0
ms_per_claim: 14.0
ms_per_delete: 12.1
reqs_per_sec: 80.3
successful_reqs: 9637.0
total_reqs: 9637.0

2) zaqar + mongo (message) + mongo (management):

不使用keystone,基于物理环境,消息后端使用mongo,管理后端使用mongo

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
zaqar-bench -pp 1 -pw 1 -cp 1 -cw 1 -op 0 -ow 0 -t 120
Using 'noauth' authentication method
Resetting queues...
Benchmarking Zaqar API v2...
Starting producer (pp=1, pw=1)...
Starting consumers (cp=1, cw=1)...
Producer
========
duration_sec: 120.0
ms_per_req: 25.3
reqs_per_sec: 39.5
successful_reqs: 4740.0
total_reqs: 4740.0
Consumer
========
claim_total_requests: 1206.0
duration_sec: 120.0
messages_processed: 4707.0
ms_per_claim: 25.1
ms_per_delete: 19.0
reqs_per_sec: 49.3
successful_reqs: 5913.0
total_reqs: 5913.0

3) zaqar + swift (message) + mysql (management):

不使用keystone,基于物理环境,消息后端使用swift,管理后端使用mysql

1
2
3
4
uwsgi :
threads = 4
processes = 2
noauth

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
zaqar-bench -pp 1 -pw 1 -cp 1 -cw 1 -op 0 -ow 0 -t 120
Producer
========
duration_sec: 120.1
ms_per_req: 82.4
reqs_per_sec: 12.1
successful_reqs: 1456.0
total_reqs: 1456.0
Consumer
========
claim_total_requests: 138.0
duration_sec: 120.8
messages_processed: 669.0
ms_per_claim: 421.1
ms_per_delete: 93.5
reqs_per_sec: 6.7
successful_reqs: 807.0
total_reqs: 812.0

物理环境与虚拟环境对比

物理机的测试结果好于虚拟机。

1) zaqar server + no keystone authentication + physical environment:

不使用keystone,基于物理环境,消息后端使用redis,管理后端使用mongodb

1
2
3
4
5
6
7
8
9
Stable test: send 484 messages per second, receive 758 messages per second.
Physical machine / virtual machine send: 484/115 = 4.2 times
Physical machine / virtual machine reception: 758/118 = 6.4 times
Based on the physical machine, The combination of redis (message) + mongodb (management) has the highest performance,
it is more than the combination of mongodb (message) + mongodb (management):
send 795 messages per second, receive 1001 messages per second, but always the problem.
Load average:65 cpu%:74%

1
2
3
uwsgi:
threads = 48
processes = 40
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
venv/bin/zaqar-bench -pp 4 -pw 48 -cp 4 -cw 48 -op 0 -ow 0 -t 120
Using 'noauth' authentication method
Benchmarking Zaqar API v2...
"consumer": {
"claim_total_requests": 33853,
"ms_per_claim": 9.782496440415192,
"total_reqs": 91839,
"reqs_per_sec": 758.1115688847759,
"successful_reqs": 91839,
"duration_sec": 121.14179992675781,
"ms_per_delete": 6.696917968526063,
"messages_processed": 57986
},
"producer": {
"duration_sec": 120.50231504440307,
"ms_per_req": 8.220760190922908,
"total_reqs": 58429,
"successful_reqs": 58429,
"reqs_per_sec": 484.8786513227559
},

2) zaqar server + no keystone authentication + virtual machine: (storage based on mongo)

不使用keystone,基于虚拟机环境,后端使用mongo

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
zaqar-bench -pp 4 -pw 10 -cp 0 -cw 0 -op 4 -ow 10 -t 120
Producer
========
duration_sec: 121.0
ms_per_req: 45.3
reqs_per_sec: 115.3
successful_reqs: 13957.0
total_reqs: 13959.0
Consumer
========
claim_total_requests: 2530.0
duration_sec: 121.3
messages_processed: 11842.0
ms_per_claim: 99.9
ms_per_delete: 35.0
reqs_per_sec: 118.4
successful_reqs: 14372.0
total_reqs: 14372.0

3) zaqar server + no keystone authentication + virtual machine: (storage based on redis)

不使用keystone,基于虚拟机环境,后端使用redis

1
2
3
4
5
uwsgi :
threads = 4
processes = 8
noauth
CPU 61%

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
zaqar-bench -pp 6 -pw 10 -cp 6 -cw 10 -op 0 -ow 0 -t 120
Producer
========
duration_sec: 120.7
ms_per_req: 37.2
reqs_per_sec: 161.7
successful_reqs: 19522.0
total_reqs: 19522.0
Consumer
========
claim_total_requests: 3312.0
duration_sec: 121.2
messages_processed: 15874.0
ms_per_claim: 100.8
ms_per_delete: 39.2
reqs_per_sec: 158.3
successful_reqs: 19186.0
total_reqs: 19186.0

测试总结

  • keystone的认证对zaqar有很大的性能影响,如果取消keystone的认证,性能能够提升20倍。
  • redis的性能优于mongo,但是mongo较为稳定,redis会出现503错误。
  • zaqar部署在物理机,并且消息后端使用Redis,管理后端使用MongoDB具有最高的性能,

热评文章