[BTC-dev] Reference Implementation NMON Performance Tests
Shane Kinney
modsix at gmail.com
Thu May 28 02:29:46 UTC 2015
URL: <http://therealbitcoin.org/ml/btc-dev/attachments/20150527/attachment_b94f6d4b2e8df13c0cfe0cfd4efe5a85cec3757b.txt>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
..::[ The Bitcoin Foundation: NMON Performance Tests ]::..
[ Date: 2015.05.26 ]
[ Author: mod6 [R.01] ]
0x0] Introduction
Over the month of May, mod6 conducted a series of system level performance
tests during a full sync of the bitcoin blockchain for each version. The
metrics were captured on a three second interval with NMON [R.02] against
three different versions of the Reference Implementation:
0x01: v0.5.3 [Original] [R.03] with SHA256 [R.04],
0x02: v0.5.3.1-RELEASE [R.05] with SHA256 [R.06],
0x03: v0.5.3.1-RELEASE [R.05] with SHA256 [R.06] plus asciilifeform's [R.07]
OrphanageThermonuke Patch submitted [R.08] to the btc-dev [R.09] email
list.
All of the charts and captured data have been posted to the Bitcoin
Foundation website in the following web directories:
0x01: v0.5.3 [Original] [R.0A]
0x02: v0.5.3.1-RELEASE [R.0B]
0x03: v0.5.3.1-RELEASE + patch { OrphanageThermonuke } [R.0C]
mod6 measured only one significant difference between the three variants:
while 0x01 and 0x02 exhaust system memory and trigger OOM-kills, 0x03 syncs
the full blockchain without requiring manual intervention.
0x1] Test Environment
[ Host Environment ]:
The performance tests were conducted on an `m3.medium' AWS Debian-6
``Squeeze'' AMD64 EBS AMI. This size of instance comes with 3.75Gb of RAM
allocated.
Main disk storage [/dev/sda] for the host environment was not utilized;
all disk IO for the tests were conducted on attached volume and not
reflected in disk R/W charts [Disk.png].
[ Local Environment Processes ]:
During the conducting of these performance tests, the only third party
processes running were `bitcoind' and `nmon'.
0x2] Notable Test Observations & Test Results
[ OOM-KILL ]:
The `bitcoind' process received the ``OOM-KILL'' signal from the kernel two
times during the full sync of the blockchain for versions:
[ 0x01: v0.5.3 [R.03] ]:
In the following charts for the 0x01 performance test, the OOM-KILL
signal occurred on 20150520 & 20150522. The times where this occurred
is correlated in the graph data as follows:
All_CPUs.png [R.0D] : CPU Utilization drop to 5% on 20150520 and 10%
on 20150522.
CPU.png [R.0E] : CPU Utilization drop to 5% on 20150520 and 10%
on 20150522.
Memory.png [R.0F] : Active and Inactive Memory drops to about 200MB
on 20150520 and about 1`200MB on 20150522. It
seems that the ``OOM-KILL'' signal is received
when Active Memory nears the 3`000MB boundary.
Network.png [R.10] : Total Network Utilization drops to about 60KB/s
on 20150520 and about 40KB/s on 20150522.
Process_Switches.png [R.11] : Process Switches spiked to about
2900 Switches/s on 20150520 and about
3300 Switches/s on 20150522.
Run_Queue.png [R.12] : The Run Queue length dropped to about 1.2 on
20150520 and about 1.3 on 20150522. There was
a third drop on 20150524, but is not related
to a OOM-KILL signal.
[ 0x02: v0.5.3.1-RELEASE [R.05] ]:
All_CPUs.png [R.13] : CPU Utilization drop to 0% on 20150515 and 0%
on 20150517.
CPU.png [R.14] : CPU Utilization drop to 0% on 20150515 and 0%
on 20150517.
Memory.png [R.15] : Active and Inactive Memory drops to about 0MB
on 20150515 and about 0MB on 20150517. It seems
that the ``OOM-KILL'' signal is received when
Active Memory nears the 3`000MB boundary.
Network.png [R.16] : Total Network Utilization drops to about 0KB/s
on 20150515 and about 0KB/s on 20150517.
Process_Switches.png [R.17] : Process Switches spiked to about
4`000 Switches/s on 20150515 and about
2`000 Switches/s on 20150517; both of
these spikes occurred right before they
went to near 0.
Run_Queue.png [R.18] : The Run Queue length dropped to about 1.0 on
20150515 and about 1.0 on 20150517.
[ Differences Noted in Test Phase ]:
A major difference between these two performance test runs: when the
``OOM-KILL'' signal occured in test 0x02 there was a greater time
interval [hours] between the signal and when `bitcoind' was restarted.
This may account for why CPU/Mem/Network/Process Switches/Run Queue
reached minimal values, where they did not completely in test 0x01.
During 0x01, `bitcoind' was restarted much quicker, within about 60
minutes.
[ 0x03: No ``OOM-KILL'' with OrphanageThermonuke patch applied ]:
It is well worth noting here that this performance test of 0x03 never
never received an ``OOM-KILL'' signal during full blockchain sync.
The charts for this test also reflect this fact as there are no
precipitous graph fluctuations as there were with 0x01 and 0x02.
[ CPU Utilization ]:
Across all three full blockchain sync tests the data shows that the range
of CPU Utilization hovers in the range between 30% to 50% [R.0D] [R.13]
[R.19], with the highest spike near 60% [R.13] on test 0x02.
[ Memory Utilization ]:
Across all three full blockchain sync tests the Memory charts [R.0F] [R.15]
[R.1A] show that the range of utilized Active Memory is approximately
1`800MB to 3`000MB. However, there is a notable difference with 0x03:
Active Memory seems to peak around 2`400MB and then drop and stabilize
around 1`800MB for the duration of the sync process. As was noted in the
above section on ``OOM-KILL'' it seems to be that when Active Memory
approaches 3`000MB there is a ``OOM-KILL'' signal following shortly
thereafter. This environment has 3.75Gb of RAM available to it [3`750MB]
which translates to about an 80% utilization boundary of Active Memory.
[ Total Network Utilization [Reads/Writes] ]:
Network utilization during 0x01 and 0x02 the tests range [while running]
between troughs of around 80KB/s and peaks of about 340KB/s. The 0x03 test,
has a notably higher utilization range of around 150KB/s troughs with peaks
about 600KB/s. It should also be noted that Total Network Utilization
during test 0x03, the Reference Implementation operated in the upper bounds
of the range for about the first day and a half of the blockchain sync
process and then proceeded to taper to lower levels.
0x3] Conclusion
NMON proved to be a useful tool for gathering System Level performance
metrics while the Reference Implementation was achieving full blockchain
sync. In the future, we may utilize this tool again. However, we plan to
do further testing and analysis with tools that are aimed at the process
level to achieve better granularity.
[ References ]:
[R.01]: 027A 8D7C 0FB8 A166 4372 0F40 7217 05A8 B71E ADAF
[R.02]: http://nmon.sourceforge.net/pmwiki.php
[R.03] http://thebitcoin.foundation/v0.5.3-0-gd05c03a.tar.gz
[R.04]: aab1f8ea8c7f131ff69dfa3b9437ba35531018be760132dd6373f41a591f6382
[R.05]: http://thebitcoin.foundation/bitcoin-v0_5_3_1-RELEASE.tar.gz
[R.06]: 5c41fe6cf286770a25bf61ab0c35747d0c760f8656754296d2e1d3c4274b5686
[R.07]: 1721 5D11 8B72 3950 7FAF ED98 B982 28A0 01AB FFC7
[R.08]: http://thebitcoin.foundation/ml/btc-dev/2015-May/000088.html
[R.09]: http://thebitcoin.foundation/mailman/listinfo/btc-dev
URL="http://thebitcoin.foundation/test/perf"
[R.0A]: $URL/v0_5_3-Orig/20150526
[R.0B]: $URL/v0_5_3_1-RELEASE/20150519
[R.0C]: $URL/v0_5_3_1-RELEASE-Plus-OrphanageThermonuke/20150512
[R.0D]: $URL/v0_5_3-Orig/20150526/All_CPUs.png
[R.0E]: $URL/v0_5_3-Orig/20150526/CPU.png
[R.0F]: $URL/v0_5_3-Orig/20150526/Memory.png
[R.10]: $URL/v0_5_3-Orig/20150526/Network.png
[R.11]: $URL/v0_5_3-Orig/20150526/Process_Switches.png
[R.12]: $URL/v0_5_3-Orig/20150526/Run_Queue.png
[R.13]: $URL/v0_5_3_1-RELEASE/20150519/All_CPUs.png
[R.14]: $URL/v0_5_3_1-RELEASE/20150519/CPU.png
[R.15]: $URL/v0_5_3_1-RELEASE/20150519/Memory.png
[R.16]: $URL/v0_5_3_1-RELEASE/20150519/Network.png
[R.17]: $URL/v0_5_3_1-RELEASE/20150519/Process_Switches.png
[R.18]: $URL/v0_5_3_1-RELEASE/20150519/Run_Queue.png
[R.19]: $URL/v0_5_3_1-RELEASE-Plus-OrphanageThermonuke/20150512/All_CPUs.png
[R.1A]: $URL/v0_5_3_1-RELEASE-Plus-OrphanageThermonuke/20150512/Memory.png
[R.1B]: $URL/v0_5_3_1-RELEASE-Plus-OrphanageThermonuke/20150512/Network.png
EOF
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (FreeBSD)
iQIcBAEBCgAGBQJVZmxZAAoJEHIXBai3Hq2vFhEQAIePvmYMe4YjNZZMOmayA97m
q6OBVxekCu7x4OR8Lij1oDGvLtugkGqn5c2f8eHGVGBSv7AgAer2ZIQlWfwKSNjL
mdDxK0BTGCbk2VGRG9ONInUUOtsoNQvSdHNqlMeQPmtdyNaCa8FuF7oQXoYTOIMO
e9X3Isj52U/Ojlv5KSQTSxEkZ9BDV4x5pEGt5gHB6JTzPCghJI5QSGmBe1LGe7/y
InTRXdESiifERgY+nqQhxl2hzaYPufsdJ0fyRn+sYPOFI1zu/rMPN+jQ51f7sakk
SAFS8gUUlrBNDpL9BimxbvOk3mXgoekr4pv01h1WNHHvGA73xe3X1SKxVPFFOuzP
CtY+QKao7ZR8Gg6JfCIChkbKBK5pEQ2Y/qt70Oki0TF+App7YNtjX4Im26RJt/Gx
MhbI7Q/5ZmdNe62x8k6oJ3iO0lrGfIiQIKUM7F727e53tPKRpHwbAGm0uWYG7Ul1
5l2BC6DW6K16fB6/XYCdTfjrdO0bQeRbNNZtrJHo0q7hqjvaL4pWaccU7ISJ+miN
X6maYnAQCey6MC88xsMVjbH2dxClwyiTUdoHI5QsNXj19UIEKXs/EiXFfbLhJ1mg
KqDlGP/LtcbeMBNICLbtQ4pLrsAPf7LyNJ51ZW8noX+XaFhgDbeSvajKHwymEP28
pFg8V5opdZrPfrPERVIk
=hOY6
-----END PGP SIGNATURE-----
More information about the BTC-dev
mailing list