[Gnuk-users] benchmarking security tokens speed

Szczepan Zalega | Nitrokey szczepan at nitrokey.com
Sat Aug 26 08:34:25 UTC 2017


On 08/26/2017 12:35 AM, Antoine Beaupré wrote:
> It looks like the Yubikey 4 is the fastest, being (only?) 10 times
> slower than the CPU (i3-6100U). That slows down another order of
> magnitude with 4096 keys. The NEO is as slow in 2048 as the 4 is in
> 4096, and of course doesn't support 4096 at all. The FST-01 is the
> slowest of the bunch, taking more than a full second to kick decryption
> in 2048bit RSA and 8 seconds in 4096 bits.
> 
> I'm looking for feedback on the results and the test procedure, which is
> a Python script, attached. I'm aware of the limitations of the script,
> namely that it treats the *whole* GPG decryption process as a blackbox,
> which includes AES and all sorts of stuff. In my tests, GPG chooses
> AES-256 which is why I chose a 16 bytes filesize. Since the timings
> seems to be fairly consistent, I am assuming the delays are consistent.
> 
Hi!

Nice initiative! It is good you have a script already. The more it is
automated the better.
To make the tests reproducible please give environment details: OS name,
bits and version, GPG version and from hardware side firmware versions
of used devices.
I would also remove the `pv` from pipeline since it does its own
buffering and could influence the test results. The tests should be done
on ramdisk (/dev/shm etc) to exclude disk access sharing with OS - with
so small times this is a necessity.
Why not using PKCS#11 directly (and measure real RSA speed of the
device, since AES is done in CPU anyway) instead of blackboxing the GPG?
How many runs have you done for each device? Have you removed the outliers?

You can also try to compare the results with another benchmarking tool,
like graphene-cli [1]. They have some test results already but I cannot
find it right now.

[1] https://github.com/PeculiarVentures/graphene-cli

-- 
Best regards,
Szczepan



More information about the gnuk-users mailing list