Following the work done by Dan Berstein to compare various cryptographic primitives in SUPERCOP, from stream ciphers, hash functions, and more recently CAESAR candidates, we introduce a Python tool that we call SUPERCOPy that allows to parse SUPERCOP results and draw custom graphs to compare various primitives.
The source code can be downloaded here: SUPERCOPy.tar.gz.
Presentation of SUPERCOPy
We show below the graphical interface of SUPERCOPy, which allows to select:
- the type of data to display,
- the lengths of inputs,
- the oldest SUPERCOP version to consider,
- the types of architectures to include in the graph,
- the primitives to add.
Example: CAESAR competition
Selection of algorithms
The current algorithms selected to appear in the SUPERCOPy graphs are reported in the following table.
|Versions of Round 3 CAESAR candidates selected in the graphs|
|CLOC and SILC||aes128n12t8clocv3, aes128n12t8silcv3|
- For AES-JAMBU, while a version 2 exists (aesjambuv2):
- there is no recorded benchmarks for it in SUPERCOP yet,
- and to the best of our knowledge there is no design difference between v1 and v2.
- For Ketje Sr, a version 2 also exists (ketjesrv2), however:
- the change between v1 and v2 mainly reorders the way the data is inputed/outputed from the internal state, which should not affect the performances by a significant factor,
- and the only SUPERCOP benchmark available for ketjesrv2 are extremely slow (reference implementation only).
- COLM is excluded because software implementations have not yet been benchmarked in SUPERCOP.
Case study: long messages, no associated data
We give an example of the graphs the tool can output, by showing the results from SUPERCOP (after 2016/10/26) on the CAESAR competition Round 3 candidates.
More graphs comparing the CAESAR Round 3 candidates can be found below. Data "X+Y" refers to a message either long or containing X bytes and associated data either long or containing Y bytes. The graphs include aes128gcmv1 (for comparison) as well as the 15 CAESAR candidates that advanced to the Round 3.
|Comparison of Round 3 CAESAR candidates (main versions)|
Example: Stream Ciphers
We give another example by showing the results from SUPERCOP (after 2016/09/10) on a selection of stream ciphers for long messages.
Details about the source code
The source code of SUPERCOPy can be found here: SUPERCOPy.tar.gz.
Generating all the graphs can be achieved by running the Bash script
This file contains a sequence of commands described below.
First, the script fetches the SUPERCOP data as HTML files:
curl -s -O https://bench.cr.yp.to/computers.html curl -s -O https://bench.cr.yp.to/results-caesar.html curl -s -O https://bench.cr.yp.to/results-hash.html curl -s -O https://bench.cr.yp.to/results-sha3.html curl -s -O https://bench.cr.yp.to/results-stream.html
Then, the script calls two Python script (given here with the example of stream ciphers):
python parse.py -f results-stream.html -c computers.html -a ALL > stream.txt, which parses the input file to format it for input to the next script.
python generate_graphs.py stream.txt 00000000 messages long, which generates the graph as PNG and PDF files, with an explicit filename.
The script that triggers a graphical interface is called
generate_graphs_interface.py and runs under Python3 and relies on Tkinter.
Details about the data
SUPERCOPy starts by parsing the page of results from SUPERCOP and discard all the benchmarks marked as "unstable", "normally" or the values with high variance (in red).
Then, we print on a graph the selected algorithms (e.g., the main proposal for the Round 3 CAESAR candidates), sorted by architectures (amd64, x86, arm, etc.), then chronologically.