From Syllab
Jump to: navigation, search

Following the work done by Dan Berstein to compare various cryptographic primitives in SUPERCOP, from stream ciphers, hash functions, and more recently CAESAR candidates, we introduce a Python tool that we call SUPERCOPy that allows to parse SUPERCOP results and draw custom graphs to compare various primitives.

The source code can be downloaded here: SUPERCOPy.tar.gz.

Authors: Jérémy Jean and Thomas Peyrin.


Presentation of SUPERCOPy

We show below the graphical interface of SUPERCOPy, which allows to select:

  • the type of data to display,
  • the lengths of inputs,
  • the oldest SUPERCOP version to consider,
  • the types of architectures to include in the graph,
  • the primitives to add.
Graphical interface of SUPERCOPy for CAESAR Round 3 candidates
Graphical interface of SUPERCOPy for stream ciphers

Example: CAESAR competition

Selection of algorithms

The current algorithms selected to appear in the SUPERCOPy graphs are reported in the following table.

Versions of Round 3 CAESAR candidates selected in the graphs
ACORN acorn128v3
AEGIS aegis128l
AEZ aezv4
Ascon ascon128v12
CLOC and SILC aes128n12t8clocv3, aes128n12t8silcv3
COLM (*)
Deoxys deoxysi128v141
AES-JAMBU (*) aesjambuv1
Ketje (*) ketjesrv1
Keyak lakekeyakv2
MORUS morus1280128v2
NORX norx6441v3
OCB aeadaes128ocbtaglen128v1
AES-OTR aes128otrpv3
Tiaoxin tiaoxinv2

(*) Notes:

  • For AES-JAMBU, while a version 2 exists (aesjambuv2):
    • there is no recorded benchmarks for it in SUPERCOP yet,
    • and to the best of our knowledge there is no design difference between v1 and v2.
  • For Ketje Sr, a version 2 also exists (ketjesrv2), however:
    • the change between v1 and v2 mainly reorders the way the data is inputed/outputed from the internal state, which should not affect the performances by a significant factor,
    • and the only SUPERCOP benchmark available for ketjesrv2 are extremely slow (reference implementation only).
  • COLM is excluded because software implementations have not yet been benchmarked in SUPERCOP.

Case study: long messages, no associated data

We give an example of the graphs the tool can output, by showing the results from SUPERCOP (after 2016/10/26) on the CAESAR competition Round 3 candidates.

Other cases

More graphs comparing the CAESAR Round 3 candidates can be found below. Data "X+Y" refers to a message either long or containing X bytes and associated data either long or containing Y bytes. The graphs include aes128gcmv1 (for comparison) as well as the 15 CAESAR candidates that advanced to the Round 3.

Comparison of Round 3 CAESAR candidates (main versions)
Encrypt Decrypt Forgery Attempt
long+0 Here Here Here
long+long Here Here Here
0+long Here Here Here
1536+0 Here Here Here
1536+1536 Here Here Here
0+1536 Here Here Here
64+0 Here Here Here
64+64 Here Here Here
0+64 Here Here Here

Example: Stream Ciphers

We give another example by showing the results from SUPERCOP (after 2016/09/10) on a selection of stream ciphers for long messages.


Details about the source code

The source code of SUPERCOPy can be found here: SUPERCOPy.tar.gz.

Generating all the graphs can be achieved by running the Bash script

This file contains a sequence of commands described below.

First, the script fetches the SUPERCOP data as HTML files:

curl -s -O
curl -s -O
curl -s -O
curl -s -O
curl -s -O

Then, the script calls two Python script (given here with the example of stream ciphers):

  • python -f results-stream.html -c computers.html -a ALL > stream.txt, which parses the input file to format it for input to the next script.
  • python stream.txt 00000000 messages long, which generates the graph as PNG and PDF files, with an explicit filename.

The script that triggers a graphical interface is called and runs under Python3 and relies on Tkinter.

Details about the data

SUPERCOPy starts by parsing the page of results from SUPERCOP and discard all the benchmarks marked as "unstable", "normally" or the values with high variance (in red).

Then, we print on a graph the selected algorithms (e.g., the main proposal for the Round 3 CAESAR candidates), sorted by architectures (amd64, x86, arm, etc.), then chronologically.