trstk is a toolkit is conceived for these purposes:
- Plot the DET curve for a particular system
- Check the consistency between score files w.r.t. the filenames scores refer to
To install from the command line on a machine you have access to the python installation tree (e.g., on a Windows machine):
pip install trstk
If you don't have adminstrative rights on the Python installation directory, you can create an isolated virtual environment using virtualenv. Follow instructions there to download and create a virtual environment and then either easy_install or pip install this package.
We describe a few scenarios for using the Toolkit in specific cases. Read the full documentation in the doc directory for instructions on how to create your own scripts that can re-use the readout functionality available in the kit.
Example 1: Plotting a DET Curve
The following command will plot a single DET curve for a given input score file:
This command should produce a single plot in PDF file named det.pdf calculated using the contents of the input score file test.scores. The plot title will be empty. You can change the output filename and its type (we support either .png files or .jpg) or add a plot title like this:
plotDET.py --title="My Test DET" --output=test.png test.scores
You can plot a series of overlayed DET curves in the following manner:
plotDET.py --title="My Test DET" --output=overlayed.pdf \
--label=devel development.scores --label=test test.scores
This command will produce a single plot in a PDF file, with the overlayed DET curves generated using each of the score files given as input parameters. A legend will be drawn at a convenient location in the plot using the labels for each of the curves as determined by your input. By default the program generates black-and-white plots, but can be instructed to produce coloured plots using the --colour option (see plotDET.py --help message).
Example 2: Checking score set consistency
You can check the consistency between two (or more) score sets that are supposed to provide scores for multiple biometric modalities using the checkModalities.py script. This tool will compare two input files and will stop on the first error it finds:
checkModalities.py faceverif.scores speechverif.scores
If you sort all files before calling the program, huge score files can be checked in a much faster way as we will avoid the sorting step within the program. You can do this using the sort and uniq unix utilities to sort all score files before using checkModalities.py like this:
sort my-scores.txt | uniq > sorted-scores.txt
sort other-scores.txt | uniq > other-sorted-scores.txt
checkModalities.py --sorted sorted-scores.txt other-sorted-scores.txt