Training and evaluation process

The objective is to run each signature verification tool on the evaluation signature dataset. The training and evaluation data are collected offline and online. Participants are asked to return similarity scores that are ranked numerically, where a higher score indicates a higher similarity between the questioned signature and the reference signature. Similarity scores should be ranked numerically, where higher indicates a higher similarity between the questioned signature and the reference signature. A second binary score is requested for the decision if the questioned signature is written by the same writer as the reference signature (score 1) or not (score 0).

Submission of executables

Participants are required to submit a software tool that is able to compare one (1) questioned signature against one (1) reference signature and that outputs a similarity score and a decision score. The tool must be made available as a Linux or Windows-win32 command line application (.o /.exe file) or a standalone java J2SE application (.jar file).

The interface of the verification tool (called sigVerify) consists of 4 parameters, separated by spaces. Read more...

The executable will not be used for purposes other than the ICDAR2009 competition. Performances will be published in the ICDAR proceedings and in an extended journal publication. After the competition, all executables will be destroyed. If a team chooses to use some expiration mechanism, the expiration should be set to 31 December 2009.

Evaluation of performance

In the training and the evaluation phase, signatures can either be authentic: written by a authentic writer, or forged: forgery of a signature of the authentic writer. The training set consists of a small group of authentic writers and a large number of skilled forgeries whereas the evaluation set consists of a large group of authentic writers and a smaller number of skilled forgeries.

Performance will be evaluated in Detection Error Trade-off (DET) curves containing Equal Error Rates based on the similarity scores. Additional experiments will be conducted for an extended journal publication.

Schedule

Training set available: January 20th, 2009
Deadline for submitting verification tool: March 9th, 2009
Presentation of performance at ICDAR 2009 in a special session.