The recent revelations of the NSA’s BULLRUN and PRISM projects, highlight the great need to ensure an attack-free environment of software implementations. The latter task is not to be considered an easy process especially bearing in mind that software testing may consume up to half of the overall software development cost. Unfortunately, there are relatively few good methods for evaluation test set quality after ensuring basic requirements traceability. Towards that end, a number of combinatorial strategies have been devised that can maximize the probability of detecting faults, with combinatorial testing being the most prominent one. Combinatorial testing has been successfully applied for testing (critical) software systems in large organizations and is an already proven method for providing quality assurance of secure software.
In this talk, we will consider a number of research problems and challenges coming from security testing that seek solutions (also) from combinatorial mathematics. In particular, the talk will evolve around:
The problem of measuring the combinatorial coverage of existing test data. This metric is based on measuring the combinations covered by different input parameters of the system under test. The problem is exemplified in the case of security protocols where new (general) methods for estimating the combinatorial coverage will also be presented.
The problem of efficient test generation when software or hardware defects that can trigger security vulnerabilities depend on a small number of parameters is in general NP-hard. Therefore, seeking theoretical and algorithmic solutions to this problem from the field of discrete mathematics is a challenging task. We will also demonstrate that combinatorial testing can provide the means for constructing large test suites needed for triggering FPGA cryptographic Trojans.
The challenge of deriving accurate models of software systems and designing efficient security testing methods considerably reducing the amount of resources needed—with mathematical levels of trustworthiness in the evaluation results. We will present recent advances on web application security testing.
The talk is concluded with some open research problems and directions for future research on prioritized combinatorial testing (in conjunction with combinatorial optimization techniques) and fault-localization via discrete mathematical methods. These directions, if successful, can be applied ad-hoc as a general toolset in different applications domains, asides information security, e.g. in the case of mechanical engineering and automotive testing.
Dr. Dimitris E. Simos is a Key Researcher for the Applied Discrete Mathematics for Information Security research area with SBA Research, Austria where he is leading the combinatorial security testing research team. He is also an Adjunct Lecturer with Vienna University of Technology. He holds a Ph.D. in Discrete Mathematics and Combinatorics (2011) from the National Technical University of Athens. Prior to joining SBA Research, he was within the Project Team SECRET of INRIA Paris-Rocquencourt Research Center working on the design and analysis of cryptographic algorithms. His research was supported by a 3-year Marie Curie Fellow grant (2012-2015) awarded by the ERCIM through the EU-funded “Alain Bensoussan” Fellowship Programme. He is also a Fellow of the Institute of Combinatorics and its Applications (FTICA) since 2012.
This talk is organized by the Information and Software Engineering Group at the Institute of Software Technology and Interactive Systems.