Automated Test Case Generation for Digital System Designs: A Mapping Study on VHDL, Verilog and SystemVerilog Description Languages

Researchers have proposed different methods for testing digital systems and circuits in the last couple of decades. The need for testing digital logic circuits has become more important than ever due to the growing complexity of such systems. During the design phase, testing is focusing on design defects as well as manufacturing and wear out type of defects. Failures in digital systems could be caused, for example, by design errors, the use of inherently probabilistic devices and manufacturing variability. As a way to test digital systems in a more efficient way, automated test generation has been proposed to automatically create tests that can quickly and accurately identify faulty components. Examples of such techniques are the sequential test generation, the scan path testing and the random test generation techniques. With the research domain becoming more mature and growing, it is essential to systematically identify, analyse and classify these contributions. We perform a systematic mapping study of automated test generation for digital circuits aimed at providing an overview of the application of these techniques. We focus on three of the most widely-used and well-supported hardware description languages (HDLs) for digital systems: Verilog, SystemVerilog and VHDL. Our results suggest that the majority of the test generation methods for digital circuits are focused on the behavioral and register-transfer design levels. Fault-independent and fault-oriented test generation are the most frequently reported types of test generation methods, while HDL model simulation is the most common test generation technology used to search for test cases in these academic studies. While the results are suggesting a growing interest in this area, the majority of articles are published as conferences papers. Our results show that only 31% of the methods are implemented as software tools and only 63% of all contributions are actually generating executable test cases. This study makes three important contributions, (i) a state-of-the-art of test generation for digital system designs research is provided, (ii) the reported characteristics are identified in both the primary papers and experimental reports, (iii) gaps and opportunities for future test generation for digital system designs research are identified.