SecondDEP: Resilient Computing that Prevents Shellcode Execution in Cyber-Attacks

This paper proposes a novel method of preventing shellcode execution even if DEP is bypassed. The method prevents Windows APIs from calling on a data area by API hooking, based on evidence that shellcode is executed in a data area and that the shellcode calls Windows APIs. Performance tests indicated that all samples of shellcode provided by Metasploit Framework, as well asthe 18 most recent attacks using Metasploit Framework, can be detected. Comparison of this method with anti-virus products showed that this method prevented shellcode execution, whereas anti-virus products failed. Another test showed that the overhead of the method has little effect on the performance of computer operations.