Due to the still increasing interconnectedness of systems it is very much important to further strengthen activities towards assuring security requirements of those systems. Quality assurance methods like coding guidelines with a focus on security related issues, and static analysis tools are necessary but not sufficient because of the fact that security is a system property. Therefore, it is important to also perform system tests focusing on security threads. When carrying out in a manual way testing is very labor intensive and the question arise whether it is possible to automate security testing? In this paper we take up this question, discuss the underlying challenges, and introduce current work dealing with the automation of security testing. In particular, we present work on using combinatorial testing and AI planning for detecting vulnerabilities in systems. In addition, we discuss shortcomings of the present approaches, open research challenges and further research directions.