SANS has published an excellent guide for evaluating open source products and applications.  A partial list of several insightful questions are shared below:

https://isc.sans.edu/forums/diary/Free+Software+Quick+Security+Checklist/21751/

Free software (open source or not) is interesting for many reasons. It can be adapted to your own needs, it can be easily integrated within complex architectures but the most important remains, of course, the price. Even if they are many hidden costs related to “free” software. In case of issues, a lot of time may be spent in searching for a solution or diving into the source code

Today, more organisations are not afraid anymore to deploy free software in their infrastructure but are those solutions really secure? A customer came to me with an interesting question about performing a security audit of free software. The idea is to validate the software before deploying it in infrastructure.

The idea is not to perform a deep source code review or to pentest the tool but more to establish a checklist of key points. I already compiled a rough list of questions that I’d like to share with you:

* What is the programming language used?  
* Architecture and security framework?
* Regular updates?
* Roadmap for the coming months?
* How big is the community around the project?
* How big is the current users base?
* The documentation and quality of documentation within the code? 
* Are external pieces of code (like libraries) used?