One of the great things about the annual RSA conference is meeting people smarter then you. Simple, informal conversations or structured presentations are a tremendous way to learn and come up with new ideas. The other night I had a chance to have dinner with Andy Jaquith, author of Security Metrics, often considered the bible of, well security metrics. Obviously the challenge of measuring human behavior popped up, and during the discussion we had a cool idea, your P / T ratio (prevent vs. detection ratio). It goes something like this.
1. When I first started measuring human behavior, my main focus was on measuring the effectiveness of prevention. The classic example is phishing assessments, if your awareness training is effective the number of people who fall victim to a phishing assessment should go down over time.
2. During the process of running such tests I saw another, just as valuable metric, the number of people who reported the phishing assessments. In other words, not only did they not fall victim, but because of their training they knew they should report it, and how to report it. This measured detection, an added bonus I was not expecting.
The idea of a P / T ratio is track that combination over time. For example, when you first start your phishing assessments you may have seven people fall victim for every one person that identifies and reports it, so your ratio is 7/1. Over time (say nine months) you could potentially reverse that, so for example for every one person that falls victim seven report it, so your ratio is 1/7. What I like about this metric is you can use it for other assessments that test behavior (social engineering phone calls, SMS scams, etc). In addition this ratio works well for comparing your score other organizations, regardless of how much larger or smaller they are than you.