Security Awareness Blog

Security Awareness Metric - What is Your Prevention / Detection Ratio?

One of the great things about the annual RSA conference is meeting people smarter then you. Simple, informal conversations or structured presentations are a tremendous way to learn and come up with new ideas. The other night I had a chance to have dinner with Andy Jaquith, author of Security Metrics, often considered the bible of, well security metrics. Obviously the challenge of measuring human behavior popped up, and during the discussion we had a cool idea, your P / T ratio (prevent vs. detection ratio). It goes something like this.

1. When I first started measuring human behavior, my main focus was on measuring the effectiveness of prevention. The classic example is phishing assessments, if your awareness training is effective the number of people who fall victim to a phishing assessment should go down over time.

2. During the process of running such tests I saw another, just as valuable metric, the number of people who reported the phishing assessments. In other words, not only did they not fall victim, but because of their training they knew they should report it, and how to report it. This measured detection, an added bonus I was not expecting.

The idea of a P / T ratio is track that combination over time. For example, when you first start your phishing assessments you may have seven people fall victim for every one person that identifies and reports it, so your ratio is 7/1. Over time (say nine months) you could potentially reverse that, so for example for every one person that falls victim seven report it, so your ratio is 1/7. What I like about this metric is you can use it for other assessments that test behavior (social engineering phone calls, SMS scams, etc). In addition this ratio works well for comparing your score other organizations, regardless of how much larger or smaller they are than you.

2 Comments

Posted March 2, 2012 at 4:22 AM | Permalink | Reply

Kenta Little

I've read a large amount posts on your website and I'm strongly able to connect security-awareness with occupational-disaster-prevention (for lack of better words). However, I'm not quite able to bridge the gap between the two. So my question is: Can a company's Occupational Safety and Health branch and the company's Security branch work as one team or should they remain two separate branches.
Thanks,

Posted March 2, 2012 at 3:19 PM | Permalink | Reply

lspitzner

Kenta, great question! To be honest it depends. There are tremendous advantages combining programs, including cutting down costs and sharing resources. But challenges include your message can get diluted or just how well do the two different branches work together. I tend to see smaller organizations combine such programs (including other topics such as privacy, disaster recovery or business continuity) while larger organizations tend to have separate departments for each of these topics.