<— Back

If The Hype Doesn’t Kill You, Flawed or Missing Analysis Will

Many vendors like to use dazzling terms that describe the same technology their competitors have and/or are doing the same thing that their competitors do. As an analogy, one company that sells bottled water calls theirs pure while another says purified. The one that wants to dazzle with science will talk about how “we’ve combined 2 sets of hydrogen atoms with one oxygen molecule having the specific atomic weight of 8 in order to create the purest water that is scientifically attainable. Or, my favorite, “Now with twice as much hydrogen as oxygen.” The last one actually was on a billboard, albeit deliberately humorous.

When testing security products, the quality of the hype doesn’t cut it. The quality of protection is king. But deciding on a solution requires more than beautiful numbers. If one product protects against 100% of the attacks against it, while another scores 94%. Which product is best? What about the ones scoring better or worse than 94%? The answer is that you don’t have enough data to reach a logical decision. For example, in some cases regulatory compliance requirements must prioritized. If a product must be “accredited” as providing PCI DSS requirements, then non-accredited products are not viable solutions, regardless of quality. There are a variety of considerations that must be evaluated alone and in conjunction with each other. In this presentation we will provide anti-hype information designed to help IT practitioners improve the quality and comprehensiveness of their analysis of the results of security product test data, regardless of what test organization is providing results. Data doesn’t lie, but numbers laugh at those who make purchase decisions based on data without analysis.

Randy Abrams

Randy Abrams is a 25+ veteran of the cyber security industry. During his 12-year career at Microsoft Randy designed, implemented, and managed the multiscanning system used by Microsoft to ensure that infected software is not released, and worked as the Operations Manager for the Global Infrastructure Alliance for Internet Security, a program that provided security information to ISPs across the globe. Randy has also served as the Director of Technical Education at ESET, a Research Director at NSS Labs, a Senior Security Analyst at Webroot, a senior security analyst at OPSWAT, and is currently a senior security analyst for cloud security company SecureIQLab. From 2000 to 2019 Randy served on the board of directors for the Association of Anti Virus Asia Researchers and remains on the education outreach advisory board for the organization.