Science

In Search of Evidence-Based IT-Security

IT security is largely a science-free field. This needs to change.
Applied IT security is largely a science-free field. The IT-Security industry is selling a range of products with often very questionable and sometimes outright ridiculous claims. Yet it's widely accepted practice among users and companies that protection with security appliances, antivirus products and firewalls is a necessity. There are no rigorous scientific studies that try to evaluate the effectiveness of most security products or strategies. Evidence-based IT security could provide a way out of the security nihilism that's often dominating the debate – however it doesn't exist yet.
From Next-Generation APT-Defense to Machine Learning and Artificial Intelligence: The promises of IT security product vendors are often bold. Some marketing promises are simply impossible, because they violate a fundamental theorem of computer science, the halting problem. Many IT security professionals are skeptical of security appliances, antivirus software and other IT security products and call them snake oil. Furthermore security products often have security vulnerabilities themselves, which has lately been shown by the impressive work done by Tavis Ormandy from Google's Project Zero. When there's disagreement about the effectiveness of an approach then rational people should ask for scientific evidence. However, surprisingly this evidence largely doesn't exist. While there obviously is a lot of scientific research in IT security it rarely tries to answer practical questions most relevant to users. Decisions are made in an ad-hoc way and are usually based on opinions rather than rigorous scientific evidence. It is quite ironic that given the medical analogies this field likes to use (viruses, infections etc.), nobody is looking how medicine solves these problems. The gold standard of scientific evidence in medicine (and many other fields) is to do randomized controlled trials (RCTs) and meta-analyses of those trials. An RCT divides patients in groups and a treatment – for example a new drug – is compared against a placebo treatment or against the current best practice. Single trials are usually not considered sufficient, therefore meta-analyses pool together the results of all trials done on a particular question. There's no reason RCTs couldn't be applied to the question whether a particular security product works. Evidence-based medicine is undoubtedly the right approach, but these methods aren't without problems. Publication Bias skews results, many studies cannot be replicated and the scientific publishing and career system is often supporting poor scientific practices. But this doesn't question the scientific approach itself, it just means that more rigorous scientific practices need to be implemented. Unfortunately, in the few cases where controlled studies are done in the Infosec world they often suffer from the most basic methodological problems like being underpowered (too few participants), never being independently replicated or not measuring relevant outcomes. (There are a few studies on password security and similar questions.) Applying rigorous science to IT security could provide a way out of the security nihilism that dominates the debate so often these days - “Everything is broken, everyone's going to get hacked eventually”. And by learning from other fields Evidence-Based IT Security could skip the flaws that rife other fields of science.

Additional information

Type lecture
Language English

More sessions

12/27/16
Science
Ulf Treger
Saal G
What are the politics and aesthetics of mapping? An introduction how cartography shapes cities and landscapes, creates borders and determines the perception of our environment. How an evolving mix of high-resolution satellite imagery, algorithm-based mappings and the huge amount of data of digitized cities will enhance these effects? And in contrast, how can maps be designed, that question the “objectivity” and “correctness” of conventional cartography?
12/27/16
Science
Aylin Caliskan
Saal 2
Artificial intelligence and machine learning are in a period of astounding growth. However, there are concerns that these technologies may be used, either with or without intention, to perpetuate the prejudice and unfairness that unfortunately characterizes many human institutions. We show for the first time that human-like semantic biases result from the application of standard machine learning to ordinary language—the same sort of language humans are exposed to every day. We replicate a ...
12/27/16
Science
Bernd Sieker
Saal 1
Legend has it that most airline pilots will at one time have uttered the sentence "What's it Doing now?", whenever the autopilot or one of its related systems did something unexpected. I will be exploring some high-profile accidents in which wrong expectations of automation behaviour contributed to the outcome.
12/28/16
Science
André Lampe
Saal 1
Jeder weiß ungefähr was ein Mikroskop ist und vielleicht hat man auch mal davon gehört das da immernoch dran geforscht wird – Stichwort Hochauflösungsmikroskopie (Nobelpreis 2014 in Chemie). Es gibt deutlich mehr Mikroskope in der professionellen Forschung als es Teleskope gibt, deutlich mehr – und da könnte man sich jetzt fragen: "Warum sehe ich so viele Bilder von Sterne, aber kaum Mikroskopiebilder von öffentlichen Einrichtungen und Stellen?". Um diese Frage zu beantworten will ich ...
12/28/16
Science
Axel
Saal 1
Physicists are not computer scientists. But at CERN and worldwide, they need to analyze petabytes of data, efficiently. Since more than 20 years now, ROOT helps them with interactive development of analysis algorithms (in the context of the experiments' multi-gigabyte software libraries), serialization of virtually any C++ object, fast statistical and general math tools, and high quality graphics for publications. I.e. ROOT helps physicists transform data into knowledge. The presentation will ...
12/28/16
Science
KaLeiMai
Saal 2
The Anthropocene is widely understood to mean the current <em>&quot;period of Earth's history during which humans have a decisive influence on the state, dynamics and future&quot;</em> of this planet. For several years, scientists in the <a href="http://quaternary.stratigraphy.org/workinggroups/anthropocene/" title="Website of the Working Group on the &#39;Anthropocene&#39; (AWG)">Working Group on the 'Anthropocene' (AWG)</a> have <a ...
12/29/16
Science
Anja Drephal
Saal 2
Used in cell phone technology, bluetooth devices, and WiFi, Frequency Hopping Spread Spectrum (FHSS) is often said to have been invented in the early 1940s by none other than Hollywood actress and sex symbol Hedy Lamarr. This talk will present the undeniably entertaining history of a well-known actress moonlighting as a military inventor as well as give an overview of the 100-year-old history of frequency hopping and its past and present uses.