Yesterday’s post was about the exciting keynote at this year’s MetriSec. Today’s post is about another highlight, the panel.
One of the biggest problems in empirical studies about computer security is the data. Usually you can’t control the data acquisition process yourself; instead, you need to take other people’s work and use that. For example, you could be using Mozilla Foundation Security Advisories, or the National Vulnerability Database. Then the question is, to what extent can you trust this information to be complete and unbiased?
The answer is that you cannot, at least not without knowing the process by which these databases are created. For example, many researchers have for years believed that the NVD constitutes some kind of ground truth. If that were true, then one would expect that entries that have been in the NVD for some time will in general not change. Work currently being done here at ETH indicates, however, that the amount of change, or churn, in the NVD is quite high, and that even very old entries get changed!
The panel discussion at MetriSec will discuss these problems. I will moderate, and participants will be at least Laurie Williams, Peter Gutmann, and Fabio Massacci. All three have much experience with empirical work, so I expect a high-class discussion.
And this is why you should come to MetriSec 2012, too!
[Edit 2012-07-18: Added Fabio as panelist.]