Confusion Matrix

Confusion Matrix

The confusion matrix is used in cybersecurity to describe the performance of a detection system that provides a Boolean outcome (normal or malicious). While there are only two valid results from a Boolean test, there are four possible outcomes based on whether the test was correct in its identification.

In a confusion matrix, positive and negative are used to describe the outcome of the test, for example if an AV was scanning a file to see if it was malware and detected that it was malware the result would be positive.

Positive/Negative tests like those in cybersecurity detection systems and many other fields like healthcare and AI are not always correct. To describe if the outcome of the test is actually correct, true and false are used. Using the previous example of an AV scanning a file, if the AV scanned the file and detected that it was malware, however, upon further inspection by a skilled analyst it was determined to be a normal file and not malware, then we could describe the result as a false positive.

Below is an example of a confusion matrix.

The four states of the matrix are:

True Positive: the test correctly detected the presence of what it was testing for, in cybersecurity this means it properly detected malicious activity.

True Negative: the test correctly detected the absence of what it was testing for, in cybersecurity this means it properly detected normal activity.

False Positive: the test incorrectly detected the presence of what it was testing for, in cybersecurity this means that it classified normal (sometimes referred to as benign) activity as malicious. False positives can be a nuisance and also use up resources for security teams to triage. Detection tools are frequently tuned to reduce the amount of false positives they produce.

False Negative: the test incorrectly detected the absence of what it was testing for, in cybersecurity this means that it classified malicious activity as normal (benign). False negatives are much more dangerous than false positives as they mean that malicious activity went undetected by the system. When tuning detection systems to reduce false positives it is important to not over tune and introduce false negatives.