Smoke Detector: Cross-Product Intrusion Detection With Weak Indicators

The central task of a Security Incident and Event Manager (SIEM) or Managed Security Service Provider (MSSP) is to detect security incidents on the basis of tens of thousands of event types coming from many kinds of security products. We present Smoke Detector, which processes trillions of security events with the Random Walk with Restart (RWR) algorithm, inferring high order relationships between known security incidents and imperfect secondary security events (smoke) to find undiscovered security incidents (fire). Smoke Detector successfully reproduces 96.4% of the critical security incidents identified by a mature MSSP on the basis of rules involving primary-indicators at a 1% False Positive rate. We found that enough of these "False Positives" correspond to previously unidentified security incidents that Smoke Detector is capable of raising the MSSP's critical incident count by 19%. ur algorithm also advances the state of the art in the following ways. (1) It provides a robust approach for leveraging Big Data sensor nets to increase adversarial resistance of protected networks. (2) Our event-scoring techniques enable efficient discovery of primary indicators of compromise. (3) By providing intuition and tuning capabilities into Smoke Detector's discovered security incidents, aiding incident display and response.