An industrial control system (ICS) is an automated network of devices that make up a complex industrial process. For example, a large-scale electrical grid may contain thousands of instruments, sensors, and controls that transfer and distribute power, along with computing systems that capture data transmitted across these devices. Monitoring the ICS network for new device connections, device performance, or adversarial attacks requires sophisticated data analysis.

A team of LLNL researchers—Brian Kelley from the Global Security Computing Applications Division and Indrasis Chakraborty, Brian Gallagher, and Dan Merl from the Center for Applied Scientific Computing (CASC)—recently published a patent (pending) for a novel machine learning (ML) framework that discovers and predicts key data about networked devices. Keeping a data-driven eye on an ICS helps ensure its reliability and security.

Contextual Connections

Web users are accustomed to seeing contextual suggestions when typing a word into a search engine. A natural language processing algorithm known as Word2Vec makes this possible by learning word associations from previous searches and common usage.

Figure 1. Conceptual workflow of the team’s ML framework, which can accommodate any number of connected devices in an ICS.

Similarly, LLNL’s ML framework uses context to infer meaning between ICS data values. “When one device in the network begins communicating data, a device connected to it must also begin communicating. They are contextually connected,” states Chakraborty. The framework’s IP2Vec (internet protocol) and DNP2Vec (distributed network protocol) algorithms train on, and learn from, device data related to these communication protocols (e.g., endpoint host IP address, target IP address, device manufacturer, device model number, number of bytes being transferred).

First, the workflow passively captures packets of data transmitted by the network’s devices and extracts values that help it classify devices as ICS or non-ICS (i.e., those related to IT [information technology] functions instead of those monitoring or controlling a physical process). Then devices are further identified by their manufacturers and model numbers. The resulting characterization of device features helps ICS operators manage or audit a network’s dynamics without disturbing it.

A crucial step in these classification algorithms is the transformation of device data into latent representations, also called latent vectors. These vectors are composed of machine-readable ones and zeros, and are labeled to assist the classification process. They also provide a predictive capability for ICS network dynamics and properties.

Kelley explains, “By training the ML model on a variety of datasets across different ICS types, such as data from large utility companies, we want it to learn about the characteristics of those systems. Then when our model is presented with data from a system it hasn’t seen before, it could recognize relevant devices and tell us about the devices’ provenance or metadata.”

Data-Rich Systems

The team has tested the ML framework by training it on LLNL datasets and comparing the outputs to ground truth. “Any physical device that stores and transmits data has a communication protocol, and the Lab has a lot of networked devices that communicate with each other,” says Chakraborty. Kelley adds, “Our model’s predictions are, so far, in line with ground truth data from several networks.”

Additionally, a user can tailor the framework to their communication protocols and amount of data, as well as integrate it into an existing ICS assessment process. The framework is flexible enough for a variety of data types and computing elements including laptops, tablets, touch screens, and so on.

The pending patent enables the technology to be easily adopted in many use cases: public utilities, building systems, sensors that send and receive signal data, and more. “We’re seeing a lot of interest in applying this method in practice to new types of networks,” notes Kelley, who along with Gallagher came up with the idea for this technology, then led a subsequent Laboratory Directed Research and Development project to expand it.

The team is particularly keen to apply their ML framework to cyber security systems, where implementation can make a difference in both detection and prevention of suspicious activity. Gathering information about the network’s status and continuously re-evaluating the data over time can lead to detection of outlier behavior in new devices—a possible indication of unusual activity, Kelley points out. “There aren’t enough cyber security analysts to secure and defend all of our nation’s critical infrastructure. These types of automation techniques can help,” he says. “Cyber security will only become more important in the coming years.”

Figure 2. In this example implementation, IP2Vec has been extended to include DNP version 3 features (e.g., network traffic statistics, data fields) in the output layer. Latent vectors (comprising weights w1–w4) convey device data in machine-readable representations.

—Holly Auten