IDS Triggers

The purpose of an IDS system is to alert the appropriate personnel once an intrusion is detected. Burglar alarm systems trigger an alarm based on a motion detector, a broken window, or an opened door. IDS also have two types of triggering mechanisms.

IDSs are merely packet sniffers with the capability to do some basic analyzing. These systems don’t inherently know the difference between normal traffic and malicious traffic. For the IDS to recognize malicious traffic or activity, you must first “teach” the IDS what constitutes an attack. The IDS then compares actual network traffic or computer activity with what has been defined as malicious and, if a match is made, an alarm is triggered.

Not all systems use the same method of triggering an alarm and each type of triggering system has its own strengths and weaknesses. To choose the appropriate IDS system for your environment, you must first understand each type of triggering system, as well as the benefits and drawbacks of each. Modern IDSs use two types of triggering mechanisms:

Anomaly Detection (Profile based)
Misuse Detection (Signature based)

STUDY TIP
The terms “anomaly detection” and “profile based,” and the terms “misuse detection” and “signature based” are used interchangeably.

Misuse Detection

Misuse detection is commonly referred to as signature-based detection. Misuse detection requires the use of signature files that identify intrusive activity. The signature files used in misuse detection are analogous to signature files commonly used in virus-scanning software to identify viruses on computer systems.

A signature file is a set of rules used to identify common intrusive activity. The research of highly skilled engineers discovered attacks, patterns, and methods to write signature files to identify them. As more attack methods and exploits are discovered, the IDS vendor will provide updates to signature files, just as virus-scanning vendors provide updates to their own software. Once the signature files are updated, the IDS system will begin analyzing all activity searching for a match. If activity or traffic is found that matches the signature, an alarm is triggered. IDS systems typically come with a database of signatures for common attacks and exploits.

Signature-Based Benefits

Signature files are created based on known attack methods and activity, so if a match is made, a high probability exists that an attack is underway. Misuse detection, unlike anomaly detection, will have fewer false positive reports because matches are based on a known intrusive activity, not just unusual traffic. Signature-based detection doesn’t monitor traffic patterns or look for anomalies. Instead, it monitors activity simply looking for a match to any configured signature.

Because misuse detection relies on signatures—not traffic patterns—the IDS system can be configured and can begin protecting the network immediately. The signatures contained in the signature database contain the known intrusive activity and a description of the signature. Each signature in the database can be viewed, enabled, or disabled. Different levels of alarms, as well as different preventative actions, can be configured for individual signatures, giving security administrators granular control of their IDS systems.

Misuse detection is easier to understand and configure than anomaly-based systems. Signature files can be viewed so administrators can understand what actions must be matched for an alarm to be generated. Security administrators can enable signatures, and then perform a test on the network and view the resulting alarm that’s generated. Because misuse detection is easier to understand, implement, and test, administrators have a higher degree of control and more confidence in their IDSs.

Misuse Detection Drawbacks

While there are many benefits to misuse detection triggers, some drawbacks exist to this form of intrusion detection. Misuse detection is simpler to configure and understand, but this simplicity comes at a cost of lost functionality and administrative overhead. Misuse detection has the following drawbacks:

Inability to detect new or unknown attacks

Inability to detect variations of known attacks

Signature database administration

Sensors must maintain state information

Inability to Detect New Attacks

Misuse detection accomplishes its mission by comparing computer network activity to known intrusive activity defined in the signature database. If an attack is instigated that doesn’t match a known intrusive activity, the sensors typically won’t generate and alert. The IDS system using misuse detection must be aware of the activity of an attack before it can identify that attack. New attacks that haven’t previously been used or discovered normally won’t be detected by a misuse detection IDS. Signature files are created to be as flexible as possible and, in some cases, a previously unknown attack will be detected by the IDS. Even though an exact match might not occur, the IDS could detect a previously unknown attack that uses a similar method of attack or intrusion activity. IDS systems must be updated with the latest signatures to be effective. Even if an IDS system has been updated with the latest signature database, it’s possible that new types of attacks will won’t generate an alert.

Inability to Detect Attack Variations

Intruders also have access to the signature files and IDS systems used by security administrators. Because this information is available to everyone, hackers can use this information to test and alter their attack. By altering the attack in some minor way, an intruder might be able to perform an intrusion without being detected (false positive). Signature files are static—they don’t adapt as some anomaly-based systems do. If an attack doesn’t match a signature file, the sensors won’t generate an alarm. Because the signature files are included with the IDS systems, intruders are aware of what will and what won’t generate an alarm. Armed with this knowledge, an intruder can customize their attacks to defeat the IDS.

Signature File Administration

The responsibility of the security administrator is to ensure the database file is current. The security administrator must also configure the probes with the signatures they want the probes to use, as well as the severity level of each matched signature. Keeping the signature database current with constant updates and applying those updates to all sensors can be a difficult and time-consuming task.

State Information

Just like firewalls, sensors must maintain state data. Sensors simply match activity or traffic to preconfigured signatures. In some cases, the amount of data to match a signature could be spread across multiple packets and a variable amount of time. Additionally, hackers might fragment their packets before sending them across the network in an attempt to prevent the packets from being analyzed. Sensors must record this information and recompile it to match it against any signatures. The maximum amount of time a probe must record the state—from the first packet until a match is made—is called the event horizon.

The event horizon can range from minutes for some signatures to days or weeks for others. For example, a security administrator might want to be alerted if anyone performs a port scan against their network, but might not want to be notified if only one or two ports are scanned. Some patient hackers might only scan two ports every four hours for a month. Within a couple of weeks, this hacker could have found all the services available on the network, which means it’s important for the sensor to remember what ports have been scanned and by whom. But how long should the sensor remember this information? The amount of time the sensor is configured to keep state information for a given signature is called the event horizon for that signature. Some signature files, such as those that detect reconnaissance attacks, have an event horizon that spans weeks, while other attacks have an event horizon that spans the time the user is logged into the network. The event horizon is a variable contained in the signature files.

Sensors have a limited amount of storage available. Most sensors keep this state information recorded in memory for fast retrieval, but the storage space is limited. Hackers might attempt to disable your IDS systems by sending them so much information that the sensor(s) run out of resources and can no longer record state data. Once the sensor has reached its memory limits, it no longer analyzes any additional information. This would be an example of a DoS attack against the IDS itself.

Anomaly Detection

Anomaly- or profile-based triggering analyzes computer activity and network traffic looking for anomalies. If an anomaly is found, an alarm is triggered. An anomaly is any deviation or departure from the normal or common order, form, or rule. Because this type of detection is looking for any activity or traffic that isn’t normal, the security administrator must first define what is normal activity or traffic. Security administrators can define normal activity by creating user group profiles.

A user group profile represents a baseline of normal computer activity and network traffic for a given user group. User groups are defined by the security engineer and can be used to represent users or computers with common job functions, or users and computers within the same departments. Typically, user groups should be divided according to the activities and network resources each group uses. A web server farm could have its own profile based on web traffic, while mail servers could have another profile based on SMTP. You wouldn’t expect telnet traffic destined for your web servers or SSH traffic destined for your mail servers. For these reasons, you should have different profiles for each type of service offered on your network.

Various techniques are used for building user profiles and some IDS can be configured to build their own profiles. The typical methods used to build user group profiles are statistical sampling, rule-based, and neural networks. Each profile is used as a definition for normal user and network activity. If a user deviates too far from their defined profile, the IDS system will generate an alert.

Building Profiles Using Statistical Sampling

With statistical sampling, alarms are generated based on deviations from a normal state. Normal state is defined by sampling normal activity and traffic for a given period of time. Normalcy is based on the average or median of all activity or traffic. Deviations are measured by calculating the standard deviations.

Standard deviations are simply the amount of activity or data that matches, or doesn’t match, a sample, and they measure the deviation from a normal state. For example, if 60 percent of all your data falls within one standard deviation and 35 percent of all your traffic falls within two standard deviations, then only 5 percent of your data falls within three or more deviations. Using this method, the IDS system can detect how abnormal specific activity or traffic is.

Building Profiles Using the Rule-Based Approach

Rule-based profile building is accomplished by defining rules to define normal user behavior. You must create rules that define normal user activity, and these are created by sampling computer and network activity for a given amount of time. Once the data set has been collected, rules can be created to define normal activity. The rules are models representing normal computer and network activity. Any traffic that doesn’t match the rules is considered abnormal and generates alarms.

Building Profiles Using Neural Networks

Just as a psychologist can use inkblots to discover how you relate information in your mind, neural networks can use matrix(s) to relate normal activity on your network or computer systems. Neural networks are built or trained by presenting the IDS system with large amounts of data and rules about data relationships. Neural networks attempt to use artificial intelligence to build matrixes based on the given information. Relationships between these data inputs are used to build a matrix modeled after the biological neurons, such as those found in the human brain. Once the neural network is established, it can be used as a model or definition of normal activity. Any activity that doesn’t map correctly to the matrix or neural network is considered abnormal and generates an alarm.

Anomaly Detection Benefits

Using anomaly detection as the triggering mechanism has many benefits. With anomaly-based detection, the intruder never knows what might or might not generate an alarm, because he or she doesn’t have access to the profiles used to detect an attack. User group profiles are much like a dynamic signature database that changes as your network changes. With signature-based detection, the intruder can test on their own IDS system what will generate an alert. Signature files are provided with a purchased IDS system, so a hacker could use their own IDS system to perform testing. Once the hacker understands what will generate an alert, the attacker can then customize his or her attack methodology and tools to defeat the IDS. Because anomaly detection doesn’t use a preconfigured signature database, intruders can’t be sure what activity will generate an alert.

Anomaly detection can quickly detect an internal attack using a compromised user account. If a user account belonging to an administrative assistant is being used to perform system administration, the IDS system using anomaly detection will generate an alarm as long as that account isn’t normally used for system administration.

The biggest advantage to anomaly- or profile-based detection is it isn’t based on a set of preconfigured signatures or known attacks. Profiles can be dynamic and can use artificial intelligence to determine what normal activity is. Because profile-based detection isn’t based on known signatures, it’s better suited to detect previously unknown or unpublished attacks as long as the attack deviates from normal activity (profile). Profile-based detection can be used to detect new attack methods, which signature-based detection won’t detect.

Anomaly-Based Drawbacks

While many benefits exist to using anomaly- or profile-based detection, many drawbacks also exist with this method of intrusion detection. Many of the drawbacks of anomaly detection have to do with the creation of user group profiles, as well as the quality of these profiles. Drawbacks with anomaly detection include the following:

High initial prep time
No protection during initial training time
Constant update of profiles as users’ habits change
Defining normal behavior can be difficult
False positives, false negatives
Hard to understand

Difficulties with User Group Profiles

Anomaly-based detection relies on the use of user group profiles. The IDS is only as good as the profiles being used to define what normal activity is. Profiles are a baseline of normal activity, created by sampling network traffic and activity over a set period of time. While creating the user profiles, it’s vital no intrusive activity occurs on the network and all systems are free of backdoors or Trojan horses. If intrusive activity occurs on the network during the initial training time, the intrusive activity will be included in the profile and, therefore, the activity will seen as normal activity.

The initial training time should consist of enough data to truly represent normal activity and traffic. The training time could range from days to weeks or even months. Defining normal activity can be a daunting task. What normal activity is in one month could or could not be normal the next month. Users aren’t compelled to use the same applications and perform the same functions without deviation. Defining normal activity is even more challenging in environments where users’ jobs or responsibilities change often. As users’ habits change, the profiles describing normal activity for those users must also change. Additionally, while the system is being “trained,” the IDS provides no protection, so it’s vital no intrusive activity occurs during this training period.

Creating user profiles can be difficult for advanced users or diverse groups of users. If a user group contains a vast amount of users that all perform different functions, then it’s difficult to differentiate normal activity from intrusive activity. System administrators, network engineers, and Unix administrators all generate activity that wouldn’t be permissible for other types of users. For this reason, segregating different users according to resources and applications each group uses is important.

Some systems can be configured to update the profile constantly, based on traffic and activity as it’s being measured. Statistical sampling, discussed in the previous section, constantly monitors the network and uses the data collected to update the profile. The benefit is this: the profile is always kept current with user activity changes, however, a hacker can use this feature to manipulate the IDS. A hacker could slowly begin performing intrusive activity over a long period of time. Starting with small amounts of activity, and slowly increasing the amount of traffic and activity, the hacker can train the IDS to ignore the intrusive activity. The IDS system will slowly begin to consider the intrusive activity as normal, which will result in false negatives.

False Reporting

A false negative is a situation when intrusive activity is on the network or systems, yet the intrusive activity goes undetected by the IDS system. If the activity is considered normal, then an alert won’t be generated. Anomaly detection is only as good as the profile used to detect intrusive activity. Signature-based detection systems tend to have more false negatives than anomaly-based systems because they aren’t suited to discovering new methods of attack.

A false positive occurs when the IDS system generates an alarm for activity that isn’t considered intrusive. Car alarms, for example, commonly report false positives. IDS systems should be continually tuned to strike a balance between false negatives and false positives. Too many false positives and the IDS system will soon be ignored, much like car alarms are today. Even worse, too many false negatives could result in a great deal of damage. Anomaly-based systems tend to have more false positives because they’re looking for anything out of the ordinary.

Difficult to Understand

The last major drawback to anomaly-based detection is its complexity. Statistical sampling, rule-based, and neural networks are all profile- building strategies that are hard to explain and understand. Signature-based detection is much simpler to understand: if a given activity matches a signature, then an alarm is sent, along with a notification of which signature was matched. Anomaly detection requires a more in-depth understanding and it’s harder to discover why the system generated an alert. Because of its complexity, many security administrators have a difficult time understanding the system and are uncomfortable with the IDS. This lack of understanding might also cause lack of confidence in their IDS.