2.2.24 AppId Preprocessor

With increasingly complex networks and growing network traffic, network administrators require application awareness in managing networks. An administrator may allow only applications that are business relevant, low bandwidth and/or deal with certain subject matter.

AppId preprocessor adds application level view to manage networks. It does this by adding the following features

Network control: The preprocessor provides simplified single point application awareness by making a set of application identifiers (AppId) available to Snort Rule writers.
Network usage awareness: the preprocessor outputs statistics to show network bandwidth used by each application seen on network. Administrators can monitor bandwidth usage and may decide to block applications that are wasteful.
Custom applications: The preprocessor enables administrators to create their own application detectors to detect new applications. The detectors are written in Lua and interface with Snort using a well-defined C-Lua API.

2.2.24.1 Dependency Requirements

For proper functioning of the preprocessor:

Stream session tracking must be enabled, i.e. stream5. TCP or UDP must be enabled in stream5. The preprocessor requires a session tracker to keep its data.
Protocol Aware Flushing (PAF) must be enabled.
IP defragmentation should be enabled, i.e. the frag3 preprocessor should be enabled and configured.
HTTP preprocessor must be enabled and configured. The processor does not require any AppId specific configuration. The preprocessor provides parsed HTTP headers for application determination. Without HTTP preprocessor, AppId preprocessor will identify only non-HTTP applications.
LuaJIT version 2.0.2 must be installed on host where snort is being compiled and run. Newer versions of LuaJIT are not tested for compatibility.

2.2.24.2 Preprocessor Configuration

AppId dynamic preprocessor is enabled by default(from snort-2.9.12). The preprocessor can be disabled during build time by including the following option in ./configure:

--disable-open-appid

The configuration name is ;SPMquot;appid;SPMquot;:

The preprocessor name is appid.

verbatim447#

Option syntax

<#21535#><#21498#><#21498#><#21535#>

Option	Argument	Required	Default
app_detector_dir	;SPMlt;directory;SPMgt;	NO	app_detector_dir { /usr/local/etc/appid }
app_stats_filename	;SPMlt;filename;SPMgt;	NO	NULL
app_stats_period	;SPMlt;time in seconds;SPMgt;	NO	300 seconds
app_stats_rollover_size	;SPMlt;disk size in bytes;SPMgt;	NO	20 MB
app_stats_rollover_time	;SPMlt;time in seconds;SPMgt;	NO	1 day
memcap	;SPMlt;memory limit bytes;SPMgt;	NO	256 MB
debug	;SPMlt;;SPMquot;yes;SPMquot;;SPMgt;	NO	disabled
dump_ports	No	NO	disabled

Option explanations

<#21616#><#21536#><#21536#><#21616#>

app_detector_dir

<#21571#><#21568#><#21537#><#21537#><#21568#><#21571#>	specifies base path where Cisco provided detectors and application configuration files are installed by ODP (Open Detector Package) package. The package contains Lua detectors and some application metadata. Customer written detectors are stored in subdirectory ;SPMquot;custom;SPMquot; under the same base path.
<#21572#><#21569#><#21538#><#21538#><#21569#><#21572#>	Syntax verbatim448#
<#21573#><#21570#><#21539#><#21539#><#21570#><#21573#>	Examples verbatim449#

<#21617#><#21540#><#21540#><#21617#>

app_stats_filename

<#21577#><#21574#><#21541#><#21541#><#21574#><#21577#>	name of file. If this configuration is missing, application stats are disabled.
<#21578#><#21575#><#21542#><#21542#><#21575#><#21578#>	Syntax verbatim450#
<#21579#><#21576#><#21543#><#21543#><#21576#><#21579#>	Examples verbatim451#

<#21618#><#21544#><#21544#><#21618#>

app_stats_period

<#21583#><#21580#><#21545#><#21545#><#21580#><#21583#>	bucket size in seconds. Default 5 minutes.
<#21584#><#21581#><#21546#><#21546#><#21581#><#21584#>	Syntax verbatim452#
<#21585#><#21582#><#21547#><#21547#><#21582#><#21585#>	Examples verbatim453#

<#21619#><#21548#><#21548#><#21619#>

app_stats_rollover_size

<#21589#><#21586#><#21549#><#21549#><#21586#><#21589#>	file size which will cause file rollover. Default 20 MB.
<#21590#><#21587#><#21550#><#21550#><#21587#><#21590#>	Syntax verbatim454#
<#21591#><#21588#><#21551#><#21551#><#21588#><#21591#>	Examples verbatim455#

<#21620#><#21552#><#21552#><#21620#>

app_stats_rollover_time ;SPMgt;

<#21595#><#21592#><#21553#><#21553#><#21592#><#21595#>	time since file creation which will cause rollover. Default 1 day.
<#21596#><#21593#><#21554#><#21554#><#21593#><#21596#>	Syntax verbatim456#
<#21597#><#21594#><#21555#><#21555#><#21594#><#21597#>	Examples verbatim457#

<#21621#><#21556#><#21556#><#21621#>

memcap ;SPMgt;

<#21601#><#21598#><#21557#><#21557#><#21598#><#21601#>	upper bound for memory used by AppId internal structures. Default 32MB.
<#21602#><#21599#><#21558#><#21558#><#21599#><#21602#>	Syntax verbatim458#
<#21603#><#21600#><#21559#><#21559#><#21600#><#21603#>	Examples verbatim459#

<#21622#><#21560#><#21560#><#21622#>

dump_ports ;SPMgt;

<#21607#><#21604#><#21561#><#21561#><#21604#><#21607#>	prints port only detectors and information on active detectors. Used for troubleshooting.
<#21608#><#21605#><#21562#><#21562#><#21605#><#21608#>	Syntax verbatim460#
<#21609#><#21606#><#21563#><#21563#><#21606#><#21609#>	Examples verbatim461#

<#21623#><#21564#><#21564#><#21623#>

debug

<#21613#><#21610#><#21565#><#21565#><#21610#><#21613#>	Used in some old detectors for debugging.
<#21614#><#21611#><#21566#><#21566#><#21611#><#21614#>	Syntax verbatim462#
<#21615#><#21612#><#21567#><#21567#><#21612#><#21615#>	Examples verbatim463#

Default configuration

verbatim464#

2.2.24.3 Rule Options

The AppId preprocessor adds 1 new rule option as follows:

<#21626#><#21625#><#21625#><#21626#>
verbatim465#

The preprocessor must be enabled for this rule option to work.

appid

<#21628#><#21627#><#21627#><#21628#> The rule option allows users to customize rules to specific application in a simple manner. The option can take up to 10 application names separated by spaces, tabs, or commas. Application names in rules are the names you will see in last column in appMapping.data file. A rule is considered a match if one of the appId in a rule match an appId in a session.
For client side packets, payloadAppId in a session is matched with all AppIds in a rule. Thereafter miscAppId, clientAppId and serviceAppId are matched. Since Alert Events contain one AppId, only the first match is reported. If rule without appId option matches, then the most specific appId (in order of payload, misc, client, server) is reported.
The same logic is followed for server side packets with one exception. Order of matching is changed to make serviceAppId higher then clientAppId.

Syntax

verbatim466#

Examples

verbatim467#

2.2.24.4 Application Rule Events

A new event type is defined for logging application name in Snort Alerts in unified2 format only. These events contain only one application name. The Events can be enabled for unified2 output using 'appid_event_types keyword.

For example, the following configuration will log alert in my.alert file with application name.

verbatim468#

u2spewfoo, u2openappid, u2streamer tools can be used to print alerts in new format. Each event will display additional application name at the end of the event.

Examples

verbatim469#

2.2.24.5 Application Usage Statistics

AppId preprocessor prints application network usage periodically in snort log directory in unified2 format. File name, time interval for statistic and file rollover are controlled by appId preprocessor configuration. u2spewfoo, u2openappid, u2streamer tools can be used to print contents of these files. An example output from u2openappid tools is as follows:

verbatim470#

2.2.24.6 Open Detector Package (ODP) Installation

Application detectors from Snort team will be delivered in a separate package called Open Detector Package. ODP is a package that contains the following artifacts:

Application detectors in Lua language.
Port detectors, which are port only application detectors, in meta-data in YAML format.
appMapping.data file containing application metadata. This file should not be modified. The first column contains application identifier and second column contains application name. Other columns contain internal information.
Lua library files DetectorCommon.lua, flowTrackerModule.lua and hostServiceTrackerModule.lua

User can install ODP package in any directory of its choosing and configure this directory in app_detector_dir option in appId preprocessor configuration. Installing ODP will not modify any subdirectory named custom, where user-created detectors are located.

When installed, ODP will create following sub-directories:

verbatim471#

2.2.24.7 User Created Application Detectors

Users can create new applications by coding detectors in Lua language. Users can also copy Snort team provided detectors into custom subdirectory and customize the detector. A document will be posted on Snort Website with details on API usage.

Users must organize their Lua detectors and libraries by creating the following directory structure, under ODP installation directory.

verbatim472#