AlienVault Unified Security Managementв„ў Solution
Complete. Simple. Affordable
How to create a data source plugin
CopyrightВ© 2014 AlienVault. All rights reserved.
AlienVaultв„ў, В AlienVault В Unified В Security В Managementв„ў, В AlienVault В USMв„ў, В AlienVault В Open В Threat В Exchangeв„ў, В AlienVault В OTXв„ў, В Open В Threat В Exchangeв„ў, В AlienVault В OTX В Reputation В Monitorв„ў, В AlienVault В OTX В Reputation В Monitor В Alertв„ў, В AlienVault В OSSIMв„ў В and В OSSIMв„ў В are В trademarks В or В service В marks В of В AlienVault.
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
CONTENTS
1.
INTRODUCTION ..................................................................................................... 4
2.
TYPES OF DATA SOURCE PLUGINS .................................................................. 4
2.1. Detector Plugins ....................................................................................................... 5
2.2. Monitor Plugins ...................................................................................................... 23
3.
HOW TO CREATE A CUSTOM DATA SOURCE PLUGIN .................................. 23
3.1. Exchange Web SMTP server logs .......................................................................... 24
3.2. Creation of the plugin configuration file exchangews.cfg ........................................ 24
3.3. Create the database file exchangews.sql ............................................................... 26
3.4. Activate data source plugins................................................................................... 26
3.5. Files .local .............................................................................................................. 32
5.
HOW TO USE CUSTOM FUNCTION IN DATA SOURCE PLUGINS .................. 33
APPENDIX A - RECOMMENDATIONS BEFORE CREATING A NEW PLUGIN ......... 35
APPENDIX B - LIST OF DATA SOURCE PLUGINS ................................................... 37
B.1. Database Plugins ................................................................................................... 37
B.2. Log Plugins ............................................................................................................ 37
B.3. Monitor Plugins ...................................................................................................... 38
B.4. Remote Plugins ...................................................................................................... 39
B.5. SDEE Plugins ........................................................................................................ 39
B.6. WMI Plugins ........................................................................................................... 39
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 3 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
1.
INTRODUCTION
The objective of this document is to explain how to create plugins supported by AlienVault
USM.
A plugin is a software component that adds a specific feature to AlienVault USM. Plugins are
used to improve the collection capabilities of the AlienVault Sensors and to indicate to the
system, how to understand and to collect events generated by each application and device.
Sensors receive events from remote hosts using the Syslog, WMI or any other protocols. The
sensors use the Collection Plugins (also called Data Source connectors) in order to support the
maximum possible number of applications and devices.
For any system that consumes logs, it is needed a parser to read those logs and extract
information from them into standard information fields (username, IP addresses, etc.).
AlienVault does this via Agent plugin that defines how to collect events from the application or
device as well as how events should be normalized before sending them to the AlienVault
USM central Server. Log Normalization is essentially breaking down a log message into
common fields.
It is necessary to enable a plugin in order to indicate to the system that must collect events
generated by an application or device. Plugins may be pre-configured by AlienVault or defined
by users.
AlienVault plugins are text configuration files and have the extension *.cfg. These files are
located in /etc/ossim/agent/plugins in the  Sensor’s  file  system.
2.
TYPES OF DATA SOURCE PLUGINS
There are 2 types: monitor and detector:
Detector. В These В plugins В receive В logs, В information В and В extract В events В from В them. В They В process В text В log В information В from В log В files В created В by В RSyslog В collection В system;Нѕ В and В from В log В data В retrieved В from В remote В systems В via В one В of В the В remote В collection В protocols В such В as В SDEE В and В SFTP. В These В plugins В can В be:
Database. В They В monitor В a В file В in В external В databases.
Logs. В They В monitor В a В file, В usually В receiving В data В through В syslog.
Remote В Logs. В They В monitor В a В file В in В a В remote В appliance.
SDEE В (Security В Device В Event В Exchange). В CISCO В device В logs.
WMI В (Windows В Management В Instrumentation). В They В collect В remotely В Microsoft В Windows В events В and В data В in В an В agent-less В way.
Monitor. В These В plugins В request В information В from В systems, В checking В the В status В of В the В things В they В monitored В at В the В time В of В the В request. В They В generate В text В logs В that В are В fed В into В the В DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 4 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
syslog В like В normal В logs В and В they В are В often В used В to В correlate В log В events В into В alarms В by В matching В events В against В the В current В status В of В systems.
2.1.
Detector Plugins
DATABASE В PLUGINS
2.1.1.
It is easier to understand how this type of plugin works by means of an example:
;; PCI Trace
[DEFAULT]
plugin_id=1698
[config]
type=detector
enable=yes
source=database
source_type=mssql
source_ip=
source_port=3306
user=
password=
db=
sleep=60
process=
start=no
stop=no
[start_query]
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 5 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
query="select TOP 1 pci.RowNumber from pcitrace as pci ORDER BY
pci.RowNumber desc"
regexp=
userdata1={$3}
log= Virus {$3} detected on {$2}, path: {$4} {$5}
[query]
query="select pci.RowNumber, pci.EventClass, pci.TextData,
pci.ApplicationName, pci.NTUserName, pci.LoginName, pci.CPU, pci.Reads
from pcitrace as pci ORDER BY pci.RowNumber"
regexp=
ref=0
plugin_sid=1
username={$5}
userdata1={$2}
userdata2={$3}
userdata3={$4}
userdata4={$6}
userdata5={$7}
userdata6={$8}
log={$1},{$2},{$3},{$4},{$5},{$6},{$7},{$8}
The fields related to database fields are an example for mssql. In case of having mysql, it must
be indicated.
Indicate the point to start to capture. It must be a query to obtain the last event identified by a
sequence number. In this case will be:
select TOP 1 pci.RowNumber from pcitrace as pci ORDER BY pci.RowNumber
desc
The  last  “RowNumer”  is  obtained  from  a  table.
A  query  for  getting  all  values  is  needed.  The  same  field  used  in  the  “start  query”  must  be  selected as first element.
query="select pci.RowNumber, pci.EventClass, pci.TextData,
pci.ApplicationName, pci.NTUserName, pci.LoginName, pci.CPU, pci.Reads
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 6 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
from pcitrace as pci ORDER BY pci.RowNumber"
Regexp field must be empty:
regexp=
ref=0
plugin_sid=1
$2 is the second element in the query. In this example is the value of pci.EventClass
username={$5}
userdata1={$2}
userdata2={$3}
userdata3={$4}
userdata4={$6}
userdata5={$7}
userdata6={$8}
log={$1},{$2},{$3},{$4},{$5},{$6},{$7},{$8}
2.1.2.
LOGS
This is an example of a log plugin:
[DEFAULT]
plugin_id=1563
[config]
enable=yes
type=detector
source=log
location=/var/log/optenet.log
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 7 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
create_file=false
process=
start=no
; launch plugin process when agent starts
stop=no
; shutdown plugin process when agent stops
startup=
shutdown=
[optenet - spam detected]
regexp="^(?P<domain>\S+)\t(?P<SRC_IP>\IPV4)\t*\[\S+\]*\t\[(?P<src_mail>\S+)\]
\t\[(?P<dst_mail>\S+)\].*"
event_type=event
plugin_sid=1
device={resolv($SRC_IP)}
src_ip={resolv($SRC_IP)}
userdata1={$domain}
userdata2={$src_mail}
userdata3={$dst_mail}
Plugins extract events (SIDs) from logs by matching each line in the log according to a
regular expression, and then normalizing out data fields from the text. So when the
following log message arrives:
Feb 8 10:09:06 golgotha sshd[24472]: Failed password for dgil from
192.168.6.69 port 33992 ssh2
It matches the following SID from the SSH plugin.
[01 - Failed password]
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 8 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
The information in a log entry to be normalized into field is specified in the regular
expression1:
regexp="(?P<date>\w{3}\s+\d{1,2}\s\d\d:\d\d:\d\d)\s+(?P<dst>\S+).*ssh.*Failed
(?P<type>publickey|password|none) for\s+(?P<info>invalid
user)?\s*(?P<user>\S+)\s.*from\s+(?P<src>\S+)\s.*port\s+(?P<sport>\d{1,5})"
And these values are normalized out of it:
Date = Feb 8 10:09:06
src_ip =192.168.6.69
Username = dgil
The level of information that can be extracted from a log source is dependent on the level
of detail in the plugin. The more SIDs defined, the greater the ability to extract meaning
from processed logs.
REMOTE В LOGS
2.1.3.
This is an example of a remote log plugin:
# Alienvault plugin
# Author: Alienvault Team at devel@alienvault.com
# Plugin ssh-remote id:4003 version: 0.0.1
# Last modification: 2013-06-05 11:43
#
# Accepted products:
# openbsd - openssh 5.4
# openbsd - openssh 5.5
# openbsd - openssh 5.6
1
The bolded fields in the regexp indicate that the matching text will be mapped to information fields during
normalization.
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 9 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
# openbsd - openssh 5.7
# openbsd - openssh 5.8
# openbsd - openssh 5.8p2
# openbsd - openssh 5.9
# Description:
#
# Ssh (Secure Shell) is a program for logging into a remote machine
# and for executing commands on a remote machine.
# URL: http://www.openssh.com
#
#
$Id: ssh.cfg,v 1.12 2010/03/23 16:42:18 juanmals Exp $
#
#
[DEFAULT]
plugin_id=4003
dst_ip=\_CFG(plugin-defaults,sensor)
dst_port=22
[config]
type=detector
enable=yes
source=remote-log
location=/var/log/auth.log
create_file=false
process=sshd
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 10 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
start=no
stop=no
startup=/etc/init.d/ssh start
shutdown=/etc/init.d/ssh stop
host=
user=root
passwd=
readAll=false
[ssh - Failed password]
event_type=event
regexp="(\SYSLOG_DATE)\s+(?P<sensor>[^\s]*).*?ssh.*?Failed password for
(?P<user>\S+)\s+from\s+.*?(?P<src>\IPV4).*?port\s+(?P<sport>\PORT)"
plugin_sid=1
device={resolv($sensor)}
date={normalize_date($1)}
src_ip={$src}
dst_ip={resolv($sensor)}
src_port={$sport}
username={$user}
Entries marked in bold must always appear because they are used for connecting to remote
host:
source=remote-log
host=
user=root
passwd=
readAll=false
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 11 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
SDEE В (SECURITY В DEVICE В EVENT В EXCHANGE)
2.1.4.
SDEE 2 is a standard that specifies the format of messages and protocol used to
communicate events generated by security devices. This protocol is used in the Cisco
Systems IPS Sensor 5.0. AlienVault support this type of logs collection. AlienVault USM
captures events from:
Cisco В Network В Prevention В Systems В (IPS)
Cisco В Network В Detection В Systems В (IPS)
Cisco В Switch В IDS
Cisco В IOS В routers В with В the В Inline В Intrusion В Prevention В System В (IPS) В functions
Cisco В IDS В modules В for В routers
Cisco В PIX В Firewalls
Cisco В Catalyst В 6500 В Series В firewall В service В modules В (FWSMs)
Cisco В Management В Center В for В Cisco В security В agents
CiscoWorks В Monitoring В Center В for В Security В servers
If you have your own update package from your vendor, you can populate the AlienVault
database with the new signatures.
Go to /usr/share/ossim/scripts/ to update the plugin sid information:
python createCiscoIPSSidmap.py IOS-S416-CLI.pkg.xml
DELETE FROM plugin WHERE id = "1597";
DELETE FROM plugin_sid where plugin_id = "1597";
INSERT INTO plugin (id, type, name, description) VALUES (1597, 1, 'CiscoIPS', 'Cisco Intrusion Prevention System');
INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name,
priority, reliability) VALUES (1597, 5986, NULL, NULL, 'Cisco-IPS:
Microsoft GDI GIF Parsing Vulnerability', 3, 4);
2
This protocol is used in the Cisco Systems IPS Sensor 5.0 to replace Remote Data Exchange Protocol
(RDEP).
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 12 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name,
priority, reliability) VALUES (1597, 5984, NULL, NULL, 'Cisco-IPS: IE COM
Object Code Execution', 3, 4);
INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name,
priority, reliability) VALUES (1597, 5985, NULL, NULL, 'Cisco-IPS:
Quicktime RTSP Content-Type Excessive Length', 3, 4);
INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name,
priority, reliability) VALUES (1597, 19159, NULL, NULL, 'Cisco-IPS: Green
Dam Youth Escort Software Update Check', 1, 4);
INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name,
priority, reliability) VALUES (1597, 19401, NULL, NULL, 'Cisco-IPS:
Microsoft Publisher File Parsing Vulnerability', 3, 4);
This is script generates the needed SQL information to update AlienVault database. Write
the following to insert information:
python createCiscoIPSSidmap.py IOS-S416-CLI.pkg.xml > sdee.sql
ossim-db < sdee.sql
If you want to update cross-correlation information:
python ciscoIPSOsMap.py IOS-S416-CLI.pkg.xml
replace into plugin_reference values (1597, 1109, 3001, 3);
replace into plugin_reference values (1597, 1109, 3001, 3);
replace into plugin_reference values (1597, 1109, 3001, 3);
replace into plugin_reference values (1597, 1109, 3001, 3);
replace into plugin_reference values (1597, 2156, 3001, 1);
replace into plugin_reference values (1597, 2157, 3001, 3);
replace into plugin_reference values (1597, 2157, 3001, 3);
replace into plugin_reference values (1597, 2157, 3001, 3);
...
...
python ciscoIPSOsMap.py IOS-S416-CLI.pkg.xml > sdee-os.sql
ossim-db < sdee-os.sql
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 13 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
Do В not В forget В to В restart В ossim-server В in В order В to В update В AlienVault В Server В cache.
Follow the instructions below to configure AlienVault Agent and collect events from SDEE
capable device:
1.
Add В SDEE В reference В to В this В file: В /etc/ossim/agent/config.cfg
2.
Edit В this В file: В /etc/ossim/agent/plugins/cisco-ips.cfg
[DEFAULT]
plugin_id=1597
[config]
type=detector
enable=yes
source=sdee
source_ip=
user=
password=
sleep=5
process=
start=no
stop=no
3.
Insert  the  credentials:  your  “source_ip”,  “user”  and  “password”  data.
4.
Restart В AlienVault В Agent В for В receiving В data В from В SDEE В device.
Keep in mind the following points:
Each В time В a В new В session В begins В with В a В SDEE В device, В a В Subscription В ID В will В be В provided. В If В the В device В closes В the В connection В or В the В connectivity В is В lost, В you В have В to В close В the В session В DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 14 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
in В order В to В continue В collecting В from В the В device. В The В AlienVault В Agent В closes В the В session В automatically, В but В if В not, В you В should В do В it В manually.
The В latest В Subscription В ID В can В be В found В here: В /etc/ossim/agent/sdee_sid.data
Execute В the В following:
python /usr/share/ossim/scripts/closeSDEEsession.py SubscriptionID
This В closes В the В last В session. В If В you В still В have В problems, В execute В the В following:
grep subs /var/log/ossim/agent.log
The В agent В debugging В can В also В turn В on, В stopping В the В current В agent В and В starting В it В manually В on В verbose В mode:
ossim-agent -v
You В should В get В something В like В this:
2012-05-07 05:15:40,925 Agent [DEBUG]: <?xml version="1.0" encoding="UTF8"?><env:Envelope xmlns="http://www.cisco.com/cids/2006/08/cidee"
xmlns:env="http://www.w3.org/2003/05/soap-envelope"
xmlns:sd="http://example.org/2003/08/sdee"
xmlns:cid="http://www.cisco.com/cids/2006/08/cidee"><env:Header><sd:oobInfo><
sd:sessionId>373397c2f80a792a4029fbcc0cd027e5</sd:sessionId><sd:remainingevents>0</sd:remainingevents></sd:oobInfo></env:Header><env:Body><sd:events></sd:events></env:Body>
</env:Envelope>
2.1.5.
WMI В (WINDOWS В MANAGEMENT В INSTRUMENTATION)
They collect remotely Microsoft Windows events and data in an agent-less way.
This is an example of a WMI plugin:
[DEFAULT]
plugin_id=1518
[config]
type=detector
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 15 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
enable=yes
source=wmi
credentials_file=/etc/ossim/agent/wmi_credentials.csv
sleep=10
process=
start=no
stop=no
[start_cmd]
cmd=wmic -U OSS_WMI_USER%OSS_WMI_PASS //OSS_WMI_HOST "Select
LogFile,RecordNumber from Win32_NTLogEvent Where Logfile = 'Application'" |
head -n 3 | tail -n 1 | cut -f 2 -d \|
regexp=
[cmd]
cmd = wmic -U OSS_WMI_USER%OSS_WMI_PASS //OSS_WMI_HOST "Select
ComputerName,EventCode,Logfile,Message,RecordNumber,SourceName,TimeWritten,Us
er from Win32_NTLogEvent Where Logfile = 'Application' and RecordNumber >
OSS_COUNTER" | cat
start_regexp=^([^\|]+)\|(\d+)\|([^\|]+)\|
regexp="^(?P<system_name>[^\|]+)\|(?P<plugin_sid>\d+)\|(?P<logfile>[^\|]+)\|(
?P<message>[^\|]+)\|(?P<recordnumber>[^\|]+)\|(?P<sourcename>[^\|]+)\|(?P<tim
ewritten>[^\|]+)\|(?P<username>.*)$"
src_ip={resolv($0)}
plugin_sid={$1}
userdata2={$2}
userdata3={$3}
userdata4={$4}
userdata5={$5}
userdata6={$6}
username={$7}
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 16 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
2.1.5.1.
3
DC-00138
PREPARING В WINDOWS
1.
Create В a В new В limited В user В for В not В using В an В administrator В account В for В remote В connections В and В make В the В installation В much В more В secure.
2.
For  this  example,  the  user  “wmiuser”  and  password  “wmi”  have  been  created.
3.
Configure В DCOM В to В allow В the В user В access В to В the В computer В remotely.
4.
Grant В remote В launch В to В DCOM В and В activation В permissions В for В our В user:
a)
Run В Dcomcnfg В by В selecting В Run В on В the В Start В menu В and В typing В in В Dcomcnfg. В Then В click В OK.
b)
Open  “Administrative  Tools” 3  and  expand  “Component  Services”.  Click  the  secondary  button  of  the  mouse  over  “My  Computer”  and  select  “Properties”.
This option is not always in the same place. It depends on the Windows version.
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 17 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
c)
DC-00138
Edition 03
Click  on  “COM  Security”  tab.  Then  click  on  Edit  Limits  under  “Access  Permissions”:
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 18 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
d)
DC-00138
Edition 03
Click  on  “Anonymous  Logon”  and  then  on  “Remote  Access”:
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 19 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
4
DC-00138
e)
Click В OK.
f)
Click В Apply.
g)
Click В OK.
5.
Run В Dcomcnfg В by В selecting В Run В on В the В Start В menu В and В typing В in В Dcomcnfg. В Then В click В OK.
6.
Open  “Administrative  Tools”4  and  expand  “Component  Services”.  Click  the  secondary  button  of  the  mouse  over  “My  Computer”  and  select  “Properties”.
7.
Click  on  “COM  Security”  tab  and  then  click  on  Edit  Limits  under  “Launch  and  Activation  Permissions”.
This option is not always in the same place. It depends on the Windows version.
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 20 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
8.
DC-00138
Click В Add В button.
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 21 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
9.
Enter В the В user В name В and В click В OK.
10. In  the  “Launch  and  Activation  Permisson”  screen,  click  the  options:  “Remote  Launch”,  “Local  Activation”  and  “Remote  Activation”.  Then  click  OK.
11. Click В OK.
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 22 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
12. Click В Apply.
13. Click В OK.
2.1.5.2.
CONFIGURING В ALIENVAULT В USM
Before the activation of wmi plug-ins, it is necessary to create a file having Windows IPs
and credentials.
1.
Create В a В wmi_credentials.csv В file.
vim /etc/ossim/agent/wmi_credentials.csv
2.
Add В ips, В users В and В password В with В following В formats:
127.0.0.1,user,pass
127.0.0.2,domain/user, pass
127.0.0.3,domain/user ,pass
2.2.
Monitor Plugins
These plugins are used to execute actions in sensors in correlation time through directives. For
instance, the 2005 plugin monitor is used in these 2 files:
ntop-monitor.cfg
session-monitor.cfg
This is used from monitor directives like:
/etc/ossim/server/alienvault-attacks.xml: <rule type="monitor" name="AV
More than 10 secs persistence" reliability="5" from="1:SRC_IP"
to="1:DST_IP" port_from="1:SRC_PORT" port_to="1:DST_PORT"
plugin_id="2005" plugin_sid="248" condition="ge" value="10" interval="15"
time_out="30" absolute="true">
3.
HOW TO CREATE A CUSTOM DATA SOURCE PLUGIN
This section explains how to create a custom plugin to process Exchange Web Server logs
through the SIEM engine.
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 23 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
3.1.
EXCHANGE WEB SMTP SERVER LOGS
The log file used for the following hands on exercise can be downloaded from here:
exchangews.log
Once the file has been downloaded, open it to see the logs we are going to parse. Here are
some sample lines:
2011-10-09 05:00:19 1.1.1.1 36A42160 SMTPSVC1 MEE-PDC 192.168.1.2 0 QUIT
- 36A42160 240 6219 68 4 0 SMTP - - - 1.1.1.10 - 1.1.1.9 [11/Oct/2011:13:16:40 -0600] "HELO -?+1.1.1.9 SMTP"
250 46
3.2.
CREATION OF THE PLUGIN CONFIGURATION FILE EXCHANGEWS.CFG
1.
Global В plugin В configuration В settings:
Copy  the  file  “ssh.cfg  “  and  name  this  new  one  as  “exchangews.cfg”.
Change  the  “plugin id“  field  (use  9001  as  it  is  part  of  the  user  range  that  goes  up  to  10000).
Change  location  to  point  to  the  log  file  “/var/log/exchangews.log”
Delete В startup В and В shutdown В fields. В These В fields В are В not В going В to В be В used В (there В is В no В application В associated В with В this В plugin).
Create В a В new В translation В table5, В as В it В is В shown В below. В This В step В is В optional:
HELO=1
MAIL=2
RCPT=3
DATA=4
QUIT=5
5
A translation table is used for translating a string to a number in order to use it as plugin_sid. It is necessary to
include the function {translate($field_to_translate)} for using a translation table.
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 24 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
xxxx=6
DEFAULT_=9999
2.
Create В new В rules, В filling В up В the В fields В below. В Create В two В regular В expressions В to В parse В the В data, В because В there В are В two В different В formats В in В the В log В file.
[exchangews - Generic rule]
#2011-10-09 05:00:15 1.1.1.1 36A42160 SMTPSVC1 MEE-PDC 192.168.1.2 0 HELO +36A42160 250 0 48 13 0 SMTP - - - #2011-10-09 05:00:16 1.1.1.1 36A42160 SMTPSVC1 MEE-PDC 192.168.1.2 0 MAIL +FROM:+<test@sample1.com> 250 0 57 45 0 SMTP - - - event_type=event
regexp="(?P<date>\d{4}-\d{2}\d{2}\s\d{2}:\d{2}:\d{2})\s(?P<src_ip>\IPV4)\s(?P<userdata2>\S+)\s(?P<hostnam
e>\S+)\s(?P<userdata3>\S+)\s(?P<dst_IP>\IPV4)\s\d\s(?P<type>\w+)"
date={normalize_date($date)}
plugin_sid={translate($type)}
dst_ip={resolv($dst_ip)}
src_ip={resolv($src_ip)}
hostname={$hostname}
userdata2={$userdata2}
userdata3={$userdata3}
[exchangews = Generic rule 2 NCSA Format]
#1.1.1.10 - 1.1.1.9 [11/Oct/2011:13:16:40 -0600] "HELO -?+1.1.1.9 SMTP" 250
46
#1.1.1.10 - 1.1.1.9 [11/Oct/2011:13:16:41 -0600] "MAIL ?+FROM:+<Keith@testdomain.com> SMTP" 250 46
event_type=event
regexp="(?P<src_ip>\IPV4)\s\s(?P<dst_ip>\S+)\s\[(?P<date>\d\d\/\w{3}\/\d{4}:\d\d:\d\d:\d\d)\s\d{4}\]\s\"(?P<type>\w+)"
date={normalize_date($date)}
plugin_sid={translate($type)}
dst_ip={resolv($dst_ip)}
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 25 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
src_ip={resolv($src_ip)}
3.
3.3.
Check  regular  expressions  with  logs  inside  the  file  “/var/log/exchangews.log”.  There  are  several  utilities  on  the  Internet  to  test  regular  expressions  written  in  Python.  It  is  recommended  to  use  one  of  these  utilities  to  check  that  the  created  regular  expressions  match  the  logs.
CREATE THE DATABASE FILE EXCHANGEWS.SQL
1.
Create В a В file В using В the В following В examples:
INSERT INTO plugin (id, type, name, description) VALUES (9001, 1,
'exchangews', 'Exchange E-mail Web server');
INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name,
priority, reliability) VALUES (9001, 1, NULL, NULL, 'exchangews: HELO'
,3, 2);
INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name,
priority, reliability) VALUES (9001, 9999, NULL, NULL, 'exchangews:
Generic exchange event' ,3, 2);
2.
Insert В file В values В into В the В database В in В the В server В box.
cat exchangews.sql | ossim-db
3.
Apply В changes В in В SIEM.
/etc/init.d/ossim-server restart
3.4.
ACTIVATE DATA SOURCE PLUGINS
Choose one of the following options to activate plugins:
Through В command В line В console
Through В web
3.4.1.
ACTIVATE В PLUGINS В THROUGH В COMMAND В LINE В CONSOLE
To activate plugins by a console, the instructions below should be followed:
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 26 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
1.
Open В a В console В terminal В application В and В connect В to В the В AlienVault В System В by В running В the В following В command:
ssh root@IP_address
IP_address refers to the default IP of your appliance.
2.
Next, В it В appears В a В screen В which В includes В the В main В menu:
3.
By  using  the  arrow  keys  on  the  keyboard,  select  the  option  “Configure  Sensor”.  Accept  the  selection  (<OK>)  by  pressing  Enter  key.
Use the Tab key on the keyboard to move from <OK> to <Exit> or vice versa:
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 27 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
DC-00138
4.
Select  the  option  “Configure  Data  Source  Plugins”.  Accept  the  selection  (<OK>)  by  pressing  Enter  key.
5.
Select В the В plugins В to В activate. В To В move В between В them В use В the В arrow В keys В on В the В keyboard В and В select/deselect В it В by В pressing В the В Space В Bar В on В the В keyboard. В Accept В the В selection В (<OK>) В by В pressing В Enter В key. В It В is В possible В to В select В several В plugins. В Accept В the В selection В (<OK>) В by В pressing В Enter В key.
6.
The  �Configure  Sensor’  window  appears.  Move  from  <OK>  to  <Back>  by  using  the  Tab  key  on  the  keyboard.  Back  to  the  AlienVault  Setup  Screen.
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 28 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
DC-00138
7.
Select  the  option  “Apply  all  changes”.  Accept  the  selection  (<OK>)  by  pressing  Enter  key.
8.
Apply В all В changes В (<Yes>) В by В pressing В Enter В key.
9.
The В process В can В take В several В minutes В depending В on В the В number В of В plugins В to В activate:
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 29 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
10. Once В the В process В finishes, В the В following В screen В appears:
11. Press В Enter В key. В The В AlienVault В Setup В screen В appears.
12. Move В from В <OK> В to В <Exit> В by В using В the В Tab В key В on В the В keyboard. В Press В Enter В key.
3.4.2.
ACTIVATE В PLUGINS В BY В WEB
To activate plugins by the web, the instructions below should be followed:
DC-00138
1.
Use В a В web В browser В access В to В your В AlienVault В console В at В https://your_ip/
2.
Write В a В valid В IP В address В in В the В navigation В bar В of В a В web В browser.
3.
Enter В a В valid В user В name В and В password В and В click В on В Login.
4.
Navigate  to  “Configuration  >  Deployment”:
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 30 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
DC-00138
5.
The В following В window В appears:
6.
Click  on  one  of  the  “Node  Name”  then,  on  Sensor  Configuration  link,  and  finally  on  Collection  link.  A  table  appears:
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 31 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
This table displays 2 columns. The left column shows plugins that are enabled and the
right column shows plugins that are available to be enabled.
To pass an item from one side to the other, drag and drop the item or use the links [+]
or [-] which are next to each item.
7.
3.5.
To В make В all В changes В take В effect, В click В the В APPLY В CHANGES В button.
FILES .LOCAL
Whenever a plugin file is going to be changed, it is recommended to copy the filename.cfg into
another file named filename.cfg.local. Make all necessary changes in the .local file and keep
the .cfg file without any change. By copying the file, you preserve the original plugin version
over the updated version. The new updates will never overwrite your plugin customization.
Keep В in В mind В that В the В original В plugins В can В be В modified В by В AlienVault В when В an В update В process В has В been В done.
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 32 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
5.
HOW TO USE CUSTOM FUNCTION IN DATA SOURCE PLUGINS
The instructions below should be followed:
1.
Insert В a В custom В function В field В in В the В plugin В .cfg В file:
[config]
custom_functions_file=/etc/ossim/agent/plugin/ssh_custom_functions.cfg
2.
Create  a  function  file  having  in  mind  that  a  function  must  start  with  “Start  Function  <func name>”  and  must  end  with  “End  function”:
Start Function
log_hello
def log_hello(self):
return "Hello log!"
End Function
Start Function
log_hello_data
def log_hello_data(self,data):
return "Hello log: %s" % data
End Function
3.
Edit В the В plugin В rules В to В use В the В function В by В using В two В points:
[0021 - ssh - Failed password]
# Feb 8 10:09:06 golgotha sshd[24472]: Failed password for dgil from
192.168.6.69 port 33992 ssh2
event_type=event
regexp="(\w{3}\s+\d{1,2}\s\d\d:\d\d:\d\d)\s+(?P<sensor>\S*).*ssh.*Failed
password for
(?P<user>\S+)\s+from\s+.*?(?P<src>\IPV4).*port\s+(?P<sport>\d{1,5})"
plugin_sid=1
sensor={resolv($sensor)}
date={normalize_date($1)}
src_ip={$src}
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 33 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
dst_ip={resolv($sensor)}
src_port={$sport}
username={$user}
userdata1={:log_hello()}
userdata2={:log_hello_data($user)}
It В is В not В possible В to В apply В a В built in В function В to В a В custom В function В (for В instance, В translate(:log_hello()) В ), В as В the В last В function В which В are В executed В are В the В custom В ones.
So В translate В will В receive В :log_hello() В as В a В simple В chain В of В chars.
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 34 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
APPENDIX A - RECOMMENDATIONS BEFORE CREATING A NEW
PLUGIN
Follow the following recommendations before creating a new plugin:
Copy В a В log В as В big В as В possible.
Extract В events В from В the В log В using В consecutive В grep В -v, В until В the В command В does В not В return В anything.
Use В grep В to В check В individually В every В event. В Try В to В find В different В values В that В a В same В event В may В take.
Discard В repeated В events.
Look В for В event В patterns В in В order В to В group В them В using В some В identifier В such В as В the В same В field В distribution, В for В instance.
Take В into В account В that В your В target В will В be В identifying В individual В events В using В a В plugin_sid, В you В may В need В to В think В what В translates В you В will В be В using.
For В every В event, В find В out В the В number В of В times В it В is В repeated В within В the В log, В using В the В following В command В to В count В the В lines.
wc -l
Think В if В it's В worth В using В a В single В regex В for В an В event В or В if В several В can В be В grouped В together В without В making В the В regex В very В complex.
Only В capture В the В fields В that В are В going В to В be В used В in В correlation В later В on.
Create В a В rule В with В a В generic В regex В at В the В end В to В capture В any В remaining В event.
Choose В the В right В pre-check, В keeping В in В mind В that В it В applies В a В first В filter В to В the В events.
Make В sure В the В rules В are В alphabetically В ordered, В starting В with В 0001 В and В finishing В with В 9999, В creating В 0002, В 0003... В groups, В leaving В room В for В future В expressions.
The В rules В are В loaded В and В applied В in В alphabetical В order, В so В events В captured В by В a В rule В will В not В be В processed В by В the В rules В loaded В after В that В one. В Rule В order В must В be В chosen В carefully В to В avoid В event В masking В due В to В generic В rules В being В loaded В before В specific В ones.
The В SQL В does В not В need В the В sids В to В be В correlative. В Gaps В can В be В left В in В order В to В make В it В more В maintainable.
Let’s  say  from  1000  to  1999  for  A  event  types,  from  2000  to  2999  for  B  event  types,  etc.
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 35 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
Be В careful В if В you В add В a В custom В function В into В a В plugin;Нѕ В or В if В you В access В to В a В proprietary В database. В This В may В deteriorate В the В performance В if В it В is В not В well В designed.
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 36 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
APPENDIX B - LIST OF DATA SOURCE PLUGINS
B.1. Database Plugins
drupal-wiki.cfg
eljefe.cfg
forensics-db-1.cfg
mcafee-epo.cfg
moodle.cfg
motion.cfg
oracle-sql.cfg
panda-se.cfg
post_correlation.cfg
vmware-vcenter-sql.cfg
B.2. Log Plugins
airlock.cfg
aix-audit.cfg
aladdin.cfg
allot.cfg
alteonos.cfg
amun-honeypot.cfg
apache.cfg
apache-syslog.cfg
arpalert.cfg
arpwatch.cfg
artemisa.cfg
aruba.cfg
aruba-6.cfg
ascenlink.cfg
avast.cfg
axigen-mail.cfg
bind.cfg
bit9.cfg
bluecoat.cfg
bro-ids.cfg
cisco-3030.cfg
cisco-ace.cfg
cisco-acs.cfg
cisco-acs-idm.cfg
cisco-asa.cfg
cisco-asr.cfg
cisco-fw.cfg
cisco-ids.cfg
cisco-ips-syslog.cfg
cisco-nexus-nx-os.cfg
cisco-pix.cfg
cisco-router.cfg
cisco-vpn.cfg
cisco-wlc.cfg
citrix-netscaler.cfg
clamav.cfg
clurgmgr.cfg
courier.cfg
cyberguard.cfg
dhcp.cfg
dionaea.cfg
dovecot.cfg
dragon.cfg
enterasys-rmatrix.cfg
exchange.cfg
extreme-switch.cfg
extreme-wireless.cfg
f5.cfg
f5-firepass.cfg
fidelis.cfg
fortigate.cfg
fortiguard.cfg
fortimail.cfg
fw1-alt.cfg
fw1ngr60.cfg
gfi.cfg
glastopng.cfg
heartbeat.cfg
honeyd.cfg
hp-eva.cfg
iis.cfg
impervasecuresphere.cfg
intrushield.cfg
ipfw.cfg
iphone.cfg
iptables.cfg
ironport.cfg
isa.cfg
juniper-srx.cfg
juniper-vpn.cfg
kismet.cfg
linuxdhcp.cfg
lucent-brick.cfg
m0n0wall.cfg
mcafee.cfg
mcafee-antispam.cfg
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 37 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
modsecurity.cfg
monit.cfg
motorola-firewall.cfg
mwcollect.cfg
nagios.cfg
nepenthes.cfg
nessus.cfg
nessus-detector.cfg
netgear.cfg
netkeeper-fw.cfg
netkeeper-nids.cfg
netscreen-firewall.cfg
netscreen-igs.cfg
netscreen-manager.cfg
netscreen-nsm.cfg
nfs.cfg
nortel-switch.cfg
ntsyslog.cfg
openldap.cfg
optenet.cfg
oracle-syslog.cfg
osiris.cfg
ossec.cfg
ossec-idm.cfg
ossec-idm-single-line.cfg
ossec-single-line.cfg
ossim-agent.cfg
p0f.cfg
pads.cfg
paloalto.cfg
pam_unix.cfg
panda-as.cfg
pf.cfg
postfix.cfg
prads.cfg
prads_eth0.cfg
proxim-orinoco.cfg
pureftpd.cfg
radiator.cfg
radware-ips.cfg
raslogd.cfg
realsecure.cfg
rrd.cfg
rsa-secureid.cfg
sap.cfg
sendmail.cfg
serviceguard.cfg
shrubbery-tacacs.cfg
sidewinder.cfg
siteprotector.cfg
siteprotector-snmp.cfg
sitescope.cfg
smbd.cfg
snare.cfg
snare-idm.cfg
snare-mssql.cfg
snare-msssis.cfg
snort_syslog.cfg
sonicwall.cfg
sophos.cfg
spamassassin.cfg
squid.cfg
squidGuard.cfg
ssh.cfg
stonegate.cfg
stonegate_ips.cfg
storewize-V7000.cfg
sudo.cfg
suhosin.cfg
suricata-http.cfg
symantec-ams.cfg
symantec-epm.cfg
syslog.cfg
tacacs-plus.cfg
tarantella.cfg
tippingpoint.cfg
token-rsa.cfg
trendmicro.cfg
usbudev.cfg
vandyke-vshell.cfg
vmware-esxi.cfg
vmware-vcenter.cfg
vmware-workstation.cfg
vplus.cfg
vsftpd.cfg
vyatta.cfg
W2003DNS.cfg
watchguard.cfg
webmin.cfg
websense.cfg
wuftp.cfg
B.3. Monitor Plugins
The following plugins are monitor plugins:
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 38 of 39
AlienVault Unified Security Managementв„ў Solution
How to create a data source plugin
6
nessus-monitor.cfg
nmap-monitor.cfg
ntop-monitor.cfg
ocs-monitor.cfg
opennms-monitor.cfg
ossim-monitor.cfg
ping-monitor.cfg
session-monitor.cfg
tcptrack-monitor.cfg
whois-monitor.cfg
wmi-monitor.cfg
B.4. Remote Plugins
ssh-remote.cfg
B.5. SDEE Plugins
cisco-ips.cfg
B.6. WMI Plugins
wmi-application-logger.cfg
wmi-security-logger.cfg
wmi-security-logger-srv2008.cfg
wmi-system-logger.cfg
6
This plugin is used by certain directives to ascertain whether a DOS or DDoS attack was successful. See
directive 34031 for use-case.
DC-00138
Edition 03
CopyrightВ© 2014 AlienVault. All rights reserved.
Page 39 of 39
© Copyright 2024 Paperzz
