Security Onion: Validating EXE/DLL Download Alerts

Posted on Wed 17 May 2017 in howto

As I've mentioned before, Security Onion is a fantastic network security-focused Linux distribution which can monitor your network and/or hosts for malicious activity.

The Onion can run Snort or Suricata as a network IDS, and it can also run bro alongside those traditional IDS engines to add another layer of intelligence. This article will highlight one way in which these two engines can be combined to quickly triage IDS alerts.

Snort and Suricata both make use of the Talos rule set, and can also use the Emerging Threats (aka ET) rule set. Both of those IDS rulesets include rules to generate alerts when an executable file is downloaded. On a network of nearly any size, those rules can get to be .... noisy. I don't want to remove the rules, since they can be valuable, but I do want to quickly determine if any of the executables downloaded might pose a risk.

One way to evaluate the risk a file might introduce is to look at where it was downloaded from. By default in the Onion, Snort and Suricata don't show the server host information, but we can query the sguil DB and then take the resulting set and grep the bro logs for hostnames.

First, we need to generate the list of rule signature ID's (or SIDs) that indicate the download of an exe or dll file:

jma@onion-server:~$ grep -Ei '(exe|dll).*download' /etc/nsm/rules/*.rules | grep -Po 'sid:[0-9]+' | cut -f2 -d: > ~/sids

I imagine there are more elegant ways to accomplish this, but the above works well enough for me. :-)

Someday, maybe I'll incorporate the above command into the script below so that the entire process is fully automated. But for now, take the SIDs that are stored in the ~/sids file and update the ET_EXE_SIDS variable in the script. I've stored the script in the /usr/local/bin/ folder on the onion server and saved it as check-exe-downloads.sh.

Stand Alone Installation

Security Onion can be installed as a stand-alone system or as a distributed environment. We'll start with the basic, simpler stand-alone installation. In this scenario, the bro logs are stored on the same server as the DB, which makes them easy to query.

Links to the full script can be found at the end of the article

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
#!/bin/bash

# Update this list with any SIDs that indicate EXE or dll
# or other file downloads
#
# grep -Ei '(exe|dll).*download' /etc/nsm/rules/*.rules | grep -Po 'sid:[0-9]+' | cut -f2 -d: > ~/sids
#

# NOTE: This should really all be on one line, but is broken up here for display

ET_EXE_SIDS="2000419, 2018959, 2014518, 2014819, 2007998, 2008407,
2008408, 2008409, 2010011, 2010190, 2015537, 2015566, 2015567,
2016197, 2009568, 2009651, 2012610, 2014909, 2015688, 2016696,
2017057, 2017093, 2017297, 2017318, 2017672, 2017673, 2017674,
2017675, 2017676, 2017677, 2017679, 2017680, 2017681, 2017682,
2017683, 2018103, 2018104, 2018191, 2018556, 2018963, 2019714,
2020573, 2022050, 2022051, 2022053, 2022653, 2022884, 2023745,
2023817, 2000423, 2000424, 2000425, 2000427, 2010342, 2010447,
2010716, 2010869, 2011900, 2011919, 2011923, 2011980, 2011981,
2011983, 2011984, 2011985, 2011986, 2011990, 2013442, 2014181,
2015752, 2017678, 2017795, 2017961, 2022052, 2002068, 2008438,
2014514, 2014515, 2014516, 2014517, 2014518, 2014567, 2014819,
2016141, 2016538, 2016767, 2021216, 2023454, 2023455, 2023456,
2023457, 2023458, 2023459, 2023460, 2023461, 2023462, 2023463,
2023464, 2000371, 2001533, 2009091, 2014735, 2014810, 2018324,
2018333, 2018339, 2018367, 2000418, 2000419, 2000426, 2003179,
2003595, 2012523, 2012524, 2013037, 2014059, 2014313, 2014471,
2016856, 2018959, 2019240, 2020202, 2020914, 2009174, 2010050,
2010059, 2010741, 2011495, 2011496, 2011982, 2011989, 2011991,
2012208, 2012227, 2012389, 2013036, 2013291, 2013352, 2013441,
2013560, 2013770, 2013826, 2013827, 2014150, 2014230, 2014525,
2015547, 2015862, 2016029, 2016475, 2016844, 2017190, 2017583,
2017598, 2017617, 2017962, 2018241, 2018254, 2018395, 2018403,
2018464, 2018572, 2018581, 2018982, 2019103, 2020198, 2020199,
2020200, 2020201, 2021697, 2021774, 2021952, 2021954, 2022037,
2022207, 2022239, 2022482, 2022483, 2022640, 2022692, 2001048, 27982,
28000, 8737, 8738, 8740, 24501, 24791, 25140, 26040, 26041, 26043,
26526, 26534, 26891, 26962, 27005, 27069, 27082, 27083, 27084,
27936, 11192, 16313, 21173, 16425, 25513, 25514, 32947, 27255,
28806, 28945, 35737, 35738, 16096, 16670, 21425, 21554, 26880,
28983, 28984, 28985, 39856, 39857, 24259, 24260, 25782, 26470, 31487,
31488, 33941, 33942, 33943, 26257, 32951, 38033, 38034, 18648, 23209,
23210, 29501, 31091, 7129, 7849, 36454, 36455, 2067, 24520, 27862"

IP_FILE="${HOME}/ip"

if [ -e ${IP_FILE} ]; then
  rm ${IP_FILE} 2&>/dev/null
fi

# Build the SELECT statement using the SIDs above
QUERY="SELECT INET_NTOA(src_ip) as sip FROM event WHERE signature_id IN (${ET_EXE_SIDS}) AND status="0" GROUP BY sip;"

# The grep command strips the results of the header line 
# store the results in a file for grep to reference
mysql --defaults-file=/etc/mysql/debian.cnf -Dsecurityonion_db -e "${QUERY}" | grep -v sip > ${IP_FILE}

# This is ugly and not opitmized. But it works
zcat -f /nsm/bro/logs/*/http_eth1.* | bro-cut id.resp_h host | grep -f ${IP_FILE} | awk '{print $2}' | sort | uniq -c | sort -rn

rm ${IP_FILE}

The results will look something like this:

jma@onion-server:~$ /usr/local/bin/check-exe-downloads.sh
3501 13.107.4.50     au.download.windowsupdate.com
 236 13.107.4.50     download.windowsupdate.com
  74 8.254.242.142   download.windowsupdate.com
  44 8.254.242.174   download.windowsupdate.com
  19 8.254.242.142   au.download.windowsupdate.com
  15 209.116.186.211 r8---sn-mv-qxoe.gvt1.com
   2 8.254.242.174   au.download.windowsupdate.com

Distributed Onion Environment

In a distributed environment, the sensors forward the Snort/Suricata alerts back to the central server, but keep the Bro logs locally. That means that the SQL query is run from the central server and the Bro logs are grep'd via SSH to report back the findings. A few tweaks to the script will allow you to still gather the results, as seen below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
#!/bin/bash

# Update this list with any SIDs that indicate EXE or dll or other file
# downloads
#
# grep -Ei '(exe|dll).*download' /etc/nsm/rules/*.rules | grep -Po 'sid:[0-9]+' | cut -f2 -d: > ~/sids
#

# NOTE: This should really all be on one line, but is broken up here for display
ET_EXE_SIDS="2000419, 2018959, 2014518, 2014819, 2007998, 2008407,
2008408, 2008409, 2010011, 2010190, 2015537, 2015566, 2015567,
2016197, 2009568, 2009651, 2012610, 2014909, 2015688, 2016696,
2017057, 2017093, 2017297, 2017318, 2017672, 2017673, 2017674,
2017675, 2017676, 2017677, 2017679, 2017680, 2017681, 2017682,
2017683, 2018103, 2018104, 2018191, 2018556, 2018963, 2019714,
2020573, 2022050, 2022051, 2022053, 2022653, 2022884, 2023745,
2023817, 2000423, 2000424, 2000425, 2000427, 2010342, 2010447,
2010716, 2010869, 2011900, 2011919, 2011923, 2011980, 2011981,
2011983, 2011984, 2011985, 2011986, 2011990, 2013442, 2014181,
2015752, 2017678, 2017795, 2017961, 2022052, 2002068, 2008438,
2014514, 2014515, 2014516, 2014517, 2014518, 2014567, 2014819,
2016141, 2016538, 2016767, 2021216, 2023454, 2023455, 2023456,
2023457, 2023458, 2023459, 2023460, 2023461, 2023462, 2023463,
2023464, 2000371, 2001533, 2009091, 2014735, 2014810, 2018324,
2018333, 2018339, 2018367, 2000418, 2000419, 2000426, 2003179,
2003595, 2012523, 2012524, 2013037, 2014059, 2014313, 2014471,
2016856, 2018959, 2019240, 2020202, 2020914, 2009174, 2010050,
2010059, 2010741, 2011495, 2011496, 2011982, 2011989, 2011991,
2012208, 2012227, 2012389, 2013036, 2013291, 2013352, 2013441,
2013560, 2013770, 2013826, 2013827, 2014150, 2014230, 2014525,
2015547, 2015862, 2016029, 2016475, 2016844, 2017190, 2017583,
2017598, 2017617, 2017962, 2018241, 2018254, 2018395, 2018403,
2018464, 2018572, 2018581, 2018982, 2019103, 2020198, 2020199,
2020200, 2020201, 2021697, 2021774, 2021952, 2021954, 2022037,
2022207, 2022239, 2022482, 2022483, 2022640, 2022692, 2001048, 27982,
28000, 8737, 8738, 8740, 24501, 24791, 25140, 26040, 26041, 26043,
26526, 26534, 26891, 26962, 27005, 27069, 27082, 27083, 27084,
27936, 11192, 16313, 21173, 16425, 25513, 25514, 32947, 27255,
28806, 28945, 35737, 35738, 16096, 16670, 21425, 21554, 26880,
28983, 28984, 28985, 39856, 39857, 24259, 24260, 25782, 26470, 31487,
31488, 33941, 33942, 33943, 26257, 32951, 38033, 38034, 18648, 23209,
23210, 29501, 31091, 7129, 7849, 36454, 36455, 2067, 24520, 27862"

IP_FILE="${HOME}/ip"

if [ -e ${IP_FILE} ]; then
  rm ${IP_FILE} 2&>/dev/null
fi

# Build the SELECT statement using the SIDs above
QUERY="SELECT INET_NTOA(src_ip) as sip FROM event WHERE signature_id IN (${ET_EXE_SIDS}) AND status="0" GROUP BY sip;"

# The grep command strips the results of the header line 
# store the results in a file for grep to reference

# This is the first change to the remote version of the script. Since the
# output must be read into an environment variable, we send it through awk
# for additional formatting suitable to the environment.
sudo mysql --defaults-file=/etc/mysql/debian.cnf -Dsecurityonion_db -e "${QUERY}" | awk '/^[0-9]/ { ORS="|"; print $1 } END { ORS=""; print $0 }' > ${IP_FILE}

# The grep regex can't be read from a local file, since the grep process is
# actually being run remotely. Instead, read the results into the environment
# variable which can be passed on the command line to grep through the SSH
# session.
REGEX="(`cat ${IP_FILE}`)"

# This is ugly and not opitmized (at all!). But it works
# SSH to each sensor and run the grep command against the bro logs on the
# sensor
echo
echo "Sensor1 results"
ssh sensor1 "zcat -f /nsm/bro/logs/*/http* | /opt/bro/bin/bro-cut id.resp_h host | grep -E \"${REGEX}\"| awk '{print $2}' | sort | uniq -c | sort -rn"
echo
echo "Sensor2 results"
ssh sensor2 "zcat -f /nsm/bro/logs/*/http* | /opt/bro/bin/bro-cut id.resp_h host | grep -E \"${REGEX}\"| awk '{print $2}' | sort | uniq -c | sort -rn"

rm ${IP_FILE}

The results would now look something like this:

jma@onion-server:~$ /usr/local/bin/check-exe-downloads.sh

Sensor1 results
24224 208.111.171.148  d1.sophosupd.com
11781 208.111.171.148  d2.sophosupd.com
 4706 208.111.171.148  dci.sophosupd.com
 3702 13.107.4.50      au.download.windowsupdate.com
 1614 208.111.171.148  d3.sophosupd.com
  237 13.107.4.50      download.windowsupdate.com
  61 8.253.104.30      download.windowsupdate.com
   48 8.254.242.158    download.windowsupdate.com
   27 38.100.7.132     00100d-1.l.windowsupdate.com
   24 209.116.186.210  r7---sn-mv-qxoe.gvt1.com
   13 208.111.170.216  000ee3-1.l.windowsupdate.com
   13 208.111.168.89   00138b-1.l.windowsupdate.com
   13 208.111.168.78   001b40-1.l.windowsupdate.com
   13 208.111.168.72   001b40-1.l.windowsupdate.com
    8 8.253.104.30     au.download.windowsupdate.com
    2 69.164.19.161    000866-1.l.windowsupdate.com
    2 38.100.7.163     001389-1.l.windowsupdate.com
    2 208.111.170.203  000ed5-1.l.windowsupdate.com

Sensor2 results
6838 208.111.171.148  d1.sophosupd.com
3409 208.111.171.148  d2.sophosupd.com
2945 13.107.4.50      au.download.windowsupdate.com
1293 208.111.171.148  dci.sophosupd.com
 429 208.111.171.148  d3.sophosupd.com
 170 13.107.4.50      download.windowsupdate.com
  86 208.111.161.190  00100d-1.l.windowsupdate.com
  75 38.100.7.149     0015b0-1.l.windowsupdate.com
  58 208.111.161.146  000793-1.l.windowsupdate.com
  55 8.254.242.158    download.windowsupdate.com
  35 8.253.104.30     download.windowsupdate.com
  34 208.111.161.206  000793-1.l.windowsupdate.com
  25 69.164.19.161    0007c2-1.l.windowsupdate.com
  24 8.254.242.158    au.download.windowsupdate.com
  24 208.111.161.206  000238-1.l.windowsupdate.com
  24 208.111.161.154  0004d3-1.l.windowsupdate.com
  23 69.28.159.143    00100d-1.l.windowsupdate.com
  14 74.125.155.199   r1---sn-p5qs7n7z.gvt1.com
  13 38.100.7.177     00100d-1.l.windowsupdate.com
  13 208.111.168.72   000793-1.l.windowsupdate.com
  10 69.28.159.200    001c17-1.l.windowsupdate.com
   6 208.111.161.146  00100d-1.l.windowsupdate.com
   4 208.111.161.138  000935-1.l.windowsupdate.com
   3 38.100.7.132     000935-1.l.windowsupdate.com
   2 8.253.104.30     au.download.windowsupdate.com
   2 69.28.159.20     000935-1.l.windowsupdate.com
   2 208.111.161.206  000ed5-1.l.windowsupdate.com
   2 208.111.161.203  00246b-1.l.windowsupdate.com
   1 208.111.161.206  00115a-1.l.windowsupdate.com

It's not guaranteed that malicious files won't be downloaded from "normal-looking" sites, but this is a good quick sanity check for anything that might stick out from the norm.

Script Downloads

Standalone check-exe-downloads.sh

Distributed check-exe-downloads.sh