All of this was prior to the latest versions of Security Onion which now run inside docker instances. I’ve not yet looked to see how this would be replicated there. But I’m leaving this up for historical purposes.
As I’ve mentioned before, Security Onion is a fantastic network security-focused Linux distribution which can monitor your network and/or hosts for malicious activity.
The Onion can run Snort or Suricata as a network IDS, and it can also run bro Zeek alongside those traditional IDS engines to add another layer of intelligence. This article will highlight one way in which these two engines can be combined to quickly triage IDS alerts.
Snort and Suricata both make use of the Talos rule set, and can also use the Emerging Threats (aka ET) rule set. Both of those IDS rulesets include rules to generate alerts when an executable file is downloaded. On a network of nearly any size, those rules can get to be …. noisy. I don’t want to remove the rules, since they can be valuable, but I do want to quickly determine if any of the executables downloaded might pose a risk.
One way to evaluate the risk a file might introduce is to look at where it was downloaded from. By default in the Onion, Snort and Suricata don’t show the server host information, but we can query the sguil DB and then take the resulting set and grep the bro logs for hostnames.
First, we need to generate the list of rule signature ID’s (or SIDs) that indicate the download of an exe or dll file:
jma@onion-server:~$ grep -Ei '(exe|dll).*download' /etc/nsm/rules/*.rules | grep -Po 'sid:[0-9]+' | cut -f2 -d: > ~/sids
I imagine there are more elegant ways to accomplish this, but the above works well enough for me. :-)
Someday, maybe I’ll incorporate the above command into the script below so that the entire process is fully automated. But for now, take the SIDs that are stored in the ~/sids file and update the ET_EXE_SIDS variable in the script. I’ve stored the script in the /usr/local/bin/ folder on the onion server and saved it as check-exe-downloads.sh.
Stand Alone Installation
Security Onion can be installed as a stand-alone system or as a distributed environment. We’ll start with the basic, simpler stand-alone installation. In this scenario, the bro logs are stored on the same server as the DB, which makes them easy to query.
Links to the full script can be found at the end of the article
#!/bin/bash
# Update this list with any SIDs that indicate EXE or dll
# or other file downloads
#
# grep -Ei '(exe|dll).*download' /etc/nsm/rules/*.rules | grep -Po 'sid:[0-9]+' | cut -f2 -d: > ~/sids
#
# NOTE: This should really all be on one line, but is broken up here for display
ET_EXE_SIDS="2000419, 2018959, 2014518, 2014819, 2007998, 2008407,
2008408, 2008409, 2010011, 2010190, 2015537, 2015566, 2015567,
2016197, 2009568, 2009651, 2012610, 2014909, 2015688, 2016696,
2017057, 2017093, 2017297, 2017318, 2017672, 2017673, 2017674,
2017675, 2017676, 2017677, 2017679, 2017680, 2017681, 2017682,
2017683, 2018103, 2018104, 2018191, 2018556, 2018963, 2019714,
2020573, 2022050, 2022051, 2022053, 2022653, 2022884, 2023745,
2023817, 2000423, 2000424, 2000425, 2000427, 2010342, 2010447,
2010716, 2010869, 2011900, 2011919, 2011923, 2011980, 2011981,
2011983, 2011984, 2011985, 2011986, 2011990, 2013442, 2014181,
2015752, 2017678, 2017795, 2017961, 2022052, 2002068, 2008438,
2014514, 2014515, 2014516, 2014517, 2014518, 2014567, 2014819,
2016141, 2016538, 2016767, 2021216, 2023454, 2023455, 2023456,
2023457, 2023458, 2023459, 2023460, 2023461, 2023462, 2023463,
2023464, 2000371, 2001533, 2009091, 2014735, 2014810, 2018324,
2018333, 2018339, 2018367, 2000418, 2000419, 2000426, 2003179,
2003595, 2012523, 2012524, 2013037, 2014059, 2014313, 2014471,
2016856, 2018959, 2019240, 2020202, 2020914, 2009174, 2010050,
2010059, 2010741, 2011495, 2011496, 2011982, 2011989, 2011991,
2012208, 2012227, 2012389, 2013036, 2013291, 2013352, 2013441,
2013560, 2013770, 2013826, 2013827, 2014150, 2014230, 2014525,
2015547, 2015862, 2016029, 2016475, 2016844, 2017190, 2017583,
2017598, 2017617, 2017962, 2018241, 2018254, 2018395, 2018403,
2018464, 2018572, 2018581, 2018982, 2019103, 2020198, 2020199,
2020200, 2020201, 2021697, 2021774, 2021952, 2021954, 2022037,
2022207, 2022239, 2022482, 2022483, 2022640, 2022692, 2001048, 27982,
28000, 8737, 8738, 8740, 24501, 24791, 25140, 26040, 26041, 26043,
26526, 26534, 26891, 26962, 27005, 27069, 27082, 27083, 27084,
27936, 11192, 16313, 21173, 16425, 25513, 25514, 32947, 27255,
28806, 28945, 35737, 35738, 16096, 16670, 21425, 21554, 26880,
28983, 28984, 28985, 39856, 39857, 24259, 24260, 25782, 26470, 31487,
31488, 33941, 33942, 33943, 26257, 32951, 38033, 38034, 18648, 23209,
23210, 29501, 31091, 7129, 7849, 36454, 36455, 2067, 24520, 27862"
IP_FILE="${HOME}/ip"
if [ -e ${IP_FILE} ]; then
rm ${IP_FILE} 2&>/dev/null
fi
# Build the SELECT statement using the SIDs above
QUERY="SELECT INET_NTOA(src_ip) as sip FROM event WHERE signature_id IN (${ET_EXE_SIDS}) AND status="0" GROUP BY sip;"
# The grep command strips the results of the header line
# store the results in a file for grep to reference
mysql --defaults-file=/etc/mysql/debian.cnf -Dsecurityonion_db -e "${QUERY}" | grep -v sip > ${IP_FILE}
# This is ugly and not opitmized. But it works
zcat -f /nsm/bro/logs/*/http_eth1.* | bro-cut id.resp_h host | grep -f ${IP_FILE} | awk '{print $2}' | sort | uniq -c | sort -rn
rm ${IP_FILE}
The results will look something like this:
jma@onion-server:~$ /usr/local/bin/check-exe-downloads.sh
3501 13.107.4.50 au.download.windowsupdate.com
236 13.107.4.50 download.windowsupdate.com
74 8.254.242.142 download.windowsupdate.com
44 8.254.242.174 download.windowsupdate.com
19 8.254.242.142 au.download.windowsupdate.com
15 209.116.186.211 r8---sn-mv-qxoe.gvt1.com
2 8.254.242.174 au.download.windowsupdate.com
Distributed Onion Environment
In a distributed environment, the sensors forward the Snort/Suricata alerts back to the central server, but keep the Bro logs locally. That means that the SQL query is run from the central server and the Bro logs are grep’d via SSH to report back the findings. A few tweaks to the script will allow you to still gather the results, as seen below.
#!/bin/bash
# Update this list with any SIDs that indicate EXE or dll or other file
# downloads
#
# grep -Ei '(exe|dll).*download' /etc/nsm/rules/*.rules | grep -Po 'sid:[0-9]+' | cut -f2 -d: > ~/sids
#
# NOTE: This should really all be on one line, but is broken up here for display
ET_EXE_SIDS="2000419, 2018959, 2014518, 2014819, 2007998, 2008407,
2008408, 2008409, 2010011, 2010190, 2015537, 2015566, 2015567,
2016197, 2009568, 2009651, 2012610, 2014909, 2015688, 2016696,
2017057, 2017093, 2017297, 2017318, 2017672, 2017673, 2017674,
2017675, 2017676, 2017677, 2017679, 2017680, 2017681, 2017682,
2017683, 2018103, 2018104, 2018191, 2018556, 2018963, 2019714,
2020573, 2022050, 2022051, 2022053, 2022653, 2022884, 2023745,
2023817, 2000423, 2000424, 2000425, 2000427, 2010342, 2010447,
2010716, 2010869, 2011900, 2011919, 2011923, 2011980, 2011981,
2011983, 2011984, 2011985, 2011986, 2011990, 2013442, 2014181,
2015752, 2017678, 2017795, 2017961, 2022052, 2002068, 2008438,
2014514, 2014515, 2014516, 2014517, 2014518, 2014567, 2014819,
2016141, 2016538, 2016767, 2021216, 2023454, 2023455, 2023456,
2023457, 2023458, 2023459, 2023460, 2023461, 2023462, 2023463,
2023464, 2000371, 2001533, 2009091, 2014735, 2014810, 2018324,
2018333, 2018339, 2018367, 2000418, 2000419, 2000426, 2003179,
2003595, 2012523, 2012524, 2013037, 2014059, 2014313, 2014471,
2016856, 2018959, 2019240, 2020202, 2020914, 2009174, 2010050,
2010059, 2010741, 2011495, 2011496, 2011982, 2011989, 2011991,
2012208, 2012227, 2012389, 2013036, 2013291, 2013352, 2013441,
2013560, 2013770, 2013826, 2013827, 2014150, 2014230, 2014525,
2015547, 2015862, 2016029, 2016475, 2016844, 2017190, 2017583,
2017598, 2017617, 2017962, 2018241, 2018254, 2018395, 2018403,
2018464, 2018572, 2018581, 2018982, 2019103, 2020198, 2020199,
2020200, 2020201, 2021697, 2021774, 2021952, 2021954, 2022037,
2022207, 2022239, 2022482, 2022483, 2022640, 2022692, 2001048, 27982,
28000, 8737, 8738, 8740, 24501, 24791, 25140, 26040, 26041, 26043,
26526, 26534, 26891, 26962, 27005, 27069, 27082, 27083, 27084,
27936, 11192, 16313, 21173, 16425, 25513, 25514, 32947, 27255,
28806, 28945, 35737, 35738, 16096, 16670, 21425, 21554, 26880,
28983, 28984, 28985, 39856, 39857, 24259, 24260, 25782, 26470, 31487,
31488, 33941, 33942, 33943, 26257, 32951, 38033, 38034, 18648, 23209,
23210, 29501, 31091, 7129, 7849, 36454, 36455, 2067, 24520, 27862"
IP_FILE="${HOME}/ip"
if [ -e ${IP_FILE} ]; then
rm ${IP_FILE} 2&>/dev/null
fi
# Build the SELECT statement using the SIDs above
QUERY="SELECT INET_NTOA(src_ip) as sip FROM event WHERE signature_id IN (${ET_EXE_SIDS}) AND status="0" GROUP BY sip;"
# The grep command strips the results of the header line
# store the results in a file for grep to reference
# This is the first change to the remote version of the script. Since the
# output must be read into an environment variable, we send it through awk
# for additional formatting suitable to the environment.
sudo mysql --defaults-file=/etc/mysql/debian.cnf -Dsecurityonion_db -e "${QUERY}" | awk '/^[0-9]/ { ORS="|"; print $1 } END { ORS=""; print $0 }' > ${IP_FILE}
# The grep regex can't be read from a local file, since the grep process is
# actually being run remotely. Instead, read the results into the environment
# variable which can be passed on the command line to grep through the SSH
# session.
REGEX="(`cat ${IP_FILE}`)"
# This is ugly and not opitmized (at all!). But it works
# SSH to each sensor and run the grep command against the bro logs on the
# sensor
echo
echo "Sensor1 results"
ssh sensor1 "zcat -f /nsm/bro/logs/*/http* | /opt/bro/bin/bro-cut id.resp_h host | grep -E \"${REGEX}\"| awk '{print $2}' | sort | uniq -c | sort -rn"
echo
echo "Sensor2 results"
ssh sensor2 "zcat -f /nsm/bro/logs/*/http* | /opt/bro/bin/bro-cut id.resp_h host | grep -E \"${REGEX}\"| awk '{print $2}' | sort | uniq -c | sort -rn"
rm ${IP_FILE}
The results would now look something like this:
jma@onion-server:~$ /usr/local/bin/check-exe-downloads.sh
Sensor1 results
24224 208.111.171.148 d1.sophosupd.com
11781 208.111.171.148 d2.sophosupd.com
4706 208.111.171.148 dci.sophosupd.com
3702 13.107.4.50 au.download.windowsupdate.com
1614 208.111.171.148 d3.sophosupd.com
237 13.107.4.50 download.windowsupdate.com
61 8.253.104.30 download.windowsupdate.com
48 8.254.242.158 download.windowsupdate.com
27 38.100.7.132 00100d-1.l.windowsupdate.com
24 209.116.186.210 r7---sn-mv-qxoe.gvt1.com
13 208.111.170.216 000ee3-1.l.windowsupdate.com
13 208.111.168.89 00138b-1.l.windowsupdate.com
13 208.111.168.78 001b40-1.l.windowsupdate.com
13 208.111.168.72 001b40-1.l.windowsupdate.com
8 8.253.104.30 au.download.windowsupdate.com
2 69.164.19.161 000866-1.l.windowsupdate.com
2 38.100.7.163 001389-1.l.windowsupdate.com
2 208.111.170.203 000ed5-1.l.windowsupdate.com
Sensor2 results
6838 208.111.171.148 d1.sophosupd.com
3409 208.111.171.148 d2.sophosupd.com
2945 13.107.4.50 au.download.windowsupdate.com
1293 208.111.171.148 dci.sophosupd.com
429 208.111.171.148 d3.sophosupd.com
170 13.107.4.50 download.windowsupdate.com
86 208.111.161.190 00100d-1.l.windowsupdate.com
75 38.100.7.149 0015b0-1.l.windowsupdate.com
58 208.111.161.146 000793-1.l.windowsupdate.com
55 8.254.242.158 download.windowsupdate.com
35 8.253.104.30 download.windowsupdate.com
34 208.111.161.206 000793-1.l.windowsupdate.com
25 69.164.19.161 0007c2-1.l.windowsupdate.com
24 8.254.242.158 au.download.windowsupdate.com
24 208.111.161.206 000238-1.l.windowsupdate.com
24 208.111.161.154 0004d3-1.l.windowsupdate.com
23 69.28.159.143 00100d-1.l.windowsupdate.com
14 74.125.155.199 r1---sn-p5qs7n7z.gvt1.com
13 38.100.7.177 00100d-1.l.windowsupdate.com
13 208.111.168.72 000793-1.l.windowsupdate.com
10 69.28.159.200 001c17-1.l.windowsupdate.com
6 208.111.161.146 00100d-1.l.windowsupdate.com
4 208.111.161.138 000935-1.l.windowsupdate.com
3 38.100.7.132 000935-1.l.windowsupdate.com
2 8.253.104.30 au.download.windowsupdate.com
2 69.28.159.20 000935-1.l.windowsupdate.com
2 208.111.161.206 000ed5-1.l.windowsupdate.com
2 208.111.161.203 00246b-1.l.windowsupdate.com
1 208.111.161.206 00115a-1.l.windowsupdate.com
It’s not guaranteed that malicious files won’t be downloaded from “normal-looking” sites, but this is a good quick sanity check for anything that might stick out from the norm.