Like eat our vegetables. Wait. No, that’s actually good for you.

Whatever.

You get the idea.

Encrypting a file with 7zip is simple. Use the -p option:

~$ 7z a archive.7z -pSTRONG_PASSWORD_THAT_IS_REALLY_LONG_HERE! folder/

7-Zip [64] 17.04 : Copyright (c) 1999-2021 Igor Pavlov : 2017-08-28
p7zip Version 17.04 (locale=utf8,Utf16=on,HugeFiles=on,64 bits,8 CPUs x64)

Scanning the drive:
1 folder, 4 files, 0 bytes

Creating archive: archive.7z

Items to compress: 5

    
Files read from disk: 0
Archive size: 186 bytes (1 KiB)
Everything is Ok

I have always loved 7zip’s attempt to make me feel better.

“Don’t worry. Everything is Ok.” :thumbs_up: :grinning_squinting_face:

Note: There is no space between the option and the password. That’s an oddity with 7z that I just don’t see with many other CLI tools.

You may notice, however, that when you go to list the contents of the archive, the folder and file names are listed without prompting for the password!

~$ 7z l archive.7z                   

7-Zip [64] 17.04 : Copyright (c) 1999-2021 Igor Pavlov : 2017-08-28
p7zip Version 17.04 (locale=utf8,Utf16=on,HugeFiles=on,64 bits,8 CPUs x64)

Scanning the drive for archives:
1 file, 186 bytes (1 KiB)

Listing archive: archive.7z

--
Path = archive.7z
Type = 7z
Physical Size = 186
Headers Size = 186
Solid = -
Blocks = 0

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2023-08-23 16:01:10 D....            0            0  folder
2023-08-23 16:01:10 ....A            0            0  folder/bank_info.xlsx
2023-08-23 16:01:10 ....A            0            0  folder/classified.txt
2023-08-23 16:01:10 ....A            0            0  folder/file1.txt
2023-08-23 16:01:10 ....A            0            0  folder/file2.txt
------------------- ----- ------------ ------------  ------------------------
2023-08-23 16:01:10                  0            0  4 files, 1 folders

Showing folder and file names without requiring the password is probably not acceptable for encrypted files. (Then again, ideally we wouldn’t be using 7zip for encryption, but….I already made that point.)

To encrypt the folder and file names, too, we need to use an option that only shows up on the man page of 7zip and not in the CLI help output:

-mhe=on|off
   7z format only : enables or disables archive header encryption (Default : off)

So all of this only works on the 7z format vs traditional zip, etc…

With this, our example changes to:

7z a archive.7z -mhe=on -pSTRONG_PASSWORD_THAT_IS_REALLY_LONG_HERE! folder/

The output is the same until the file contents are listed:

7z l archive.7z                           

7-Zip [64] 17.04 : Copyright (c) 1999-2021 Igor Pavlov : 2017-08-28
p7zip Version 17.04 (locale=utf8,Utf16=on,HugeFiles=on,64 bits,8 CPUs x64)

Scanning the drive for archives:
1 file, 221 bytes (1 KiB)

Listing archive: archive.7z


Enter password (will not be echoed):
--
Path = archive.7z
Type = 7z
Physical Size = 221
Headers Size = 221
Solid = -
Blocks = 0

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2023-08-23 16:01:10 D....            0            0  folder
2023-08-23 16:01:10 ....A            0            0  folder/bank_info.xlsx
2023-08-23 16:01:10 ....A            0            0  folder/classified.txt
2023-08-23 16:01:10 ....A            0            0  folder/file1.txt
2023-08-23 16:01:10 ....A            0            0  folder/file2.txt
------------------- ----- ------------ ------------  ------------------------
2023-08-23 16:01:10                  0            0  4 files, 1 folders

One other thing to note: encrypting a file in this manner will leave the password in your shell history file. I will leave as an exercise to the reader the following search: “HISTCONTROL=ignorespace” 😏

Rather than login to the web GUI of my OPNSense firewall, I created an alias to check for updates from the CLI.

Add the following line to .bash_aliases or directly to .bashrc:

alias check-update='sudo /usr/local/opnsense/scripts/firmware/launcher.sh check'

There are a few small, but interesting, additional steps required for the alias to work over a “non-interactive” SSH session.

First, include the -t parameter with ssh to force the use of a psuedo-tty (so sudo will work). And instead of calling the alias directly, invoke bash and pass the alias as the command. The combined flags of -ic to bash make all that work together. (Credit goes to Cyberciti.biz for explaining these options to me.)

All of the above results in allowing me to run this from my terminal:

me@macbookpro:~ ssh -t opnsense /usr/local/bin/bash -ic 'check-update'

resulting in output similar to the following:

Updating OPNsense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: .......... done
Processing entries: .......... done
OPNsense repository update completed. 822 packages processed.
All repositories are up to date.
Checking integrity... done (0 conflicting)
Your packages are up to date.
Checking for upgrades (0 candidates): . done
Processing candidates (0 candidates): . done
Checking integrity... done (0 conflicting)
Your packages are up to date.

I am re-evaluating Firefox as my everyday browser (moving back from Brave) and was reminded of the Multi-account container functionality today. That is a killer feature, IMHO. I’d been running several instances of Brave under different profiles to keep things separate, but this is so much easier (and better with RAM, maybe?).

However, I was also reminded of an incredibly annoying and frustrating side-effect of Firefox’s now-default privacy feature to prevent browser/device fingerprinting: the timezone for my device is mis-reported to websites, so the displayed timestamps of messages are off by hours. It’s incredibly disorienting to look at your email inbox and see messages arrive from the future!

Spoofing the client’s timezone is only one part of Mozilla’s attempt to prevent browser fingerprinting. Fingerprinting is (probably?) a genuine privacy/tracking concern, but I just can’t handle incorrect timestamps in my emails and other web-based message clients (e.g. Facebook, Twitter, Mastodon, GroupMe, and whatever else). Mozilla reports this is a feature under “heavy development” right now and even they recommend disabling it if it’s causing problems.

So as a reminder/shortcut to myself, here is how to fix it.

The Fix

1. Navigate to about:config and reassure Firefox that you “know what you’re doing”

2. Then set the privacy.resistFingerprinting option to False.

3. Reload the impacted tabs (or just restart the whole browser) and you’re done.

I spent some time looking to see if there was an option to limit my changes to only not meddle with the timezone rather than taking a sledge-hammer to the whole thing. But it doesn’t look like it’s configurable at the moment. I even tried to whitelist specific sites. Granted it is entirely possible I didn’t implement the whitelist correctly or simply muffed up the test some other way.

Hopefully, Mozilla will introduce flexibility with this privacy feature in the future.

macOS supports the use of Touch ID for sudo authentication. Yes, I still love to live in the terminal, so this is an everyday, multiple times a day thing for me.

Fortunately, it’s a very quick edit to enable the functionality and it saves a TON of time (vs entering your password each time you run the sudo command - or worse, setting the NOPASSWD option.)

Edit PAM

Initial file contents

# sudo: auth account password session
auth       sufficient     pam_smartcard.so
auth       required       pam_opendirectory.so
account    required       pam_permit.so
password   required       pam_deny.so
session    required       pam_permit.so

me@macbookpro:~ # sudo vi /etc/pam.d/sudo

Insert as the first uncommented line: auth sufficient pam_tid.so

Resulting file contents

# sudo: auth account password session
auth       sufficient     pam_tid.so
auth       sufficient     pam_smartcard.so
auth       required       pam_opendirectory.so
account    required       pam_permit.so
password   required       pam_deny.so
session    required       pam_permit.so

Force save (:w!) then quit (:q)

And you’re done!

But there is a minor caveat….

In macOS, PAM cofiguration files are protected by System Integrity Protection (SIP) so they will be overwritten with every OS upgrade - by design.

It’s a bit annyoing and maybe some day I’ll dig into better options. But for now it’s a quick and generally painless edit after an OS update.

However, with the migration away from a VM, I lost the ability to create a snapshot of the system prior to an upgrade - just in case something went wrong I could easily roll back.

Recently I learned about BSD Boot Environments and the bectl utility and how they can help fill that gap!

1. Login via SSH and start a shell session

2. List any current/previoius boot environment (BE) snapshots

root@opnsense-fw:~ # bectl list
BE     Active Mountpoint Space Created
22.7.2 -      -          1.04G 2021-09-05 10:57
22.7.4 -      -          388M  2022-09-02 09:28
22.7.5 NR     /          4.09G 2022-10-05 08:58

In the “Active” column, the “N” stands for “Now” and “R” is “Reboot”, showing which BE is active now vs after the next reboot.

3. Create and activate a new BE

root@opnsense-fw:~ # bectl create [new_version]
root@opnsense-fw:~ # bectl activate [new_version]

4. Reboot and upgrade
5. Validate BE; Remove oldest BE

root@opnsense-fw:~ # bectl list
BE     Active Mountpoint Space Created
22.7.2 -      -          1.04G 2021-09-05 10:57
22.7.4 -      -          896M  2022-09-02 09:28
22.7.5 NR     /          4.84G 2022-10-05 08:58
root@opnsense-fw:~ # bectl destroy 22.7.2
root@opnsense-fw:~ # bectl list
BE     Active Mountpoint Space Created
22.7.4 -      -          1.13G 2022-09-02 09:28
22.7.5 NR     /          3.80G 2022-10-05 08:58

I expect to write more about what I learned in that exercise, but I wanted to start with the problem of directory index files, since this solution didn’t come up often in my search for solutions. There are lots of pages out there talking about hosting static pages on S3 and CloudFront has a Default root object setting for each Distribution. But I discovered that only works for the root of the distribution - not any subdirectories.

The simplest solution I found (there may be others) was the use of CloudFront functions.

What is it?

As described in the AWS documentation:

CloudFront Functions is ideal for lightweight, short-running functions for use cases like the following:

• Cache key normalization – You can transform HTTP request attributes (headers, query strings, cookies, even the URL path) to create an optimal cache key, which can improve your cache hit ratio.
• Header manipulation – You can insert, modify, or delete HTTP headers in the request or response. For example, you can add a True-Client-IP header to every request.
• URL redirects or rewrites – You can redirect viewers to other pages based on information in the request, or rewrite all requests from one path to another.
• Request authorization – You can validate hashed authorization tokens, such as JSON web tokens (JWT), by inspecting authorization headers or other request metadata.

CloudFront Functions support two types of event:

• Viewer request
• Viewer response

CloudFront also supports lambda functions via Labmda@Edge executions. Lambda@Edge supports the viewer request and response events as well as Origin requests and response events. But a directory index redirect is simple enough that Lambda@Edge wasn’t needed.

Of note, CloudFront Functions cannot include response body content. Hence, the Lambda@Edge options.

CloudFront functions are written in Javascript. I am no Javascript expert (to put it mildly), but fortunately this particular use case was included in the CloudFront documentation as an example! (Unfortunately, this use case didn’t come up much in my support queries, so I was glad to have finally found it!)

CloudFront Function

Name: folder_index

function handler(event) {
    // Copied from https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/example-function-add-index.html
    // Retrieved on 2021-12-18
    var request = event.request;
    var uri = request.uri;

    // Check whether the URI is missing a file name.
    if (uri.endsWith('/')) {
        request.uri += 'index.html';
    }
    // Check whether the URI is missing a file extension.
    else if (!uri.includes('.')) {
        request.uri += '/index.html';
    }

    return request;
}

Once this function is saved, it must be tested and then published before attaching it to the CloudFront distribution.

Testing and Deployment

Testing is very straightforward in the CloudFront Functions page:

Press “Test Function” and look for results simliar to below.

Once you’ve validated the function returns the expected results, it can be associated to the Distribution from the Functions page, or back on the Distribution Behaviors tab:

Once that was done, all the pretty URL functionality worked perfectly.

Cost

As with all things AWS, CloudFront is (generally) a pay-as-you-go service. But GeekCabinet is a fairly low traffic site, so I should be able to stay well within the Free Tier for CloudFront:

• 1 TB of data transfer out
• 10,000,000 HTTP or HTTPS Requests
• 2,000,000 CloudFront Function Invocations

But even if GeekCabinet had more traffic, pricing on CloudFront and Functions wouldn’t break the bank.

You are charged for the total number of invocations across all your functions. CloudFront Functions counts an invocation each time it starts executing in response to a CloudFront event globally. Invocation pricing is $0.10 per 1 million invocations ($0.0000001 per invocation).

Conclusion

I’m pleased with what I’ve learned and CloudFront Functions ended up being a simple solution to my problem. And a solution I was suprised wasn’t highlighted in other forums, since I’d imagine this is a common problem.

Disclaimer

Personally, I’m biased toward not starting in Security, but starting in System Administration or Network Administration or development or….

The reason is that if you haven’t actually spent time running an IT shop, you won’t really have an idea of what you’re defending or the balance between keeping things running smoothly and patching (for example).

When a Security professional “demands” that a patch be applied, we should understand the implications of those demands and be able to work with the Operations teams to ensure the patches are tested properly - again, just as an example.

So with that disclaimer out of the way…

Home Lab

First of all, you need a home lab. You need a place to test out new tools and technologies - WITH PERMISSION - and a home lab is probably the best place to do that.

VirtualBox is a great, free way to start a home lab. VMWare is fine if you have some money to start with, but I still use VirtualBox on a regular basis, and I have a license for VMWare, too.

Both virtualziation platforms allow you to create internal, virtual networks where you can see the interaction between attack tools and defense tools, and get familiar with how to configure both.

What should go in your lab? I’m glad you asked!

Below, I’ve started a list of tools and technologies I’ve used in production and/or learning environments that will give you plenty to explore. Many of these resources are free and/or Open Source, though obviously not all of them. These are also not listed in order of preference or performance. For most of these, I’ve included a brief description or thoughts around the tool, but I’m sure this will be a bit of a living list.

And who knows? Maybe I’ll get around to writing up some more in-depth info on these as time goes on. :-)

Tools / Resources

*Last updated: 2021-09-09

Virtualization

• VirtualBox
  • Great for running virtual machines on your laptop / small server
• Proxmox
  • Run a virtual data center in a single server
  • This is really the next step up for home labs and suitable for larger labs (or even home production use)
• VMWare
  • The defacto standard for local virtual machines. Not free.
• AWS
  • um…..duh.
  • Not as expensive as you think it might be, once you get the hang of it.
• Vultr and/or DigitalOcean
  • Good places to run a virtual server for $5 / month
  • Perfect if you don’t want to dive into AWS just yet.

Network Routing and Firewalls

• iptables
  • The famous Linux firewall.
  • There are a ton of resources and tutorials out there on this, but I included the one for CentOS.
  • It’s not necessarily the best, but it’s a start.
• OPNSense
  • BSD-based Open Source firewall.
  • Includes a web GUI for management.
  • Fork of PFSense, but probably my pick between the two.
• PFSense
  • Still a solid, supported Web GUI-based BSD firewall.
• MicroTik
  • Not free, but their Cloud Hosted Router (CHR) is excellently priced.
  • There are also cheap(er) hardware devices that can be purchased to run this production-supported solution.
• VyOS
  • Command line Linux-based router and firewall. Love this software, but it doesn’t have a frequent update cycle.

Utilities

• grep / Regular Expressions
  • Learn regular expressions. You’ll need them.
  • And you’ll hate them.
• Let’s Encrypt
  • Free, programmable TLS certificates.
  • No reason not to have a cert on any website you create.
• nmap - Network Mapper
  • The defacto standard for network scanners.
  • The NSE (Nmap Scripting Engine) has added quite a bit to this and you can do some basic vulnerability scanning here.
  • My go-to for initial local network reconnaissance.
• PowerShell
  • I strongly prefer Linux or Mac over Windows, but the rest of the world runs Windows and you need to know how to manage and secure these boxen.
  • PowerShell is actually pretty awesome and definitely worth learning!
• python
  • My personal preference for script languages. There are others, and you can choose them if you like. I promise it won’t hurt my feelings.
• Shell scripts (bash / zsh)
  • Know your way around a command line.
  • Waay too many tutorials out there right now. Google it.
• tcpdump
  • The original tool to capture network traffic. Still awesome.
• Wireshark
  • The simplest way to examine the packets you captured with tcpdump.
  • Of course, Wireshark can capture on it’s own (use tshark on the command line).

Forensics

• SANS SIFTKit
  • Created and maintained by SANS, the SIFT Kit is an excellent forensics-machine-in-a-box and includes a ton of fantastic tools and documentation.
• Sleuthkit
  • Included in the SIFT Kit and is a basic tool in file system forensics.

Configuration Management

• Boxstarter (Built on Chocolatey)
  • “Repeatable, reboot resilient windows environment installations made easy using Chocolatey packages” (from their site).
  • Honestly, I haven’t mastered this one yet, but it looks really interesting.
• Chocolatey (Windows)
  • Like yum or apt-get for Windows.
  • I even created a scheduled task on my family’s computers to automatically patch all their software on a regular basis.
• Docker
  • Allows you to use pre-packaged applications, alone or together with other “containers” to quickly build and manage applications in a standard fashion.
• Github
  • Version control.
  • Super important to maintain versions of your scripts, configuration, notes, whatever.
  • You will love this. And hate it.
• Gitlab
  • See above.
  • Gitlab also supports private repositories on their free plan (something you have to pay for also now available on Github, since Microsoft bought them out).
  • Using a private repo, you could keep your configurations on all your workstations in a version control system.
  • All you need to do to make your new computer setup just like your old one is to clone the repo!
  • Gitlab (and Github) also support hosting static sites - Geek Cabinet is was hosted here on Gitlab, for example. Gitlab even supports custom certifcates (e.g. Let’s Encrypt) where Github does not (yet?).
• Puppet
  • Using Puppet (or Salt Stack - see below), you can ensure servers and workstations are configured consistently and correctly from one central place.
  • This changes the way you administer your network.
  • Runs on Ruby, but you don’t need to know that (or the language) in order to use it.
• Salt Stack
  • See Above
  • Python-based version of Puppet.

Offense

• Kali Linux
  • Network Penetration Testing in a box.
  • When you’re ready to start practice attacking systems (with permission), this is a great place to start.
• Tenable Nessus Home Scanner
  • Maybe should’ve even put this in the “Defense” section…¯\_(ツ)_/¯
  • Free for home use, a quality network vulnerability scanner to find and (lightly) test the systems in your lab
  • Or make sure your IDS is working…
  • Again, use only with permission. If you own what you’re scanning, you’re fine. If you don’t - ask first. Or just skip it.

Defense

• Zeek IDS
• OSSEC HIDS
• Security Onion
  • Network Intrusion Detection System in a box.
  • Includes Snort, Suricata, Zeek, OSSEC, + …..
• Snort
• Suricata

Learning

• Local Community Meetups
• SANS Reading Room
• SANS Facilitator Program

All of this was prior to the latest versions of Security Onion which now run inside docker instances. I’ve not yet looked to see how this would be replicated there. But I’m leaving this up for historical purposes.

Another tool in the arsenal of Security Onion is OSSEC, a “scalable, multi-platform, open source Host-based Intrusion Detection System (HIDS).” OSSEC examines log and alert events and correlates them against pre-built (or custom) rules and sends alerts as configured. When installed on the Security Onion server, OSSEC alerts are logged in the sguil database and managed alongside alerts from the network IDS.

An important aspect of any NIDS/HIDS is the ability to tune out expected traffic, to keep noise to a minimum so that real alerts can be triaged and mitigated. Whitelisting the IP address of known and approved vulnerability scanners can reduce the analyst’s workload tremendously. For the Snort/Suricata NIDS, those IPs can be whitelisted through local rules or even completely ignored via BPF filtering (something I’ll cover another time).

Since OSSEC generates alerts by reviewing log entries, the NIDS filters won’t apply to OSSEC. Instead, a reference list must be created/maintained for OSSEC to utilize.

There is a <white_list> directive in the ossec.conf file, but that directive only controls the Active Response module of OSSEC. Hosts identified in the <white_list> tags will not be blocked by any Active Response script, but alerts may still be generated for activity from those sources.

The procedure documented here instead prevents alerts from being generated by specified IPs.

Creating the list file

Create a file to store the key-value paired IPs and labels in the /var/ossec/lists directory. For my example, I will use approved_scanners_list as the file name.

Reference lists in OSSEC must be entered in the format:

key1:value
key2:value
key3:value

Each key must be unique, but the values can be duplicated.

Individual hosts can be entered like so:

10.5.0.99:KaliScanner
10.15.0.120:NessusScanner

IP ranges that break on classes are also supported, like this:

192.168.23.:SecurityToolNetwork
172.12.:InternalNetwork

CIDR entries are not supported, so ranges that do not end on a class border must be either combined with a class entry or listed individually.

Update OSSEC config to reference the list

Once the list has been created, ossec.conf must be updated to reference the list:

<rules>
   [...trimmed output...]
   <list>lists/approved_scanners_list</list>
</rules>

When the list has been created and referenced in the ossec.conf file, the ossec-makelists command must be run to compile the list into a readable format for OSSEC.

The output below shows the result when a reference list needs to be updated:

root@so-onion:/var/ossec/bin# ./ossec-makelists
 * File lists/ip_list.cdb does not need to be compiled
 * File lists/approved_scanners_list.cdb needs to be updated

The output below shows the result when no lists need to be updated:

root@so-onion:/var/ossec/bin# ./ossec-makelists
 * File lists/ip_list.cdb does not need to be compiled
 * File lists/approved_scanners_list.cdb does not need to be compiled

The OSSEC service should be restarted after adding a completely new list to the configuration, but does not need to be restarted if an existing list has been updated. Restarting the OSSEC service is as simple as:

root@so-onion:/var/ossec/bin# ./ossec-control restart

Referencing a list file in the rules

Whitelisting an IP address in OSSEC is accomplished by setting the alert level to “0” for a given rule. We can piggy back on previous rules that have triggered, so that not all traffic from an IP is whitelisted, but only that traffic that matches expected behavior.

A review of the /var/ossec/rules/web_rules.xml file shows that SID group 31100 indicates web access logs:

<group name="web,accesslog,">
   <rule id="31100" level="0">
    <category>web-log</category>
      <description>Access log messages grouped.</description>
   </rule>

Using that information, we can whitelist the scanners for all web-based traffic like so:

<rule id="131100" level="0">
   <if_sid>31100</if_sid>
   <list field="srcip" lookup="address_match_key">lists/approved_scanners_list</list>
   <description>Quiet: Approved security scanners</description>
</rule>

More than one SID group can be referenced in the same rule by separating them with a comma:

<rule id="131100" level="0">
   <!--
      5706    [OSSEC] SSH insecure connection attempt (scan).
      5710    [OSSEC] Attempt to login using a non-existent user
      5712    [OSSEC] SSHD brute force trying to get access to the system.
      31100   web_rules.xml Alerts
   -->
   <if_sid>5706, 5710, 5712, 31100</if_sid>
   <list field="srcip" lookup="address_match_key">lists/approved_scanners_list</list>
   <description>Quiet: Approved security scanners</description>
</rule>

Further Reading

For more information regarding OSSEC reference lists, please see the full OSSEC documentation page.

OSSEC, in general, has been written about rather extensively in the SANS Reading Room, and my paper can be found there as well.

All of this was prior to the latest versions of Security Onion which now run inside docker instances. I’ve not yet looked to see how this would be replicated there. But I’m leaving this up for historical purposes.

As I’ve mentioned before, Security Onion is a fantastic network security-focused Linux distribution which can monitor your network and/or hosts for malicious activity.

The Onion can run Snort or Suricata as a network IDS, and it can also run bro Zeek alongside those traditional IDS engines to add another layer of intelligence. This article will highlight one way in which these two engines can be combined to quickly triage IDS alerts.

Snort and Suricata both make use of the Talos rule set, and can also use the Emerging Threats (aka ET) rule set. Both of those IDS rulesets include rules to generate alerts when an executable file is downloaded. On a network of nearly any size, those rules can get to be …. noisy. I don’t want to remove the rules, since they can be valuable, but I do want to quickly determine if any of the executables downloaded might pose a risk.

One way to evaluate the risk a file might introduce is to look at where it was downloaded from. By default in the Onion, Snort and Suricata don’t show the server host information, but we can query the sguil DB and then take the resulting set and grep the bro logs for hostnames.

First, we need to generate the list of rule signature ID’s (or SIDs) that indicate the download of an exe or dll file:

jma@onion-server:~$ grep -Ei '(exe|dll).*download' /etc/nsm/rules/*.rules | grep -Po 'sid:[0-9]+' | cut -f2 -d: > ~/sids

I imagine there are more elegant ways to accomplish this, but the above works well enough for me. :-)

Someday, maybe I’ll incorporate the above command into the script below so that the entire process is fully automated. But for now, take the SIDs that are stored in the ~/sids file and update the ET_EXE_SIDS variable in the script. I’ve stored the script in the /usr/local/bin/ folder on the onion server and saved it as check-exe-downloads.sh.

Stand Alone Installation

Security Onion can be installed as a stand-alone system or as a distributed environment. We’ll start with the basic, simpler stand-alone installation. In this scenario, the bro logs are stored on the same server as the DB, which makes them easy to query.

Links to the full script can be found at the end of the article

#!/bin/bash

# Update this list with any SIDs that indicate EXE or dll
# or other file downloads
#
# grep -Ei '(exe|dll).*download' /etc/nsm/rules/*.rules | grep -Po 'sid:[0-9]+' | cut -f2 -d: > ~/sids
#

# NOTE: This should really all be on one line, but is broken up here for display

ET_EXE_SIDS="2000419, 2018959, 2014518, 2014819, 2007998, 2008407,
2008408, 2008409, 2010011, 2010190, 2015537, 2015566, 2015567,
2016197, 2009568, 2009651, 2012610, 2014909, 2015688, 2016696,
2017057, 2017093, 2017297, 2017318, 2017672, 2017673, 2017674,
2017675, 2017676, 2017677, 2017679, 2017680, 2017681, 2017682,
2017683, 2018103, 2018104, 2018191, 2018556, 2018963, 2019714,
2020573, 2022050, 2022051, 2022053, 2022653, 2022884, 2023745,
2023817, 2000423, 2000424, 2000425, 2000427, 2010342, 2010447,
2010716, 2010869, 2011900, 2011919, 2011923, 2011980, 2011981,
2011983, 2011984, 2011985, 2011986, 2011990, 2013442, 2014181,
2015752, 2017678, 2017795, 2017961, 2022052, 2002068, 2008438,
2014514, 2014515, 2014516, 2014517, 2014518, 2014567, 2014819,
2016141, 2016538, 2016767, 2021216, 2023454, 2023455, 2023456,
2023457, 2023458, 2023459, 2023460, 2023461, 2023462, 2023463,
2023464, 2000371, 2001533, 2009091, 2014735, 2014810, 2018324,
2018333, 2018339, 2018367, 2000418, 2000419, 2000426, 2003179,
2003595, 2012523, 2012524, 2013037, 2014059, 2014313, 2014471,
2016856, 2018959, 2019240, 2020202, 2020914, 2009174, 2010050,
2010059, 2010741, 2011495, 2011496, 2011982, 2011989, 2011991,
2012208, 2012227, 2012389, 2013036, 2013291, 2013352, 2013441,
2013560, 2013770, 2013826, 2013827, 2014150, 2014230, 2014525,
2015547, 2015862, 2016029, 2016475, 2016844, 2017190, 2017583,
2017598, 2017617, 2017962, 2018241, 2018254, 2018395, 2018403,
2018464, 2018572, 2018581, 2018982, 2019103, 2020198, 2020199,
2020200, 2020201, 2021697, 2021774, 2021952, 2021954, 2022037,
2022207, 2022239, 2022482, 2022483, 2022640, 2022692, 2001048, 27982,
28000, 8737, 8738, 8740, 24501, 24791, 25140, 26040, 26041, 26043,
26526, 26534, 26891, 26962, 27005, 27069, 27082, 27083, 27084,
27936, 11192, 16313, 21173, 16425, 25513, 25514, 32947, 27255,
28806, 28945, 35737, 35738, 16096, 16670, 21425, 21554, 26880,
28983, 28984, 28985, 39856, 39857, 24259, 24260, 25782, 26470, 31487,
31488, 33941, 33942, 33943, 26257, 32951, 38033, 38034, 18648, 23209,
23210, 29501, 31091, 7129, 7849, 36454, 36455, 2067, 24520, 27862"

IP_FILE="${HOME}/ip"

if [ -e ${IP_FILE} ]; then
  rm ${IP_FILE} 2&>/dev/null
fi

# Build the SELECT statement using the SIDs above
QUERY="SELECT INET_NTOA(src_ip) as sip FROM event WHERE signature_id IN (${ET_EXE_SIDS}) AND status="0" GROUP BY sip;"

# The grep command strips the results of the header line 
# store the results in a file for grep to reference
mysql --defaults-file=/etc/mysql/debian.cnf -Dsecurityonion_db -e "${QUERY}" | grep -v sip > ${IP_FILE}

# This is ugly and not opitmized. But it works
zcat -f /nsm/bro/logs/*/http_eth1.* | bro-cut id.resp_h host | grep -f ${IP_FILE} | awk '{print $2}' | sort | uniq -c | sort -rn

rm ${IP_FILE}

The results will look something like this:

jma@onion-server:~$ /usr/local/bin/check-exe-downloads.sh
3501 13.107.4.50     au.download.windowsupdate.com
 236 13.107.4.50     download.windowsupdate.com
  74 8.254.242.142   download.windowsupdate.com
  44 8.254.242.174   download.windowsupdate.com
  19 8.254.242.142   au.download.windowsupdate.com
  15 209.116.186.211 r8---sn-mv-qxoe.gvt1.com
   2 8.254.242.174   au.download.windowsupdate.com

Distributed Onion Environment

In a distributed environment, the sensors forward the Snort/Suricata alerts back to the central server, but keep the Bro logs locally. That means that the SQL query is run from the central server and the Bro logs are grep’d via SSH to report back the findings. A few tweaks to the script will allow you to still gather the results, as seen below.

#!/bin/bash

# Update this list with any SIDs that indicate EXE or dll or other file
# downloads
#
# grep -Ei '(exe|dll).*download' /etc/nsm/rules/*.rules | grep -Po 'sid:[0-9]+' | cut -f2 -d: > ~/sids
#

# NOTE: This should really all be on one line, but is broken up here for display
ET_EXE_SIDS="2000419, 2018959, 2014518, 2014819, 2007998, 2008407,
2008408, 2008409, 2010011, 2010190, 2015537, 2015566, 2015567,
2016197, 2009568, 2009651, 2012610, 2014909, 2015688, 2016696,
2017057, 2017093, 2017297, 2017318, 2017672, 2017673, 2017674,
2017675, 2017676, 2017677, 2017679, 2017680, 2017681, 2017682,
2017683, 2018103, 2018104, 2018191, 2018556, 2018963, 2019714,
2020573, 2022050, 2022051, 2022053, 2022653, 2022884, 2023745,
2023817, 2000423, 2000424, 2000425, 2000427, 2010342, 2010447,
2010716, 2010869, 2011900, 2011919, 2011923, 2011980, 2011981,
2011983, 2011984, 2011985, 2011986, 2011990, 2013442, 2014181,
2015752, 2017678, 2017795, 2017961, 2022052, 2002068, 2008438,
2014514, 2014515, 2014516, 2014517, 2014518, 2014567, 2014819,
2016141, 2016538, 2016767, 2021216, 2023454, 2023455, 2023456,
2023457, 2023458, 2023459, 2023460, 2023461, 2023462, 2023463,
2023464, 2000371, 2001533, 2009091, 2014735, 2014810, 2018324,
2018333, 2018339, 2018367, 2000418, 2000419, 2000426, 2003179,
2003595, 2012523, 2012524, 2013037, 2014059, 2014313, 2014471,
2016856, 2018959, 2019240, 2020202, 2020914, 2009174, 2010050,
2010059, 2010741, 2011495, 2011496, 2011982, 2011989, 2011991,
2012208, 2012227, 2012389, 2013036, 2013291, 2013352, 2013441,
2013560, 2013770, 2013826, 2013827, 2014150, 2014230, 2014525,
2015547, 2015862, 2016029, 2016475, 2016844, 2017190, 2017583,
2017598, 2017617, 2017962, 2018241, 2018254, 2018395, 2018403,
2018464, 2018572, 2018581, 2018982, 2019103, 2020198, 2020199,
2020200, 2020201, 2021697, 2021774, 2021952, 2021954, 2022037,
2022207, 2022239, 2022482, 2022483, 2022640, 2022692, 2001048, 27982,
28000, 8737, 8738, 8740, 24501, 24791, 25140, 26040, 26041, 26043,
26526, 26534, 26891, 26962, 27005, 27069, 27082, 27083, 27084,
27936, 11192, 16313, 21173, 16425, 25513, 25514, 32947, 27255,
28806, 28945, 35737, 35738, 16096, 16670, 21425, 21554, 26880,
28983, 28984, 28985, 39856, 39857, 24259, 24260, 25782, 26470, 31487,
31488, 33941, 33942, 33943, 26257, 32951, 38033, 38034, 18648, 23209,
23210, 29501, 31091, 7129, 7849, 36454, 36455, 2067, 24520, 27862"

IP_FILE="${HOME}/ip"

if [ -e ${IP_FILE} ]; then
  rm ${IP_FILE} 2&>/dev/null
fi

# Build the SELECT statement using the SIDs above
QUERY="SELECT INET_NTOA(src_ip) as sip FROM event WHERE signature_id IN (${ET_EXE_SIDS}) AND status="0" GROUP BY sip;"

# The grep command strips the results of the header line 
# store the results in a file for grep to reference

# This is the first change to the remote version of the script. Since the
# output must be read into an environment variable, we send it through awk
# for additional formatting suitable to the environment.
sudo mysql --defaults-file=/etc/mysql/debian.cnf -Dsecurityonion_db -e "${QUERY}" | awk '/^[0-9]/ { ORS="|"; print $1 } END { ORS=""; print $0 }' > ${IP_FILE}

# The grep regex can't be read from a local file, since the grep process is
# actually being run remotely. Instead, read the results into the environment
# variable which can be passed on the command line to grep through the SSH
# session.
REGEX="(`cat ${IP_FILE}`)"

# This is ugly and not opitmized (at all!). But it works
# SSH to each sensor and run the grep command against the bro logs on the
# sensor
echo
echo "Sensor1 results"
ssh sensor1 "zcat -f /nsm/bro/logs/*/http* | /opt/bro/bin/bro-cut id.resp_h host | grep -E \"${REGEX}\"| awk '{print $2}' | sort | uniq -c | sort -rn"
echo
echo "Sensor2 results"
ssh sensor2 "zcat -f /nsm/bro/logs/*/http* | /opt/bro/bin/bro-cut id.resp_h host | grep -E \"${REGEX}\"| awk '{print $2}' | sort | uniq -c | sort -rn"

rm ${IP_FILE}

The results would now look something like this:

jma@onion-server:~$ /usr/local/bin/check-exe-downloads.sh

Sensor1 results
24224 208.111.171.148  d1.sophosupd.com
11781 208.111.171.148  d2.sophosupd.com
 4706 208.111.171.148  dci.sophosupd.com
 3702 13.107.4.50      au.download.windowsupdate.com
 1614 208.111.171.148  d3.sophosupd.com
  237 13.107.4.50      download.windowsupdate.com
  61 8.253.104.30      download.windowsupdate.com
   48 8.254.242.158    download.windowsupdate.com
   27 38.100.7.132     00100d-1.l.windowsupdate.com
   24 209.116.186.210  r7---sn-mv-qxoe.gvt1.com
   13 208.111.170.216  000ee3-1.l.windowsupdate.com
   13 208.111.168.89   00138b-1.l.windowsupdate.com
   13 208.111.168.78   001b40-1.l.windowsupdate.com
   13 208.111.168.72   001b40-1.l.windowsupdate.com
    8 8.253.104.30     au.download.windowsupdate.com
    2 69.164.19.161    000866-1.l.windowsupdate.com
    2 38.100.7.163     001389-1.l.windowsupdate.com
    2 208.111.170.203  000ed5-1.l.windowsupdate.com

Sensor2 results
6838 208.111.171.148  d1.sophosupd.com
3409 208.111.171.148  d2.sophosupd.com
2945 13.107.4.50      au.download.windowsupdate.com
1293 208.111.171.148  dci.sophosupd.com
 429 208.111.171.148  d3.sophosupd.com
 170 13.107.4.50      download.windowsupdate.com
  86 208.111.161.190  00100d-1.l.windowsupdate.com
  75 38.100.7.149     0015b0-1.l.windowsupdate.com
  58 208.111.161.146  000793-1.l.windowsupdate.com
  55 8.254.242.158    download.windowsupdate.com
  35 8.253.104.30     download.windowsupdate.com
  34 208.111.161.206  000793-1.l.windowsupdate.com
  25 69.164.19.161    0007c2-1.l.windowsupdate.com
  24 8.254.242.158    au.download.windowsupdate.com
  24 208.111.161.206  000238-1.l.windowsupdate.com
  24 208.111.161.154  0004d3-1.l.windowsupdate.com
  23 69.28.159.143    00100d-1.l.windowsupdate.com
  14 74.125.155.199   r1---sn-p5qs7n7z.gvt1.com
  13 38.100.7.177     00100d-1.l.windowsupdate.com
  13 208.111.168.72   000793-1.l.windowsupdate.com
  10 69.28.159.200    001c17-1.l.windowsupdate.com
   6 208.111.161.146  00100d-1.l.windowsupdate.com
   4 208.111.161.138  000935-1.l.windowsupdate.com
   3 38.100.7.132     000935-1.l.windowsupdate.com
   2 8.253.104.30     au.download.windowsupdate.com
   2 69.28.159.20     000935-1.l.windowsupdate.com
   2 208.111.161.206  000ed5-1.l.windowsupdate.com
   2 208.111.161.203  00246b-1.l.windowsupdate.com
   1 208.111.161.206  00115a-1.l.windowsupdate.com

It’s not guaranteed that malicious files won’t be downloaded from “normal-looking” sites, but this is a good quick sanity check for anything that might stick out from the norm.

Script Downloads

Standalone check-exe-downloads.sh

Distributed check-exe-downloads.sh

Circle is an extremely easy and simple to use home web filter, and one which is at least relatively effective with time controls, since I can hear my kids groan every time their alloted time has been spent.

Installation

Installation is super simple:

1. Download the phone app
2. Plugin the device
3. Use the phone app to tell Circle which network to pair with (wired or wireless)

Profiles

That’s really it. But unless you want to have the same Internet limitations as your kids (including time spent), you should setup User Profiles on the phone app and assign devices to those profiles. That’s also super easy within the app.

With those profiles, you can specify time limits or website categories that are permitted or denied per profile. But there’s not a lot of customization available here. Circle comes with several family friendly websites listed as “apps” that you can allow or deny per profile. You can also whitelist a given domain or URL per profile. Not much else there, really.

Unfortunately, any network activity counts as Internet time used. So even running Windows Updates takes up the time alloted to that profile. If you’re an advanced home user, and you have any sort of centralized management software in place, that will get blocked once the profile’s Internet time is up for the day.

Reporting

Ick.

You can see how much time a profile has spent on the Internet (today, this week, or this month) via the phone app. You can also see the list of recent connections. But there’s no good way to export that data.

Pros

• Installation
  • No agent required
• Excellent mobile notifications
  • When a new device shows up on your network
  • When a user reached his/her activity timelimit
• Works for wired and wireless devices seemlessly

Cons

• Must be managed via the phone app - no web interface available
• Filtering is done via ARP spoofing, so it’s VERY noisy on the network
• Customization of the filter is limited
• Detailed reporting is lacking
• Devices must belong to one user/profile
• ARP spoofing(?!)
• Infrequent device updates (has it updated at all since I received it?)
• Can be bypassed via static ARP entry on the host
• Can only manage one network segment (e.g. one wireless network)
  • This obviously only comes into play if you have more than one VLAN on your home network, which most people don’t have. But for geeks, this can be a problem.

Conclusion

Circle could be a fine fit for your home, provided you don’t have a complicated home network, and don’t require detailed logging of activity.

I was looking through the various options for ncat and came across an option I hadn’t seen before: Broker mode. After reading through the examples, I learned that the --chat mode is really just a special mode of connection brokering.

In ncat, “brokering” a connection allows multiple connections to the same listening instance, and takes the input from one connection and send it as output to all the other connections. The ncat chat server adds labels to who said what to help keep the confusion down, but in every other way, it is simply a brokered connection.

This mode of operation could come in handy when two systems cannot connect directly to each other, perhaps due to NAT’ing or other firewall restrictions. Utilizing a third system can help move data efficiently between systems.

Use of the --broker flag implies the -l flag:

    ubahmapk@laptop:~ > ncat --broker -v 
    Ncat: Version 7.25SVN ( https://nmap.org/ncat )
    Ncat: Listening on :::31337
    Ncat: Listening on 0.0.0.0:31337

If no listening port is specified, the highly popular port of 31337 is used as the default.

Using broker mode, we can transfer files from one host to another (or multiple!) through an intermediate host. First set up the listening broker on port 443 (since that was the only other port open on my firewall):

    ubahmapk@broker-host:~$ sudo ncat -l -v --broker -p 443
    Ncat: Version 7.25SVN ( https://nmap.org/ncat )
    Ncat: Listening on :::443
    Ncat: Listening on 0.0.0.0:443

Get ready to receive the file:

    ubahmapk@receiving-host:~$ ncat --recv-only 192.168.2.99 443 | tee outputfile

By piping the output through tee, we can see the data as it comes in while also saving it to the file.

We can see on the broker that the receiving host connected:

    Ncat: Connection from 192.168.1.6.
    Ncat: Connection from 192.168.1.6:40868.

Now that everything’s setup, we can send our precious data! Just for fun, I’m using the HTTP headers from the SANS Internet Storm Center website:

    ubahmapk@sending-host:~$ cat headers
    HTTP/1.1 200 OK
    Date: Wed, 09 Nov 2016 21:42:25 GMT
    Content-Type: text/html; charset=UTF-8
    Connection: keep-alive
    Server: nc -6 -l 80
    X-Content-Type-Options: nosniff
    X-XSS-Protection: 1; mode=block
    X-HeyJason: DEV522 rocks
    Permitted-Cross-Domain-Policies: none
    Public-Key-Pins: pin-sha256="yPygUehClEHV8rvCx38NfHm7VA6IQN65Jkp2W4czLl4=";pin-sha256="ujF0jpR9Bfbrlj2annpMzkLl1DZr1y80DAqNkoAw9IA=";pin-sha256="oBPvhtvElQwtqQAFCzmHX7iaOgvmPfYDRPEMP5zVMBQ=";pin-sha256="Ofki57ad70COg0ke3x80cbJ62Tt3c/f3skTimJdpnTw="; pin-sha256="kS2Xhr6z68kfHmJMGRYw5Gept+QuLctgg7RQaHUfYHc="; max-age=2592000; report-uri="https://sansisc.report-uri.io/r/default/hpkp/enforce"
    Strict-Transport-Security: max-age=63072000
    X-Do-Not-Hack: 18 U.S.C. Parag 1030
    X-Frame-Options: SAMEORIGIN
    Content-Security-Policy-Report-Only: default-src 'self' www.sans.org; report-uri https://sansisc.report-uri.io/r/default/csp/reportOnly; script-src 'unsafe-inline' 'unsafe-eval' 'self'; style-src 'unsafe-inline' 'self'; frame-src www.sans.org
    Cache-Control: max-age=0, public
    Expires: Wed, 09 Nov 2016 21:42:25 GMT
    X-IPv6-Geekiness: FALSE

So, let’s send the file:

    ubahmapk@sending-host:~$ ncat --send-only 192.168.2.99 443 < headers

The broker console confirms the sending host’s connection:

    Ncat: Connection from 10.1.0.4.
    Ncat: Connection from 10.1.0.4:59048.

and the resulting file is stored in ‘outfile’.

Broker mode is certainly not the only way to send a file, and probably not even the most common. But it is certainly interesting. Not to mention the fact that it is always good to know all the available options. :-)

I originally posted this entry on our family’s Wordpress blog, back in 2015. I decided to update and repost the content here, since it’s still relevant and it fits better here anyway…

miniLock is no longer actively maintained and is no longer recommended as an encrypton option.

There are lots of different ways to encrypt a file and today I want to cover one of the more obscure, and I believe more unique methods.

miniLock is was a Chrome app and relies on the combination of an email address and a strong passphrase as the key. This is both a positive and a negative.

That’s great because it means you can encrypt and decrypt files on a Chromebook, or any other machine without storing the private key locally. It’s bad because most people don’t have good passwords – much less passphrases.

Still, it’s a simple way to encrypt files, and I like to have options.

At the risk of making the process more complicated than it really is, I took some screenshots and walked through the installation and encryption process below.

Installation

Step 1: Install

Installation is super simple. Visit minilock.io and click the link to the Chrome App Store

Click “Add to Chrome”

Step 2: Launch

Go to the installed Apps in Chrome and click on the light blue padlock

Step 3: Create key

This is done by entering your email address and a strong passphrase.

Encryption

Step 4: Select file

Your miniLock ID is shown at the bottom of this window. This key can be posted or sent anywhere – and should be if you want others to be able to send you encrypted files with miniLock. I currently have even had my ID in my Twitter bio line - and in fact, the entire key ID fits in a single tweet.

My miniLock ID is UXTFUyqcoM3spXNMtqpZGqqNzjZyZsGiQQwezVH1UYNR5.

Now you can either drag the file you want to encrypt, or click the big square box to find your file.

Step 5: Encrypt

Paste in the miniLock IDs of the file recipients. I strongly recommend keeping these in a text file or email or Evernote or something similar. Again, they are not sensitive, so it really doesn’t matter where you keep them. A file in Dropbox would work just fine. :-)

You can add more miniLock IDs if you need to.

Step 7: Save encrypted file

Once complete, you can download the encrypted file by clicking on the down arrow. This “.minilock” file can be attached to an email or sent via Dropbox or put on a USB stick or stored and distributed anywhere, really.

Decryption

Decrypting a “.minilock” file is just as easy. Simply drag and drop the file onto the window from Step 4. The file will decrypt (assuming it was encrypted to your miniLock ID) and you can download and save the original file.

Conclusion

That’s really all there is to it.

I believe the greatest weakness and strength of the entire miniLock system is the keys. (Of course, that should be said of any good encryption system - and this solution hasn’t been elevated to that status yet.)

It’s a strength because the private key isn’t stored anywhere - it’s generated by the login process. It’s a weakness because there isn’t a good way to store the keys for others. Which means key distribution with miniLock has the same problem as PGP. Except there isn’t a way to sign the keys of others, so there’s no good way to establish trust, making the miniLock less ideal than PGP.

I still prefer PGP (or GnuPG) for encryption, but it’s good to have alternatives to work with. So I consider this just another option.

Why would you want to limit access to an ncat listener?

If you’re competing in a Capture the Flag event and you’ve managed to establish an ncat listener, it would be no good to take a host you worked so hard to gain and gice it away to the other competitors.

Worse still if it was not a Capture the Flag event but a live pentest, in which you opened a hole to a live shell console to real attackers while simply conducting your test!

Connections to the ncat session can be restricted via the use of the following flags:

• --allow
• --deny
• --allowfile
• --denyfile

These largely work as you might expect. Per the ncat man page, each of these options support the standard host specification as used by Nmap.

    ubahmapk@laptop:~ > ncat -v --chat -l -p 8888 --deny 192.168.0.30
    Ncat: Version 7.12 ( https://nmap.org/ncat )
    Ncat: Listening on :::8888
    Ncat: Listening on 0.0.0.0:8888
    Ncat: Connection from 192.168.0.30 on file descriptor 5.
    Ncat: Connection from 192.168.0.30:64554.
    Ncat: New connection denied: not allowed

When the client attempts to connect, the only message received is:

    C:\USERS\ubahmapk> ncat 192.168.0.10 8888
    close: No error

Which actually differs from when the port isn’t open:

    C:\USERS\ubahmapk> ncat 192.168.0.10 8888
    Ncat: No connection could be made because the target machine actively refused it. .

(The extra period at the end is not a typo on my part. It’s really included in the output…)

Obviously, in most cases permitting or restricting a single IP is not so helpful. So you can also use the --allowfile or --denyfile options. The files specified here should contain one entry (either IP, hostname, or CIDR range) per line.

Combining all the techniques we’ve seen so far, we can very quickly and easily set up a private, encrypted chat server for use in….well doing almost anything you want. It could be used as a poor man’s red team dumping ground, so that all members of the team have access to all the accounts and hosts compromised so far, for example.

    ncat -v --ssl --allowfile allowed-ips -l -p 8443 --chat

I used port 8443 here to help the encrypted chat traffic possibly blend in with other potential traffic.

We still have a few more options to cover in the coming weeks, but we’re off to a great start! :-)

Start it up

Start the server using the -l and --chat flags:

    ubahmapk@laptop:~ > ncat --chat -l -p 8888

Note that you will not see any of the chat traffic in this window, but if you add the -v flag, you will at least see the connections as they are established:

    ubahmapk@laptop:~ > ncat -v -l --chat -p 8888
    Ncat: Version 7.12 ( https://nmap.org/ncat )
    Ncat: Listening on :::8888
    Ncat: Listening on 0.0.0.0:8888

Since this server console does not actively participate in the conversation, adding the -v flag does not send the verbose info to the participants.

Open up a new terminal window and make a standard ncat (or even a traditional nc) connection to the chat server:

    ubahmapk@laptop:~ > ncat 192.168.0.10 8888
    <announce> 192.168.0.10 is connected as <user5>.
    <announce> already connected: nobody.

ncat informs you of the previously connected users, in this case there were no others. Since this is an ncat session, when new users connect and the previously existing connections are announced, you will see your own connection listed as new uers arrive:

    <announce> 192.168.0.20 is connected as <user6>.
    <announce> already connected: 192.168.0.10 as <user5>.

ncat reports the connections on the server console thusly:

    Ncat: Connection from 192.168.0.10 on file descriptor 5.
    Ncat: Connection from 192.168.0.10:57850.
    Ncat: Connection from 192.168.0.20 on file descriptor 6.
    Ncat: Connection from 192.168.0.20:83853.

As participants chat, the content is prepended by the user ID so everyone knows who said what:

    <user7> It's me, it's me, it's Ernest T!

And lastly, departures from the chat room are also announced:

    <announce> <user7> is disconnected.

Encryption

As described above, all of this traffic would traverse the network in plain text by default. But, as you may remember, ncat can establish SSL connections, too!

    ubahmapk@laptop:~ > ncat -v -l --ssl --chat -p 8888
    Ncat: Version 7.12 ( https://nmap.org/ncat )
    Ncat: Generating a temporary 1024-bit RSA key. Use --ssl-key and --ssl-cert to use a permanent one.
    Ncat: SHA-1 fingerprint: FB24 84E8 D3F2 F77D DB1B 9C8B 00A4 7C89 E5D0 4A69
    Ncat: Listening on :::8888
    Ncat: Listening on 0.0.0.0:8888

Note that ncat will generate a temporary SSL certificate unless you specify a key and cert to use.

Client connections would just need to include the --ssl flag and everything else works the same as before:

    Ncat: Connection from 192.168.0.10 on file descriptor 5.
    Ncat: Connection from 192.168.0.10:58874.
    Ncat: Connection from 192.168.0.20 on file descriptor 6.
    Ncat: Connection from 192.168.0.20:85876

Access Control

But what if you don’t want just anyone to be able to connect to this chat server? Well, it turns out that ncat also supports IP restrictions, which can be applied to listening chat servers. But we’ll save that for another day. Or, you can check out ncat Access Control here.

ncat is full of really slick new features, but the one I will cover here is the ability to do all the wonderful things nc could do, but over an SSL connection. (Yes, yes, yes: I should call it a TLS connection instead, but since the ncat documentation still refers to it as “SSL”, I will do the same here.)

To establish an encrypted ncat session, simply pass the “–ssl” option to ncat, along with the hostname or IP and destination port, like you would with any other connection:

ubahmapk@laptop:~ > echo -e "GET / HTTP/1.1\r\nHost: \
ubahmapk.github.io\r\nUser-Agent: ncat\r\nAccept: */*\r\nReferer: \
https://www.google.com/?gws_rd=ssl#q=ncat+ssl\r\n\r\n" \ 
| ncat --ssl -v ubahmapk.github.io 443
Ncat: Version 6.49SVN ( http://nmap.org/ncat )
Ncat: SSL connection to 23.235.44.133:443. Fastly, Inc.
Ncat: SHA-1 fingerprint: 2199 1384 6372 1713 B9ED 0E8F 00A5 9B73 0DD0 5658
HTTP/1.1 200 OK
Server: GitHub.com
Content-Type: text/html; charset=utf-8
Last-Modified: Mon, 21 Mar 2016 02:14:16 GMT
Access-Control-Allow-Origin: *
Expires: Mon, 21 Mar 2016 03:42:50 GMT
Cache-Control: max-age=600
X-GitHub-Request-Id: 17EB2C2C:38FF:984CD29:56EF6B60
Content-Length: 12733
Accept-Ranges: bytes
Date: Mon, 21 Mar 2016 03:32:50 GMT
Via: 1.1 varnish
Age: 0
Connection: keep-alive
X-Served-By: cache-dfw1826-DFW
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1458531170.944139,VS0,VE49
Vary: Accept-Encoding
X-Fastly-Request-ID: f9a65b919e4649f5ef6f24397ecdf953fee840dd

[trimmed output]

The ‘-v’ option above caused ncat to include the three lines at the top beginning with “Ncat: “, which confirm the version running, along with the SSL connection connection information. Adding three ‘-v’ options would add information like this:

 NCAT DEBUG: Using system default trusted CA certificates and those in /usr/local/share/ncat/ca-bundle.crt.
 Ncat: Subject: C=US, O=DigiCert Inc, OU=www.digicert.com, CN=DigiCert High Assurance EV Root CA
 Ncat: Issuer: C=US, O=DigiCert Inc, OU=www.digicert.com, CN=DigiCert High Assurance EV Root CA
 Ncat: SHA-1 fingerprint: 5FB7 EE06 33E2 59DB AD0C 4C9A E6D3 8F1A 61C7 DC25
 Ncat: Subject: C=US, O=DigiCert Inc, OU=www.digicert.com, CN=DigiCert SHA2 High Assurance Server CA
 Ncat: Issuer: C=US, O=DigiCert Inc, OU=www.digicert.com, CN=DigiCert High Assurance EV Root CA
 Ncat: SHA-1 fingerprint: A031 C467 82E6 E6C6 62C2 C87C 76DA 9AA6 2CCA BD8E
 Ncat: Subject: C=US, ST=California, L=San Francisco, O=Fastly, Inc., CN=www.github.com
 Ncat: Issuer: C=US, O=DigiCert Inc, OU=www.digicert.com, CN=DigiCert SHA2 High Assurance Server CA
 Ncat: SHA-1 fingerprint: 2199 1384 6372 1713 B9ED 0E8F 00A5 9B73 0DD0 5658
 NCAT DEBUG: Checking certificate DNS name "www.github.com" against "ubahmapk.github.io".
 NCAT DEBUG: Checking certificate DNS name "*.github.com" against "ubahmapk.github.io".
 NCAT DEBUG: Checking certificate DNS name "github.com" against "ubahmapk.github.io".
 NCAT DEBUG: Checking certificate DNS name "*.github.io" against "ubahmapk.github.io".
 Ncat: SSL connection to 23.235.40.133:443. Fastly, Inc.
 Ncat: SHA-1 fingerprint: 2199 1384 6372 1713 B9ED 0E8F 00A5 9B73 0DD0 5658

But it also includes a great deal of other details from the libnsock library and can greatly clutter up the output.

Inbound connections with ncat can also utilize the ssl option:

ncat --ssl -l -p 443 -e /bin/bash

(The above is a terrible thing to run. Do NOT do that…)

The code snippet below shows the options used to specify public and private SSL keys; certificate validation behavior and which file should be used to validate certificates; and which SSL ciphers to accept (or reject):

--ssl-cert             Specify SSL certificate file (PEM) for listening
--ssl-key              Specify SSL private key (PEM) for listening
--ssl-verify           Verify trust and domain name of certificates
--ssl-trustfile        PEM file containing trusted SSL certificates
--ssl-ciphers          Cipherlist containing SSL ciphers to use

If you needed to make SSL connections with the traditional nc client, you could either create an stunnel connection or utilize the openssl s_client command as a ncat client:

ubahmapk@laptop:~ > openssl s_client -host ubahmapk.github.io -port 443 
CONNECTED(00000003)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert High Assurance EV Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 High Assurance Server CA
verify return:1
depth=0 C = US, ST = California, L = San Francisco, O = "Fastly, Inc.", CN = www.github.com
verify return:1
---
Certificate chain
0 s:/C=US/ST=California/L=San Francisco/O=Fastly, Inc./CN=www.github.com
i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA
---
Server certificate

[output trimmed]

But this method isn’t nearly as clean, and doesn’t allow for all the other functionality of netcat!

In fact, I’ll probably start a series on the different ncat options just for fun… :-)