trinitor/DetectionLab
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Merge pull request #149 from clong/suricata_and_ip_fixes

Browse Source 
Multiple fixes, additions [ci skip]
This commit is contained in:
Chris Long
2018-09-07 22:16:22 -07:00
committed by GitHub
parent ef84579aa7 6ad4dcf2b4
commit 053205a025
15 changed files with 280 additions and 241 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
Packer/scripts/debloat-windows.ps1
View File

@@ -8,21 +8,27 @@ if ($env:PACKER_BUILDER_TYPE -And $($env:PACKER_BUILDER_TYPE).startsWith("hyperv
(New-Object System.Net.WebClient).DownloadFile($url, "$env:TEMP\debloat.zip") (New-Object System.Net.WebClient).DownloadFile($url, "$env:TEMP\debloat.zip")
Expand-Archive -Path $env:TEMP\debloat.zip -DestinationPath $env:TEMP -Force Expand-Archive -Path $env:TEMP\debloat.zip -DestinationPath $env:TEMP -Force
#Write-Host Disable scheduled tasks # Disable Windows Defender
#. $env:TEMP\Debloat-Windows-10-master\utils\disable-scheduled-tasks.ps1
#Write-Host Block telemetry
#. $env:TEMP\Debloat-Windows-10-master\scripts\block-telemetry.ps1
#Write-Host Disable services
#. $env:TEMP\Debloat-Windows-10-master\scripts\disable-services.ps1
Write-host Disable Windows Defender Write-host Disable Windows Defender
#. $env:TEMP\Debloat-Windows-10-master\scripts\disable-windows-defender.ps1 $os = (gwmi win32_operatingsystem).caption
if ($os -like "*Windows 10*") {
set-MpPreference -DisableRealtimeMonitoring $true
} else {
Uninstall-WindowsFeature Windows-Defender-Features Uninstall-WindowsFeature Windows-Defender-Features
}
# Optimize Windows Update
Write-host Optimize Windows Update Write-host Optimize Windows Update
. $env:TEMP\Debloat-Windows-10-master\scripts\optimize-windows-update.ps1 . $env:TEMP\Debloat-Windows-10-master\scripts\optimize-windows-update.ps1
#Write-host Disable Windows Update Write-host Disable Windows Update
#Set-Service wuauserv -StartupType Disabled Set-Service wuauserv -StartupType Disabled
#Write-Host Remove OneDrive
#. $env:TEMP\Debloat-Windows-10-master\scripts\remove-onedrive.ps1 # Turn off shutdown event tracking
if ( -Not (Test-Path 'registry::HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT\Reliability'))
{
New-Item -Path 'registry::HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT' -Name Reliability -Force
}
Set-ItemProperty -Path 'registry::HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT\Reliability' -Name ShutdownReasonOn -Value 0
rm $env:TEMP\debloat.zip rm $env:TEMP\debloat.zip
rm -recurse $env:TEMP\Debloat-Windows-10-master rm -recurse $env:TEMP\Debloat-Windows-10-master

2
Packer/windows_2016.json
View File

@@ -32,6 +32,7 @@
"enable_secure_boot":true "enable_secure_boot":true
}, },
{ {
"vm_name":"WindowsServer2016",
"type": "vmware-iso", "type": "vmware-iso",
"communicator": "winrm", "communicator": "winrm",
"iso_url": "{{user `iso_url`}}", "iso_url": "{{user `iso_url`}}",
@@ -70,6 +71,7 @@
} }
}, },
{ {
"vm_name":"WindowsServer2016",
"type": "virtualbox-iso", "type": "virtualbox-iso",
"communicator": "winrm", "communicator": "winrm",
"iso_url": "{{user `iso_url`}}", "iso_url": "{{user `iso_url`}}",

14
README.md
View File

@@ -85,9 +85,9 @@ $ packer build --only=[vmware|virtualbox]-iso windows_2016.json
* Provision the WEF host and configure it as a Windows Event Collector in the Servers OU * Provision the WEF host and configure it as a Windows Event Collector in the Servers OU
* Provision the Win10 host and configure it as a computer in the Workstations OU * Provision the Win10 host and configure it as a computer in the Workstations OU
7. Navigate to https://192.168.38.5:8000 in a browser to access the Splunk instance on logger. Default credentials are admin:changeme (you will have the option to change them on the next screen) 7. Navigate to https://192.168.38.105:8000 in a browser to access the Splunk instance on logger. Default credentials are admin:changeme (you will have the option to change them on the next screen)
8. Navigate to https://192.168.38.5:8412 in a browser to access the Fleet server on logger. Default credentials are admin:admin123#. Query packs are pre-configured with queries from [palantir/osquery-configuration](https://github.com/palantir/osquery-configuration). 8. Navigate to https://192.168.38.105:8412 in a browser to access the Fleet server on logger. Default credentials are admin:admin123#. Query packs are pre-configured with queries from [palantir/osquery-configuration](https://github.com/palantir/osquery-configuration).
9. Navigate to https://192.168.38.5:8888 in a browser to access the Caldera server on logger. Default credentials are admin:caldera. 9. Navigate to https://192.168.38.105:8888 in a browser to access the Caldera server on logger. Default credentials are admin:caldera.
## Basic Vagrant Usage ## Basic Vagrant Usage
Vagrant commands must be run from the "Vagrant" folder. Vagrant commands must be run from the "Vagrant" folder.
@@ -108,10 +108,10 @@ Vagrant commands must be run from the "Vagrant" folder.
## Lab Information ## Lab Information
* Domain Name: windomain.local * Domain Name: windomain.local
* Admininstrator login: vagrant:vagrant * Admininstrator login: vagrant:vagrant
* Fleet login: https://192.168.38.5:8412 - admin:admin123# * Fleet login: https://192.168.38.105:8412 - admin:admin123#
* Splunk login: https://192.168.38.5:8000 - admin:changeme * Splunk login: https://192.168.38.105:8000 - admin:changeme
* Caldera login: https://192.168.38.5:8888 - admin:caldera * Caldera login: https://192.168.38.105:8888 - admin:caldera
* MS ATA login: https://192.168.38.3 - wef\vagrant:vagrant * MS ATA login: https://192.168.38.103 - wef\vagrant:vagrant
## Lab Hosts ## Lab Hosts
* DC - Windows 2016 Domain Controller * DC - Windows 2016 Domain Controller

17
Vagrant/Vagrantfile vendored
View File

@@ -4,7 +4,7 @@ Vagrant.configure("2") do |config|
cfg.vm.box = "bento/ubuntu-16.04" cfg.vm.box = "bento/ubuntu-16.04"
cfg.vm.hostname = "logger" cfg.vm.hostname = "logger"
config.vm.provision :shell, path: "bootstrap.sh" config.vm.provision :shell, path: "bootstrap.sh"
cfg.vm.network :private_network, ip: "192.168.38.5", gateway: "192.168.38.1", dns: "8.8.8.8" cfg.vm.network :private_network, ip: "192.168.38.105", gateway: "192.168.38.1", dns: "8.8.8.8"
cfg.vm.provider "vmware_fusion" do |v, override| cfg.vm.provider "vmware_fusion" do |v, override|
v.vmx["displayname"] = "logger" v.vmx["displayname"] = "logger"
@@ -45,9 +45,9 @@ Vagrant.configure("2") do |config|
cfg.winrm.basic_auth_only = true cfg.winrm.basic_auth_only = true
cfg.winrm.timeout = 300 cfg.winrm.timeout = 300
cfg.winrm.retry_limit = 20 cfg.winrm.retry_limit = 20
cfg.vm.network :private_network, ip: "192.168.38.2", gateway: "192.168.38.1" cfg.vm.network :private_network, ip: "192.168.38.102", gateway: "192.168.38.1"
cfg.vm.provision "shell", path: "scripts/fix-second-network.ps1", privileged: false, args: "192.168.38.2" cfg.vm.provision "shell", path: "scripts/fix-second-network.ps1", privileged: false, args: "192.168.38.102"
cfg.vm.provision "shell", path: "scripts/provision.ps1", privileged: false cfg.vm.provision "shell", path: "scripts/provision.ps1", privileged: false
cfg.vm.provision "reload" cfg.vm.provision "reload"
cfg.vm.provision "shell", path: "scripts/provision.ps1", privileged: false cfg.vm.provision "shell", path: "scripts/provision.ps1", privileged: false
@@ -64,6 +64,7 @@ Vagrant.configure("2") do |config|
cfg.vm.provision "shell", path: "scripts/configure-powershelllogging.ps1", privileged: true cfg.vm.provision "shell", path: "scripts/configure-powershelllogging.ps1", privileged: true
cfg.vm.provision "shell", path: "scripts/configure-AuditingPolicyGPOs.ps1", privileged: true cfg.vm.provision "shell", path: "scripts/configure-AuditingPolicyGPOs.ps1", privileged: true
cfg.vm.provision "shell", path: "scripts/install-autorunstowineventlog.ps1", privileged: true cfg.vm.provision "shell", path: "scripts/install-autorunstowineventlog.ps1", privileged: true
cfg.vm.provision "shell", inline: 'wevtutil el | Foreach-Object {wevtutil cl "$_"}', privileged: true
cfg.vm.provision "shell", inline: "Set-SmbServerConfiguration -AuditSmb1Access $true -Force", privileged: true cfg.vm.provision "shell", inline: "Set-SmbServerConfiguration -AuditSmb1Access $true -Force", privileged: true
cfg.vm.provider "vmware_fusion" do |v, override| cfg.vm.provider "vmware_fusion" do |v, override|
@@ -102,14 +103,15 @@ Vagrant.configure("2") do |config|
cfg.winrm.basic_auth_only = true cfg.winrm.basic_auth_only = true
cfg.winrm.timeout = 300 cfg.winrm.timeout = 300
cfg.winrm.retry_limit = 20 cfg.winrm.retry_limit = 20
cfg.vm.network :private_network, ip: "192.168.38.3", gateway: "192.168.38.1", dns: "192.168.38.2" cfg.vm.network :private_network, ip: "192.168.38.103", gateway: "192.168.38.1", dns: "192.168.38.102"
cfg.vm.provision "shell", path: "scripts/fix-second-network.ps1", privileged: false, args: "-ip 192.168.38.3 -dns 192.168.38.2" cfg.vm.provision "shell", path: "scripts/fix-second-network.ps1", privileged: false, args: "-ip 192.168.38.103 -dns 192.168.38.102"
cfg.vm.provision "shell", path: "scripts/provision.ps1", privileged: false cfg.vm.provision "shell", path: "scripts/provision.ps1", privileged: false
cfg.vm.provision "reload" cfg.vm.provision "reload"
cfg.vm.provision "shell", path: "scripts/provision.ps1", privileged: false cfg.vm.provision "shell", path: "scripts/provision.ps1", privileged: false
cfg.vm.provision "shell", path: "scripts/download_palantir_wef.ps1", privileged: true cfg.vm.provision "shell", path: "scripts/download_palantir_wef.ps1", privileged: true
cfg.vm.provision "shell", path: "scripts/download_palantir_osquery.ps1", privileged: true cfg.vm.provision "shell", path: "scripts/download_palantir_osquery.ps1", privileged: true
cfg.vm.provision "shell", inline: 'wevtutil el | Foreach-Object {wevtutil cl "$_"}', privileged: true
cfg.vm.provision "shell", path: "scripts/install-wefsubscriptions.ps1", privileged: true cfg.vm.provision "shell", path: "scripts/install-wefsubscriptions.ps1", privileged: true
cfg.vm.provision "shell", path: "scripts/install-splunkuf.ps1", privileged: true cfg.vm.provision "shell", path: "scripts/install-splunkuf.ps1", privileged: true
cfg.vm.provision "shell", path: "scripts/install-windows_ta.ps1", privileged: true cfg.vm.provision "shell", path: "scripts/install-windows_ta.ps1", privileged: true
@@ -158,15 +160,16 @@ Vagrant.configure("2") do |config|
cfg.winrm.basic_auth_only = true cfg.winrm.basic_auth_only = true
cfg.winrm.timeout = 300 cfg.winrm.timeout = 300
cfg.winrm.retry_limit = 20 cfg.winrm.retry_limit = 20
cfg.vm.network :private_network, ip: "192.168.38.4", gateway: "192.168.38.1", dns: "192.168.38.2" cfg.vm.network :private_network, ip: "192.168.38.104", gateway: "192.168.38.1", dns: "192.168.38.102"
cfg.vm.provision "shell", path: "scripts/fix-second-network.ps1", privileged: false, args: "-ip 192.168.38.4 -dns 192.168.38.2" cfg.vm.provision "shell", path: "scripts/fix-second-network.ps1", privileged: false, args: "-ip 192.168.38.104 -dns 192.168.38.102"
cfg.vm.provision "shell", path: "scripts/provision.ps1", privileged: false cfg.vm.provision "shell", path: "scripts/provision.ps1", privileged: false
cfg.vm.provision "reload" cfg.vm.provision "reload"
cfg.vm.provision "shell", path: "scripts/provision.ps1", privileged: false cfg.vm.provision "shell", path: "scripts/provision.ps1", privileged: false
cfg.vm.provision "shell", path: "scripts/download_palantir_wef.ps1", privileged: true cfg.vm.provision "shell", path: "scripts/download_palantir_wef.ps1", privileged: true
cfg.vm.provision "shell", path: "scripts/download_palantir_osquery.ps1", privileged: true cfg.vm.provision "shell", path: "scripts/download_palantir_osquery.ps1", privileged: true
cfg.vm.provision "shell", path: "scripts/MakeWindows10GreatAgain.ps1", privileged: true cfg.vm.provision "shell", path: "scripts/MakeWindows10GreatAgain.ps1", privileged: true
cfg.vm.provision "shell", inline: 'wevtutil el | Foreach-Object {wevtutil cl "$_"}', privileged: true
cfg.vm.provision "shell", path: "scripts/install-splunkuf.ps1", privileged: true cfg.vm.provision "shell", path: "scripts/install-splunkuf.ps1", privileged: true
cfg.vm.provision "shell", path: "scripts/install-utilities.ps1", privileged: true cfg.vm.provision "shell", path: "scripts/install-utilities.ps1", privileged: true
cfg.vm.provision "shell", path: "scripts/install-osquery.ps1", privileged: true cfg.vm.provision "shell", path: "scripts/install-osquery.ps1", privileged: true

205
Vagrant/bootstrap.sh
View File

@@ -14,19 +14,22 @@ apt_install_prerequisites() {
fix_eth1_static_ip() { fix_eth1_static_ip() {
# There's a fun issue where dhclient keeps messing with eth1 despite the fact # There's a fun issue where dhclient keeps messing with eth1 despite the fact
# that eth1 has a static IP set. We workaround this by telling dhclient to leave it alone. # that eth1 has a static IP set. We workaround this by setting a static DHCP lease.
echo 'interface "eth1" {}' >> /etc/dhcp/dhclient.conf echo -e 'interface "eth1" {
systemctl restart networking.service send host-name = gethostname();
send dhcp-requested-address 192.168.38.105;
}' >> /etc/dhcp/dhclient.conf
service networking restart
# Fix eth1 if the IP isn't set correctly # Fix eth1 if the IP isn't set correctly
ETH1_IP=$(ifconfig eth1 | grep 'inet addr' | cut -d ':' -f 2 | cut -d ' ' -f 1) ETH1_IP=$(ifconfig eth1 | grep 'inet addr' | cut -d ':' -f 2 | cut -d ' ' -f 1)
if [ "$ETH1_IP" != "192.168.38.5" ]; then if [ "$ETH1_IP" != "192.168.38.105" ]; then
echo "Incorrect IP Address settings detected. Attempting to fix." echo "Incorrect IP Address settings detected. Attempting to fix."
ifdown eth1 ifdown eth1
ip addr flush dev eth1 ip addr flush dev eth1
ifup eth1 ifup eth1
ETH1_IP=$(ifconfig eth1 | grep 'inet addr' | cut -d ':' -f 2 | cut -d ' ' -f 1) ETH1_IP=$(ifconfig eth1 | grep 'inet addr' | cut -d ':' -f 2 | cut -d ' ' -f 1)
if [ "$ETH1_IP" == "192.168.38.5" ]; then if [ "$ETH1_IP" == "192.168.38.105" ]; then
echo "The static IP has been fixed and set to 192.168.38.5" echo "The static IP has been fixed and set to 192.168.38.105"
else else
echo "Failed to fix the broken static IP for eth1. Exiting because this will cause problems with other VMs." echo "Failed to fix the broken static IP for eth1. Exiting because this will cause problems with other VMs."
exit 1 exit 1
@@ -35,17 +38,17 @@ fix_eth1_static_ip() {
} }
install_python() { install_python() {
# Install Python 3.6.4 # Install Python 3.6.4
if ! which /usr/local/bin/python3.6 > /dev/null; then if ! which /usr/local/bin/python3.6 > /dev/null; then
echo "Installing Python v3.6.4..." echo "Installing Python v3.6.4..."
wget https://www.python.org/ftp/python/3.6.4/Python-3.6.4.tgz wget https://www.python.org/ftp/python/3.6.4/Python-3.6.4.tgz
tar -xvf Python-3.6.4.tgz tar -xvf Python-3.6.4.tgz
cd Python-3.6.4 || exit cd Python-3.6.4 || exit
./configure && make && make install ./configure && make && make install
cd /home/vagrant || exit cd /home/vagrant || exit
else else
echo "Python seems to be downloaded already.. Skipping." echo "Python seems to be downloaded already.. Skipping."
fi fi
} }
install_golang() { install_golang() {
@@ -62,6 +65,7 @@ install_golang() {
echo 'export GOROOT=/usr/local/go' >> /home/vagrant/.bashrc echo 'export GOROOT=/usr/local/go' >> /home/vagrant/.bashrc
echo 'export GOPATH=$HOME/.go' >> /root/.bashrc echo 'export GOPATH=$HOME/.go' >> /root/.bashrc
echo 'export GOROOT=/usr/local/go' >> /root/.bashrc echo 'export GOROOT=/usr/local/go' >> /root/.bashrc
echo 'export PATH=$PATH:/opt/splunk/bin' >> /root/.bashrc
source /root/.bashrc source /root/.bashrc
sudo update-alternatives --install "/usr/bin/go" "go" "/usr/local/go/bin/go" 0 sudo update-alternatives --install "/usr/bin/go" "go" "/usr/local/go/bin/go" 0
sudo update-alternatives --set go /usr/local/go/bin/go sudo update-alternatives --set go /usr/local/go/bin/go
@@ -92,11 +96,15 @@ install_splunk() {
/opt/splunk/bin/splunk add index suricata -auth 'admin:changeme' /opt/splunk/bin/splunk add index suricata -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_forwarder/splunk-add-on-for-microsoft-windows_500.tgz -auth 'admin:changeme' /opt/splunk/bin/splunk install app /vagrant/resources/splunk_forwarder/splunk-add-on-for-microsoft-windows_500.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/add-on-for-microsoft-sysmon_800.tgz -auth 'admin:changeme' /opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/add-on-for-microsoft-sysmon_800.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/asn-lookup-generator_012.tgz -auth 'admin:changeme'
# Add a Splunk TCP input on port 9997 # Add a Splunk TCP input on port 9997
echo -e "[splunktcp://9997]\nconnection_host = ip" > /opt/splunk/etc/apps/search/local/inputs.conf echo -e "[splunktcp://9997]\nconnection_host = ip" > /opt/splunk/etc/apps/search/local/inputs.conf
# Add props.conf and transforms.conf # Add props.conf and transforms.conf
cp /vagrant/resources/splunk_server/props.conf /opt/splunk/etc/apps/search/local/ cp /vagrant/resources/splunk_server/props.conf /opt/splunk/etc/apps/search/local/
cp /vagrant/resources/splunk_server/transforms.conf /opt/splunk/etc/apps/search/local/ cp /vagrant/resources/splunk_server/transforms.conf /opt/splunk/etc/apps/search/local/
cp /opt/splunk/etc/system/default/limits.conf /opt/splunk/etc/system/local/limits.conf
# Bump the memtable limits to allow for the ASN lookup table
sed -i .bak 's/max_memtable_bytes = 10000000/max_memtable_bytes = 30000000/g' /opt/splunk/etc/system/local/limits.conf
# Skip Splunk Tour and Change Password Dialog # Skip Splunk Tour and Change Password Dialog
touch /opt/splunk/etc/.ui_login touch /opt/splunk/etc/.ui_login
# Enable SSL Login for Splunk # Enable SSL Login for Splunk
@@ -105,6 +113,8 @@ install_splunk() {
# Reboot Splunk to make changes take effect # Reboot Splunk to make changes take effect
/opt/splunk/bin/splunk restart /opt/splunk/bin/splunk restart
/opt/splunk/bin/splunk enable boot-start /opt/splunk/bin/splunk enable boot-start
# Generate the ASN lookup table
/opt/splunk/bin/splunk search "|asngen | outputlookup asn" -auth 'admin:changeme'
fi fi
} }
@@ -158,30 +168,30 @@ import_osquery_config_into_fleet() {
cd /home/vagrant/osquery-configuration/Endpoints/Windows/ || exit cd /home/vagrant/osquery-configuration/Endpoints/Windows/ || exit
# Fleet requires you to login before importing packs # Fleet requires you to login before importing packs
# Login # Login
curl 'https://192.168.38.5:8412/api/v1/kolide/login' -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/login' -H 'authority: 192.168.38.5:8412' --data-binary '{"username":"admin","password":"admin123#"}' --compressed --insecure curl 'https://192.168.38.105:8412/api/v1/kolide/login' -H 'origin: https://192.168.38.105:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.105:8412/login' -H 'authority: 192.168.38.105:8412' --data-binary '{"username":"admin","password":"admin123#"}' --compressed --insecure
sleep 1 sleep 1
curl 'https://192.168.38.5:8412/setup' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'upgrade-insecure-requests: 1' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' -H 'authority: 192.168.38.5:8412' --compressed --insecure curl 'https://192.168.38.105:8412/setup' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'upgrade-insecure-requests: 1' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' -H 'authority: 192.168.38.105:8412' --compressed --insecure
sleep 1 sleep 1
# Setup organization name and email address # Setup organization name and email address
curl 'https://192.168.38.5:8412/api/v1/setup' -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/setup' -H 'authority: 192.168.38.5:8412' --data-binary '{"kolide_server_url":"https://192.168.38.5:8412","org_info":{"org_name":"detectionlab"},"admin":{"admin":true,"email":"example@example.com","password":"admin123#","password_confirmation":"admin123#","username":"admin"}}' --compressed --insecure curl 'https://192.168.38.105:8412/api/v1/setup' -H 'origin: https://192.168.38.105:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.105:8412/setup' -H 'authority: 192.168.38.105:8412' --data-binary '{"kolide_server_url":"https://192.168.38.105:8412","org_info":{"org_name":"detectionlab"},"admin":{"admin":true,"email":"example@example.com","password":"admin123#","password_confirmation":"admin123#","username":"admin"}}' --compressed --insecure
sleep 3 sleep 3
# Import all Windows configs # Import all Windows configs
/home/vagrant/configimporter/configimporter -host https://localhost:8412 -user 'admin' -config osquery_to_import.conf /home/vagrant/configimporter/configimporter -host https://localhost:8412 -user 'admin' -config osquery_to_import.conf
# Get auth token # Get auth token
TOKEN=$(curl 'https://192.168.38.5:8412/api/v1/kolide/login' -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/login' -H 'authority: 192.168.38.5:8412' --data-binary '{"username":"admin","password":"admin123#"}' --compressed --insecure | grep token | cut -d '"' -f 4) TOKEN=$(curl 'https://192.168.38.105:8412/api/v1/kolide/login' -H 'origin: https://192.168.38.105:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.105:8412/login' -H 'authority: 192.168.38.105:8412' --data-binary '{"username":"admin","password":"admin123#"}' --compressed --insecure | grep token | cut -d '"' -f 4)
# Set all packs to be targeted to Windows hosts # Set all packs to be targeted to Windows hosts
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/1' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/3/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure curl 'https://192.168.38.105:8412/api/v1/kolide/packs/1' -X PATCH -H 'origin: https://192.168.38.105:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.105:8412/packs/3/edit' -H 'authority: 192.168.38.105:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure
sleep 1 sleep 1
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/2' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/3/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure curl 'https://192.168.38.105:8412/api/v1/kolide/packs/2' -X PATCH -H 'origin: https://192.168.38.105:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.105:8412/packs/3/edit' -H 'authority: 192.168.38.105:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure
sleep 1 sleep 1
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/3' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/3/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure curl 'https://192.168.38.105:8412/api/v1/kolide/packs/3' -X PATCH -H 'origin: https://192.168.38.105:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.105:8412/packs/3/edit' -H 'authority: 192.168.38.105:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure
sleep 1 sleep 1
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/4' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/3/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure curl 'https://192.168.38.105:8412/api/v1/kolide/packs/4' -X PATCH -H 'origin: https://192.168.38.105:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.105:8412/packs/3/edit' -H 'authority: 192.168.38.105:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure
sleep 1 sleep 1
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/5' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/3/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure curl 'https://192.168.38.105:8412/api/v1/kolide/packs/5' -X PATCH -H 'origin: https://192.168.38.105:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.105:8412/packs/3/edit' -H 'authority: 192.168.38.105:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure
# Rename primary pack # Rename primary pack
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/5' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/5/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"name":"windows-pack"}' --compressed --insecure curl 'https://192.168.38.105:8412/api/v1/kolide/packs/5' -X PATCH -H 'origin: https://192.168.38.105:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.105:8412/packs/5/edit' -H 'authority: 192.168.38.105:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"name":"windows-pack"}' --compressed --insecure
# Add Splunk monitors for Fleet # Add Splunk monitors for Fleet
/opt/splunk/bin/splunk add monitor "/home/vagrant/kolide-quickstart/osquery_result" -index osquery -sourcetype 'osquery:json' -auth 'admin:changeme' /opt/splunk/bin/splunk add monitor "/home/vagrant/kolide-quickstart/osquery_result" -index osquery -sourcetype 'osquery:json' -auth 'admin:changeme'
/opt/splunk/bin/splunk add monitor "/home/vagrant/kolide-quickstart/osquery_status" -index osquery-status -sourcetype 'osquery:status' -auth 'admin:changeme' /opt/splunk/bin/splunk add monitor "/home/vagrant/kolide-quickstart/osquery_status" -index osquery-status -sourcetype 'osquery:status' -auth 'admin:changeme'
@@ -218,127 +228,144 @@ install_caldera() {
} }
install_bro() { install_bro() {
# environment variables # Environment variables
NODECFG=/opt/bro/etc/node.cfg NODECFG=/opt/bro/etc/node.cfg
SPLUNK_BRO_JSON=/opt/splunk/etc/apps/TA-bro_json SPLUNK_BRO_JSON=/opt/splunk/etc/apps/TA-bro_json
SPLUNK_BRO_MONITOR='monitor:///opt/bro/spool/manager' SPLUNK_BRO_MONITOR='monitor:///opt/bro/spool/manager'
SPLUNK_SURICATA_MONITOR='monitor:///var/log/suricata' SPLUNK_SURICATA_MONITOR='monitor:///var/log/suricata'
echo "deb http://download.opensuse.org/repositories/network:/bro/xUbuntu_16.04/ /" > /etc/apt/sources.list.d/bro.list echo "deb http://download.opensuse.org/repositories/network:/bro/xUbuntu_16.04/ /" > /etc/apt/sources.list.d/bro.list
curl -s http://download.opensuse.org/repositories/network:/bro/xUbuntu_16.04/Release.key |apt-key add - curl -s http://download.opensuse.org/repositories/network:/bro/xUbuntu_16.04/Release.key |apt-key add -
# update APT repositories
# Update APT repositories
apt-get -qq -ym update apt-get -qq -ym update
apt-get -qq -ym install \ # Install tools to build and configure bro
bro \ apt-get -qq -ym install bro crudini
crudini \ # Load bro scripts
# install tools to build and configure bro echo '
@load protocols/ftp/software
@load protocols/smtp/software
@load protocols/ssh/software
@load protocols/http/software
@load tuning/json-logs
@load policy/integration/collective-intel
@load policy/frameworks/intel/do_notice
@load frameworks/intel/seen
@load frameworks/intel/do_notice
@load frameworks/files/hash-all-files
@load policy/protocols/smb
@load policy/protocols/conn/vlan-logging
@load policy/protocols/conn/mac-logging
# load bro scripts redef Intel::read_files += {
cat<<EOF >> /opt/bro/share/bro/site/local.bro
@load protocols/ftp/software
@load protocols/smtp/software
@load protocols/ssh/software
@load protocols/http/software
@load tuning/json-logs
@load policy/integration/collective-intel
@load policy/frameworks/intel/do_notice
@load frameworks/intel/seen
@load frameworks/intel/do_notice
@load frameworks/files/hash-all-files
@load policy/protocols/smb
@load policy/protocols/conn/vlan-logging
@load policy/protocols/conn/mac-logging
redef Intel::read_files += {
"/opt/bro/etc/intel.dat" "/opt/bro/etc/intel.dat"
}; };
' >> /opt/bro/share/bro/site/local.bro
EOF # Configure Bro
# configure bro
crudini --del $NODECFG bro crudini --del $NODECFG bro
crudini --set $NODECFG manager type manager crudini --set $NODECFG manager type manager
crudini --set $NODECFG manager host localhost crudini --set $NODECFG manager host localhost
crudini --set $NODECFG proxy type proxy crudini --set $NODECFG proxy type proxy
crudini --set $NODECFG proxy host localhost crudini --set $NODECFG proxy host localhost
CPUS=$(lscpu -e |awk /yes/'{print $1'} |wc -l)
# setup $CPUS numbers of bro workers # Setup $CPUS numbers of bro workers
for i in eth1 crudini --set $NODECFG worker-eth1 type worker
do crudini --set $NODECFG worker-eth1 host localhost
crudini --set $NODECFG worker-$i type worker crudini --set $NODECFG worker-eth1 interface eth1
crudini --set $NODECFG worker-$i host localhost crudini --set $NODECFG worker-eth1 lb_method pf_ring
crudini --set $NODECFG worker-$i interface $i crudini --set $NODECFG worker-eth1 lb_procs "$(nproc)"
crudini --set $NODECFG worker-$i lb_method pf_ring
crudini --set $NODECFG worker-$i lb_procs $CPUS
done
# setup bro to run at boot # Setup bro to run at boot
cp /vagrant/resources/bro/bro.service /lib/systemd/system/bro.service cp /vagrant/resources/bro/bro.service /lib/systemd/system/bro.service
systemctl enable bro
systemctl start bro
for i in bro # Setup splunk TA to ingest bro and suricata data
do
systemctl enable $i
systemctl start $i
done
# setup splunk TA to ingest bro and suricata data
git clone https://github.com/jahshuah/splunk-ta-bro-json $SPLUNK_BRO_JSON git clone https://github.com/jahshuah/splunk-ta-bro-json $SPLUNK_BRO_JSON
mkdir -p $SPLUNK_BRO_JSON/local mkdir -p $SPLUNK_BRO_JSON/local
cp $SPLUNK_BRO_JSON/default/inputs.conf $SPLUNK_BRO_JSON/local/inputs.conf cp $SPLUNK_BRO_JSON/default/inputs.conf $SPLUNK_BRO_JSON/local/inputs.conf
crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_BRO_MONITOR index bro crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_BRO_MONITOR index bro
crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_BRO_MONITOR sourcetype json_bro crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_BRO_MONITOR sourcetype json_bro
crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_BRO_MONITOR whitelist '.*\.log$' crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_BRO_MONITOR whitelist '.*\.log$'
crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_BRO_MONITOR blacklist '.*(communication|stderr)\.log$' crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_BRO_MONITOR blacklist '.*(communication|stderr)\.log$'
crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_BRO_MONITOR disabled 0 crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_BRO_MONITOR disabled 0
crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_SURICATA_MONITOR index suricata crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_SURICATA_MONITOR index suricata
crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_SURICATA_MONITOR sourcetype json_suricata crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_SURICATA_MONITOR sourcetype json_suricata
crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_SURICATA_MONITOR whitelist 'eve.json' crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_SURICATA_MONITOR whitelist 'eve.json'
crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_SURICATA_MONITOR disabled 0 crudini --set $SPLUNK_BRO_JSON/local/inputs.conf $SPLUNK_SURICATA_MONITOR disabled 0
# ensure permissions are correct and restart splunk # Ensure permissions are correct and restart splunk
chown -R splunk $SPLUNK_BRO_JSON chown -R splunk $SPLUNK_BRO_JSON
/opt/splunk/bin/splunk restart /opt/splunk/bin/splunk restart
# Verify that Bro is running
if ! pgrep -f bro > /dev/null; then
echo "Bro attempted to start but is not running. Exiting"
exit 1
fi
} }
install_suricata() { install_suricata() {
# install yq to maniuplate the suricata.yaml inline # Run iwr -Uri testmyids.com -UserAgent "BlackSun" in Powershell to generate test alerts
# Install yq to maniuplate the suricata.yaml inline
/usr/bin/go get -u github.com/mikefarah/yq /usr/bin/go get -u github.com/mikefarah/yq
# install suricata # Install suricata
add-apt-repository -y ppa:oisf/suricata-stable add-apt-repository -y ppa:oisf/suricata-stable
apt-get -qq -y update && apt-get -qq -y install suricata crudini apt-get -qq -y update && apt-get -qq -y install suricata crudini
# install suricata-update # Install suricata-update
pip3.6 install --pre --upgrade suricata-update cd /home/vagrant || exit 1
# add DC_SERVERS variable to suricata.yaml in support et-open signatures git clone https://github.com/OISF/suricata-update.git
cd /home/vagrant/suricata-update || exit 1
python setup.py install
# Add DC_SERVERS variable to suricata.yaml in support et-open signatures
/root/go/bin/yq w -i /etc/suricata/suricata.yaml vars.address-groups.DC_SERVERS '$HOME_NET' /root/go/bin/yq w -i /etc/suricata/suricata.yaml vars.address-groups.DC_SERVERS '$HOME_NET'
sed -i '0,/^/s//%YAML 1.1\n---\n/' /etc/suricata/suricata.yaml
# It may make sense to store the suricata.yaml file as a resource file if this begins to become too complex
# Add more verbose alert logging
/root/go/bin/yq w -i /etc/suricata/suricata.yaml outputs.1.eve-log.types.0.alert.payload true
/root/go/bin/yq w -i /etc/suricata/suricata.yaml outputs.1.eve-log.types.0.alert.payload-buffer-size 4kb
/root/go/bin/yq w -i /etc/suricata/suricata.yaml outputs.1.eve-log.types.0.alert.payload-printable yes
/root/go/bin/yq w -i /etc/suricata/suricata.yaml outputs.1.eve-log.types.0.alert.packet yes
/root/go/bin/yq w -i /etc/suricata/suricata.yaml outputs.1.eve-log.types.0.alert.http yes
/root/go/bin/yq w -i /etc/suricata/suricata.yaml outputs.1.eve-log.types.0.alert.tls yes
/root/go/bin/yq w -i /etc/suricata/suricata.yaml outputs.1.eve-log.types.0.alert.ssh yes
/root/go/bin/yq w -i /etc/suricata/suricata.yaml outputs.1.eve-log.types.0.alert.smtp yes
# Turn off traffic flow logging (duplicative of Bro and wrecks Splunk trial license)
/root/go/bin/yq d -i /etc/suricata/suricata.yaml outputs.1.eve-log.types.1 # Remove HTTP
/root/go/bin/yq d -i /etc/suricata/suricata.yaml outputs.1.eve-log.types.1 # Remove DNS
/root/go/bin/yq d -i /etc/suricata/suricata.yaml outputs.1.eve-log.types.1 # Remove TLS
/root/go/bin/yq d -i /etc/suricata/suricata.yaml outputs.1.eve-log.types.2 # Remove SMTP
/root/go/bin/yq d -i /etc/suricata/suricata.yaml outputs.1.eve-log.types.2 # Remove SSH
/root/go/bin/yq d -i /etc/suricata/suricata.yaml outputs.1.eve-log.types.2 # Remove Stats
/root/go/bin/yq d -i /etc/suricata/suricata.yaml outputs.1.eve-log.types.2 # Remove Flow
# AF packet monitoring should be set to eth1
/root/go/bin/yq w -i /etc/suricata/suricata.yaml af-packet.0.interface eth1
crudini --set --format=sh /etc/default/suricata '' iface eth1 crudini --set --format=sh /etc/default/suricata '' iface eth1
# update suricata signature sources # update suricata signature sources
suricata-update update-sources suricata-update update-sources
# disable protocol decode as it is duplicative of bro # disable protocol decode as it is duplicative of bro
echo re:protocol-command-decode >> /etc/suricata/disable.conf echo re:protocol-command-decode >> /etc/suricata/disable.conf
# enable et-open and attackdetection sources # enable et-open and attackdetection sources
for i in et/open ptresearch/attackdetection suricata-update enable-source et/open
do suricata-update enable-source ptresearch/attackdetection
suricata-update enable-source $i # Add the YAML header to the top of the suricata config
echo "Adding the YAML header to /etc/suricata/suricata.yaml"
echo -e "%YAML 1.1\n---\n$(cat /etc/suricata/suricata.yaml)" > /etc/suricata/suricata.yaml
done # Update suricata and restart
# update suricata and restart
suricata-update suricata-update
systemctl restart suricata service suricata stop
service suricata start
# Verify that Suricata is running
if ! pgrep -f suricata > /dev/null; then
echo "Suricata attempted to start but is not running. Exiting"
exit 1
fi
} }
main() { main() {

BIN
Vagrant/resources/splunk_server/asn-lookup-generator_012.tgz Normal file
View File

Binary file not shown.

2
Vagrant/scripts/install-caldera-agent.ps1
View File

@@ -2,7 +2,7 @@
If (-not (Test-Path 'C:\Program Files\cagent\cagent.exe')) { If (-not (Test-Path 'C:\Program Files\cagent\cagent.exe')) {
# Add /etc/hosts entry # Add /etc/hosts entry
Add-Content "c:\windows\system32\drivers\etc\hosts" " 192.168.38.5 logger" Add-Content "c:\windows\system32\drivers\etc\hosts" " 192.168.38.105 logger"
# Make the directory # Make the directory
New-Item "c:\Program Files\cagent" -type directory New-Item "c:\Program Files\cagent" -type directory

2
Vagrant/scripts/install-osquery.ps1
View File

@@ -19,7 +19,7 @@ If (-not ($service)) {
### --- TLS CONFIG BEGINS --- ### --- TLS CONFIG BEGINS ---
### COMMENT ALL LINES BELOW UNTIL "TLS CONFIG ENDS" if using local configuration ### COMMENT ALL LINES BELOW UNTIL "TLS CONFIG ENDS" if using local configuration
## Add entry to hosts file for Kolide for SSL validation ## Add entry to hosts file for Kolide for SSL validation
Add-Content "c:\windows\system32\drivers\etc\hosts" " 192.168.38.5 kolide" Add-Content "c:\windows\system32\drivers\etc\hosts" " 192.168.38.105 kolide"
## Add kolide secret and avoid BOM ## Add kolide secret and avoid BOM
$Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False $Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False
[System.IO.File]::WriteAllLines("c:\ProgramData\osquery\kolide_secret.txt", "enrollmentsecret", $Utf8NoBomEncoding) [System.IO.File]::WriteAllLines("c:\ProgramData\osquery\kolide_secret.txt", "enrollmentsecret", $Utf8NoBomEncoding)

2
Vagrant/scripts/install-splunkuf.ps1
View File

@@ -6,7 +6,7 @@ If (-not (Test-Path "C:\Program Files\SplunkUniversalForwarder\bin\splunk.exe"))
Write-Host "Installing & Starting Splunk" Write-Host "Installing & Starting Splunk"
(New-Object System.Net.WebClient).DownloadFile('https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=windows&version=7.1.0&product=universalforwarder&filename=splunkforwarder-7.1.0-2e75b3406c5b-x64-release.msi&wget=true', $msiFile) (New-Object System.Net.WebClient).DownloadFile('https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=windows&version=7.1.0&product=universalforwarder&filename=splunkforwarder-7.1.0-2e75b3406c5b-x64-release.msi&wget=true', $msiFile)
Start-Process -FilePath "c:\windows\system32\msiexec.exe" -ArgumentList '/i', "$msiFile", 'RECEIVING_INDEXER="192.168.38.5:9997" WINEVENTLOG_SEC_ENABLE=1 WINEVENTLOG_SYS_ENABLE=1 WINEVENTLOG_APP_ENABLE=1 AGREETOLICENSE=Yes SERVICESTARTTYPE=1 LAUNCHSPLUNK=1 SPLUNKPASSWORD=changeme /quiet' -Wait Start-Process -FilePath "c:\windows\system32\msiexec.exe" -ArgumentList '/i', "$msiFile", 'RECEIVING_INDEXER="192.168.38.105:9997" WINEVENTLOG_SEC_ENABLE=1 WINEVENTLOG_SYS_ENABLE=1 WINEVENTLOG_APP_ENABLE=1 AGREETOLICENSE=Yes SERVICESTARTTYPE=1 LAUNCHSPLUNK=1 SPLUNKPASSWORD=changeme /quiet' -Wait
} Else { } Else {
Write-Host "Splunk is already installed. Moving on." Write-Host "Splunk is already installed. Moving on."
} }

2
Vagrant/scripts/join-domain.ps1
View File

@@ -4,7 +4,7 @@
Write-Host 'Join the domain' Write-Host 'Join the domain'
Write-Host "First, set DNS to DC to join the domain" Write-Host "First, set DNS to DC to join the domain"
$newDNSServers = "192.168.38.2" $newDNSServers = "192.168.38.102"
$adapters = Get-WmiObject Win32_NetworkAdapterConfiguration | Where-Object {$_.IPAddress -match "192.168.38."} $adapters = Get-WmiObject Win32_NetworkAdapterConfiguration | Where-Object {$_.IPAddress -match "192.168.38."}
$adapters | ForEach-Object {$_.SetDNSServerSearchOrder($newDNSServers)} $adapters | ForEach-Object {$_.SetDNSServerSearchOrder($newDNSServers)}

2
Vagrant/scripts/provision.ps1
View File

@@ -26,7 +26,7 @@ if ($env:COMPUTERNAME -imatch 'vagrant') {
} }
if ($env:COMPUTERNAME -imatch 'dc') { if ($env:COMPUTERNAME -imatch 'dc') {
. c:\vagrant\scripts\create-domain.ps1 192.168.38.2 . c:\vagrant\scripts\create-domain.ps1 192.168.38.102
} else { } else {
. c:\vagrant\scripts\join-domain.ps1 . c:\vagrant\scripts\join-domain.ps1
} }

8
build.ps1
View File

@@ -394,19 +394,19 @@ function download {
function post_build_checks { function post_build_checks {
Write-Verbose '[post_build_checks] Running Caldera Check.' Write-Verbose '[post_build_checks] Running Caldera Check.'
$CALDERA_CHECK = download -URL 'https://192.168.38.5:8888' -PatternToMatch '<title>CALDERA</title>' $CALDERA_CHECK = download -URL 'https://192.168.38.105:8888' -PatternToMatch '<title>CALDERA</title>'
Write-Verbose "[post_build_checks] Cladera Result: $CALDERA_CHECK" Write-Verbose "[post_build_checks] Cladera Result: $CALDERA_CHECK"
Write-Verbose '[post_build_checks] Running Splunk Check.' Write-Verbose '[post_build_checks] Running Splunk Check.'
$SPLUNK_CHECK = download -URL 'https://192.168.38.5:8000/en-US/account/login?return_to=%2Fen-US%2F' -PatternToMatch 'This browser is not supported by Splunk' $SPLUNK_CHECK = download -URL 'https://192.168.38.105:8000/en-US/account/login?return_to=%2Fen-US%2F' -PatternToMatch 'This browser is not supported by Splunk'
Write-Verbose "[post_build_checks] Splunk Result: $SPLUNK_CHECK" Write-Verbose "[post_build_checks] Splunk Result: $SPLUNK_CHECK"
Write-Verbose '[post_build_checks] Running Fleet Check.' Write-Verbose '[post_build_checks] Running Fleet Check.'
$FLEET_CHECK = download -URL 'https://192.168.38.5:8412' -PatternToMatch 'Kolide Fleet' $FLEET_CHECK = download -URL 'https://192.168.38.105:8412' -PatternToMatch 'Kolide Fleet'
Write-Verbose "[post_build_checks] Fleet Result: $FLEET_CHECK" Write-Verbose "[post_build_checks] Fleet Result: $FLEET_CHECK"
Write-Verbose '[post_build_checks] Running MS ATA Check.' Write-Verbose '[post_build_checks] Running MS ATA Check.'
$ATA_CHECK = download -URL 'https://192.168.38.3' -SuccessOn401 $ATA_CHECK = download -URL 'https://192.168.38.103' -SuccessOn401
Write-Verbose "[post_build_checks] ATA Result: $ATA_CHECK" Write-Verbose "[post_build_checks] ATA Result: $ATA_CHECK"

48
build.sh
View File

@@ -8,12 +8,12 @@
# https://github.com/clong/DetectionLab/issues # https://github.com/clong/DetectionLab/issues
print_usage() { print_usage() {
echo "Usage: ./build.sh <virtualbox|vmware_desktop>" echo "Usage: ./build.sh <virtualbox | vmware_desktop> <--vagrant-only | --packer-only>"
exit 0 exit 0
} }
check_packer_path() { check_packer_path() {
# Check for existence of Packer in PATH # Check for existence of Packer in PATH
if ! which packer >/dev/null; then if ! which packer >/dev/null; then
(echo >&2 "Packer was not found in your PATH.") (echo >&2 "Packer was not found in your PATH.")
(echo >&2 "Please correct this before continuing. Quitting.") (echo >&2 "Please correct this before continuing. Quitting.")
@@ -23,16 +23,16 @@ check_packer_path() {
} }
check_vagrant_path() { check_vagrant_path() {
# Check for existence of Vagrant in PATH # Check for existence of Vagrant in PATH
if ! which vagrant >/dev/null; then if ! which vagrant >/dev/null; then
(echo >&2 "Vagrant was not found in your PATH.") (echo >&2 "Vagrant was not found in your PATH.")
(echo >&2 "Please correct this before continuing. Quitting.") (echo >&2 "Please correct this before continuing. Quitting.")
exit 1 exit 1
fi fi
# Ensure Vagrant >= 2.0.0 # Ensure Vagrant >= 2.0.0
if [ "$(vagrant --version | grep -o "[0-9]" | head -1)" -lt 2 ]; then if [ "$(vagrant --version | grep -o "[0-9]" | head -1)" -lt 2 ]; then
(echo >&2 "WARNING: It is highly recommended to use Vagrant 2.0.0 or above before continuing") (echo >&2 "WARNING: It is highly recommended to use Vagrant 2.0.0 or above before continuing")
fi fi
} }
# Returns 0 if not installed or 1 if installed # Returns 0 if not installed or 1 if installed
@@ -69,9 +69,9 @@ check_vmware_desktop_vagrant_plugin_installed() {
(echo >&2 "Visit https://www.hashicorp.com/blog/introducing-the-vagrant-vmware-desktop-plugin for more information on how to purchase and install it") (echo >&2 "Visit https://www.hashicorp.com/blog/introducing-the-vagrant-vmware-desktop-plugin for more information on how to purchase and install it")
(echo >&2 "VMWare Fusion will not be listed as a provider until the vagrant-vmware-desktop plugin has been installed.") (echo >&2 "VMWare Fusion will not be listed as a provider until the vagrant-vmware-desktop plugin has been installed.")
echo "0" echo "0"
else else
echo "$VAGRANT_VMWARE_DESKTOP_PLUGIN_PRESENT" echo "$VAGRANT_VMWARE_DESKTOP_PLUGIN_PRESENT"
fi fi
} }
# List the available Vagrant providers present on the system # List the available Vagrant providers present on the system
@@ -251,10 +251,10 @@ vagrant_reload_host() {
post_build_checks() { post_build_checks() {
# If the curl operation fails, we'll just leave the variable equal to 0 # If the curl operation fails, we'll just leave the variable equal to 0
# This is needed to prevent the script from exiting if the curl operation fails # This is needed to prevent the script from exiting if the curl operation fails
CALDERA_CHECK=$(curl -ks -m 2 https://192.168.38.5:8888 | grep -c '302: Found' || echo "") CALDERA_CHECK=$(curl -ks -m 2 https://192.168.38.105:8888 | grep -c '302: Found' || echo "")
SPLUNK_CHECK=$(curl -ks -m 2 https://192.168.38.5:8000/en-US/account/login?return_to=%2Fen-US%2F | grep -c 'This browser is not supported by Splunk' || echo "") SPLUNK_CHECK=$(curl -ks -m 2 https://192.168.38.105:8000/en-US/account/login?return_to=%2Fen-US%2F | grep -c 'This browser is not supported by Splunk' || echo "")
FLEET_CHECK=$(curl -ks -m 2 https://192.168.38.5:8412 | grep -c 'Kolide Fleet' || echo "") FLEET_CHECK=$(curl -ks -m 2 https://192.168.38.105:8412 | grep -c 'Kolide Fleet' || echo "")
ATA_CHECK=$(curl --fail --write-out "%{http_code}" -ks https://192.168.38.3 -m 2) ATA_CHECK=$(curl --fail --write-out "%{http_code}" -ks https://192.168.38.103 -m 2)
[[ $ATA_CHECK == 401 ]] && ATA_CHECK=1 [[ $ATA_CHECK == 401 ]] && ATA_CHECK=1
BASH_MAJOR_VERSION=$(/bin/bash --version | grep 'GNU bash' | grep -o version\.\.. | cut -d ' ' -f 2 | cut -d '.' -f 1) BASH_MAJOR_VERSION=$(/bin/bash --version | grep 'GNU bash' | grep -o version\.\.. | cut -d ' ' -f 2 | cut -d '.' -f 1)
@@ -340,17 +340,17 @@ build_packer_boxes() {
(echo >&2 "To file an issue, please visit https://github.com/clong/DetectionLab/issues/") (echo >&2 "To file an issue, please visit https://github.com/clong/DetectionLab/issues/")
exit 1 exit 1
fi fi
elif [ "$(hostname)" == "packerwindows2016" ]; then # Workaround for CI environment elif [ "$(hostname)" == "packerwindows2016" ]; then # Workaround for CI environment
(echo >&2 "CI Environment detected. If you are a user and are seeing this, please file an issue on GitHub.") (echo >&2 "CI Environment detected. If you are a user and are seeing this, please file an issue on GitHub.")
RET=$(packer_build_box "windows_2016") RET=$(packer_build_box "windows_2016")
if [ "$RET" -eq 0 ]; then if [ "$RET" -eq 0 ]; then
(echo >&2 "Good news! The windows_2016 box was built with Packer successfully!") (echo >&2 "Good news! The windows_2016 box was built with Packer successfully!")
else else
(echo >&2 "Something went wrong while attempting to build the windows_2016 box.") (echo >&2 "Something went wrong while attempting to build the windows_2016 box.")
(echo >&2 "To file an issue, please visit https://github.com/clong/DetectionLab/issues/") (echo >&2 "To file an issue, please visit https://github.com/clong/DetectionLab/issues/")
exit 1 exit 1
fi fi
else else
for PACKER_BOX in "${PACKER_BOXES[@]}"; do # Normal user workflow for PACKER_BOX in "${PACKER_BOXES[@]}"; do # Normal user workflow
RET=$(packer_build_box "$PACKER_BOX") RET=$(packer_build_box "$PACKER_BOX")
if [ "$RET" -eq 0 ]; then if [ "$RET" -eq 0 ]; then
@@ -360,8 +360,8 @@ build_packer_boxes() {
(echo >&2 "To file an issue, please visit https://github.com/clong/DetectionLab/issues/") (echo >&2 "To file an issue, please visit https://github.com/clong/DetectionLab/issues/")
exit 1 exit 1
fi fi
done done
fi fi
} }
choose_md5_tool() { choose_md5_tool() {
@@ -467,7 +467,7 @@ main() {
build_vagrant_hosts build_vagrant_hosts
post_build_checks post_build_checks
fi fi
} }
main "$@" main "$@"
exit 0 exit 0

8
ci/build_machine_bootstrap.sh
View File

@@ -51,8 +51,8 @@ if [ "$PACKER_ONLY" -eq 0 ]; then
# Install Vagrant # Install Vagrant
mkdir /opt/vagrant mkdir /opt/vagrant
cd /opt/vagrant || exit 1 cd /opt/vagrant || exit 1
wget https://releases.hashicorp.com/vagrant/2.1.2/vagrant_2.1.2_x86_64.deb wget https://releases.hashicorp.com/vagrant/2.1.4/vagrant_2.1.4_x86_64.deb
dpkg -i vagrant_2.1.2_x86_64.deb dpkg -i vagrant_2.1.4_x86_64.deb
vagrant plugin install vagrant-reload vagrant plugin install vagrant-reload
# Make the Vagrant instances headless # Make the Vagrant instances headless
@@ -64,8 +64,8 @@ if [ "$VAGRANT_ONLY" -eq 0 ]; then
# Install Packer # Install Packer
mkdir /opt/packer mkdir /opt/packer
cd /opt/packer || exit 1 cd /opt/packer || exit 1
wget https://releases.hashicorp.com/packer/1.2.3/packer_1.2.3_linux_amd64.zip wget https://releases.hashicorp.com/packer/1.2.5/packer_1.2.5_linux_amd64.zip
unzip packer_1.2.3_linux_amd64.zip unzip packer_1.2.5_linux_amd64.zip
cp packer /usr/local/bin/packer cp packer /usr/local/bin/packer
# Make the Packer images headless # Make the Packer images headless

5
ci/circle_workflows/vagrant_changes.sh
View File

@@ -36,7 +36,7 @@ ssh -i ~/.ssh/id_rsa root@"$IP_ADDRESS" 'bash -s' -- < ci/build_machine_bootstra
## Waiting for Packet server to post build results ## Waiting for Packet server to post build results
MINUTES_PAST=0 MINUTES_PAST=0
while [ "$MINUTES_PAST" -lt 120 ]; do while [ "$MINUTES_PAST" -lt 180 ]; do
STATUS=$(curl $IP_ADDRESS) STATUS=$(curl $IP_ADDRESS)
if [ "$STATUS" == "building" ]; then if [ "$STATUS" == "building" ]; then
echo "$STATUS" echo "$STATUS"
@@ -44,9 +44,10 @@ while [ "$MINUTES_PAST" -lt 120 ]; do
sleep 300 sleep 300
((MINUTES_PAST += 5)) ((MINUTES_PAST += 5))
else else
scp -i ~/.ssh/id_rsa root@"$IP_ADDRESS":/opt/DetectionLab/Vagrant/vagrant_up_*.log /tmp/artifacts/ || echo "Vagrant log not yet present"
break break
fi fi
if [ "$MINUTES_PAST" -gt 120 ]; then if [ "$MINUTES_PAST" -gt 180 ]; then
echo "Serer timed out. Uptime: $MINUTES_PAST minutes." echo "Serer timed out. Uptime: $MINUTES_PAST minutes."
scp -i ~/.ssh/id_rsa root@"$IP_ADDRESS":/opt/DetectionLab/Vagrant/vagrant_up_*.log /tmp/artifacts/ scp -i ~/.ssh/id_rsa root@"$IP_ADDRESS":/opt/DetectionLab/Vagrant/vagrant_up_*.log /tmp/artifacts/
curl -X DELETE --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/devices/'"$DEVICE_ID" curl -X DELETE --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/devices/'"$DEVICE_ID"