Add the ability to use the hyperv provider.

This commit is contained in:
man715
2020-08-25 13:03:34 -07:00
parent 85a5990b2f
commit 53b0ec43a3
11 changed files with 885 additions and 0 deletions

36
HyperV/README.md Normal file
View File

@@ -0,0 +1,36 @@
# Detection Lab for Hyper-V
This hypervisor does not undergo the same level of weekly testing as what is found in the main part of Detectionlab.
Hyper-V support is only in beta and needs more testing by more users to ensure its stability and usability.
## Requirements
The version of Hyper-V will need to be compatible with Hyper-V VM configuration version 9.0.
You will need to be running one of the following operating systems:
* Windows 10 1809 or later
* Windows Server 2019
* Windows Hyper-V Server 2019
For a breakdown of what Operating Systems support which VM configuration versions please visit: (https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/deploy/upgrade-virtual-machine-version-in-hyper-v-on-windows-or-windows-server)
This build also requires vagrant-reload. If you do not have it installed, you will be prompted to install it.
## Known Issues Important!
Unfortunately, due to some issues with Vagrant and Microsoft, running DetectionLab with Hyper-V is not as easy as it is with other providers. You will need to do the following BEFORE trying to run `vagrant up`.
1) Patch Vagant
Vagrant does not play nice with Windows SMB share authentication. It uses cmdkey which is not properly implimented. More details about the issue can be found in this issue:(https://github.com/hashicorp/vagrant/issues/10661)
To patch this, go find the mount_shared_folder.rb file and replace `"cmdkey /add:#{options[:smb_host]} /user:#{options[:smb_username]} /pass:\"#{options[:smb_password]}\""` with `"cmdkey '/add:\"#{options[:smb_host]}\"' '/user:\"#{options[:smb_username]}\"' '/pass:\"#{options[:smb_password]}\"'"`
2) Windows will require you to enter an administrator username and password to be able to create and mount the SMB share.
This means the build will not be fully automated. One thing you can do is add `config.vm.synced_folder '../Vagrant', '/vagrant', smb_username: "username", smb_password: "password"` to the Vagrantfile on line 2. For security reasons, this is not a good idea as you will be storing your username and password in plaintext.
By not having this line added to the Vagrantfile you will be required to put in your username and password at least 2 times per machine.
3) Selecting Virtual Switch
During the build you will also have to select the virtual switch you want to use for each server. This cannot be avoided. There is a option that can be used to force the network adapter to use a particular switch; however, using that option breaks this build process.
A really hacky workaround is, if you are using the smb_username and smb_password options, would be to press the option number corresponding with the virtual switch you want to use then enter four times after `vagrant up`. So if you know you want the virtual switch 1 `vagrant up` 1 enter 1 enter 1 enter 1 enter
Yes, I know hacky but it works.
## How this build works
The majority of this build works the same as the VirtualBox build. The most notable difference is is on the Windows builds. There is a script that will create an internal virtual switch called "NATSwitch." Throughout the build process, a script will create a second network adapter and attach it to the NATSwitch on the VM being built. After the machine is built the original network adapter will be removed from the VM.
## Note
This build will run two scripts on the host machine. It is advisable to always check any scripts that will be run on your machine before running them.

221
HyperV/Vagrantfile vendored Normal file
View File

@@ -0,0 +1,221 @@
Vagrant.configure("2") do |config|
config.vm.synced_folder '../Vagrant', '/vagrant'
config.vagrant.plugins = "vagrant-reload"
# Set up the Hyper-V switch
config.trigger.before :up do |trigger|
trigger.info = "Creating 'NATSwitch' Hyper-V switch if it does not exist..."
trigger.run = {privileged: "true", powershell_elevated_interactive: "true", path: "./hyperv-create-nat-switch.ps1"}
end
config.vm.define "logger" do |cfg|
cfg.vm.box = "bento/ubuntu-18.04"
cfg.vm.hostname = "logger"
cfg.vm.boot_timeout = 600
cfg.vm.provision :shell, path: "fix-eth0-static-ip.sh"
cfg.vm.provision "reload"
cfg.vm.provision :shell, path: "check-eth0-ip.sh"
cfg.vm.provision :shell, path: "../Vagrant/bootstrap.sh"
cfg.vm.provision :shell, path: "fix-motd.sh"
# Change the switch to the NATSwitch
cfg.trigger.before :reload do |trigger|
trigger.info = "Setting Hyper-V switch to 'NATSwitch' to allow for static IP..."
trigger.run = {
privileged: "true",
powershell_elevated_interactive: "true",
path: "./hyperv-change-switch.ps1" ,
args: "-vmname logger"
}
end
cfg.vm.provider "hyperv" do |h, override|
h.vmname = "logger"
h.memory = 4096
h.cpus = 2
h.ip_address_timeout = 60
end
end
config.vm.define "dc" do |cfg|
cfg.vm.box = "man715/Windows2016"
cfg.vm.hostname = "dc"
cfg.vm.boot_timeout = 600
cfg.winrm.transport = :plaintext
cfg.vm.communicator = "winrm"
cfg.winrm.basic_auth_only = true
cfg.winrm.timeout = 300
cfg.winrm.retry_limit = 20
# Create a new network adapter on the NATSwitch
cfg.trigger.before :reload do |trigger|
trigger.info = "Setting Hyper-V switch to 'NATSwitch' to allow for static IP..."
trigger.run = {
privileged: "true",
powershell_elevated_interactive: "true",
path: "./hyperv-set-switch.ps1",
args: "-vmname dc.windomain.local"
}
end
# Remove the network adapter that was used to set up the box
cfg.trigger.after :up do |trigger|
trigger.info = "Removing the maintenance ethernet adapter"
trigger.run = {
privileged: "true",
powershell_elevated_interactive: "true",
path: "./hyperv-remove-networkadapter.ps1",
args: "-vmname dc.windomain.local"
}
end
cfg.vm.provision "reload"
cfg.vm.provision "shell", path: "hyperv-set-static-ip.ps1", privileged: true, args: "-ip 192.168.38.102 -dns 8.8.8.8"
cfg.vm.provision "shell", path: "../Vagrant/scripts/provision.ps1", privileged: true
cfg.vm.provision "reload"
cfg.vm.provision "shell", path: "../Vagrant/scripts/provision.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/download_palantir_wef.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-utilities.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-redteam.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-choco-extras.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-osquery.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-sysinternals.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-velociraptor.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/configure-ou.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/configure-wef-gpo.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/configure-powershelllogging.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/configure-AuditingPolicyGPOs.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/configure-rdp-user-gpo.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/configure-disable-windows-defender-gpo.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-autorunstowineventlog.ps1", privileged: true
cfg.vm.provision "shell", inline: 'wevtutil el | Select-String -notmatch "Microsoft-Windows-LiveId" | Foreach-Object {wevtutil cl "$_"}', privileged: true
cfg.vm.provision "shell", inline: "Set-SmbServerConfiguration -AuditSmb1Access $true -Force", privileged: true
cfg.vm.provision "shell", inline: 'cscript c:\windows\system32\slmgr.vbs /dlv', privileged: true
cfg.vm.provider "hyperv" do |h, override|
h.vmname = "dc.windomain.local"
h.memory = 4096
h.cpus = 2
h.ip_address_timeout = 600
end
end
config.vm.define "wef" do |cfg|
cfg.vm.box = "man715/Windows2016"
cfg.vm.hostname = "wef"
cfg.vm.boot_timeout = 600
cfg.vm.communicator = "winrm"
cfg.winrm.basic_auth_only = true
cfg.winrm.timeout = 300
cfg.winrm.retry_limit = 20
# Create a new network adapter on the NATSwitch
cfg.trigger.before :reload do |trigger|
trigger.info = "Setting Hyper-V switch to 'NATSwitch' to allow for static IP..."
trigger.run = {
privileged: "true",
powershell_elevated_interactive: "true",
path: "./hyperv-set-switch.ps1",
args: "-vmname wef.windomain.local"
}
end
# Remove the network adapter that was used to set up the box
cfg.trigger.after :up do |trigger|
trigger.info = "Removing the maintenance ethernet adapter"
trigger.run = {
privileged: "true",
powershell_elevated_interactive: "true",
path: "./hyperv-remove-networkadapter.ps1",
args: "-vmname wef.windomain.local"
}
end
cfg.vm.provision "reload"
cfg.vm.provision "shell", path: "hyperv-set-static-ip.ps1", privileged: true, args: "-ip 192.168.38.103 -dns 8.8.8.8"
cfg.vm.provision "shell", path: "../Vagrant/scripts/provision.ps1", privileged: true
cfg.vm.provision "reload"
cfg.vm.provision "shell", path: "../Vagrant/scripts/provision.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/download_palantir_wef.ps1", privileged: true
cfg.vm.provision "shell", inline: 'wevtutil el | Select-String -notmatch "Microsoft-Windows-LiveId" | Foreach-Object {wevtutil cl "$_"}', privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-wefsubscriptions.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-splunkuf.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-windows_ta.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-utilities.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-redteam.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-choco-extras.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-osquery.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-sysinternals.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-velociraptor.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/configure-pslogstranscriptsshare.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-autorunstowineventlog.ps1", privileged: true
cfg.vm.provision "shell", inline: "Set-SmbServerConfiguration -AuditSmb1Access $true -Force", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-microsoft-ata.ps1", privileged: true
cfg.vm.provision "shell", inline: 'cscript c:\windows\system32\slmgr.vbs /dlv', privileged: true
cfg.vm.provider "hyperv" do |h, override|
h.vmname = "wef.windomain.local"
h.memory = 4096
h.cpus = 2
h.ip_address_timeout = 600
end
end
config.vm.define "win10" do |cfg|
cfg.vm.box = "man715/Windows10"
cfg.vm.hostname = "win10"
cfg.vm.boot_timeout = 600
cfg.vm.communicator = "winrm"
cfg.winrm.basic_auth_only = true
cfg.winrm.timeout = 300
cfg.winrm.retry_limit = 20
# Create a new network adapter on the NATSwitch
cfg.trigger.before :reload do |trigger|
trigger.info = "Setting Hyper-V switch to 'NATSwitch' to allow for static IP..."
trigger.run = {
privileged: "true",
powershell_elevated_interactive: "true",
path: "./hyperv-set-switch.ps1",
args: "-vmname win10.windomain.local"
}
end
# Remove the network adapter that was used to set up the box
cfg.trigger.after :up do |trigger|
trigger.info = "Removing the maintenance ethernet adapter"
trigger.run = {
privileged: "true",
powershell_elevated_interactive: "true",
path: "./hyperv-remove-networkadapter.ps1",
args: "-vmname win10.windomain.local"
}
end
cfg.vm.provision "reload"
cfg.vm.provision "shell", path: "hyperv-set-static-ip.ps1", privileged: true, args: "-ip 192.168.38.104 -dns 8.8.8.8"
cfg.vm.provision "shell", path: "../Vagrant/scripts/MakeWindows10GreatAgain.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/provision.ps1", privileged: true
cfg.vm.provision "reload"
cfg.vm.provision "shell", path: "../Vagrant/scripts/provision.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/download_palantir_wef.ps1", privileged: true
cfg.vm.provision "shell", inline: 'wevtutil el | Select-String -notmatch "Microsoft-Windows-LiveId" | Foreach-Object {wevtutil cl "$_"}', privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-utilities.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-redteam.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-choco-extras.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-osquery.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-sysinternals.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-velociraptor.ps1", privileged: true
cfg.vm.provision "shell", path: "../Vagrant/scripts/install-autorunstowineventlog.ps1", privileged: true
cfg.vm.provision "shell", inline: 'cscript c:\windows\system32\slmgr.vbs /dlv', privileged: true
cfg.vm.provider "hyperv" do |h, override|
h.vmname = "win10.windomain.local"
h.memory = 4096
h.cpus = 2
h.ip_address_timeout = 600
end
end
end

498
HyperV/bootstrap.sh Normal file
View File

@@ -0,0 +1,498 @@
#! /bin/bash
# Override existing DNS Settings using netplan, but don't do it for Terraform builds
if ! curl -s 169.254.169.254 --connect-timeout 2 >/dev/null; then
echo -e " eth0:\n dhcp4: true\n nameservers:\n addresses: [8.8.8.8,8.8.4.4]" >>/etc/netplan/01-netcfg.yaml
netplan apply
fi
sed -i 's/nameserver 127.0.0.53/nameserver 8.8.8.8/g' /etc/resolv.conf && chattr +i /etc/resolv.conf
# Get a free Maxmind license here: https://www.maxmind.com/en/geolite2/signup
# Required for the ASNgen app to work: https://splunkbase.splunk.com/app/3531/
export MAXMIND_LICENSE=
if [ -n "$MAXMIND_LICENSE" ]; then
echo "Note: You have not entered a MaxMind license key on line 5 of bootstrap.sh, so the ASNgen Splunk app may not work correctly."
echo "However, it is not required and everything else should function correctly."
fi
export DEBIAN_FRONTEND=noninteractive
echo "apt-fast apt-fast/maxdownloads string 10" | debconf-set-selections
echo "apt-fast apt-fast/dlflag boolean true" | debconf-set-selections
sed -i "2ideb mirror://mirrors.ubuntu.com/mirrors.txt bionic main restricted universe multiverse\ndeb mirror://mirrors.ubuntu.com/mirrors.txt bionic-updates main restricted universe multiverse\ndeb mirror://mirrors.ubuntu.com/mirrors.txt bionic-backports main restricted universe multiverse\ndeb mirror://mirrors.ubuntu.com/mirrors.txt bionic-security main restricted universe multiverse" /etc/apt/sources.list
apt_install_prerequisites() {
echo "[$(date +%H:%M:%S)]: Adding apt repositories..."
# Add repository for apt-fast
add-apt-repository -y ppa:apt-fast/stable
# Add repository for yq
add-apt-repository -y ppa:rmescandon/yq
# Add repository for suricata
add-apt-repository -y ppa:oisf/suricata-stable
# Install prerequisites and useful tools
echo "[$(date +%H:%M:%S)]: Running apt-get clean..."
apt-get clean
echo "[$(date +%H:%M:%S)]: Running apt-get update..."
apt-get -qq update
apt-get -qq install -y apt-fast
echo "[$(date +%H:%M:%S)]: Running apt-fast install..."
apt-fast -qq install -y jq whois build-essential git unzip htop yq mysql-server redis-server python-pip
}
modify_motd() {
echo "[$(date +%H:%M:%S)]: Updating the MOTD..."
# Force color terminal
sed -i 's/#force_color_prompt=yes/force_color_prompt=yes/g' /root/.bashrc
sed -i 's/#force_color_prompt=yes/force_color_prompt=yes/g' /home/vagrant/.bashrc
# Remove some stock Ubuntu MOTD content
chmod -x /etc/update-motd.d/10-help-text
# Copy the DetectionLab MOTD
cp /vagrant/resources/logger/20-detectionlab /etc/update-motd.d/
chmod +x /etc/update-motd.d/20-detectionlab
}
test_prerequisites() {
for package in jq whois build-essential git unzip yq mysql-server redis-server python-pip; do
echo "[$(date +%H:%M:%S)]: [TEST] Validating that $package is correctly installed..."
# Loop through each package using dpkg
if ! dpkg -S $package >/dev/null; then
# If which returns a non-zero return code, try to re-install the package
echo "[-] $package was not found. Attempting to reinstall."
apt-get -qq update && apt-get install -y $package
if ! which $package >/dev/null; then
# If the reinstall fails, give up
echo "[X] Unable to install $package even after a retry. Exiting."
exit 1
fi
else
echo "[+] $package was successfully installed!"
fi
done
}
install_splunk() {
# Check if Splunk is already installed
if [ -f "/opt/splunk/bin/splunk" ]; then
echo "[$(date +%H:%M:%S)]: Splunk is already installed"
else
echo "[$(date +%H:%M:%S)]: Installing Splunk..."
# Get download.splunk.com into the DNS cache. Sometimes resolution randomly fails during wget below
dig @8.8.8.8 download.splunk.com >/dev/null
dig @8.8.8.8 splunk.com >/dev/null
dig @8.8.8.8 www.splunk.com >/dev/null
# Try to resolve the latest version of Splunk by parsing the HTML on the downloads page
echo "[$(date +%H:%M:%S)]: Attempting to autoresolve the latest version of Splunk..."
LATEST_SPLUNK=$(curl https://www.splunk.com/en_us/download/splunk-enterprise.html | grep -i deb | grep -Eo "data-link=\"................................................................................................................................" | cut -d '"' -f 2)
# Sanity check what was returned from the auto-parse attempt
if [[ "$(echo "$LATEST_SPLUNK" | grep -c "^https:")" -eq 1 ]] && [[ "$(echo "$LATEST_SPLUNK" | grep -c "\.deb$")" -eq 1 ]]; then
echo "[$(date +%H:%M:%S)]: The URL to the latest Splunk version was automatically resolved as: $LATEST_SPLUNK"
echo "[$(date +%H:%M:%S)]: Attempting to download..."
wget --progress=bar:force -P /opt "$LATEST_SPLUNK"
else
echo "[$(date +%H:%M:%S)]: Unable to auto-resolve the latest Splunk version. Falling back to hardcoded URL..."
# Download Hardcoded Splunk
wget --progress=bar:force -O /opt/splunk-8.0.2-a7f645ddaf91-linux-2.6-amd64.deb 'https://download.splunk.com/products/splunk/releases/8.0.2/linux/splunk-8.0.2-a7f645ddaf91-linux-2.6-amd64.deb&wget=true'
fi
if ! ls /opt/splunk*.deb 1>/dev/null 2>&1; then
echo "Something went wrong while trying to download Splunk. This script cannot continue. Exiting."
exit 1
fi
if ! dpkg -i /opt/splunk*.deb >/dev/null; then
echo "Something went wrong while trying to install Splunk. This script cannot continue. Exiting."
exit 1
fi
/opt/splunk/bin/splunk start --accept-license --answer-yes --no-prompt --seed-passwd changeme
/opt/splunk/bin/splunk add index wineventlog -auth 'admin:changeme'
/opt/splunk/bin/splunk add index osquery -auth 'admin:changeme'
/opt/splunk/bin/splunk add index osquery-status -auth 'admin:changeme'
/opt/splunk/bin/splunk add index sysmon -auth 'admin:changeme'
/opt/splunk/bin/splunk add index powershell -auth 'admin:changeme'
/opt/splunk/bin/splunk add index zeek -auth 'admin:changeme'
/opt/splunk/bin/splunk add index suricata -auth 'admin:changeme'
/opt/splunk/bin/splunk add index threathunting -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_forwarder/splunk-add-on-for-microsoft-windows_700.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/splunk-add-on-for-microsoft-sysmon_1062.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/asn-lookup-generator_110.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/lookup-file-editor_331.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/splunk-add-on-for-zeek-aka-bro_400.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/force-directed-app-for-splunk_200.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/punchcard-custom-visualization_130.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/sankey-diagram-custom-visualization_130.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/link-analysis-app-for-splunk_161.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/threathunting_144.tgz -auth 'admin:changeme'
# Install the Maxmind license key for the ASNgen App
if [ -n "$MAXMIND_LICENSE" ]; then
mkdir /opt/splunk/etc/apps/TA-asngen/local
cp /opt/splunk/etc/apps/TA-asngen/default/asngen.conf /opt/splunk/etc/apps/TA-asngen/local/asngen.conf
sed -i "s/license_key =/license_key = $MAXMIND_LICENSE/g" /opt/splunk/etc/apps/TA-asngen/local/asngen.conf
fi
# Replace the props.conf for Sysmon TA and Windows TA
# Removed all the 'rename = xmlwineventlog' directives
# I know youre not supposed to modify files in "default",
# but for some reason adding them to "local" wasnt working
cp /vagrant/resources/splunk_server/windows_ta_props.conf /opt/splunk/etc/apps/Splunk_TA_windows/default/props.conf
cp /vagrant/resources/splunk_server/sysmon_ta_props.conf /opt/splunk/etc/apps/TA-microsoft-sysmon/default/props.conf
# Add custom Macro definitions for ThreatHunting App
cp /vagrant/resources/splunk_server/macros.conf /opt/splunk/etc/apps/ThreatHunting/default/macros.conf
# Fix props.conf in ThreatHunting App
sed -i 's/EVAL-host_fqdn = Computer/EVAL-host_fqdn = ComputerName/g' /opt/splunk/etc/apps/ThreatHunting/default/props.conf
# Fix Windows TA macros
mkdir /opt/splunk/etc/apps/Splunk_TA_windows/local
cp /opt/splunk/etc/apps/Splunk_TA_windows/default/macros.conf /opt/splunk/etc/apps/Splunk_TA_windows/local
sed -i 's/wineventlog_windows/wineventlog/g' /opt/splunk/etc/apps/Splunk_TA_windows/local/macros.conf
# Fix Force Directed App until 2.0.1 is released (https://answers.splunk.com/answers/668959/invalid-key-in-stanza-default-value-light.html#answer-669418)
rm /opt/splunk/etc/apps/force_directed_viz/default/savedsearches.conf
# Add a Splunk TCP input on port 9997
echo -e "[splunktcp://9997]\nconnection_host = ip" >/opt/splunk/etc/apps/search/local/inputs.conf
# Add props.conf and transforms.conf
cp /vagrant/resources/splunk_server/props.conf /opt/splunk/etc/apps/search/local/
cp /vagrant/resources/splunk_server/transforms.conf /opt/splunk/etc/apps/search/local/
cp /opt/splunk/etc/system/default/limits.conf /opt/splunk/etc/system/local/limits.conf
# Bump the memtable limits to allow for the ASN lookup table
sed -i.bak 's/max_memtable_bytes = 10000000/max_memtable_bytes = 30000000/g' /opt/splunk/etc/system/local/limits.conf
# Skip Splunk Tour and Change Password Dialog
echo "[$(date +%H:%M:%S)]: Disabling the Splunk tour prompt..."
touch /opt/splunk/etc/.ui_login
mkdir -p /opt/splunk/etc/users/admin/search/local
echo -e "[search-tour]\nviewed = 1" >/opt/splunk/etc/system/local/ui-tour.conf
# Source: https://answers.splunk.com/answers/660728/how-to-disable-the-modal-pop-up-help-us-to-improve.html
if [ ! -d "/opt/splunk/etc/users/admin/user-prefs/local" ]; then
mkdir -p "/opt/splunk/etc/users/admin/user-prefs/local"
fi
echo '[general]
render_version_messages = 1
dismissedInstrumentationOptInVersion = 4
notification_python_3_impact = false
display.page.home.dashboardId = /servicesNS/nobody/search/data/ui/views/logger_dashboard' >/opt/splunk/etc/users/admin/user-prefs/local/user-prefs.conf
# Enable SSL Login for Splunk
echo -e "[settings]\nenableSplunkWebSSL = true" >/opt/splunk/etc/system/local/web.conf
# Copy over the Logger Dashboard
if [ ! -d "/opt/splunk/etc/apps/search/local/data/ui/views" ]; then
mkdir -p "/opt/splunk/etc/apps/search/local/data/ui/views"
fi
cp /vagrant/resources/splunk_server/logger_dashboard.xml /opt/splunk/etc/apps/search/local/data/ui/views || echo "Unable to find dashboard"
# Reboot Splunk to make changes take effect
/opt/splunk/bin/splunk restart
/opt/splunk/bin/splunk enable boot-start
fi
}
download_palantir_osquery_config() {
if [ -f /opt/osquery-configuration ]; then
echo "[$(date +%H:%M:%S)]: osquery configs have already been downloaded"
else
# Import Palantir osquery configs into Fleet
echo "[$(date +%H:%M:%S)]: Downloading Palantir osquery configs..."
cd /opt && git clone https://github.com/palantir/osquery-configuration.git
fi
}
install_fleet_import_osquery_config() {
if [ -f "/opt/fleet" ]; then
echo "[$(date +%H:%M:%S)]: Fleet is already installed"
else
cd /opt || exit 1
echo "[$(date +%H:%M:%S)]: Installing Fleet..."
echo -e "\n127.0.0.1 kolide" >>/etc/hosts
echo -e "\n127.0.0.1 logger" >>/etc/hosts
# Set MySQL username and password, create kolide database
mysql -uroot -e "ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'kolide';"
mysql -uroot -pkolide -e "create database kolide;"
# Always download the latest release of Fleet
curl -s https://api.github.com/repos/kolide/fleet/releases/latest | grep 'https://github.com' | grep "/fleet.zip" | cut -d ':' -f 2,3 | tr -d '"' | wget --progress=bar:force -i -
unzip fleet.zip -d fleet
cp fleet/linux/fleetctl /usr/local/bin/fleetctl && chmod +x /usr/local/bin/fleetctl
cp fleet/linux/fleet /usr/local/bin/fleet && chmod +x /usr/local/bin/fleet
# Prepare the DB
fleet prepare db --mysql_address=127.0.0.1:3306 --mysql_database=kolide --mysql_username=root --mysql_password=kolide
# Copy over the certs and service file
cp /vagrant/resources/fleet/server.* /opt/fleet/
cp /vagrant/resources/fleet/fleet.service /etc/systemd/system/fleet.service
mkdir /var/log/fleet
/bin/systemctl enable fleet.service
/bin/systemctl start fleet.service
echo "[$(date +%H:%M:%S)]: Waiting for fleet service to start..."
while true; do
result=$(curl --silent -k https://127.0.0.1:8412)
if echo "$result" | grep -q setup; then break; fi
sleep 1
done
fleetctl config set --address https://192.168.38.105:8412
fleetctl config set --tls-skip-verify true
fleetctl setup --email admin@detectionlab.network --username admin --password 'admin123#' --org-name DetectionLab
fleetctl login --email admin@detectionlab.network --password 'admin123#'
# Set the enrollment secret to match what we deploy to Windows hosts
mysql -uroot --password=kolide -e 'use kolide; update enroll_secrets set secret = "enrollmentsecret" where active=1;'
echo "Updated enrollment secret"
# Change the query invervals to reflect a lab environment
# Every hour -> Every 3 minutes
# Every 24 hours -> Every 15 minutes
sed -i 's/interval: 3600/interval: 180/g' osquery-configuration/Fleet/Endpoints/MacOS/osquery.yaml
sed -i 's/interval: 3600/interval: 180/g' osquery-configuration/Fleet/Endpoints/Windows/osquery.yaml
sed -i 's/interval: 28800/interval: 900/g' osquery-configuration/Fleet/Endpoints/MacOS/osquery.yaml
sed -i 's/interval: 28800/interval: 900/g' osquery-configuration/Fleet/Endpoints/Windows/osquery.yaml
# Don't log osquery INFO messages
# Fix snapshot event formatting
fleetctl get options > /tmp/options.yaml
/usr/bin/yq w -i /tmp/options.yaml 'spec.config.options.enroll_secret' 'enrollmentsecret'
/usr/bin/yq w -i /tmp/options.yaml 'spec.config.options.logger_snapshot_event_type' 'true'
# Fleet 3.0 requires the "kind" to be "options" instead of "option"
sed -i 's/kind: option/kind: options/g' /tmp/options.yaml
fleetctl apply -f /tmp/options.yaml
# Use fleetctl to import YAML files
fleetctl apply -f osquery-configuration/Fleet/Endpoints/MacOS/osquery.yaml
fleetctl apply -f osquery-configuration/Fleet/Endpoints/Windows/osquery.yaml
for pack in osquery-configuration/Fleet/Endpoints/packs/*.yaml; do
fleetctl apply -f "$pack"
done
# Add Splunk monitors for Fleet
# Files must exist before splunk will add a monitor
touch /var/log/fleet/osquery_result
touch /var/log/fleet/osquery_status
/opt/splunk/bin/splunk add monitor "/var/log/fleet/osquery_result" -index osquery -sourcetype 'osquery:json' -auth 'admin:changeme'
/opt/splunk/bin/splunk add monitor "/var/log/fleet/osquery_status" -index osquery-status -sourcetype 'osquery:status' -auth 'admin:changeme'
fi
}
install_zeek() {
echo "[$(date +%H:%M:%S)]: Installing Zeek..."
# Environment variables
NODECFG=/opt/zeek/etc/node.cfg
sh -c "echo 'deb http://download.opensuse.org/repositories/security:/zeek/xUbuntu_18.04/ /' > /etc/apt/sources.list.d/security:zeek.list"
wget -nv https://download.opensuse.org/repositories/security:zeek/xUbuntu_18.04/Release.key -O /tmp/Release.key
apt-key add - </tmp/Release.key &>/dev/null
# Update APT repositories
apt-get -qq -ym update
# Install tools to build and configure Zeek
apt-get -qq -ym install zeek crudini
export PATH=$PATH:/opt/zeek/bin
pip install zkg==2.1.1
zkg refresh
zkg autoconfig
zkg install --force salesforce/ja3
# Load Zeek scripts
echo '
@load protocols/ftp/software
@load protocols/smtp/software
@load protocols/ssh/software
@load protocols/http/software
@load tuning/json-logs
@load policy/integration/collective-intel
@load policy/frameworks/intel/do_notice
@load frameworks/intel/seen
@load frameworks/intel/do_notice
@load frameworks/files/hash-all-files
@load base/protocols/smb
@load policy/protocols/conn/vlan-logging
@load policy/protocols/conn/mac-logging
@load ja3
redef Intel::read_files += {
"/opt/zeek/etc/intel.dat"
};
' >>/opt/zeek/share/zeek/site/local.zeek
# Configure Zeek
crudini --del $NODECFG zeek
crudini --set $NODECFG manager type manager
crudini --set $NODECFG manager host localhost
crudini --set $NODECFG proxy type proxy
crudini --set $NODECFG proxy host localhost
# Setup $CPUS numbers of Zeek workers
crudini --set $NODECFG worker-eth0 type worker
crudini --set $NODECFG worker-eth0 host localhost
crudini --set $NODECFG worker-eth0 interface eth0
crudini --set $NODECFG worker-eth0 lb_method pf_ring
crudini --set $NODECFG worker-eth0 lb_procs "$(nproc)"
# Setup Zeek to run at boot
cp /vagrant/resources/zeek/zeek.service /lib/systemd/system/zeek.service
systemctl enable zeek
systemctl start zeek
# Configure the Splunk inputs
mkdir -p /opt/splunk/etc/apps/Splunk_TA_bro/local && touch /opt/splunk/etc/apps/Splunk_TA_bro/local/inputs.conf
crudini --set /opt/splunk/etc/apps/Splunk_TA_bro/local/inputs.conf monitor:///opt/zeek/spool/manager index zeek
crudini --set /opt/splunk/etc/apps/Splunk_TA_bro/local/inputs.conf monitor:///opt/zeek/spool/manager sourcetype bro:json
crudini --set /opt/splunk/etc/apps/Splunk_TA_bro/local/inputs.conf monitor:///opt/zeek/spool/manager whitelist '.*\.log$'
crudini --set /opt/splunk/etc/apps/Splunk_TA_bro/local/inputs.conf monitor:///opt/zeek/spool/manager blacklist '.*(communication|stderr)\.log$'
crudini --set /opt/splunk/etc/apps/Splunk_TA_bro/local/inputs.conf monitor:///opt/zeek/spool/manager disabled 0
# Ensure permissions are correct and restart splunk
chown -R splunk:splunk /opt/splunk/etc/apps/Splunk_TA_bro
/opt/splunk/bin/splunk restart
# Verify that Zeek is running
if ! pgrep -f zeek >/dev/null; then
echo "Zeek attempted to start but is not running. Exiting"
exit 1
fi
}
install_velociraptor() {
echo "[$(date +%H:%M:%S)]: Installing Velociraptor..."
if [ ! -d "/opt/velociraptor" ]; then
mkdir /opt/velociraptor
fi
echo "[$(date +%H:%M:%S)]: Attempting to determine the URL for the latest release of Velociraptor"
LATEST_VELOCIRAPTOR_LINUX_URL=$(curl -sL https://github.com/Velocidex/velociraptor/releases/latest | grep 'linux-amd64' | grep -Eo "/(?[^\"]+)" | grep amd | sed 's#^#https://github.com#g')
echo "[$(date +%H:%M:%S)]: The URL for the latest release was extracted as $LATEST_VELOCIRAPTOR_LINUX_URL"
echo "[$(date +%H:%M:%S)]: Attempting to download..."
#wget -P /opt/velociraptor --progress=bar:force "$LATEST_VELOCIRAPTOR_LINUX_URL"
# Harcoding until the release after v0.4.7
wget -P /opt/velociraptor --progress=bar:force "https://github.com/Velocidex/velociraptor/releases/download/v0.4.7/velociraptor-v0.4.7-1-linux-amd64"
if [ "$(file /opt/velociraptor/velociraptor*linux-amd64 | grep -c 'ELF 64-bit LSB executable')" -eq 1 ]; then
echo "[$(date +%H:%M:%S)]: Velociraptor successfully downloaded!"
else
echo "[$(date +%H:%M:%S)]: Failed to download the latest version of Velociraptor. Please open a DetectionLab issue on Github."
return
fi
cd /opt/velociraptor || exit 1
mv velociraptor-*-linux-amd64 velociraptor
chmod +x velociraptor
cp /vagrant/resources/velociraptor/server.config.yaml /opt/velociraptor
echo "[$(date +%H:%M:%S)]: Creating Velociraptor dpkg..."
./velociraptor --config /opt/velociraptor/server.config.yaml debian server
echo "[$(date +%H:%M:%S)]: Installing the dpkg..."
if dpkg -i velociraptor_*_server.deb >/dev/null; then
echo "[$(date +%H:%M:%S)]: Installation complete!"
else
echo "[$(date +%H:%M:%S)]: Failed to install the dpkg"
return
fi
}
install_suricata() {
# Run iwr -Uri testmyids.com -UserAgent "BlackSun" in Powershell to generate test alerts from Windows
echo "[$(date +%H:%M:%S)]: Installing Suricata..."
# Install suricata
apt-get -qq -y install suricata crudini
test_suricata_prerequisites
# Install suricata-update
cd /opt || exit 1
git clone https://github.com/OISF/suricata-update.git
cd /opt/suricata-update || exit 1
pip install pyyaml
python setup.py install
cp /vagrant/resources/suricata/suricata.yaml /etc/suricata/suricata.yaml
crudini --set --format=sh /etc/default/suricata '' iface eth0
# update suricata signature sources
suricata-update update-sources
# disable protocol decode as it is duplicative of Zeek
echo re:protocol-command-decode >>/etc/suricata/disable.conf
# enable et-open and attackdetection sources
suricata-update enable-source et/open
suricata-update enable-source ptresearch/attackdetection
# Configure the Splunk inputs
mkdir -p /opt/splunk/etc/apps/SplunkLightForwarder/local && touch /opt/splunk/etc/apps/SplunkLightForwarder/local/inputs.conf
crudini --set /opt/splunk/etc/apps/SplunkLightForwarder/local/inputs.conf monitor:///var/log/suricata index suricata
crudini --set /opt/splunk/etc/apps/SplunkLightForwarder/local/inputs.conf monitor:///var/log/suricata sourcetype suricata:json
crudini --set /opt/splunk/etc/apps/SplunkLightForwarder/local/inputs.conf monitor:///var/log/suricata whitelist 'eve.json'
crudini --set /opt/splunk/etc/apps/SplunkLightForwarder/local/inputs.conf monitor:///var/log/suricata disabled 0
crudini --set /opt/splunk/etc/apps/SplunkLightForwarder/local/props.conf json_suricata TRUNCATE 0
# Update suricata and restart
suricata-update
service suricata stop
service suricata start
sleep 3
# Verify that Suricata is running
if ! pgrep -f suricata >/dev/null; then
echo "Suricata attempted to start but is not running. Exiting"
exit 1
fi
}
test_suricata_prerequisites() {
for package in suricata crudini; do
echo "[$(date +%H:%M:%S)]: [TEST] Validating that $package is correctly installed..."
# Loop through each package using dpkg
if ! dpkg -S $package >/dev/null; then
# If which returns a non-zero return code, try to re-install the package
echo "[-] $package was not found. Attempting to reinstall."
apt-get clean && apt-get -qq update && apt-get install -y $package
if ! which $package >/dev/null; then
# If the reinstall fails, give up
echo "[X] Unable to install $package even after a retry. Exiting."
exit 1
fi
else
echo "[+] $package was successfully installed!"
fi
done
}
install_guacamole() {
echo "[$(date +%H:%M:%S)]: Installing Guacamole..."
cd /opt || exit 1
apt-get -qq install -y libcairo2-dev libjpeg62-dev libpng-dev libossp-uuid-dev libfreerdp-dev libpango1.0-dev libssh2-1-dev libssh-dev tomcat8 tomcat8-admin tomcat8-user
wget --progress=bar:force "http://apache.org/dyn/closer.cgi?action=download&filename=guacamole/1.0.0/source/guacamole-server-1.0.0.tar.gz" -O guacamole-server-1.0.0.tar.gz
tar -xf guacamole-server-1.0.0.tar.gz && cd guacamole-server-1.0.0 || echo "[-] Unable to find the Guacamole folder."
./configure &>/dev/null && make --quiet &>/dev/null && make --quiet install &>/dev/null || echo "[-] An error occurred while installing Guacamole."
ldconfig
cd /var/lib/tomcat8/webapps || echo "[-] Unable to find the tomcat8/webapps folder."
wget --progress=bar:force "http://apache.org/dyn/closer.cgi?action=download&filename=guacamole/1.0.0/binary/guacamole-1.0.0.war" -O guacamole.war
mkdir /etc/guacamole
mkdir /usr/share/tomcat8/.guacamole
cp /vagrant/resources/guacamole/user-mapping.xml /etc/guacamole/
cp /vagrant/resources/guacamole/guacamole.properties /etc/guacamole/
cp /vagrant/resources/guacamole/guacd.service /lib/systemd/system
sudo ln -s /etc/guacamole/guacamole.properties /usr/share/tomcat8/.guacamole/
sudo ln -s /etc/guacamole/user-mapping.xml /usr/share/tomcat8/.guacamole/
systemctl enable guacd
systemctl enable tomcat8
systemctl start guacd
systemctl start tomcat8
}
postinstall_tasks() {
# Include Splunk and Zeek in the PATH
echo export PATH="$PATH:/opt/splunk/bin:/opt/zeek/bin" >>~/.bashrc
echo "export SPLUNK_HOME=/opt/splunk" >>~/.bashrc
# Ping DetectionLab server for usage statistics
curl -s -A "DetectionLab-logger" "https:/ping.detectionlab.network/logger"
}
main() {
apt_install_prerequisites
modify_motd
test_prerequisites
install_splunk
download_palantir_osquery_config
install_fleet_import_osquery_config
install_velociraptor
install_suricata
install_zeek
install_guacamole
postinstall_tasks
}
main
exit 0

18
HyperV/check-eth0-ip.sh Normal file
View File

@@ -0,0 +1,18 @@
check_eth0_ip() {
ETH1_IP=$(ip -4 addr show eth1 | grep -oP '(?<=inet\s)\d+(\.\d+){3}' | head -1)
if [ "$ETH1_IP" != "192.168.38.105" ]; then
echo "Incorrect IP Address settings detected. Attempting to fix."
ifdown eth1
ip addr flush dev eth1
ifup eth1
ETH1_IP=$(ifconfig eth1 | grep 'inet addr' | cut -d ':' -f 2 | cut -d ' ' -f 1)
if [ "$ETH1_IP" == "192.168.38.105" ]; then
echo "[$(date +%H:%M:%S)]: The static IP has been fixed and set to 192.168.38.105"
else
echo "[$(date +%H:%M:%S)]: Failed to fix the broken static IP for eth1. Exiting because this will cause problems with other VMs."
exit 1
fi
fi
}
check_eth0_ip

View File

@@ -0,0 +1,31 @@
fix_eth0_static_ip() {
# There's a fun issue where dhclient keeps messing with eth0 despite the fact
# that eth0 has a static IP set. We workaround this by setting a static DHCP lease.
echo -e 'interface "eth0" {
send host-name = gethostname();
send dhcp-requested-address 192.168.38.105;
}' >>/etc/dhcp/dhclient.conf
netplan apply
# Set the ip address on eth0 and rename the adapter to eth1
ETH0_IP=$(ip -4 addr show eth0 | grep -oP '(?<=inet\s)\d+(\.\d+){3}')
if [ "$ETH0_IP" != "192.168.38.105" ]; then
MAC=$(ip a | grep "link/ether" | cut -d ' ' -f 6)
cat > /etc/netplan/01-netcfg.yaml << EOL
network:
ethernets:
eth0:
match:
macaddress: $MAC
dhcp4: no
addresses: [192.168.38.105/24]
gateway4: 192.168.38.1
nameservers:
addresses: [8.8.8.8,8.8.4.4]
set-name: eth1
version: 2
renderer: networkd
EOL
fi
}
fix_eth0_static_ip

4
HyperV/fix-motd.sh Normal file
View File

@@ -0,0 +1,4 @@
#! /bin/sh
# Fix the motd script
cp /etc/update-motd.d/20-detectionlab .
tr -d '\15\32' < 20-detectionlab > /etc/update-motd.d/20-detectionlab

View File

@@ -0,0 +1,5 @@
# See: https://www.thomasmaurer.ch/2016/01/change-hyper-v-vm-switch-of-virtual-machines-using-powershell/
# https://superuser.com/questions/1354658/hyperv-static-ip-with-vagrant#:~:text=%20HyperV%20-%20Static%20Ip%20with%20Vagrant%20,static%20IP%20address%20will%20be%20set...%20More%20
param ([String] $vmname)
Get-VM "$vmname" | Get-VMNetworkAdapter | Connect-VMNetworkAdapter -SwitchName "NATSwitch"

View File

@@ -0,0 +1,32 @@
# See: https://www.petri.com/using-nat-virtual-switch-hyper-v
If ("NATSwitch" -in (Get-VMSwitch | Select-Object -ExpandProperty Name) -eq $FALSE) {
'Creating Internal-only switch named "NATSwitch" on Windows Hyper-V host...'
New-VMSwitch -SwitchName "NATSwitch" -SwitchType Internal
New-NetIPAddress -IPAddress 192.168.38.1 -PrefixLength 24 -InterfaceAlias "vEthernet (NATSwitch)"
New-NetNAT -Name "NATNetwork" -InternalIPInterfaceAddressPrefix 192.168.38.0/24
}
else {
'"NATSwitch" for static IP configuration already exists; skipping'
}
If ("192.168.38.1" -in (Get-NetIPAddress | Select-Object -ExpandProperty IPAddress) -eq $FALSE) {
'Registering new IP address 192.168.38.1 on Windows Hyper-V host...'
New-NetIPAddress -IPAddress 192.168.38.1 -PrefixLength 24 -InterfaceAlias "vEthernet (NATSwitch)"
}
else {
'"192.168.38.1" for static IP configuration already registered; skipping'
}
If ("192.168.38.0/24" -in (Get-NetNAT | Select-Object -ExpandProperty InternalIPInterfaceAddressPrefix) -eq $FALSE) {
'Registering new NAT adapter for 192.168.38.0/24 on Windows Hyper-V host...'
New-NetNAT -Name "NATNetwork" -InternalIPInterfaceAddressPrefix 192.168.38.0/24
}
else {
'"192.168.38.0/24" for static IP configuration already registered; skipping'
}

View File

@@ -0,0 +1,3 @@
param ([String] $vmname)
Remove-VMNetworkAdapter -VMName $vmname -Name "Network Adapter"

View File

@@ -0,0 +1,29 @@
# Source: https://github.com/StefanScherer/adfs
param ([String] $ip, [String] $dns)
Write-Host "$('[{0:HH:mm}]' -f (Get-Date))"
Write-Host "Setting IP address and DNS information."
Write-Host "If this step times out, it's because vagrant is connecting to the VM on the wrong interface"
Write-Host "See https://github.com/clong/DetectionLab/issues/114 for more information"
$subnet = $ip -replace "\.\d+$", ""
$name = (Get-NetIPAddress -AddressFamily IPv4 `
| Where-Object -FilterScript { ($_.IPAddress).StartsWith("169") } `
).InterfaceAlias
if (!$name) {
$name = (Get-NetIPAddress -AddressFamily IPv4 `
| Where-Object -FilterScript { ($_.IPAddress).StartsWith("192.168.38") } `
).InterfaceAlias
}
if ($name) {
Write-Host "Set IP address to $ip of interface $name"
& netsh.exe int ip set address $name static $ip 255.255.255.0 "$subnet.1"
Write-Host "Set DNS server address to $dns of interface $name"
& netsh.exe interface ipv4 add dnsserver $name address=192.168.38.102 index=1
if ($dns) {
& netsh.exe interface ipv4 add dnsserver $name address=$dns index=2
}
} else {
Write-Error "Could not find a interface with subnet $subnet.xx"
}

View File

@@ -0,0 +1,8 @@
# See: https://www.thomasmaurer.ch/2016/01/change-hyper-v-vm-switch-of-virtual-machines-using-powershell/
param ([String] $vmname)
# Get-VM $vmname | Get-VMNetworkAdapter | Connect-VMNetworkAdapter -SwitchName "NATSwitch"
if (((Get-VMNetworkAdapter -VMName $vmname).Name).Contains("NATAdapter")){
Write-Host "The NATAdapter already exits"
} else {
Add-VMNetworkAdapter -VMName $vmname -SwitchName "NATSwitch" -Name NATAdapter -DeviceNaming On
}