trinitor/DetectionLab
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Merge branch 'master' into H8to-patch-1

Browse Source
This commit is contained in:
Chris Long
2018-07-26 15:12:06 -07:00
committed by GitHub
parent 84297d0dc5 428b248f22
commit 199075e412
9 changed files with 284 additions and 226 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
README.md
View File

@@ -39,6 +39,7 @@ OSX 10.12.5 | 1.9.3 | 1.0.0 | VMWare Fusion (8.5.8)
OSX 10.12.6 | 2.0.1 | 1.1.3 | VMWare Fusion (8.5.9) OSX 10.12.6 | 2.0.1 | 1.1.3 | VMWare Fusion (8.5.9)
OSX 10.12.6 | 2.0.1 | 1.1.3 | VMWare Fusion (8.5.10) OSX 10.12.6 | 2.0.1 | 1.1.3 | VMWare Fusion (8.5.10)
OSX 10.12.6 | 2.0.3 | 1.2.1 | VMWare Fusion (10.1.1) OSX 10.12.6 | 2.0.3 | 1.2.1 | VMWare Fusion (10.1.1)
OSX 10.13.6 | 2.1.2 | 1.2.4 | VMWare Fusion (10.1.2)
Ubuntu 16.04 | 2.0.1 | 1.1.3 | Virtualbox (5.1) Ubuntu 16.04 | 2.0.1 | 1.1.3 | Virtualbox (5.1)
Ubuntu 16.04 | 2.0.2 | N/A | Virtualbox (5.2) Ubuntu 16.04 | 2.0.2 | N/A | Virtualbox (5.2)
Ubuntu 16.04 | 2.0.3 | 1.2.1 | Virtualbox (5.2) Ubuntu 16.04 | 2.0.3 | 1.2.1 | Virtualbox (5.2)
@@ -53,19 +54,19 @@ Ubuntu 16.04 | 2.0.3 | 1.2.1 | Virtualbox (5.2)
DetectionLab now contains build scripts for \*NIX, MacOS, and Windows users! DetectionLab now contains build scripts for \*NIX, MacOS, and Windows users!
There is a single build script that supports 3 different options: There is a single build script that supports 3 different options:
- `./build.sh <virtualbox|vmware_fusion>` - Builds the entire lab from scratch. Takes 3-5 hours depending on hardware resources and bandwidth - `./build.sh <virtualbox|vmware_desktop>` - Builds the entire lab from scratch. Takes 3-5 hours depending on hardware resources and bandwidth
- `./build.sh <virtualbox|vmware_fusion> --vagrant-only` - Downloads pre-built Packer boxes from https://detectionlab.network and builds the lab from those boxes. This option is recommended if you have more bandwidth than time or are having trouble building boxes. - `./build.sh <virtualbox|vmware_desktop> --vagrant-only` - Downloads pre-built Packer boxes from https://detectionlab.network and builds the lab from those boxes. This option is recommended if you have more bandwidth than time or are having trouble building boxes.
- `./build.sh <virtualbox|vmware_fusion> --packer-only` - This option only builds the Packer boxes and will not use Vagrant to start up the lab. - `./build.sh <virtualbox|vmware_desktop> --packer-only` - This option only builds the Packer boxes and will not use Vagrant to start up the lab.
Windows users will want to use the following script: Windows users will want to use the following script:
- `./build.ps1 -ProviderName=<virtualbox|vmware_workstation>` - Builds the entire lab from scratch. Takes 3-5 hours depending on hardware resources and bandwidth - `./build.ps1 -ProviderName=<virtualbox|vmware_desktop>` - Builds the entire lab from scratch. Takes 3-5 hours depending on hardware resources and bandwidth
- `./build.ps1 -ProviderName=<virtualbox|vmware_workstation> -VagrantOnly` - Downloads pre-built Packer boxes from https://detectionlab.network and builds the lab from those boxes. This option is recommended if you have more bandwidth than time or are having trouble building boxes. - `./build.ps1 -ProviderName=<virtualbox|vmware_desktop> -VagrantOnly` - Downloads pre-built Packer boxes from https://detectionlab.network and builds the lab from those boxes. This option is recommended if you have more bandwidth than time or are having trouble building boxes.
--- ---
## Building DetectionLab from Scratch ## Building DetectionLab from Scratch
1. Determine which Vagrant provider you want to use. 1. Determine which Vagrant provider you want to use.
* Note: Virtualbox is free, the [VMWare vagrant plugin](https://www.vagrantup.com/vmware/#buy-now) is $80. * Note: Virtualbox is free, the [VMWare Desktop vagrant plugin](https://www.vagrantup.com/vmware/#buy-now) is $80.
**NOTE:** If you'd like to save time, you can skip the building of the Packer boxes and download the boxes directly from https://detectionlab.network and put them into the `Boxes` directory: **NOTE:** If you'd like to save time, you can skip the building of the Packer boxes and download the boxes directly from https://detectionlab.network and put them into the `Boxes` directory:
@@ -107,7 +108,7 @@ $ packer build --only=[vmware|virtualbox]-iso windows_2016.json
## Basic Vagrant Usage ## Basic Vagrant Usage
Vagrant commands must be run from the "Vagrant" folder. Vagrant commands must be run from the "Vagrant" folder.
* Bring up all Detection Lab hosts: `vagrant up` (optional `--provider=[virtualbox|vmware_fusion|vmware_workstation]`) * Bring up all Detection Lab hosts: `vagrant up` (optional `--provider=[virtualbox|vmware_desktop]`)
* Bring up a specific host: `vagrant up <hostname>` * Bring up a specific host: `vagrant up <hostname>`
* Restart a specific host: `vagrant reload <hostname>` * Restart a specific host: `vagrant reload <hostname>`
* Restart a specific host and re-run the provision process: `vagrant reload <hostname> --provision` * Restart a specific host and re-run the provision process: `vagrant reload <hostname> --provision`

365
Vagrant/bootstrap.sh
View File

@@ -1,168 +1,229 @@
#! /bin/bash #! /bin/bash
# Install key and apt source for MongoDB install_mongo_db_apt_key() {
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927 # Install key and apt source for MongoDB
echo "deb http://repo.mongodb.org/apt/ubuntu "$(lsb_release -sc)"/mongodb-org/3.2 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.2.list apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927
echo "deb http://repo.mongodb.org/apt/ubuntu $(lsb_release -sc)/mongodb-org/3.2 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.2.list
}
# Install prerequisites and useful tools apt_install_prerequisites() {
apt-get update # Install prerequisites and useful tools
apt-get install -y jq whois build-essential git docker docker-compose unzip mongodb-org apt-get update
apt-get install -y jq whois build-essential git docker docker-compose unzip mongodb-org
}
# Fix static IP if it's not set correctly fix_eth1_static_ip() {
ETH1_IP=$(ifconfig eth1 | grep 'inet addr' | cut -d ':' -f 2 | cut -d ' ' -f 1) # Fix static IP if it's not set correctly
if [ "$ETH1_IP" != "192.168.38.5" ]; then
echo "Incorrect IP Address settings detected. Attempting to fix."
ifdown eth1
ip addr flush dev eth1
ifup eth1
ETH1_IP=$(ifconfig eth1 | grep 'inet addr' | cut -d ':' -f 2 | cut -d ' ' -f 1) ETH1_IP=$(ifconfig eth1 | grep 'inet addr' | cut -d ':' -f 2 | cut -d ' ' -f 1)
if [ "$ETH1_IP" == "192.168.38.5" ]; then if [ "$ETH1_IP" != "192.168.38.5" ]; then
echo "The static IP has been fixed and set to 192.168.38.5" echo "Incorrect IP Address settings detected. Attempting to fix."
else ifdown eth1
echo "Failed to fix the broken static IP for eth1. Exiting because this will cause problems with other VMs." ip addr flush dev eth1
exit 1 ifup eth1
ETH1_IP=$(ifconfig eth1 | grep 'inet addr' | cut -d ':' -f 2 | cut -d ' ' -f 1)
if [ "$ETH1_IP" == "192.168.38.5" ]; then
echo "The static IP has been fixed and set to 192.168.38.5"
else
echo "Failed to fix the broken static IP for eth1. Exiting because this will cause problems with other VMs."
exit 1
fi
fi fi
fi }
install_python() {
# Install Python 3.6.4 # Install Python 3.6.4
echo "Installing Python v3.6.4..." if ! which /usr/local/bin/python3.6 > /dev/null; then
wget https://www.python.org/ftp/python/3.6.4/Python-3.6.4.tgz echo "Installing Python v3.6.4..."
tar -xvf Python-3.6.4.tgz wget https://www.python.org/ftp/python/3.6.4/Python-3.6.4.tgz
cd Python-3.6.4 tar -xvf Python-3.6.4.tgz
./configure && make && make install cd Python-3.6.4 || exit
cd /home/vagrant ./configure && make && make install
cd /home/vagrant || exit
# Install Golang v1.8
echo "Installing GoLang v1.8..."
wget https://storage.googleapis.com/golang/go1.8.linux-amd64.tar.gz
tar -xvf go1.8.linux-amd64.tar.gz
mv go /usr/local
mkdir /home/vagrant/.go
chown vagrant:vagrant /home/vagrant/.go
mkdir /root/.go
echo 'export GOPATH=$HOME/.go' >> /home/vagrant/.bashrc
echo 'export GOROOT=/usr/local/go' >> /home/vagrant/.bashrc
echo 'export GOPATH=$HOME/.go' >> /root/.bashrc
echo '/home/vagrant/.bashrc' >> /root/.bashrc
source ~/.bashrc
sudo update-alternatives --install "/usr/bin/go" "go" "/usr/local/go/bin/go" 0
sudo update-alternatives --set go /usr/local/go/bin/go
/usr/bin/go get -u github.com/howeyc/gopass
# Check if Splunk is already installed
if [ -f "/opt/splunk/bin/splunk" ]
then echo "Splunk is already installed"
else else
echo "Installing Splunk..." echo "Python seems to be downloaded already.. Skipping."
# Get Splunk.com into the DNS cache. Sometimes resolution randomly fails during wget below
dig @8.8.8.8 splunk.com
# Download Splunk
wget --progress=bar:force -O splunk-7.1.1-8f0ead9ec3db-linux-2.6-amd64.deb 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.1.1&product=splunk&filename=splunk-7.1.1-8f0ead9ec3db-linux-2.6-amd64.deb&wget=true'
dpkg -i splunk-7.1.1-8f0ead9ec3db-linux-2.6-amd64.deb
/opt/splunk/bin/splunk start --accept-license --answer-yes --no-prompt --seed-passwd changeme
/opt/splunk/bin/splunk add index wineventlog -auth 'admin:changeme'
/opt/splunk/bin/splunk add index osquery -auth 'admin:changeme'
/opt/splunk/bin/splunk add index osquery-status -auth 'admin:changeme'
/opt/splunk/bin/splunk add index sysmon -auth 'admin:changeme'
/opt/splunk/bin/splunk add index powershell -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_forwarder/splunk-add-on-for-microsoft-windows_483.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/add-on-for-microsoft-sysmon_607.tgz -auth 'admin:changeme'
# Add a Splunk TCP input on port 9997
echo -e "[splunktcp://9997]\nconnection_host = ip" > /opt/splunk/etc/apps/search/local/inputs.conf
# Add props.conf and transforms.conf
cp /vagrant/resources/splunk_server/props.conf /opt/splunk/etc/apps/search/local/
cp /vagrant/resources/splunk_server/transforms.conf /opt/splunk/etc/apps/search/local/
# Skip Splunk Tour and Change Password Dialog
touch /opt/splunk/etc/.ui_login
# Enable SSL Login for Splunk
echo '[settings]
enableSplunkWebSSL = true' > /opt/splunk/etc/system/local/web.conf
# Reboot Splunk to make changes take effect
/opt/splunk/bin/splunk restart
/opt/splunk/bin/splunk enable boot-start
fi fi
}
# Install Fleet install_golang() {
echo "Installing Fleet..." if [ ! -f "go1.8.linux-amd64.tar.gz" ]; then
echo -e "\n127.0.0.1 kolide" >> /etc/hosts # Install Golang v1.8
git clone https://github.com/kolide/kolide-quickstart.git echo "Installing GoLang v1.8..."
cd kolide-quickstart wget https://storage.googleapis.com/golang/go1.8.linux-amd64.tar.gz
cp /vagrant/resources/fleet/server.* . tar -xvf go1.8.linux-amd64.tar.gz
sed -i 's/ -it//g' demo.sh mv go /usr/local
sed -i 's#kolide/fleet:latest#kolide/fleet:1.0.8#g' docker-compose.yml mkdir /home/vagrant/.go
./demo.sh up simple chown vagrant:vagrant /home/vagrant/.go
# Set the enrollment secret to match what we deploy to Windows hosts mkdir /root/.go
docker run --rm --network=kolidequickstart_default mysql:5.7 mysql -h mysql -u kolide --password=kolide -e 'update app_configs set osquery_enroll_secret = "enrollmentsecret" where id=1;' --batch kolide echo 'export GOPATH=$HOME/.go' >> /home/vagrant/.bashrc
# Set snapshot events to be split into multiple events echo 'export GOROOT=/usr/local/go' >> /home/vagrant/.bashrc
docker run --rm --network=kolidequickstart_default mysql:5.7 mysql -h mysql -u kolide --password=kolide -e 'insert into options (name, type, value) values ("logger_snapshot_event_type", 2, "true");' --batch kolide echo 'export GOPATH=$HOME/.go' >> /root/.bashrc
echo "Updated enrollment secret" echo '/home/vagrant/.bashrc' >> /root/.bashrc
cd /home/vagrant source /root/.bashrc
sudo update-alternatives --install "/usr/bin/go" "go" "/usr/local/go/bin/go" 0
sudo update-alternatives --set go /usr/local/go/bin/go
/usr/bin/go get -u github.com/howeyc/gopass
else
echo "GoLang seems to be downloaded already.. Skipping."
fi
}
# Import Palantir osquery configs into Fleet
echo "Downloading Palantir configs..."
git clone https://github.com/palantir/osquery-configuration.git
git clone https://github.com/kolide/configimporter.git
cd configimporter
go build
cd /home/vagrant
# Modify the config to work with config importer install_splunk() {
cat /home/vagrant/osquery-configuration/Endpoints/Windows/osquery.conf | sed 's#packs/#../packs/#g' | grep -v unwanted-chrome-extensions | grep -v security-tooling-checks | grep -v performance-metrics | grep -v logger_snapshot_event_type > /home/vagrant/osquery-configuration/Endpoints/Windows/osquery_to_import.conf # Check if Splunk is already installed
# Install configimporter if [ -f "/opt/splunk/bin/splunk" ]; then
echo "Installing configimporter" echo "Splunk is already installed"
echo "Sleeping for 5" else
sleep 5 echo "Installing Splunk..."
export CONFIGIMPORTER_PASSWORD='admin123#' # Get Splunk.com into the DNS cache. Sometimes resolution randomly fails during wget below
cd /home/vagrant/osquery-configuration/Endpoints/Windows/ dig @8.8.8.8 splunk.com
# Fleet requires you to login before importing packs # Download Splunk
# Login wget --progress=bar:force -O splunk-7.1.2-a0c72a66db66-linux-2.6-amd64.deb 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.1.2&product=splunk&filename=splunk-7.1.2-a0c72a66db66-linux-2.6-amd64.deb&wget=true'
curl 'https://192.168.38.5:8412/api/v1/kolide/login' -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/login' -H 'authority: 192.168.38.5:8412' --data-binary '{"username":"admin","password":"admin123#"}' --compressed --insecure dpkg -i splunk-7.1.2-a0c72a66db66-linux-2.6-amd64.deb
sleep 1 /opt/splunk/bin/splunk start --accept-license --answer-yes --no-prompt --seed-passwd changeme
curl 'https://192.168.38.5:8412/setup' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'upgrade-insecure-requests: 1' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' -H 'authority: 192.168.38.5:8412' --compressed --insecure /opt/splunk/bin/splunk add index wineventlog -auth 'admin:changeme'
sleep 1 /opt/splunk/bin/splunk add index osquery -auth 'admin:changeme'
# Setup organization name and email address /opt/splunk/bin/splunk add index osquery-status -auth 'admin:changeme'
curl 'https://192.168.38.5:8412/api/v1/setup' -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/setup' -H 'authority: 192.168.38.5:8412' --data-binary '{"kolide_server_url":"https://192.168.38.5:8412","org_info":{"org_name":"detectionlab"},"admin":{"admin":true,"email":"example@example.com","password":"admin123#","password_confirmation":"admin123#","username":"admin"}}' --compressed --insecure /opt/splunk/bin/splunk add index sysmon -auth 'admin:changeme'
sleep 3 /opt/splunk/bin/splunk add index powershell -auth 'admin:changeme'
# Import all Windows configs /opt/splunk/bin/splunk install app /vagrant/resources/splunk_forwarder/splunk-add-on-for-microsoft-windows_500.tgz -auth 'admin:changeme'
/home/vagrant/configimporter/configimporter -host https://localhost:8412 -user 'admin' -config osquery_to_import.conf /opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/add-on-for-microsoft-sysmon_800.tgz -auth 'admin:changeme'
# Add a Splunk TCP input on port 9997
echo -e "[splunktcp://9997]\nconnection_host = ip" > /opt/splunk/etc/apps/search/local/inputs.conf
# Add props.conf and transforms.conf
cp /vagrant/resources/splunk_server/props.conf /opt/splunk/etc/apps/search/local/
cp /vagrant/resources/splunk_server/transforms.conf /opt/splunk/etc/apps/search/local/
# Skip Splunk Tour and Change Password Dialog
touch /opt/splunk/etc/.ui_login
# Enable SSL Login for Splunk
echo '[settings]
enableSplunkWebSSL = true' > /opt/splunk/etc/system/local/web.conf
# Reboot Splunk to make changes take effect
/opt/splunk/bin/splunk restart
/opt/splunk/bin/splunk enable boot-start
fi
}
# Get auth token install_fleet() {
TOKEN=$(curl 'https://192.168.38.5:8412/api/v1/kolide/login' -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/login' -H 'authority: 192.168.38.5:8412' --data-binary '{"username":"admin","password":"admin123#"}' --compressed --insecure | grep token | cut -d '"' -f 4) # Install Fleet
# Set all packs to be targeted to Windows hosts if [ -f "/home/vagrant/kolide-quickstart" ]; then
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/1' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/3/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure echo "Fleet is already installed"
sleep 1 else
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/2' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/3/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure echo "Installing Fleet..."
sleep 1 echo -e "\n127.0.0.1 kolide" >> /etc/hosts
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/3' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/3/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure git clone https://github.com/kolide/kolide-quickstart.git
sleep 1 cd kolide-quickstart || echo "Something went wrong while trying to clone the kolide-quickstart repository"
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/4' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/3/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure cp /vagrant/resources/fleet/server.* .
sleep 1 sed -i 's/ -it//g' demo.sh
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/5' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/3/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure sed -i 's#kolide/fleet:latest#kolide/fleet:1.0.8#g' docker-compose.yml
# Rename primary pack ./demo.sh up simple
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/5' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/5/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"name":"windows-pack"}' --compressed --insecure # Set the enrollment secret to match what we deploy to Windows hosts
# Add Splunk monitors for Fleet docker run --rm --network=kolidequickstart_default mysql:5.7 mysql -h mysql -u kolide --password=kolide -e 'update app_configs set osquery_enroll_secret = "enrollmentsecret" where id=1;' --batch kolide
/opt/splunk/bin/splunk add monitor "/home/vagrant/kolide-quickstart/osquery_result" -index osquery -sourcetype 'osquery:json' -auth 'admin:changeme' # Set snapshot events to be split into multiple events
/opt/splunk/bin/splunk add monitor "/home/vagrant/kolide-quickstart/osquery_status" -index osquery-status -sourcetype 'osquery:status' -auth 'admin:changeme' docker run --rm --network=kolidequickstart_default mysql:5.7 mysql -h mysql -u kolide --password=kolide -e 'insert into options (name, type, value) values ("logger_snapshot_event_type", 2, "true");' --batch kolide
echo "Updated enrollment secret"
cd /home/vagrant || exit
fi
}
# Install Mitre's Caldera download_palantir_osquery_config() {
echo "Installing Caldera..." if [ -f /home/vagrant/osquery-configuration ]; then
cd /home/vagrant echo "osquery configs have already been downloaded"
git clone https://github.com/mitre/caldera.git else
cd /home/vagrant/caldera/caldera # Import Palantir osquery configs into Fleet
pip3.6 install -r requirements.txt echo "Downloading Palantir configs..."
git clone https://github.com/palantir/osquery-configuration.git
git clone https://github.com/kolide/configimporter.git
cd configimporter || exit
go build
cd /home/vagrant || exit
fi
}
# Add a Systemd service for MongoDB import_osquery_config_into_fleet() {
# https://www.howtoforge.com/tutorial/install-mongodb-on-ubuntu-16.04/ if [ -f "/home/vagrant/osquery-configuration/Endpoints/Windows/osquery_to_import.conf" ]; then
cp /vagrant/resources/caldera/mongod.service /lib/systemd/system/mongod.service echo "The osquery configuration has already been imported into Fleet"
# Create Systemd service for Caldera else
cp /vagrant/resources/caldera/caldera.service /lib/systemd/system/caldera.service # Modify the config to work with config importer
# Enable replication cat /home/vagrant/osquery-configuration/Endpoints/Windows/osquery.conf | sed 's#packs/#../packs/#g' | grep -v unwanted-chrome-extensions | grep -v security-tooling-checks | grep -v performance-metrics | grep -v logger_snapshot_event_type > /home/vagrant/osquery-configuration/Endpoints/Windows/osquery_to_import.conf
echo 'replication: # Install configimporter
replSetName: caldera' >> /etc/mongod.conf echo "Installing configimporter"
service mongod start echo "Sleeping for 5"
systemctl enable mongod.service sleep 5
cd /home/vagrant/caldera export CONFIGIMPORTER_PASSWORD='admin123#'
mkdir -p dep/crater/crater cd /home/vagrant/osquery-configuration/Endpoints/Windows/ || exit
wget https://github.com/mitre/caldera-crater/releases/download/v0.1.0/CraterMainWin8up.exe -O /home/vagrant/caldera/dep/crater/crater/CraterMain.exe # Fleet requires you to login before importing packs
service caldera start # Login
systemctl enable caldera.service curl 'https://192.168.38.5:8412/api/v1/kolide/login' -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/login' -H 'authority: 192.168.38.5:8412' --data-binary '{"username":"admin","password":"admin123#"}' --compressed --insecure
sleep 1
curl 'https://192.168.38.5:8412/setup' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'upgrade-insecure-requests: 1' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' -H 'authority: 192.168.38.5:8412' --compressed --insecure
sleep 1
# Setup organization name and email address
curl 'https://192.168.38.5:8412/api/v1/setup' -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/setup' -H 'authority: 192.168.38.5:8412' --data-binary '{"kolide_server_url":"https://192.168.38.5:8412","org_info":{"org_name":"detectionlab"},"admin":{"admin":true,"email":"example@example.com","password":"admin123#","password_confirmation":"admin123#","username":"admin"}}' --compressed --insecure
sleep 3
# Import all Windows configs
/home/vagrant/configimporter/configimporter -host https://localhost:8412 -user 'admin' -config osquery_to_import.conf
# Get auth token
TOKEN=$(curl 'https://192.168.38.5:8412/api/v1/kolide/login' -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/login' -H 'authority: 192.168.38.5:8412' --data-binary '{"username":"admin","password":"admin123#"}' --compressed --insecure | grep token | cut -d '"' -f 4)
# Set all packs to be targeted to Windows hosts
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/1' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/3/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure
sleep 1
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/2' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/3/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure
sleep 1
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/3' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/3/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure
sleep 1
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/4' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/3/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure
sleep 1
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/5' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/3/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"label_ids":[10]}' --compressed --insecure
# Rename primary pack
curl 'https://192.168.38.5:8412/api/v1/kolide/packs/5' -X PATCH -H 'origin: https://192.168.38.5:8412' -H 'accept-encoding: gzip, deflate, br' -H 'accept-language: en-US,en;q=0.9' -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' -H 'accept: application/json' -H 'referer: https://192.168.38.5:8412/packs/5/edit' -H 'authority: 192.168.38.5:8412' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' --data-binary '{"name":"windows-pack"}' --compressed --insecure
# Add Splunk monitors for Fleet
/opt/splunk/bin/splunk add monitor "/home/vagrant/kolide-quickstart/osquery_result" -index osquery -sourcetype 'osquery:json' -auth 'admin:changeme'
/opt/splunk/bin/splunk add monitor "/home/vagrant/kolide-quickstart/osquery_status" -index osquery-status -sourcetype 'osquery:status' -auth 'admin:changeme'
fi
}
install_caldera() {
if [ -f "/lib/systemd/system/caldera.service" ]; then
echo "Caldera is already installed... Skipping"
else
# Install Mitre's Caldera
echo "Installing Caldera..."
cd /home/vagrant || exit
git clone https://github.com/mitre/caldera.git
cd /home/vagrant/caldera/caldera || exit
pip3.6 install -r requirements.txt
# Add a Systemd service for MongoDB
# https://www.howtoforge.com/tutorial/install-mongodb-on-ubuntu-16.04/
cp /vagrant/resources/caldera/mongod.service /lib/systemd/system/mongod.service
# Create Systemd service for Caldera
cp /vagrant/resources/caldera/caldera.service /lib/systemd/system/caldera.service
# Enable replication
echo 'replication:
replSetName: caldera' >> /etc/mongod.conf
service mongod start
systemctl enable mongod.service
cd /home/vagrant/caldera || exit
mkdir -p dep/crater/crater
wget https://github.com/mitre/caldera-crater/releases/download/v0.1.0/CraterMainWin8up.exe -O /home/vagrant/caldera/dep/crater/crater/CraterMain.exe
service caldera start
systemctl enable caldera.service
fi
}
main() {
install_mongo_db_apt_key
apt_install_prerequisites
fix_eth1_static_ip
install_python
install_golang
install_splunk
install_fleet
download_palantir_osquery_config
import_osquery_config_into_fleet
install_caldera
}
main
exit 0

BIN
Vagrant/resources/splunk_forwarder/splunk-add-on-for-microsoft-windows_483.tgz
View File

Binary file not shown.

BIN
Vagrant/resources/splunk_forwarder/splunk-add-on-for-microsoft-windows_500.tgz Normal file
View File

Binary file not shown.

BIN
Vagrant/resources/splunk_server/add-on-for-microsoft-sysmon_607.tgz
View File

Binary file not shown.

BIN
Vagrant/resources/splunk_server/add-on-for-microsoft-sysmon_800.tgz Normal file
View File

Binary file not shown.

2
Vagrant/scripts/install-windows_ta.ps1
View File

@@ -9,7 +9,7 @@ If (test-path "C:\Program Files\SplunkUniversalForwarder\etc\apps\Splunk_TA_wind
} }
# Install Windows TA (this only needs to be done on the WEF server) # Install Windows TA (this only needs to be done on the WEF server)
$windowstaPath = "C:\vagrant\resources\splunk_forwarder\splunk-add-on-for-microsoft-windows_483.tgz" $windowstaPath = "C:\vagrant\resources\splunk_forwarder\splunk-add-on-for-microsoft-windows_500.tgz"
$inputsPath = "C:\Program Files\SplunkUniversalForwarder\etc\apps\Splunk_TA_windows\local\inputs.conf" $inputsPath = "C:\Program Files\SplunkUniversalForwarder\etc\apps\Splunk_TA_windows\local\inputs.conf"
Write-Host "Installing the Windows TA" Write-Host "Installing the Windows TA"
Start-Process -FilePath "C:\Program Files\SplunkUniversalForwarder\bin\splunk.exe" -ArgumentList "install app $windowstaPath -auth admin:changeme" -NoNewWindow Start-Process -FilePath "C:\Program Files\SplunkUniversalForwarder\bin\splunk.exe" -ArgumentList "install app $windowstaPath -auth admin:changeme" -NoNewWindow

44
build.ps1
View File

@@ -9,7 +9,7 @@
DetectionLab deployment. It checks: DetectionLab deployment. It checks:
* If Packer and Vagrant are installed * If Packer and Vagrant are installed
* If VirtualBox or VMWare are installed * If VirtualBox or VMware are installed
* If the proper vagrant plugins are available * If the proper vagrant plugins are available
* Various aspects of system health * Various aspects of system health
@@ -20,7 +20,7 @@
https://github.com/clong/DetectionLab/issues https://github.com/clong/DetectionLab/issues
.PARAMETER ProviderName .PARAMETER ProviderName
The Hypervisor you're using for the lab. Valid options are 'virtualbox' or 'vmware_workstation' The Hypervisor you're using for the lab. Valid options are 'virtualbox' or 'vmware_desktop'
.PARAMETER PackerPath .PARAMETER PackerPath
The full path to the packer executable. Default is C:\Hashicorp\packer.exe The full path to the packer executable. Default is C:\Hashicorp\packer.exe
@@ -33,11 +33,11 @@
This builds the DetectionLab using virtualbox and the default path for packer (C:\Hashicorp\packer.exe) This builds the DetectionLab using virtualbox and the default path for packer (C:\Hashicorp\packer.exe)
.EXAMPLE .EXAMPLE
build.ps1 -ProviderName vmware_workstation -PackerPath 'C:\packer.exe' build.ps1 -ProviderName vmware_desktop -PackerPath 'C:\packer.exe'
This builds the DetectionLab using Vmware and sets the packer path to 'C:\packer.exe' This builds the DetectionLab using VMware and sets the packer path to 'C:\packer.exe'
.EXAMPLE .EXAMPLE
build.ps1 -ProviderName vmware_workstation -VagrantOnly build.ps1 -ProviderName vmware_desktop -VagrantOnly
This command builds the DetectionLab using vmware and skips the packer process, downloading the boxes instead. This command builds the DetectionLab using vmware and skips the packer process, downloading the boxes instead.
#> #>
@@ -45,7 +45,7 @@
[cmdletbinding()] [cmdletbinding()]
Param( Param(
# Vagrant provider to use. # Vagrant provider to use.
[ValidateSet('virtualbox', 'vmware_workstation')] [ValidateSet('virtualbox', 'vmware_desktop)]
[string]$ProviderName, [string]$ProviderName,
[string]$PackerPath = 'C:\Hashicorp\packer.exe', [string]$PackerPath = 'C:\Hashicorp\packer.exe',
[switch]$VagrantOnly [switch]$VagrantOnly
@@ -120,26 +120,32 @@ function check_virtualbox_installed {
} }
function check_vmware_workstation_installed { function check_vmware_workstation_installed {
Write-Verbose '[check_vmware_workstation_installed] Running..' Write-Verbose '[check_vmware_workstation_installed] Running..'
if (install_checker -Name "VMWare Workstation") { if (install_checker -Name "VMware Workstation") {
Write-Verbose '[check_vmware_workstation_installed] Vmware found.' Write-Verbose '[check_vmware_workstation_installed] VMware Workstation found.'
return $true return $true
} }
else { else {
Write-Verbose '[check_vmware_workstation_installed] Vmware not found.' Write-Verbose '[check_vmware_workstation_installed] VMware Workstation not found.'
return $false return $false
} }
} }
function check_vmware_vagrant_plugin_installed { function check_vmware_vagrant_plugin_installed {
Write-Verbose '[check_vmware_vagrant_plugin_installed] Running..' Write-Verbose '[check_vmware_vagrant_plugin_installed] Running..'
if (vagrant plugin list | Select-String 'vagrant-vmware-workstation') { if (vagrant plugin list | Select-String 'vagrant-vmware-desktop') {
Write-Verbose '[check_vmware_vagrant_plugin_installed] VMware vagrant plugin found.' Write-Verbose 'The vagrant VMware Workstation plugin is no longer supported.'
Write-Verbose 'Please upgrade to the VMware Desktop plugin: https://www.vagrantup.com/docs/vmware/installation.html'
return $false
}
if (vagrant plugin list | Select-String 'vagrant-vmware-desktop') {
Write-Verbose '[check_vmware_vagrant_plugin_installed] Vagrant VMware Desktop plugin found.'
return $true return $true
} }
else { else {
Write-Host 'VMWare Workstation is installed, but the Vagrant plugin is not.' Write-Host 'VMware Workstation is installed, but the Vagrant plugin is not.'
Write-Host 'Visit https://www.vagrantup.com/vmware/index.html#buy-now for more information on how to purchase and install it' Write-Host 'Visit https://www.vagrantup.com/vmware/index.html#buy-now for more information on how to purchase and install it'
Write-Host 'VMWare Workstation will not be listed as a provider until the Vagrant plugin has been installed.' Write-Host 'VMware Workstation will not be listed as a provider until the Vagrant plugin has been installed.'
Write-Host 'NOTE: The plugin does not work with trial versions of VMware Workstation'
return $false return $false
} }
} }
@@ -154,17 +160,17 @@ function list_providers {
} }
if (check_vmware_workstation_installed) { if (check_vmware_workstation_installed) {
if (check_vmware_vagrant_plugin_installed) { if (check_vmware_vagrant_plugin_installed) {
Write-Host '[*] vmware_workstation' Write-Host '[*] vmware_desktop'
} }
} }
if ((-Not (check_virtualbox_installed)) -and (-Not (check_vmware_workstation_installed))) { if ((-Not (check_virtualbox_installed)) -and (-Not (check_vmware_workstation_installed))) {
Write-Error 'You need to install a provider such as VirtualBox or VMware Workstation to continue.' Write-Error 'You need to install a provider such as VirtualBox or VMware Workstation to continue.'
break break
} }
while (-Not ($ProviderName -eq 'virtualbox' -or $ProviderName -eq 'vmware_workstation')) { while (-Not ($ProviderName -eq 'virtualbox' -or $ProviderName -eq 'vmware_desktop')) {
$ProviderName = Read-Host 'Which provider would you like to use?' $ProviderName = Read-Host 'Which provider would you like to use?'
Write-Debug "ProviderName = $ProviderName" Write-Debug "ProviderName = $ProviderName"
if (-Not ($ProviderName -eq 'virtualbox' -or $ProviderName -eq 'vmware_workstation')) { if (-Not ($ProviderName -eq 'virtualbox' -or $ProviderName -eq 'vmware_desktop')) {
Write-Error "Please choose a valid provider. $ProviderName is not a valid option" Write-Error "Please choose a valid provider. $ProviderName is not a valid option"
} }
} }
@@ -222,7 +228,7 @@ function preflight_checks {
Write-Verbose '[preflight_checks] Running..' Write-Verbose '[preflight_checks] Running..'
# Check to see that no boxes exist # Check to see that no boxes exist
if (-Not ($VagrantOnly)) { if (-Not ($VagrantOnly)) {
Write-Verbose '[preflight_checks] Checking if packer is installed' Write-Verbose '[preflight_checks] Checking if Packer is installed'
check_packer check_packer
# Check Packer Version against known bad # Check Packer Version against known bad
@@ -235,7 +241,7 @@ function preflight_checks {
break break
} }
} }
Write-Verbose '[preflight_checks] Checking if vagrant is installed' Write-Verbose '[preflight_checks] Checking if Vagrant is installed'
check_vagrant check_vagrant
Write-Verbose '[preflight_checks] Checking for pre-existing boxes..' Write-Verbose '[preflight_checks] Checking for pre-existing boxes..'
@@ -425,7 +431,7 @@ if ($ProviderName -eq $Null -or $ProviderName -eq "") {
} }
# Set Provider variable for use deployment functions # Set Provider variable for use deployment functions
if ($ProviderName -eq 'vmware_workstation') { if ($ProviderName -eq 'vmware_desktop') {
$PackerProvider = 'vmware' $PackerProvider = 'vmware'
} }
else { else {

84
build.sh
View File

@@ -8,7 +8,7 @@
# https://github.com/clong/DetectionLab/issues # https://github.com/clong/DetectionLab/issues
print_usage() { print_usage() {
echo "Usage: ./build.sh <virtualbox|vmware_fusion>" echo "Usage: ./build.sh <virtualbox|vmware_desktop>"
exit 0 exit 0
} }
@@ -54,16 +54,24 @@ check_vmware_fusion_installed() {
} }
# Returns 0 if not installed or 1 if installed # Returns 0 if not installed or 1 if installed
check_vmware_vagrant_plugin_installed() { check_vmware_desktop_vagrant_plugin_installed() {
VAGRANT_VMWARE_PLUGIN_PRESENT="$(vagrant plugin list | grep -c 'vagrant-vmware-fusion')" LEGACY_PLUGIN_CHECK="$(vagrant plugin list | grep -c 'vagrant-vmware-fusion')"
if [ "$VAGRANT_VMWARE_PLUGIN_PRESENT" -eq 0 ]; then if [ "$LEGACY_PLUGIN_CHECK" -gt 0 ]; then
(echo >&2 "VMWare Fusion is installed, but the Vagrant plugin is not.") (echo >&2 "The VMware Fusion Vagrant plugin is deprecated and is no longer supported by the DetectionLab build script.")
(echo >&2 "Visit https://www.vagrantup.com/vmware/index.html#buy-now for more information on how to purchase and install it") (echo >&2 "Please upgrade to the VMware Desktop plugin: https://www.vagrantup.com/docs/vmware/installation.html")
(echo >&2 "VMWare Fusion will not be listed as a provider until the Vagrant plugin has been installed.") (echo >&2 "NOTE: The VMware plugin does not work with trial versions of VMware Fusion")
echo "0" echo "0"
else
echo "$VAGRANT_VMWARE_PLUGIN_PRESENT"
fi fi
VAGRANT_VMWARE_DESKTOP_PLUGIN_PRESENT="$(vagrant plugin list | grep -c 'vagrant-vmware-desktop')"
if [ "$VAGRANT_VMWARE_DESKTOP_PLUGIN_PRESENT" -eq 0 ]; then
(echo >&2 "VMWare Fusion is installed, but the vagrant-vmware-desktop plugin is not.")
(echo >&2 "If you are seeing this, you may have the deprecated vagrant-vmware-fusion plugin installed. Please remove it and install the vagrant-vmware-desktop plugin.")
(echo >&2 "Visit https://www.hashicorp.com/blog/introducing-the-vagrant-vmware-desktop-plugin for more information on how to purchase and install it")
(echo >&2 "VMWare Fusion will not be listed as a provider until the vagrant-vmware-desktop plugin has been installed.")
echo "0"
else
echo "$VAGRANT_VMWARE_DESKTOP_PLUGIN_PRESENT"
fi
} }
# List the available Vagrant providers present on the system # List the available Vagrant providers present on the system
@@ -75,7 +83,7 @@ list_providers() {
# Detect Providers on OSX # Detect Providers on OSX
VBOX_PRESENT=$(check_virtualbox_installed) VBOX_PRESENT=$(check_virtualbox_installed)
VMWARE_FUSION_PRESENT=$(check_vmware_fusion_installed) VMWARE_FUSION_PRESENT=$(check_vmware_fusion_installed)
VAGRANT_VMWARE_PLUGIN_PRESENT=$(check_vmware_vagrant_plugin_installed) VAGRANT_VMWARE_DESKTOP_PLUGIN_PRESENT=$(check_vmware_desktop_vagrant_plugin_installed)
else else
# Assume the only other available provider is VirtualBox # Assume the only other available provider is VirtualBox
VBOX_PRESENT=$(check_virtualbox_installed) VBOX_PRESENT=$(check_virtualbox_installed)
@@ -85,8 +93,8 @@ list_providers() {
if [ "$VBOX_PRESENT" == "1" ]; then if [ "$VBOX_PRESENT" == "1" ]; then
(echo >&2 "virtualbox") (echo >&2 "virtualbox")
fi fi
if [[ $VMWARE_FUSION_PRESENT -eq 1 ]] && [[ $VAGRANT_VMWARE_PLUGIN_PRESENT -eq 1 ]]; then if [[ $VMWARE_FUSION_PRESENT -eq 1 ]] && [[ $VAGRANT_VMWARE_DESKTOP_PLUGIN_PRESENT -eq 1 ]]; then
(echo >&2 "vmware_fusion") (echo >&2 "vmware_desktop")
fi fi
if [[ $VBOX_PRESENT -eq 0 ]] && [[ $VMWARE_FUSION_PRESENT -eq 0 ]]; then if [[ $VBOX_PRESENT -eq 0 ]] && [[ $VMWARE_FUSION_PRESENT -eq 0 ]]; then
(echo >&2 "You need to install a provider such as VirtualBox or VMware Fusion to continue.") (echo >&2 "You need to install a provider such as VirtualBox or VMware Fusion to continue.")
@@ -95,8 +103,8 @@ list_providers() {
(echo >&2 -e "\\nWhich provider would you like to use?") (echo >&2 -e "\\nWhich provider would you like to use?")
read -r PROVIDER read -r PROVIDER
# Sanity check # Sanity check
if [[ "$PROVIDER" != "virtualbox" ]] && [[ "$PROVIDER" != "vmware_fusion" ]]; then if [[ "$PROVIDER" != "virtualbox" ]] && [[ "$PROVIDER" != "vmware_desktop" ]]; then
(echo >&2 "Please choose a valid provider. \"$PROVIDER\" is not a valid option") (echo >&2 "Please choose a valid provider. \"$PROVIDER\" is not a valid option.")
exit 1 exit 1
fi fi
echo "$PROVIDER" echo "$PROVIDER"
@@ -200,29 +208,22 @@ preflight_checks() {
# Builds a box using Packer # Builds a box using Packer
packer_build_box() { packer_build_box() {
BOX="$1" BOX="$1"
if [ "$PROVIDER" == "vmware_fusion" ]; then
PROVIDER="vmware"
fi
cd "$DL_DIR/Packer" || exit 1 cd "$DL_DIR/Packer" || exit 1
(echo >&2 "Using Packer to build the $BOX Box. This can take 90-180 minutes depending on bandwidth and hardware.") (echo >&2 "Using Packer to build the $BOX Box. This can take 90-180 minutes depending on bandwidth and hardware.")
PACKER_LOG=1 PACKER_LOG_PATH="$DL_DIR/Packer/packer_build.log" $(which packer) build --only="$PROVIDER-iso" "$BOX".json >&2 PACKER_LOG=1 PACKER_LOG_PATH="$DL_DIR/Packer/packer_build.log" $(which packer) build --only="$PACKER_PROVIDER-iso" "$BOX".json >&2
echo "$?" echo "$?"
} }
# Moves the boxes from the Packer directory to the Boxes directory # Moves the boxes from the Packer directory to the Boxes directory
move_boxes() { move_boxes() {
# Hacky workaround for VMware
if [ "$PROVIDER" == "vmware_fusion" ]; then
PROVIDER="vmware"
fi
mv "$DL_DIR"/Packer/*.box "$DL_DIR"/Boxes mv "$DL_DIR"/Packer/*.box "$DL_DIR"/Boxes
# Ensure Windows 10 box exists # Ensure Windows 10 box exists
if [ ! -f "$DL_DIR"/Boxes/windows_10_"$PROVIDER".box ]; then if [ ! -f "$DL_DIR"/Boxes/windows_10_"$PACKER_PROVIDER".box ]; then
(echo >&2 "Windows 10 box is missing from the Boxes directory. Qutting.") (echo >&2 "Windows 10 box is missing from the Boxes directory. Qutting.")
exit 1 exit 1
fi fi
# Ensure Windows 2016 box exists # Ensure Windows 2016 box exists
if [ ! -f "$DL_DIR"/Boxes/windows_2016_"$PROVIDER".box ]; then if [ ! -f "$DL_DIR"/Boxes/windows_2016_"$PACKER_PROVIDER".box ]; then
(echo >&2 "Windows 2016 box is missing from the Boxes directory. Qutting.") (echo >&2 "Windows 2016 box is missing from the Boxes directory. Qutting.")
exit 1 exit 1
fi fi
@@ -298,9 +299,11 @@ parse_cli_arguments() {
case "$1" in case "$1" in
virtualbox) virtualbox)
PROVIDER="$1" PROVIDER="$1"
PACKER_PROVIDER="$1"
;; ;;
vmware_fusion) vmware_desktop)
PROVIDER="$1" PROVIDER="$1"
PACKER_PROVIDER="vmware"
;; ;;
*) *)
echo "\"$1\" is not a valid provider. Listing available providers:" echo "\"$1\" is not a valid provider. Listing available providers:"
@@ -377,57 +380,44 @@ download_boxes() {
if [ "$PROVIDER" == "virtualbox" ]; then if [ "$PROVIDER" == "virtualbox" ]; then
wget "https://www.detectionlab.network/windows_2016_virtualbox.box" -O "$DL_DIR"/Boxes/windows_2016_virtualbox.box wget "https://www.detectionlab.network/windows_2016_virtualbox.box" -O "$DL_DIR"/Boxes/windows_2016_virtualbox.box
wget "https://www.detectionlab.network/windows_10_virtualbox.box" -O "$DL_DIR"/Boxes/windows_10_virtualbox.box wget "https://www.detectionlab.network/windows_10_virtualbox.box" -O "$DL_DIR"/Boxes/windows_10_virtualbox.box
elif [ "$PROVIDER" == "vmware_fusion" ]; then elif [ "$PROVIDER" == "vmware_desktop" ]; then
wget "https://www.detectionlab.network/windows_2016_vmware.box" -O "$DL_DIR"/Boxes/windows_2016_vmware.box wget "https://www.detectionlab.network/windows_2016_vmware.box" -O "$DL_DIR"/Boxes/windows_2016_vmware.box
wget "https://www.detectionlab.network/windows_10_vmware.box" -O "$DL_DIR"/Boxes/windows_10_vmware.box wget "https://www.detectionlab.network/windows_10_vmware.box" -O "$DL_DIR"/Boxes/windows_10_vmware.box
fi fi
# Hacky workaround
if [ "$PROVIDER" == "vmware_fusion" ]; then
PROVIDER="vmware"
fi
# Ensure Windows 10 box exists # Ensure Windows 10 box exists
if [ ! -f "$DL_DIR"/Boxes/windows_10_"$PROVIDER".box ]; then if [ ! -f "$DL_DIR"/Boxes/windows_10_"$PACKER_PROVIDER".box ]; then
(echo >&2 "Windows 10 box is missing from the Boxes directory. Qutting.") (echo >&2 "Windows 10 box is missing from the Boxes directory. Qutting.")
exit 1 exit 1
fi fi
# Ensure Windows 2016 box exists # Ensure Windows 2016 box exists
if [ ! -f "$DL_DIR"/Boxes/windows_2016_"$PROVIDER".box ]; then if [ ! -f "$DL_DIR"/Boxes/windows_2016_"$PACKER_PROVIDER".box ]; then
(echo >&2 "Windows 2016 box is missing from the Boxes directory. Qutting.") (echo >&2 "Windows 2016 box is missing from the Boxes directory. Qutting.")
exit 1 exit 1
fi fi
# Verify hashes of VirtualBox boxes # Verify hashes of VirtualBox boxes
if [ "$PROVIDER" == "virtualbox" ]; then if [ "$PACKER_PROVIDER" == "virtualbox" ]; then
if [ "$("$MD5TOOL" "$DL_DIR"/Boxes/windows_10_"$PROVIDER".box | cut -d ' ' -f "$CUT_INDEX")" != "ad78b3406dd2c0e3418d1dd61e2abc2c" ]; then if [ "$("$MD5TOOL" "$DL_DIR"/Boxes/windows_10_"$PACKER_PROVIDER".box | cut -d ' ' -f "$CUT_INDEX")" != "ad78b3406dd2c0e3418d1dd61e2abc2c" ]; then
(echo >&2 "Hash mismatch on windows_10_virtualbox.box") (echo >&2 "Hash mismatch on windows_10_virtualbox.box")
fi fi
if [ "$("$MD5TOOL" "$DL_DIR"/Boxes/windows_2016_"$PROVIDER".box | cut -d ' ' -f "$CUT_INDEX")" != "f352c852ed1b849dab18442caef83712" ]; then if [ "$("$MD5TOOL" "$DL_DIR"/Boxes/windows_2016_"$PACKER_PROVIDER".box | cut -d ' ' -f "$CUT_INDEX")" != "f352c852ed1b849dab18442caef83712" ]; then
(echo >&2 "Hash mismatch on windows_2016_virtualbox.box") (echo >&2 "Hash mismatch on windows_2016_virtualbox.box")
fi fi
# Verify hashes of VMware boxes # Verify hashes of VMware boxes
elif [ "$PROVIDER" == "vmware" ]; then elif [ "$PACKER_PROVIDER" == "vmware" ]; then
if [ "$("$MD5TOOL" "$DL_DIR"/Boxes/windows_10_"$PROVIDER".box | cut -d ' ' -f "$CUT_INDEX")" != "14e1c4cc15e1dc47aead906b25c5b3cc" ]; then if [ "$("$MD5TOOL" "$DL_DIR"/Boxes/windows_10_"$PACKER_PROVIDER".box | cut -d ' ' -f "$CUT_INDEX")" != "14e1c4cc15e1dc47aead906b25c5b3cc" ]; then
(echo >&2 "Hash mismatch on windows_10_vmware.box") (echo >&2 "Hash mismatch on windows_10_vmware.box")
exit 1 exit 1
fi fi
if [ "$("$MD5TOOL" "$DL_DIR"/Boxes/windows_2016_"$PROVIDER".box | cut -d ' ' -f "$CUT_INDEX")" != "da1111c765b2fdc2ce012b6348cf74e2" ]; then if [ "$("$MD5TOOL" "$DL_DIR"/Boxes/windows_2016_"$PACKER_PROVIDER".box | cut -d ' ' -f "$CUT_INDEX")" != "da1111c765b2fdc2ce012b6348cf74e2" ]; then
(echo >&2 "Hash mismatch on windows_2016_vmware.box") (echo >&2 "Hash mismatch on windows_2016_vmware.box")
exit 1 exit 1
fi fi
# Reset PROVIDER variable if using VMware
if [ "$PROVIDER" == "vmware" ]; then
PROVIDER="vmware_fusion"
fi
fi fi
} }
build_vagrant_hosts() { build_vagrant_hosts() {
LAB_HOSTS=("logger" "dc" "wef" "win10") LAB_HOSTS=("logger" "dc" "wef" "win10")
# Change provider back to original selection if using vmware_fusion
if [ "$PROVIDER" == "vmware" ]; then
PROVIDER="vmware_fusion"
fi
# Vagrant up each box and attempt to reload one time if it fails # Vagrant up each box and attempt to reload one time if it fails
for VAGRANT_HOST in "${LAB_HOSTS[@]}"; do for VAGRANT_HOST in "${LAB_HOSTS[@]}"; do