trinitor/DetectionLab
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Actually add files

Browse Source
This commit is contained in:
Chris Long
2019-05-26 21:36:10 -07:00
parent cd722dab8b
commit e78c312bc5
10 changed files with 27 additions and 112 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.circleci/config.yml
View File

@@ -130,7 +130,7 @@ jobs:
done done
- run: - run:
name: Wait for build results name: Post the build results
command: | command: |
## Recording the build results ## Recording the build results
STATUS=$(cat /tmp/status) STATUS=$(cat /tmp/status)

4
Packer/scripts/vm-guest-tools.bat
View File

@@ -17,7 +17,7 @@ if exist "C:\Users\vagrant\windows.iso" (
) )
if not exist "C:\Windows\Temp\windows.iso" ( if not exist "C:\Windows\Temp\windows.iso" (
powershell -Command "[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; (New-Object System.Net.WebClient).DownloadFile('https://softwareupdate.vmware.com/cds/vmw-desktop/ws/15.0.4/12990004/windows/packages/tools-windows.tar', 'C:\Windows\Temp\vmware-tools.tar')" <NUL powershell -Command "(New-Object System.Net.WebClient).DownloadFile('https://softwareupdate.vmware.com/cds/vmw-desktop/ws/15.1.0/13591040/windows/packages/tools-windows.tar', 'C:\Windows\Temp\vmware-tools.tar')" <NUL
cmd /c ""C:\Program Files\7-Zip\7z.exe" x C:\Windows\Temp\vmware-tools.tar -oC:\Windows\Temp" cmd /c ""C:\Program Files\7-Zip\7z.exe" x C:\Windows\Temp\vmware-tools.tar -oC:\Windows\Temp"
FOR /r "C:\Windows\Temp" %%a in (VMware-tools-windows-*.iso) DO REN "%%~a" "windows.iso" FOR /r "C:\Windows\Temp" %%a in (VMware-tools-windows-*.iso) DO REN "%%~a" "windows.iso"
rd /S /Q "C:\Program Files (x86)\VMWare" rd /S /Q "C:\Program Files (x86)\VMWare"
@@ -38,7 +38,7 @@ if exist "C:\Users\vagrant\VBoxGuestAdditions.iso" (
) )
if not exist "C:\Windows\Temp\VBoxGuestAdditions.iso" ( if not exist "C:\Windows\Temp\VBoxGuestAdditions.iso" (
powershell -Command "[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; (New-Object System.Net.WebClient).DownloadFile('https://download.virtualbox.org/virtualbox/5.2.26/VBoxGuestAdditions_5.2.26.iso', 'C:\Windows\Temp\VBoxGuestAdditions.iso')" <NUL powershell -Command "(New-Object System.Net.WebClient).DownloadFile('https://download.virtualbox.org/virtualbox/6.0.8/VBoxGuestAdditions_6.0.8.iso', 'C:\Windows\Temp\VBoxGuestAdditions.iso')" <NUL
) )
cmd /c ""C:\Program Files\7-Zip\7z.exe" x C:\Windows\Temp\VBoxGuestAdditions.iso -oC:\Windows\Temp\virtualbox" cmd /c ""C:\Program Files\7-Zip\7z.exe" x C:\Windows\Temp\VBoxGuestAdditions.iso -oC:\Windows\Temp\virtualbox"

6
README.md
View File

@@ -2,8 +2,10 @@
# Detection Lab # Detection Lab
DetectionLab is tested weekly on Saturdays via a scheduled CircleCI workflow to ensure that builds are passing. DetectionLab is tested weekly on Saturdays via a scheduled CircleCI workflow to ensure that builds are passing.
CircleCI: [![CircleCI](https://circleci.com/gh/clong/DetectionLab/tree/master.svg?style=svg)](https://circleci.com/gh/clong/DetectionLab/tree/master) [![CircleCI](https://circleci.com/gh/clong/DetectionLab/tree/master.svg?style=shield)](https://circleci.com/gh/clong/DetectionLab/tree/master)
[![license](https://img.shields.io/github/license/clong/DetectionLab.svg?style=flat-square)](https://github.com/clong/DetectionLab/blob/master/license.md)
![Maintenance](https://img.shields.io/maintenance/yes/2019.svg?style=flat-square)
[![GitHub last commit](https://img.shields.io/github/last-commit/clong/DetectionLab.svg?style=flat-square)](https://github.com/clong/DetectionLab/commit/master)
[![Twitter](https://img.shields.io/twitter/follow/DetectionLab.svg?style=social)](https://twitter.com/DetectionLab) [![Twitter](https://img.shields.io/twitter/follow/DetectionLab.svg?style=social)](https://twitter.com/DetectionLab)
#### Donate to the project: #### Donate to the project:

4
Vagrant/Vagrantfile vendored
View File

@@ -26,7 +26,7 @@ Vagrant.configure("2") do |config|
end end
config.vm.define "dc" do |cfg| config.vm.define "dc" do |cfg|
cfg.vm.box = "detectionlab/win2016" cfg.vm.box = "../Boxes/windows_2016_vmware.box"
cfg.vm.hostname = "dc" cfg.vm.hostname = "dc"
cfg.vm.boot_timeout = 600 cfg.vm.boot_timeout = 600
cfg.winrm.transport = :plaintext cfg.winrm.transport = :plaintext
@@ -79,7 +79,7 @@ Vagrant.configure("2") do |config|
end end
config.vm.define "wef" do |cfg| config.vm.define "wef" do |cfg|
cfg.vm.box = "detectionlab/win2016" cfg.vm.box = "../Boxes/windows_2016_vmware.box"
cfg.vm.hostname = "wef" cfg.vm.hostname = "wef"
cfg.vm.boot_timeout = 600 cfg.vm.boot_timeout = 600
cfg.vm.communicator = "winrm" cfg.vm.communicator = "winrm"

9
Vagrant/bootstrap.sh
View File

@@ -13,7 +13,7 @@ apt_install_prerequisites() {
apt-get -qq update apt-get -qq update
apt-get -qq install -y apt-fast apt-get -qq install -y apt-fast
echo "[$(date +%H:%M:%S)]: Running apt-fast install..." echo "[$(date +%H:%M:%S)]: Running apt-fast install..."
apt-fast -qq install -y jq whois build-essential git docker docker-compose unzip apt-fast -qq install -y jq whois build-essential git docker docker-compose unzip htop
} }
test_prerequisites() { test_prerequisites() {
@@ -110,12 +110,17 @@ install_splunk() {
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_forwarder/splunk-add-on-for-microsoft-windows_500.tgz -auth 'admin:changeme' /opt/splunk/bin/splunk install app /vagrant/resources/splunk_forwarder/splunk-add-on-for-microsoft-windows_500.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/add-on-for-microsoft-sysmon_800.tgz -auth 'admin:changeme' /opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/add-on-for-microsoft-sysmon_800.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/asn-lookup-generator_100.tgz -auth 'admin:changeme' /opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/asn-lookup-generator_100.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/lookup-file-editor_331.tgz
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/force-directed-app-for-splunk_200.tgz -auth 'admin:changeme' /opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/force-directed-app-for-splunk_200.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/punchcard-custom-visualization_130.tgz -auth 'admin:changeme' /opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/punchcard-custom-visualization_130.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/sankey-diagram-custom-visualization_130.tgz -auth 'admin:changeme' /opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/sankey-diagram-custom-visualization_130.tgz -auth 'admin:changeme'
/opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/threathunting_13.tar.gz -auth 'admin:changeme' /opt/splunk/bin/splunk install app /vagrant/resources/splunk_server/threathunting_13.tar.gz -auth 'admin:changeme'
# Add custom Macro definitions for ThreatHunting App # Add custom Macro definitions for ThreatHunting App
cp /vagrant/resources/splunk_server/macros.conf /opt/splunk/etc/apps/ThreatHunting/default/macros.conf cp /vagrant/resources/splunk_server/macros.conf /opt/splunk/etc/apps/ThreatHunting/default/macros.conf
# Fix Windows TA macros
mkdir /opt/splunk/etc/apps/Splunk_TA_windows/local
cp /opt/splunk/etc/apps/Splunk_TA_windows/default/macros.conf /opt/splunk/etc/apps/Splunk_TA_windows/local
sed -i 's/wineventlog_windows/wineventlog/g' /opt/splunk/etc/apps/Splunk_TA_windows/local/macros.conf
# Fix Force Directed App until 2.0.1 is released (https://answers.splunk.com/answers/668959/invalid-key-in-stanza-default-value-light.html#answer-669418) # Fix Force Directed App until 2.0.1 is released (https://answers.splunk.com/answers/668959/invalid-key-in-stanza-default-value-light.html#answer-669418)
rm /opt/splunk/etc/apps/force_directed_viz/default/savedsearches.conf rm /opt/splunk/etc/apps/force_directed_viz/default/savedsearches.conf
@@ -140,7 +145,7 @@ install_splunk() {
dismissedInstrumentationOptInVersion = 2 dismissedInstrumentationOptInVersion = 2
[general_default] [general_default]
hideInstrumentationOptInModal = 1 hideInstrumentationOptInModal = 1
showWhatsNew = 0' > /opt/splunk/etc/apps/user-prefs/local/user-prefs.conf showWhatsNew = 0' > /opt/splunk/etc/system/local/user-prefs.conf
# Enable SSL Login for Splunk # Enable SSL Login for Splunk
echo -e "[settings]\nenableSplunkWebSSL = true" > /opt/splunk/etc/system/local/web.conf echo -e "[settings]\nenableSplunkWebSSL = true" > /opt/splunk/etc/system/local/web.conf

10
Vagrant/resources/splunk_forwarder/inputs.conf
View File

@@ -3,27 +3,27 @@ index = sysmon
disabled = false disabled = false
renderXml = true renderXml = true
[monitor://c:\programdata\osquery\log\osqueryd.results.log] [monitor://c:\Program Files\osquery\log\osqueryd.results.log]
index = osquery index = osquery
disabled = false disabled = false
sourcetype = osquery:json sourcetype = osquery:json
[monitor://c:\programdata\osquery\log\osqueryd.snapshots.log] [monitor://c:\Program Files\osquery\log\osqueryd.snapshots.log]
index = osquery index = osquery
disabled = false disabled = false
sourcetype = osquery:json sourcetype = osquery:json
[monitor://c:\programdata\osquery\log\osqueryd.INFO.*] [monitor://c:\Program Files\osquery\log\osqueryd.INFO.*]
index = osquery-status index = osquery-status
disabled = false disabled = false
sourcetype = osquery-info:syslog sourcetype = osquery-info:syslog
[monitor://c:\programdata\osquery\log\osqueryd.WARNING.*] [monitor://c:\Program Files\osquery\log\osqueryd.WARNING.*]
index = osquery-status index = osquery-status
disabled = false disabled = false
sourcetype = osquery-warn:syslog sourcetype = osquery-warn:syslog
[monitor://c:\programdata\osquery\log\osqueryd.ERROR.*] [monitor://c:\Program Files\osquery\log\osqueryd.ERROR.*]
index = osquery-status index = osquery-status
disabled = false disabled = false
sourcetype = osquery-error:syslog sourcetype = osquery-error:syslog

BIN
Vagrant/resources/splunk_server/lookup-file-editor_305.tgz
View File

Binary file not shown.

4
Vagrant/resources/splunk_server/macros.conf
View File

@@ -69,3 +69,7 @@ iseval = 0
[remote_thread_whitelist] [remote_thread_whitelist]
definition = search NOT [| inputlookup threathunting_remote_thread_whitelist.csv | fields mitre_technique_id host_fqdn process_name target_process_path target_process_address] definition = search NOT [| inputlookup threathunting_remote_thread_whitelist.csv | fields mitre_technique_id host_fqdn process_name target_process_path target_process_address]
iseval = 0 iseval = 0
[indextime]
definition = _index_earliest=-15m@m AND _index_latest=now
iseval = 0

98
ci/README.md
View File

@@ -5,100 +5,4 @@ for continuous integration testing by installing the prerequisites needed for
Detection Lab. After the prerequisites are installed, the build script is called Detection Lab. After the prerequisites are installed, the build script is called
and the build will begin in a tmux session. and the build will begin in a tmux session.
## Understanding the build process ![DetectionLab](../img/build.png)
Once a PR is created, the contents of that PR will be copied to a CircleCI worker to be tested.
The CircleCI worker will evaluate which files have been modified and set environment variables accordingly. There are 4 possible options and 3 different tests:
1. Code in both the Packer and Vagrant directories was modified
* In this case, the CircleCI worker will execute `ci/circle_workflows/packer_and_vagrant_changes.sh`
2. Code in neither the Packer and Vagrant directories was modified
* In this case, the CircleCI worker will execute the default test `ci/circle_worker/vagrant_changes.sh`
3. Code in only the Packer directory was modified
* In this case, the CircleCI worker will execute `ci/circle_worker/packer_changes.sh`
4. Code in only the Vagrant directory was modified
* In this case, the CircleCI worker will execute `ci/circle_worker/vagrant_changes.sh`
## Test Case Walkthroughs
### packer_and_vagrant_changes.sh
1. Spins up a single Packet server
2. Bootstraps the Packet server by calling `ci/build_machine_bootstrap.sh` with no arguments
3. Builds the Windows10 and Windows2016 images one at a time
4. Moves the resulting boxes to the Boxes directory
5. Brings each Vagrant host online one-by-one
6. CircleCI records the build results from the Packet server
### vagrant_changes.sh
1. Spins up a single Packet server
2. Bootstraps the Packet server by calling `ci/build_machine_bootstrap.sh` with the `--vagrant-only` argument
3. Downloads the pre-build Windows10 and Windows2016 boxes from Vagrant Cloud
4. Brings each Vagrant host online one-by-one
5. CircleCI records the build results from the Packet server
### packer_changes.sh
1. Spins up two separate Packet servers to allow the Packer boxes to be built in parallel
2. Bootstraps each packet Server by calling `ci/build_machine_bootstrap.sh` with the `--packer-only` argument
3. Starts the Packer build process on each server
4. CircleCI records the build result from each Packet server
```
+------------+
| |
| |
| |
| Github |
| |
| |
+------+-----+
|
|
| Pull Request
|
v
+------+-----+
| |
| |
| Circle |
+----------------------------->| Worker |
| | |
| | |
| | |
| +------+-----+
| |
| | Code changes are evaluated
| | to determine which test suite
| | to run
| |
| v
| +----------------+--------------+
Circle Worker | | packer_and_vagrant_changes.sh |
queries for | | vagrant_changes.sh |
build results | | packer_changes.sh |
| +----------------+--------------+
| |
| |
| |
| |
| |
| |
| |
| | 1. Provision Packet server(s)
| | 2. Copy repo to server
| | 3. Run server bootstrap
| | 4. Bootstrap calls build.sh with
| | the appropriate arguments
| |
| |
| +---------v---------+
| | |
| | |
| | |
+-------------------------->| Packet Server |
| |
| |
| |
+-------------------+
```

2
ci/manual_machine_bootstrap_vmware.sh
View File

@@ -6,7 +6,7 @@ export DEBIAN_FRONTEND=noninteractive
export SERIALNUMBER="SECRET" export SERIALNUMBER="SECRET"
export LICENSEFILE="SECRET" export LICENSEFILE="SECRET"
sed -i 's/archive.ubuntu.com/us.archive.ubuntu.com/g' /etc/apt/sources.list sed -i 's#http://archive.ubuntu.com#http://us.archive.ubuntu.com#g' /etc/apt/sources.list
# Install VMWare Workstation 15 # Install VMWare Workstation 15
apt-get update apt-get update