trinitor/DetectionLab
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Adding updated build scripts

Browse Source
This commit is contained in:
Chris Long
2018-03-21 23:04:38 -07:00
parent 0fd7d0647b
commit d75579f46e
10 changed files with 726 additions and 547 deletions
Download Patch File Download Diff File Expand all files Collapse all files

100
ci/README.md
View File

@@ -1,6 +1,104 @@
# ci
# Continuous Integration
The files in this directory are used to bootstrap an Ubuntu 16.04 baremetal server
for continuous integration testing by installing the prerequisites needed for
Detection Lab. After the prerequisites are installed, the build script is called
and the build will begin in a tmux session.
## Understanding the build process
Once a PR is created, the contents of that PR will be copied to a CircleCI worker to be tested.
The CircleCI worker will evaluate which files have been modified and set environment variables accordingly. There are 4 possible options and 3 different tests:
1. Code in both the Packer and Vagrant directories was modified
* In this case, the CircleCI worker will execute `ci/circle_workflows/packer_and_vagrant_changes.sh`
2. Code in neither the Packer and Vagrant directories was modified
* In this case, the CircleCI worker will execute the default test `ci/circle_worker/vagrant_changes.sh`
3. Code in only the Packer directory was modified
* In this case, the CircleCI worker will execute `ci/circle_worker/packer_changes.sh`
4. Code in only the Vagrant directory was modified
* In this case, the CircleCI worker will execute `ci/circle_worker/vagrant_changes.sh`
## Test Case Walkthroughs
### packer_and_vagrant_changes.sh
1. Spins up a single Packet server
2. Bootstraps the Packet server by calling `ci/build_machine_bootstrap.sh` with no arguments
3. Builds the Windows10 and Windows2016 images one at a time
4. Moves the resulting boxes to the Boxes directory
5. Brings each Vagrant host online one-by-one
6. CircleCI records the build results from the Packet server
### vagrant_changes.sh
1. Spins up a single Packet server
2. Bootstraps the Packet server by calling `ci/build_machine_bootstrap.sh` with the `--vagrant-only` argument
3. Downloads the pre-build Windows10 and Windows2016 boxes from https://detectionlab.network directly to the Boxes directory
4. Brings each Vagrant host online one-by-one
5. CircleCI records the build results from the Packet server
### packer_changes.sh
1. Spins up two separate Packet servers to allow the Packer boxes to be built in parallel
2. Bootstraps each packet Server by calling `ci/build_machine_bootstrap.sh` with the `--packer-only` argument
3. Starts the Packer build process on each server
4. CircleCI records the build result from each Packet server
```
+------------+
| |
| |
| |
| Github |
| |
| |
+------+-----+
|
|
| Pull Request
|
v
+------+-----+
| |
| |
| Circle |
+----------------------------->| Worker |
| | |
| | |
| | |
| +------+-----+
| |
| | Code changes are evaluated
| | to determine which test suite
| | to run
| |
| v
| +----------------+--------------+
Circle Worker | | packer_and_vagrant_changes.sh |
quries for | | vagrant_changes.sh |
build results | | packer_changes.sh |
| +----------------+--------------+
| |
| |
| | |
| |
| |
| |
| |
| | 1. Provision Packet server(s)
| | 2. Copy repo to server
| | 3. Run server bootstrap
| | 4. Bootstrap calls build.sh with
| | the appropriate arguments
| |
| |
| +---------v---------+
| | |
| | |
| | |
+-------------------------->| Packet Server |
| |
| |
| |
+-------------------+
```

54
ci/automated_install.sh
View File

@@ -1,54 +0,0 @@
#! /bin/bash
# This script is run on the Packet.net baremetal server for CI tests.
# This script will build the entire lab from scratch and takes 3-4 hours
# on a Packet.net host
# While building, the server will start a webserver on Port 80 that contains
# the text "building". Once the test is completed, the text will be replaced
# with "success" or "failed".
# Install Virtualbox 5.2
echo "deb http://download.virtualbox.org/virtualbox/debian xenial contrib" >> /etc/apt/sources.list
wget -q https://www.virtualbox.org/download/oracle_vbox_2016.asc -O- | sudo apt-key add -
apt-get update
apt-get install -y linux-headers-"$(uname -r)" virtualbox-5.2 build-essential unzip git ufw apache2
echo "building" > /var/www/html/index.html
# Set up firewall
ufw allow ssh
ufw allow http
ufw default allow outgoing
ufw --force enable
# Install Vagrant
mkdir /opt/vagrant
cd /opt/vagrant || exit 1
wget https://releases.hashicorp.com/vagrant/2.0.3/vagrant_2.0.3_x86_64.deb
dpkg -i vagrant_2.0.3_x86_64.deb
vagrant plugin install vagrant-reload
# Install Packer
mkdir /opt/packer
cd /opt/packer || exit 1
wget https://releases.hashicorp.com/packer/1.1.3/packer_1.1.3_linux_amd64.zip
unzip packer_1.1.3_linux_amd64.zip
cp packer /usr/local/bin/packer
# Make the packer images headless
for file in $(ls *.json); do
sed -i 's/"headless": false,/"headless": true,/g' "$file";
done
# Make the Vagrant instances headless
cd /opt/DetectionLab/Vagrant || exit 1
sed -i 's/vb.gui = true/vb.gui = false/g' Vagrantfile
# Ensure the script is executable
chmod +x /opt/DetectionLab/build.sh
cd /opt/DetectionLab || exit 1
# Start the build in a tmux session
sn=tmuxsession
tmux new-session -s "$sn" -d
tmux send-keys -t "$sn:0" './build.sh virtualbox | tee -a /opt/DetectionLab/Vagrant/vagrant.log && echo "success" > /var/www/html/index.html || echo "failed" > /var/www/html/index.html' Enter

42
ci/automated_install_vagrant_only.sh
View File

@@ -1,42 +0,0 @@
#! /bin/bash
# This script is run on the Packet.net baremetal server for CI tests.
# This script will bootstrap the build by downloading pre-build Packer boxes
# and should take no more than 90 minutes on a Packet.net host.
# While building, the server will start a webserver on Port 80 that contains
# the text "building". Once the test is completed, the text will be replaced
# with "success" or "failed".
# Install Virtualbox 5.2
echo "deb http://download.virtualbox.org/virtualbox/debian xenial contrib" >> /etc/apt/sources.list
wget -q https://www.virtualbox.org/download/oracle_vbox_2016.asc -O- | sudo apt-key add -
apt-get update
apt-get install -y linux-headers-"$(uname -r)" virtualbox-5.2 build-essential unzip git ufw apache2
echo "building" > /var/www/html/index.html
# Set up firewall
ufw allow ssh
ufw allow http
ufw default allow outgoing
ufw --force enable
# Install Vagrant
mkdir /opt/vagrant
cd /opt/vagrant || exit 1
wget https://releases.hashicorp.com/vagrant/2.0.3/vagrant_2.0.3_x86_64.deb
dpkg -i vagrant_2.0.3_x86_64.deb
vagrant plugin install vagrant-reload
# Make the Vagrant instances headless
cd /opt/DetectionLab/Vagrant || exit 1
sed -i 's/vb.gui = true/vb.gui = false/g' Vagrantfile
# Ensure the script is executable
chmod +x /opt/DetectionLab/build_vagrant_only.sh
cd /opt/DetectionLab || exit 1
# Start the build in a tmux session
sn=tmuxsession
tmux new-session -s "$sn" -d
tmux send-keys -t "$sn:0" './build_vagrant_only.sh virtualbox | tee -a /opt/DetectionLab/Vagrant/vagrant.log && echo "success" > /var/www/html/index.html || echo "failed" > /var/www/html/index.html' Enter

93
ci/build_machine_bootstrap.sh Executable file
View File

@@ -0,0 +1,93 @@
#! /bin/bash
# This script is run on the Packet.net baremetal server for CI tests.
# This script will build the entire lab from scratch and takes 3-4 hours
# on a Packet.net host
# While building, the server will start a webserver on Port 80 that contains
# the text "building". Once the test is completed, the text will be replaced
# with "success" or "failed".
ARGS="$1"
PACKER_ONLY=0
VAGRANT_ONLY=0
if [ ! -z "$1" ]; then
case "$1" in
--packer-only)
PACKER_ONLY=1
;;
--vagrant-only)
VAGRANT_ONLY=1
;;
*)
echo "\"$ARGS\" is not a supported argument to this script. Quitting"
exit 1
;;
esac
fi
echo "Args: $ARGS"
if [[ "$VAGRANT_ONLY" -eq 1 ]] && [[ "$PACKER_ONLY" -eq 1 ]]; then
echo "Somehow this build is configured as both packer-only and vagrant-only. This means something has gone horribly wrong."
exit 1
fi
# Install Virtualbox 5.2
echo "deb http://download.virtualbox.org/virtualbox/debian xenial contrib" >> /etc/apt/sources.list
wget -q https://www.virtualbox.org/download/oracle_vbox_2016.asc -O- | sudo apt-key add -
apt-get update
apt-get install -y linux-headers-"$(uname -r)" virtualbox-5.2 build-essential unzip git ufw apache2
echo "building" > /var/www/html/index.html
# Set up firewall
ufw allow ssh
ufw allow http
ufw default allow outgoing
ufw --force enable
if [ "$PACKER_ONLY" -eq 0 ]; then
# Install Vagrant
mkdir /opt/vagrant
cd /opt/vagrant || exit 1
wget https://releases.hashicorp.com/vagrant/2.0.3/vagrant_2.0.3_x86_64.deb
dpkg -i vagrant_2.0.3_x86_64.deb
vagrant plugin install vagrant-reload
# Make the Vagrant instances headless
cd /opt/DetectionLab/Vagrant || exit 1
sed -i 's/vb.gui = true/vb.gui = false/g' Vagrantfile
fi
if [ "$VAGRANT_ONLY" -eq 0 ]; then
# Install Packer
mkdir /opt/packer
cd /opt/packer || exit 1
wget https://releases.hashicorp.com/packer/1.1.3/packer_1.1.3_linux_amd64.zip
unzip packer_1.1.3_linux_amd64.zip
cp packer /usr/local/bin/packer
# Make the Packer images headless
cd /opt/DetectionLab/Packer || exit 1
for file in *.json; do
sed -i 's/"headless": false,/"headless": true,/g' "$file";
done
fi
# Ensure the script is executable
chmod +x /opt/DetectionLab/build.sh
cd /opt/DetectionLab || exit 1
# Start the build in a tmux session
sn=tmuxsession
tmux new-session -s "$sn" -d
if [ "$PACKER_ONLY" -eq 1 ]; then
tmux send-keys -t "$sn:0" './build.sh virtualbox --packer-only && echo "success" > /var/www/html/index.html || echo "failed" > /var/www/html/index.html' Enter
fi
if [ "$VAGRANT_ONLY" -eq 1 ]; then
tmux send-keys -t "$sn:0" './build.sh virtualbox --vagrant-only && echo "success" > /var/www/html/index.html || echo "failed" > /var/www/html/index.html' Enter
fi
if [[ "$PACKER_ONLY" -eq 0 ]] && [[ "$VAGRANT_ONLY" -eq 0 ]]; then
tmux send-keys -t "$sn:0" './build.sh virtualbox && echo "success" > /var/www/html/index.html || echo "failed" > /var/www/html/index.html' Enter
fi

80
ci/circle_workflows/packer_and_vagrant_changes.sh Normal file
View File

@@ -0,0 +1,80 @@
#! /bin/bash
set -e
# Create artifacts directory
if [ ! -d "/tmp/artifacts" ]; then
mkdir /tmp/artifacts
fi
## Delete stale servers if they exist
DELETE_DEVICE_ID=$(curl -X GET -s --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/projects/0b3f4f2e-ff05-41a8-899d-7923f620ca85/devices' | jq ."devices[0].id" | tr -d '"')
if [ "$(echo -n $DELETE_DEVICE_ID | wc -c)" -eq 36 ]; then
echo "Requesting deletion for Packet server with ID $DELETE_DEVICE_ID"
curl -X DELETE -s --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/devices/'"$DELETE_DEVICE_ID"
fi
## Provision a Type1 baremetal Packet.net server
echo "Provisioning a server on Packet.net"
DEVICE_ID=$(curl -s -X POST --header 'Accept: application/json' --header 'Content-Type: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" -d '{ "facility": "sjc1", "plan": "baremetal_1", "hostname": "detectionlab", "description": "testing", "billing_cycle": "hourly", "operating_system": "ubuntu_16_04", "userdata": "", "locked": "false", "project_ssh_keys": ["315a9565-d5b1-41b6-913d-fcf022bb89a6", "755b134a-f63c-4fc5-9103-c1b63e65fdfc"] }' 'https://api.packet.net/projects/0b3f4f2e-ff05-41a8-899d-7923f620ca85/devices' | jq ."id" | tr -d '"')
# Make sure the device ID is sane.
# TODO: maybe make this a regex
if [ "$(echo -n $DEVICE_ID | wc -c)" -ne 36 ]; then
echo "Server may have failed provisionining. Device ID is set to: $DEVICE_ID"
exit 1
fi
echo "Server successfully provisioned with ID: $DEVICE_ID"
echo "Sleeping 10 minutes to wait for Packet server to be provisioned"
sleep 300
echo "Sleeping 5 more minutes (CircleCI Keepalive)"
sleep 300
## Recording the IP address of the newly provisioned Packet server
IP_ADDRESS=$(curl -s -X GET --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" "https://api.packet.net/devices/$DEVICE_ID/ips" | jq ."ip_addresses[0].address" | tr -d '"')
# Copy repo to Packet server
# TODO: Tar up the repo and expand it remotely
cd ~/repo
rsync -Pav -e "ssh -i ~/.ssh/id_rsa" ~/repo/ root@"$IP_ADDRESS":/opt/DetectionLab
## Running install script on Packet server
ssh -i ~/.ssh/id_rsa root@"$IP_ADDRESS" 'bash -s' -- < ci/build_machine_bootstrap.sh
echo "Sleeping 5 minutes to allow the build process to start"
sleep 300
## Waiting for Packet server to post build results
MINUTES_PAST=0
while [ "$MINUTES_PAST" -lt 400 ]; do
STATUS=$(curl $IP_ADDRESS)
if [ "$STATUS" == "building" ]; then
echo "$STATUS"
scp -i ~/.ssh/id_rsa root@"$IP_ADDRESS":/opt/DetectionLab/Vagrant/vagrant_build.log /tmp/artifacts/vagrant_build.log || echo "Vagrant log not yet present"
scp -i ~/.ssh/id_rsa root@"$IP_ADDRESS":/opt/DetectionLab/Packer/packer_build.log /tmp/artifacts/packer_build.log || echo "Packer log not yet present"
sleep 300
((MINUTES_PAST += 5))
else
break
fi
done
if [ "$MINUTES_PAST" -gt 400 ]; then
echo "Serer timed out. Uptime: $MINUTES_PAST minutes."
scp -i ~/.ssh/id_rsa root@"$IP_ADDRESS":/opt/DetectionLab/Vagrant/vagrant_build.log /tmp/artifacts/vagrant_build.log || echo "Vagrant log not yet present"
scp -i ~/.ssh/id_rsa root@"$IP_ADDRESS":/opt/DetectionLab/Packer/packer_build.log /tmp/artifacts/packer_build.log || echo "Packer log not yet present"
curl -X DELETE --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/devices/'"$DEVICE_ID"
exit 1
fi
## Recording the build results
echo $STATUS
if [ "$STATUS" != "success" ]; then
echo "Build failed. Cleaning up server with ID $DEVICE_ID"
scp -i ~/.ssh/id_rsa root@"$IP_ADDRESS":/opt/DetectionLab/Vagrant/vagrant_build.log /tmp/artifacts/vagrant_build.log || echo "Vagrant log not yet present"
scp -i ~/.ssh/id_rsa root@"$IP_ADDRESS":/opt/DetectionLab/Packer/packer_build.log /tmp/artifacts/packer_build.log || echo "Packer log not yet present"
curl -X DELETE --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/devices/'"$DEVICE_ID"
exit 1
fi
echo "Build was successful. Cleaning up server with ID $DEVICE_ID"
curl -X DELETE --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/devices/'"$DEVICE_ID"
exit 0

99
ci/circle_workflows/packer_changes.sh Normal file
View File

@@ -0,0 +1,99 @@
#! /bin/bash
set -e
# Create artifacts directory
if [ ! -d "/tmp/artifacts" ]; then
mkdir /tmp/artifacts
fi
## Delete stale servers if they exist
echo "Deleting stale Packet.net servers"
DELETE_DEVICE_ID=$(curl -X GET -s --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/projects/0b3f4f2e-ff05-41a8-899d-7923f620ca85/devices' | jq ."devices[0].id" | tr -d '"')
if [ "$(echo -n $DELETE_DEVICE_ID | wc -c)" -eq 36 ]; then
echo "Requesting deletion for Packet server with ID $DELETE_DEVICE_ID"
curl -X DELETE -s --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/devices/'"$DELETE_DEVICE_ID"
fi
## Provision two Type1 baremetal Packet.net servers
echo "Provisioning packerwindows2016 on Packet.net"
SERVER1_ID=$(curl -X POST -s --header 'Accept: application/json' --header 'Content-Type: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" -d '{ "facility": "sjc1", "plan": "baremetal_1", "hostname": "packerwindows2016", "description": "testing", "billing_cycle": "hourly", "operating_system": "ubuntu_16_04", "userdata": "", "locked": "false", "project_ssh_keys":["315a9565-d5b1-41b6-913d-fcf022bb89a6", "755b134a-f63c-4fc5-9103-c1b63e65fdfc"] }' 'https://api.packet.net/projects/0b3f4f2e-ff05-41a8-899d-7923f620ca85/devices' | jq ."id" | tr -d '"')
if [ "$(echo -n $SERVER1_ID | wc -c)" -ne 36 ]; then
echo "Server may have failed provisionining. Device ID is set to: $SERVER1_ID"
exit 1
fi
echo "packerwindows2016 successfully provisioned with ID: $SERVER1_ID"
sleep 5 # Wait a bit before issuing another provision command
echo "Provisioning packerwindows10 on Packet.net"
SERVER2_ID=$(curl -X POST -s --header 'Accept: application/json' --header 'Content-Type: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" -d '{ "facility": "sjc1", "plan": "baremetal_1", "hostname": "packerwindows10", "description": "testing", "billing_cycle": "hourly", "operating_system": "ubuntu_16_04", "userdata": "", "locked": "false", "project_ssh_keys":["315a9565-d5b1-41b6-913d-fcf022bb89a6", "755b134a-f63c-4fc5-9103-c1b63e65fdfc"] }' 'https://api.packet.net/projects/0b3f4f2e-ff05-41a8-899d-7923f620ca85/devices' | jq ."id" | tr -d '"')
if [ "$(echo -n $SERVER2_ID | wc -c)" -ne 36 ]; then
echo "Server may have failed provisionining. Device ID is set to: $SERVER2_ID"
exit 1
fi
echo "packerwindows10 successfully provisioned with ID: $SERVER2_ID"
echo "Sleeping 10 minutes to wait for Packet servers to finish provisiong"
sleep 300
echo "Sleeping 5 more minutes (CircleCI Keepalive)"
sleep 300
## Recording the IP address of the newly provisioned Packet servers
SERVER1_IP_ADDRESS=$(curl -X GET --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" "https://api.packet.net/devices/$SERVER1_ID/ips" | jq ."ip_addresses[0].address" | tr -d '"')
SERVER2_IP_ADDRESS=$(curl -X GET --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" "https://api.packet.net/devices/$SERVER2_ID/ips" | jq ."ip_addresses[0].address" | tr -d '"')
# Copy repo to Packet servers
# TODO: Tar up the repo and expand it remotely
cd ~/repo
rsync -Pav -e "ssh -i ~/.ssh/id_rsa" ~/repo/ root@"$SERVER1_IP_ADDRESS":/opt/DetectionLab
rsync -Pav -e "ssh -i ~/.ssh/id_rsa" ~/repo/ root@"$SERVER2_IP_ADDRESS":/opt/DetectionLab
## Running install script on Packet server
ssh -i ~/.ssh/id_rsa root@"$SERVER1_IP_ADDRESS" 'bash -s' -- < ci/build_machine_bootstrap.sh --packer-only
ssh -i ~/.ssh/id_rsa root@"$SERVER2_IP_ADDRESS" 'bash -s' -- < ci/build_machine_bootstrap.sh --packer-only
sleep 30
## Waiting for Packet server to post build results
MINUTES_PAST=0
while [ "$MINUTES_PAST" -lt 150 ]; do
SERVER1_STATUS=$(curl $SERVER1_IP_ADDRESS)
SERVER2_STATUS=$(curl $SERVER2_IP_ADDRESS)
if [[ "$SERVER1_STATUS" == "building" ]] || [[ "$SERVER2_STATUS" == "building" ]]; then
echo "$SERVER1_STATUS" :: "$SERVER2_STATUS"
scp -i ~/.ssh/id_rsa root@"$SERVER1_IP_ADDRESS":/opt/DetectionLab/Packer/packer_build.log /tmp/artifacts/server1_packer.log
scp -i ~/.ssh/id_rsa root@"$SERVER2_IP_ADDRESS":/opt/DetectionLab/Packer/packer_build.log /tmp/artifacts/server2_packer.log
sleep 300
((MINUTES_PAST += 5))
fi
if [[ "$SERVER1_STATUS" != "building" ]] && [[ "$SERVER2_STATUS" != "building" ]]; then
break
fi
if [ "$MINUTES_PAST" -gt 150 ]; then
echo "Serer timed out. Uptime: $MINUTES_PAST minutes."
curl -X DELETE --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/devices/'"$SERVER1_ID"
curl -X DELETE --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/devices/'"$SERVER2_ID"
exit 1
fi
done
## Recording the build results
echo "Server1 Status: $SERVER1_STATUS"
echo "Server2 Status: $SERVER2_STATUS"
if [ "$SERVER1_STATUS" != "success" ]; then
echo "Build failed. Cleaning up server with ID $SERVER1_ID"
scp -i ~/.ssh/id_rsa root@"$SERVER1_IP_ADDRESS":/opt/DetectionLab/Packer/packer_build.log /tmp/artifacts/server1_packer.log || echo "Serveer1 packer_build.log not available yet"
curl -X DELETE --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/devices/'"$SERVER1_ID"
exit 1
fi
if [ "$SERVER2_STATUS" != "success" ]; then
echo "Build failed. Cleaning up server with ID $SERVER2_ID"
scp -i ~/.ssh/id_rsa root@"$SERVER2_IP_ADDRESS":/opt/DetectionLab/Packer/packer_build.log /tmp/artifacts/server2_packer.log || echo "Server2 packer_build.log not available yet"
curl -X DELETE --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/devices/'"$SERVER2_ID"
exit 1
fi
echo "Builds were successful. Cleaning up servers with IDs $SERVER1_ID and $SERVER2_ID"
curl -X DELETE --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/devices/'"$SERVER1_ID"
curl -X DELETE --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/devices/'"$SERVER2_ID"
exit 0

74
ci/circle_workflows/vagrant_changes.sh Normal file
View File

@@ -0,0 +1,74 @@
#! /bin/bash
set -e
# Create artifacts directory
if [ ! -d "/tmp/artifacts" ]; then
mkdir /tmp/artifacts
fi
## Delete stale servers if they exist
DELETE_DEVICE_ID=$(curl -X GET -s --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/projects/0b3f4f2e-ff05-41a8-899d-7923f620ca85/devices' | jq ."devices[0].id" | tr -d '"')
if [ "$(echo -n $DELETE_DEVICE_ID | wc -c)" -eq 36 ]; then
echo "Requesting deletion for Packet server with ID $DELETE_DEVICE_ID"
curl -X -s DELETE --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/devices/'"$DELETE_DEVICE_ID"
fi
## Provision a Type1 baremetal Packet.net server
echo "Provisioning a server on Packet.net"
DEVICE_ID=$(curl -s -X POST --header 'Accept: application/json' --header 'Content-Type: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" -d '{ "facility": "sjc1", "plan": "baremetal_1", "hostname": "detectionlab", "description": "testing", "billing_cycle": "hourly", "operating_system": "ubuntu_16_04", "userdata": "", "locked": "false", "project_ssh_keys": ["315a9565-d5b1-41b6-913d-fcf022bb89a6", "755b134a-f63c-4fc5-9103-c1b63e65fdfc"] }' 'https://api.packet.net/projects/0b3f4f2e-ff05-41a8-899d-7923f620ca85/devices' | jq ."id" | tr -d '"')
# Make sure the device ID is sane.
# TODO: maybe make this a regex
if [ "$(echo -n $DEVICE_ID | wc -c)" -ne 36 ]; then
echo "Server may have failed provisionining. Device ID is set to: $DEVICE_ID"
exit 1
fi
echo "Server successfully provisioned with ID: $DEVICE_ID"
echo "Sleeping 10 minutes to wait for Packet server to be provisioned"
sleep 300
echo "Sleeping 5 more minutes (CircleCI Keepalive)"
sleep 300
## Recording the IP address of the newly provisioned Packet server
IP_ADDRESS=$(curl -s -X GET --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" "https://api.packet.net/devices/$DEVICE_ID/ips" | jq ."ip_addresses[0].address" | tr -d '"')
# Copy repo to Packet server
# TODO: Tar up the repo and expand it remotely
cd ~/repo
rsync -Pav -e "ssh -i ~/.ssh/id_rsa" ~/repo/ root@"$IP_ADDRESS":/opt/DetectionLab
## Running install script on Packet server
ssh -i ~/.ssh/id_rsa root@"$IP_ADDRESS" 'bash -s' -- < ci/build_machine_bootstrap.sh --vagrant-only
## Waiting for Packet server to post build results
MINUTES_PAST=0
while [ "$MINUTES_PAST" -lt 120 ]; do
STATUS=$(curl $IP_ADDRESS)
if [ "$STATUS" == "building" ]; then
echo "$STATUS"
scp -i ~/.ssh/id_rsa root@"$IP_ADDRESS":/opt/DetectionLab/Vagrant/vagrant_build.log /tmp/artifacts/vagrant_build.log || echo "vagrant_build.log not available yet"
sleep 300
((MINUTES_PAST += 5))
else
break
fi
if [ "$MINUTES_PAST" -gt 120 ]; then
echo "Serer timed out. Uptime: $MINUTES_PAST minutes."
scp -i ~/.ssh/id_rsa root@"$IP_ADDRESS":/opt/DetectionLab/Vagrant/vagrant_build.log /tmp/artifacts/vagrant_build.log
curl -X DELETE --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/devices/'"$DEVICE_ID"
exit 1
fi
done
## Recording the build results
echo $STATUS
if [ "$STATUS" != "success" ]; then
scp -i ~/.ssh/id_rsa root@"$IP_ADDRESS":/opt/DetectionLab/Vagrant/vagrant_build.log /tmp/artifacts/vagrant_build.log
echo "Build failed. Cleaning up server with ID $DEVICE_ID"
curl -X DELETE --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/devices/'"$DEVICE_ID"
exit 1
fi
echo "Build was successful. Cleaning up server with ID $DEVICE_ID"
curl -X DELETE --header 'Accept: application/json' --header 'X-Auth-Token: '"$PACKET_API_TOKEN" 'https://api.packet.net/devices/'"$DEVICE_ID"
exit 0