added Malcolm
This commit is contained in:
3
Vagrant/resources/malcolm/scripts/beats/.gitignore
vendored
Normal file
3
Vagrant/resources/malcolm/scripts/beats/.gitignore
vendored
Normal file
@@ -0,0 +1,3 @@
|
||||
.vagrant
|
||||
data
|
||||
logs
|
||||
181
Vagrant/resources/malcolm/scripts/beats/README.md
Normal file
181
Vagrant/resources/malcolm/scripts/beats/README.md
Normal file
@@ -0,0 +1,181 @@
|
||||
# Using Beats to forward host logs to Malcolm
|
||||
|
||||
Because Malcolm uses components of the open source data analysis platform [Elastic Stack](https://www.elastic.co/elastic-stack), it can accept various host logs sent from [Beats](https://www.elastic.co/beats/#the-beats-family), Elastic Stack's lightweight data shippers. These Beats generally include prebuilt Kibana dashboards for each of their respective data sets.
|
||||
|
||||
## Examples
|
||||
|
||||
Some examples include:
|
||||
|
||||
* [Auditbeat](https://www.elastic.co/beats/auditbeat)
|
||||
- [`auditd` logs](https://www.elastic.co/guide/en/beats/auditbeat/master/auditbeat-module-auditd.html) on Linux hosts
|
||||
- [file integrity monitoring](https://www.elastic.co/guide/en/beats/auditbeat/master/auditbeat-module-file_integrity.html) on Linux, macOS (Darwin) and Windows hosts
|
||||
- [system state](https://www.elastic.co/guide/en/beats/auditbeat/master/auditbeat-module-system.html) including host, process, login, package, socket and user information on Linux, with some data sets supported on macOS and Windows hosts (apparently not available with the [Open Source Elastic license](https://www.elastic.co/subscriptions))
|
||||
* [Filebeat](https://www.elastic.co/beats/filebeat)
|
||||
- [system logs](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-system.html) (syslog and authentication logs) on Linux hosts
|
||||
- log output from [many products](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-modules.html) across Beats-supported platforms
|
||||
- arbitrary textual [log files](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html)
|
||||
* [Metricbeat](https://www.elastic.co/beats/metricbeat)
|
||||
- [system](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-system.html) resource utilization and process information
|
||||
- metrics from [many products](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-modules.html) across Beats-supported platforms
|
||||
* [Packetbeat](https://www.elastic.co/beats/packetbeat)
|
||||
- host-based packet inspection for [many protocols](https://www.elastic.co/guide/en/beats/packetbeat/current/configuration-protocols.html) (supports `libpcap` on Linux, [macOS](https://formulae.brew.sh/formula/libpcap) and [Windows](https://nmap.org/npcap/); and `af_packet` on Linux)
|
||||
* [Winlogbeat](https://www.elastic.co/downloads/beats/winlogbeat)
|
||||
* [Custom](https://www.elastic.co/guide/en/beats/devguide/current/index.html) Beats
|
||||
* [Community-contributed](https://www.elastic.co/guide/en/beats/devguide/current/community-beats.html) Beats
|
||||
|
||||
## Convenience configuration scripts and sample configurations
|
||||
|
||||
Two scripts are provided here for your convenience in configuring and running Beats to forward log data to Malcolm: [beat_config.py](./beat_config.py) and [beat_run.py](./beat_run.py). These Python scripts should run on Linux, macOS and Windows hosts with either Python 2 or Python 3.
|
||||
|
||||
Sample configurations are also provided for several beats for [Linux](./linux_vm_example) and [Windows](./windows_vm_example) hosts, as well as `Vagrantfile`s for setting up and running [VirtualBox](https://www.virtualbox.org/) VMs under [Vagrant](https://www.vagrantup.com/intro).
|
||||
|
||||
For further information, downloads, documentation or support for Beats, see the [Beats Platform Reference](https://www.elastic.co/guide/en/beats/libbeat/current/beats-reference.html) or the [Beats category](https://discuss.elastic.co/c/elastic-stack/beats) on the Elastic forums.
|
||||
|
||||
### Example: Windows configuration and run
|
||||
|
||||
```
|
||||
PS C:\Program Files\winlogbeat> dir
|
||||
|
||||
Directory: C:\Program Files\winlogbeat
|
||||
|
||||
Mode LastWriteTime Length Name
|
||||
---- ------------- ------ ----
|
||||
d----- 7/27/2020 8:49 AM kibana
|
||||
d----- 7/27/2020 8:49 AM module
|
||||
-a---- 3/26/2020 5:33 AM 41 .build_hash.txt
|
||||
-a---- 7/27/2020 8:50 AM 25799 beat_common.py
|
||||
-a---- 7/27/2020 8:50 AM 2525 beat_config.py
|
||||
-a---- 7/27/2020 8:50 AM 2244 beat_run.py
|
||||
-a---- 3/26/2020 5:32 AM 163122 fields.yml
|
||||
-a---- 7/27/2020 8:49 AM 878 install-service-winlogbeat.ps1
|
||||
-a---- 3/26/2020 4:44 AM 13675 LICENSE.txt
|
||||
-a---- 3/26/2020 4:44 AM 328580 NOTICE.txt
|
||||
-a---- 3/26/2020 5:33 AM 825 README.md
|
||||
-a---- 3/26/2020 5:33 AM 254 uninstall-service-winlogbeat.ps1
|
||||
-a---- 3/26/2020 5:33 AM 47818752 winlogbeat.exe
|
||||
-a---- 3/26/2020 5:32 AM 47900 winlogbeat.reference.yml
|
||||
-a---- 7/27/2020 8:50 AM 1349 winlogbeat.yml
|
||||
|
||||
|
||||
PS C:\Program Files\winlogbeat> .\beat_config.py -c .\winlogbeat.yml -b winlogbeat
|
||||
|
||||
Append connectivity boilerplate to .\winlogbeat.yml? (y/N): y
|
||||
|
||||
Created winlogbeat keystore
|
||||
|
||||
Configure winlogbeat Elasticsearch connectivity? (Y/n): y
|
||||
|
||||
Enter Elasticsearch connection protocol (http or https) [https]: https
|
||||
|
||||
Enter Elasticsearch SSL verification (none (for self-signed certificates) or full) [none]: none
|
||||
|
||||
Enter Elasticsearch connection host: 172.15.0.41:9200
|
||||
|
||||
Configure winlogbeat Kibana connectivity? (Y/n): y
|
||||
|
||||
Enter Kibana connection protocol (http or https) [https]: https
|
||||
|
||||
Enter Kibana SSL verification (none (for self-signed certificates) or full) [none]: none
|
||||
|
||||
Enter Kibana connection host: 172.15.0.41:5601
|
||||
|
||||
Configure winlogbeat Kibana dashboards? (Y/n): y
|
||||
|
||||
Enter directory containing Kibana dashboards [C:\Program Files\winlogbeat\kibana]: C:\Program Files\winlogbeat\kibana
|
||||
|
||||
Enter HTTP/HTTPS server username: sensor
|
||||
Enter password for sensor:
|
||||
Enter password for sensor (again):
|
||||
|
||||
Generated keystore for winlogbeat
|
||||
BEAT_KIBANA_SSL_VERIFY
|
||||
BEAT_ES_HOST
|
||||
BEAT_ES_PROTOCOL
|
||||
BEAT_ES_SSL_VERIFY
|
||||
BEAT_KIBANA_HOST
|
||||
BEAT_HTTP_PASSWORD
|
||||
BEAT_HTTP_USERNAME
|
||||
BEAT_KIBANA_DASHBOARDS_ENABLED
|
||||
BEAT_KIBANA_DASHBOARDS_PATH
|
||||
BEAT_KIBANA_PROTOCOL
|
||||
|
||||
PS C:\Program Files\winlogbeat> .\beat_run.py -c .\winlogbeat.yml -b winlogbeat
|
||||
|
||||
2020-07-27T09:00:17.472-0700 INFO instance/beat.go:622 Home path: [C:\Program Files\winlogbeat] Config path: [C:\Program Files\winlogbeat] Data path: [C:\Program Files\winlogbeat] Logs path: [C:\Program Files\winlogbeat\logs]
|
||||
2020-07-27T09:00:17.474-0700 INFO instance/beat.go:630 Beat ID: c38487f0-ea87-477b-aa93-376eb40949f4
|
||||
…
|
||||
^C
|
||||
KeyboardInterrupt
|
||||
2020-07-27T09:00:24.783-0700 INFO instance/beat.go:445 winlogbeat stopped.
|
||||
```
|
||||
|
||||
### Example: Linux configuration and run
|
||||
|
||||
```
|
||||
root@vagrant:/opt/filebeat# ls -l
|
||||
total 4
|
||||
-rw------- 1 root root 431 Jul 27 16:08 filebeat.yml
|
||||
|
||||
root@vagrant:/opt/filebeat# beat_config.py -c ./filebeat.yml -b filebeat
|
||||
|
||||
Append connectivity boilerplate to ./filebeat.yml? (y/N): y
|
||||
|
||||
Create symlink to module path /usr/share/filebeat/module as /opt/filebeat/module? (Y/n): y
|
||||
|
||||
Created filebeat keystore
|
||||
|
||||
Configure filebeat Elasticsearch connectivity? (Y/n): y
|
||||
|
||||
Enter Elasticsearch connection protocol (http or https) [https]: https
|
||||
|
||||
Enter Elasticsearch SSL verification (none (for self-signed certificates) or full) [none]: none
|
||||
|
||||
Enter Elasticsearch connection host: 172.15.0.41:9200
|
||||
|
||||
Configure filebeat Kibana connectivity? (Y/n): y
|
||||
|
||||
Enter Kibana connection protocol (http or https) [https]: https
|
||||
|
||||
Enter Kibana SSL verification (none (for self-signed certificates) or full) [none]: none
|
||||
|
||||
Enter Kibana connection host: 172.15.0.41:5601
|
||||
|
||||
Configure filebeat Kibana dashboards? (Y/n): y
|
||||
|
||||
Enter directory containing Kibana dashboards [/usr/share/filebeat/kibana]: /usr/share/filebeat/kibana
|
||||
|
||||
Enter HTTP/HTTPS server username: sensor
|
||||
Enter password for sensor:
|
||||
Enter password for sensor (again):
|
||||
|
||||
Generated keystore for filebeat
|
||||
BEAT_KIBANA_PROTOCOL
|
||||
BEAT_KIBANA_SSL_VERIFY
|
||||
BEAT_ES_PROTOCOL
|
||||
BEAT_ES_SSL_VERIFY
|
||||
BEAT_KIBANA_DASHBOARDS_ENABLED
|
||||
BEAT_KIBANA_DASHBOARDS_PATH
|
||||
BEAT_ES_HOST
|
||||
BEAT_HTTP_PASSWORD
|
||||
BEAT_HTTP_USERNAME
|
||||
BEAT_KIBANA_HOST
|
||||
|
||||
root@vagrant:/opt/filebeat# beat_run.py -c ./filebeat.yml -b filebeat
|
||||
|
||||
2020-07-27T16:12:43.270Z INFO instance/beat.go:622 Home path: [/opt/filebeat] Config path: [/opt/filebeat] Data path: [/opt/filebeat/data] Logs path: [/opt/filebeat/logs]
|
||||
2020-07-27T16:12:43.270Z INFO instance/beat.go:630 Beat ID: 759019e0-705c-4a16-87a2-52e9a5f6e799
|
||||
…
|
||||
^C
|
||||
KeyboardInterrupt
|
||||
2020-07-27T16:13:10.816Z INFO beater/filebeat.go:443 Stopping filebeat
|
||||
```
|
||||
|
||||
# <a name="Footer"></a>Copyright
|
||||
|
||||
[Malcolm](https://github.com/idaholab/Malcolm) is Copyright 2021 Battelle Energy Alliance, LLC, and is developed and released through the cooperation of the Cybersecurity and Infrastructure Security Agency of the U.S. Department of Homeland Security.
|
||||
|
||||
See [`License.txt`](https://raw.githubusercontent.com/idaholab/Malcolm/master/License.txt) for the terms of its release.
|
||||
|
||||
### Contact information of author(s):
|
||||
|
||||
[Seth Grover](mailto:malcolm.netsec@gmail.com?subject=Malcolm)
|
||||
622
Vagrant/resources/malcolm/scripts/beats/beat_common.py
Normal file
622
Vagrant/resources/malcolm/scripts/beats/beat_common.py
Normal file
@@ -0,0 +1,622 @@
|
||||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright (c) 2021 Battelle Energy Alliance, LLC. All rights reserved.
|
||||
|
||||
from __future__ import print_function
|
||||
|
||||
import getpass
|
||||
import inspect
|
||||
import os
|
||||
import platform
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
|
||||
from collections import defaultdict
|
||||
|
||||
from subprocess import (PIPE, STDOUT, Popen, CalledProcessError)
|
||||
|
||||
###################################################################################################
|
||||
ScriptPath = os.path.dirname(os.path.realpath(__file__))
|
||||
|
||||
###################################################################################################
|
||||
# python 2/3 portability
|
||||
PY3 = (sys.version_info.major >= 3)
|
||||
|
||||
# bind raw_input to input in older versions of python
|
||||
try:
|
||||
input = raw_input
|
||||
except NameError:
|
||||
pass
|
||||
|
||||
try:
|
||||
FileNotFoundError
|
||||
except NameError:
|
||||
FileNotFoundError = IOError
|
||||
|
||||
###################################################################################################
|
||||
PLATFORM_WINDOWS = "Windows"
|
||||
PLATFORM_MAC = "Darwin"
|
||||
PLATFORM_LINUX = "Linux"
|
||||
PLATFORM_LINUX_CENTOS = 'centos'
|
||||
PLATFORM_LINUX_DEBIAN = 'debian'
|
||||
PLATFORM_LINUX_FEDORA = 'fedora'
|
||||
PLATFORM_LINUX_UBUNTU = 'ubuntu'
|
||||
|
||||
OPERATION_RUN = 'run'
|
||||
OPERATION_CONFIGURE = 'config'
|
||||
|
||||
BEAT_ES_HOST = "BEAT_ES_HOST"
|
||||
BEAT_ES_PROTOCOL = "BEAT_ES_PROTOCOL"
|
||||
BEAT_ES_SSL_VERIFY = "BEAT_ES_SSL_VERIFY"
|
||||
BEAT_HTTP_PASSWORD = "BEAT_HTTP_PASSWORD"
|
||||
BEAT_HTTP_USERNAME = "BEAT_HTTP_USERNAME"
|
||||
BEAT_KIBANA_DASHBOARDS_ENABLED = "BEAT_KIBANA_DASHBOARDS_ENABLED"
|
||||
BEAT_KIBANA_DASHBOARDS_PATH = "BEAT_KIBANA_DASHBOARDS_PATH"
|
||||
BEAT_KIBANA_HOST = "BEAT_KIBANA_HOST"
|
||||
BEAT_KIBANA_PROTOCOL = "BEAT_KIBANA_PROTOCOL"
|
||||
BEAT_KIBANA_SSL_VERIFY = "BEAT_KIBANA_SSL_VERIFY"
|
||||
|
||||
BEAT_YML_TEMPLATE = """
|
||||
#================================ General ======================================
|
||||
fields_under_root: true
|
||||
|
||||
#================================ Outputs ======================================
|
||||
|
||||
#-------------------------- Elasticsearch output -------------------------------
|
||||
output.elasticsearch:
|
||||
enabled: true
|
||||
hosts: ["${BEAT_ES_HOST}"]
|
||||
protocol: "${BEAT_ES_PROTOCOL}"
|
||||
username: "${BEAT_HTTP_USERNAME}"
|
||||
password: "${BEAT_HTTP_PASSWORD}"
|
||||
ssl.verification_mode: "${BEAT_ES_SSL_VERIFY}"
|
||||
|
||||
setup.template.enabled: true
|
||||
setup.template.overwrite: false
|
||||
setup.template.settings:
|
||||
index.number_of_shards: 1
|
||||
index.number_of_replicas: 0
|
||||
|
||||
#============================== Dashboards =====================================
|
||||
setup.dashboards.enabled: "${BEAT_KIBANA_DASHBOARDS_ENABLED}"
|
||||
setup.dashboards.directory: "${BEAT_KIBANA_DASHBOARDS_PATH}"
|
||||
|
||||
#============================== Kibana =====================================
|
||||
setup.kibana:
|
||||
host: "${BEAT_KIBANA_HOST}"
|
||||
protocol: "${BEAT_KIBANA_PROTOCOL}"
|
||||
username: "${BEAT_HTTP_USERNAME}"
|
||||
password: "${BEAT_HTTP_PASSWORD}"
|
||||
ssl.verification_mode: "${BEAT_KIBANA_SSL_VERIFY}"
|
||||
|
||||
#================================ Logging ======================================
|
||||
logging.metrics.enabled: false
|
||||
"""
|
||||
|
||||
###################################################################################################
|
||||
# print to stderr
|
||||
def eprint(*args, **kwargs):
|
||||
print(*args, file=sys.stderr, **kwargs)
|
||||
|
||||
###################################################################################################
|
||||
# get interactive user response to Y/N question
|
||||
def YesOrNo(question, default=None, forceInteraction=False, acceptDefault=False):
|
||||
|
||||
if default == True:
|
||||
questionStr = "\n{} (Y/n): ".format(question)
|
||||
elif default == False:
|
||||
questionStr = "\n{} (y/N): ".format(question)
|
||||
else:
|
||||
questionStr = "\n{} (y/n): ".format(question)
|
||||
|
||||
if acceptDefault and (default is not None) and (not forceInteraction):
|
||||
reply = ''
|
||||
else:
|
||||
while True:
|
||||
reply = str(input(questionStr)).lower().strip()
|
||||
if (len(reply) > 0) or (default is not None):
|
||||
break
|
||||
|
||||
if (len(reply) == 0):
|
||||
reply = 'y' if default else 'n'
|
||||
|
||||
if reply[0] == 'y':
|
||||
return True
|
||||
elif reply[0] == 'n':
|
||||
return False
|
||||
else:
|
||||
return YesOrNo(question, default=default)
|
||||
|
||||
###################################################################################################
|
||||
# get interactive user response
|
||||
def AskForString(question, default=None, forceInteraction=False, acceptDefault=False):
|
||||
|
||||
if acceptDefault and (default is not None) and (not forceInteraction):
|
||||
reply = default
|
||||
else:
|
||||
reply = str(input('\n{}: '.format(question))).strip()
|
||||
|
||||
return reply
|
||||
|
||||
###################################################################################################
|
||||
# get interactive password (without echoing)
|
||||
def AskForPassword(prompt):
|
||||
reply = getpass.getpass(prompt=prompt)
|
||||
return reply
|
||||
|
||||
###################################################################################################
|
||||
# convenient boolean argument parsing
|
||||
def str2bool(v):
|
||||
if v.lower() in ('yes', 'true', 't', 'y', '1'):
|
||||
return True
|
||||
elif v.lower() in ('no', 'false', 'f', 'n', '0'):
|
||||
return False
|
||||
else:
|
||||
raise ValueError('Boolean value expected')
|
||||
|
||||
###################################################################################################
|
||||
# determine if a program/script exists and is executable in the system path
|
||||
def Which(cmd, debug=False):
|
||||
result = any(os.access(os.path.join(path, cmd), os.X_OK) for path in os.environ["PATH"].split(os.pathsep))
|
||||
if (not result) and (platform.system() == PLATFORM_WINDOWS):
|
||||
result = os.access(os.path.join(os.getcwd(), cmd), os.X_OK)
|
||||
if debug:
|
||||
eprint("Which {} returned {}".format(cmd, result))
|
||||
return result
|
||||
|
||||
###################################################################################################
|
||||
# run command with arguments and return its exit code, stdout, and stderr
|
||||
def check_output_input(*popenargs, **kwargs):
|
||||
|
||||
if 'stdout' in kwargs:
|
||||
raise ValueError('stdout argument not allowed, it will be overridden')
|
||||
|
||||
if 'stderr' in kwargs:
|
||||
raise ValueError('stderr argument not allowed, it will be overridden')
|
||||
|
||||
if 'input' in kwargs and kwargs['input']:
|
||||
if 'stdin' in kwargs:
|
||||
raise ValueError('stdin and input arguments may not both be used')
|
||||
inputdata = kwargs['input']
|
||||
kwargs['stdin'] = PIPE
|
||||
else:
|
||||
inputdata = None
|
||||
kwargs.pop('input', None)
|
||||
|
||||
process = Popen(*popenargs, stdout=PIPE, stderr=PIPE, **kwargs)
|
||||
try:
|
||||
output, errput = process.communicate(inputdata)
|
||||
except:
|
||||
process.kill()
|
||||
process.wait()
|
||||
raise
|
||||
|
||||
retcode = process.poll()
|
||||
|
||||
return retcode, output, errput
|
||||
|
||||
###################################################################################################
|
||||
# run command with arguments and return its exit code, stdout, and stderr
|
||||
def run_process(command, stdout=True, stderr=True, stdin=None, retry=0, retrySleepSec=5, cwd=None, env=None, debug=False):
|
||||
|
||||
retcode = -1
|
||||
output = []
|
||||
|
||||
try:
|
||||
# run the command
|
||||
retcode, cmdout, cmderr = check_output_input(command, input=stdin.encode() if (PY3 and stdin) else stdin, cwd=cwd, env=env)
|
||||
|
||||
# split the output on newlines to return a list
|
||||
if PY3:
|
||||
if stderr and (len(cmderr) > 0): output.extend(cmderr.decode(sys.getdefaultencoding()).split('\n'))
|
||||
if stdout and (len(cmdout) > 0): output.extend(cmdout.decode(sys.getdefaultencoding()).split('\n'))
|
||||
else:
|
||||
if stderr and (len(cmderr) > 0): output.extend(cmderr.split('\n'))
|
||||
if stdout and (len(cmdout) > 0): output.extend(cmdout.split('\n'))
|
||||
|
||||
except (FileNotFoundError, OSError, IOError) as e:
|
||||
if stderr:
|
||||
output.append("Command {} not found or unable to execute".format(command))
|
||||
|
||||
if debug:
|
||||
eprint("{}{} returned {}: {}".format(command, "({})".format(stdin[:80] + bool(stdin[80:]) * '...' if stdin else ""), retcode, output))
|
||||
|
||||
if (retcode != 0) and retry and (retry > 0):
|
||||
# sleep then retry
|
||||
time.sleep(retrySleepSec)
|
||||
return run_process(command, stdout, stderr, stdin, retry-1, retrySleepSec, cwd, env, debug)
|
||||
else:
|
||||
return retcode, output
|
||||
|
||||
###################################################################################################
|
||||
class Beatbox(object):
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def __init__(self, debug=False, ymlFileSpec=None, beatName=None, acceptDefaults=False):
|
||||
self.debug = debug
|
||||
self.acceptDefaults = acceptDefaults
|
||||
self.platform = platform.system()
|
||||
self.ymlFileSpec = ymlFileSpec
|
||||
self.ymlFilePath = os.path.dirname(ymlFileSpec)
|
||||
self.beatName = beatName
|
||||
self.beatExe = beatName
|
||||
self.beatInstallDir = None
|
||||
self.defaultKibanaDashboardDir = None
|
||||
self.keystoreItems = defaultdict(str)
|
||||
for initItem in [BEAT_ES_HOST,
|
||||
BEAT_ES_PROTOCOL,
|
||||
BEAT_ES_SSL_VERIFY,
|
||||
BEAT_HTTP_PASSWORD,
|
||||
BEAT_HTTP_USERNAME,
|
||||
BEAT_KIBANA_DASHBOARDS_ENABLED,
|
||||
BEAT_KIBANA_DASHBOARDS_PATH,
|
||||
BEAT_KIBANA_HOST,
|
||||
BEAT_KIBANA_PROTOCOL,
|
||||
BEAT_KIBANA_SSL_VERIFY]:
|
||||
self.keystoreItems[initItem] = ''
|
||||
self.keystorePath = None
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def __del__(self):
|
||||
# nothing for now
|
||||
pass
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def run_process(self, command, stdout=True, stderr=True, stdin=None, retry=0, retrySleepSec=5):
|
||||
return run_process(command, stdout=stdout, stderr=stderr, stdin=stdin, retry=retry, retrySleepSec=retrySleepSec, debug=self.debug)
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def build_beat_command(self, command):
|
||||
if not Which(self.beatExe, debug=self.debug):
|
||||
raise Exception("Beat executable {} does not exist".format(self.beatExe))
|
||||
|
||||
if not os.path.isfile(self.ymlFileSpec):
|
||||
raise Exception("Beat configuration {} does not exist".format(self.ymlFileSpec))
|
||||
|
||||
# convert paths to absolutes
|
||||
ymlFileSpec = os.path.abspath(self.ymlFileSpec)
|
||||
ymlFilePath = os.path.dirname(ymlFileSpec)
|
||||
|
||||
beatCmd = [self.beatExe, '--path.home', ymlFilePath, '--path.config', ymlFilePath, '--path.data', ymlFilePath if (self.platform == PLATFORM_WINDOWS) else os.path.join(ymlFilePath, 'data'), '--path.logs', os.path.join(ymlFilePath, 'logs'), '-c', ymlFileSpec, '-E', "keystore.path='{}'".format(self.keystorePath)]
|
||||
|
||||
return beatCmd + command if isinstance(command, list) else beatCmd + [ command ]
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def run_beat_command(self, command, stdout=True, stderr=True, stdin=None, retry=0, retrySleepSec=5):
|
||||
return self.run_process(self.build_beat_command(command), stdout=stdout, stderr=stderr, stdin=stdin, retry=retry, retrySleepSec=retrySleepSec)
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def configure_beat_yml(self):
|
||||
if self.debug:
|
||||
eprint("{}: {}".format(self.__class__.__name__, inspect.currentframe().f_code.co_name))
|
||||
|
||||
if (self.ymlFileSpec is not None):
|
||||
|
||||
if os.path.isfile(self.ymlFileSpec):
|
||||
# if it doesn't look like connectivity stuff (at last BEAT_ES_PROTOCOL) is in the YML file, offer to append it
|
||||
if ((len(list(filter(lambda x: BEAT_ES_PROTOCOL in x, [line.rstrip('\n') for line in open(self.ymlFileSpec)]))) == 0) and
|
||||
YesOrNo("Append connectivity boilerplate to {}?".format(self.ymlFileSpec), default=False, acceptDefault=self.acceptDefaults)):
|
||||
with open(self.ymlFileSpec, 'a') as ymlFile:
|
||||
ymlFile.write(BEAT_YML_TEMPLATE)
|
||||
|
||||
else:
|
||||
# generate a boilerplate spec file (output configured, no modules) if the YML file doesn't exist
|
||||
with open(self.ymlFileSpec, 'w') as ymlFile:
|
||||
ymlFile.write(BEAT_YML_TEMPLATE)
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def configure_keystore(self):
|
||||
if self.debug:
|
||||
eprint("{}: {}".format(self.__class__.__name__, inspect.currentframe().f_code.co_name))
|
||||
|
||||
# check if keystore already exists
|
||||
err, out = self.run_beat_command(['keystore', 'list'])
|
||||
if (err == 0) and (len(out) > 0):
|
||||
if not YesOrNo("{} keystore already exists, overwrite?".format(self.beatName), default=False, acceptDefault=self.acceptDefaults):
|
||||
raise Exception("Configuration cancelled by user")
|
||||
|
||||
# create keystore
|
||||
err, out = self.run_beat_command(['keystore', 'create', '--force'])
|
||||
if (err == 0):
|
||||
eprint('\n'.join(out))
|
||||
else:
|
||||
raise Exception("Keystore creation failed: {}".format(out))
|
||||
|
||||
# prompt for and store configuration items
|
||||
for destination in ['Elasticsearch', 'Kibana']:
|
||||
|
||||
if YesOrNo("Configure {} {} connectivity?".format(self.beatName, destination), default=True, acceptDefault=self.acceptDefaults):
|
||||
|
||||
# protocol
|
||||
tmpVal, tmpDefault = '', 'https'
|
||||
while tmpVal not in ['http', 'https']:
|
||||
tmpVal = AskForString("Enter {} connection protocol (http or https) [{}]".format(destination, tmpDefault), default=tmpDefault, acceptDefault=self.acceptDefaults).lower()
|
||||
if (len(tmpVal) == 0): tmpVal = tmpDefault
|
||||
self.keystoreItems[BEAT_ES_PROTOCOL.replace('_ES_', '_KIBANA_' if (destination == 'Kibana') else '_ES_')] = tmpVal
|
||||
|
||||
# SSL verification
|
||||
tmpVal, tmpDefault = '', 'none'
|
||||
while tmpVal not in ['none', 'full']:
|
||||
tmpVal = AskForString("Enter {} SSL verification (none (for self-signed certificates) or full) [{}]".format(destination, tmpDefault), default=tmpDefault, acceptDefault=self.acceptDefaults).lower()
|
||||
if (len(tmpVal) == 0): tmpVal = tmpDefault
|
||||
self.keystoreItems[BEAT_ES_SSL_VERIFY.replace('_ES_', '_KIBANA_' if (destination == 'Kibana') else '_ES_')] = tmpVal
|
||||
|
||||
# host
|
||||
tmpVal, tmpDefault = '', ''
|
||||
while (len(tmpVal) == 0):
|
||||
tmpVal = AskForString("Enter {} connection host".format(destination), default=tmpDefault, acceptDefault=self.acceptDefaults)
|
||||
self.keystoreItems[BEAT_ES_HOST.replace('_ES_', '_KIBANA_' if (destination == 'Kibana') else '_ES_')] = tmpVal
|
||||
|
||||
if (BEAT_KIBANA_HOST in self.keystoreItems):
|
||||
|
||||
# configure kibana dashboards
|
||||
if YesOrNo("Configure {} Kibana dashboards?".format(self.beatName), default=True, acceptDefault=self.acceptDefaults):
|
||||
self.keystoreItems[BEAT_KIBANA_DASHBOARDS_ENABLED] = 'true'
|
||||
|
||||
# kibana dashboards
|
||||
tmpVal, tmpDefault = '', self.defaultKibanaDashboardDir
|
||||
while (len(tmpVal) == 0):
|
||||
tmpVal = AskForString("Enter directory containing Kibana dashboards [{}]".format(tmpDefault), default=tmpDefault, acceptDefault=self.acceptDefaults)
|
||||
if (len(tmpVal) == 0): tmpVal = tmpDefault
|
||||
self.keystoreItems[BEAT_KIBANA_DASHBOARDS_PATH] = tmpVal
|
||||
|
||||
# username
|
||||
tmpVal, tmpDefault = '', ''
|
||||
while (len(tmpVal) == 0):
|
||||
tmpVal = AskForString("Enter HTTP/HTTPS server username", default=tmpDefault, acceptDefault=self.acceptDefaults)
|
||||
self.keystoreItems[BEAT_HTTP_USERNAME] = tmpVal
|
||||
|
||||
# password
|
||||
tmpVal, tmpValConfirm = '', 'xxxx'
|
||||
while (len(tmpVal) == 0) and (tmpVal != tmpValConfirm):
|
||||
tmpVal = AskForPassword("Enter password for {}: ".format(self.keystoreItems[BEAT_HTTP_USERNAME]))
|
||||
tmpValConfirm = AskForPassword("Enter password for {} (again): ".format(self.keystoreItems[BEAT_HTTP_USERNAME]))
|
||||
if (tmpVal != tmpValConfirm):
|
||||
eprint('Passwords do not match')
|
||||
tmpVal, tmpValConfirm = '', 'xxxx'
|
||||
self.keystoreItems[BEAT_HTTP_PASSWORD] = tmpVal
|
||||
|
||||
# write values to keystore
|
||||
for key, value in self.keystoreItems.items():
|
||||
err, out = self.run_beat_command(['keystore', 'add', key, '--stdin', '--force'], stdin=value)
|
||||
if (err != 0):
|
||||
raise Exception("Failed to add {} to {} keystore: {}".format(key, self.beatName, out))
|
||||
|
||||
# list keystore
|
||||
err, out = self.run_beat_command(['keystore', 'list'])
|
||||
if (err == 0):
|
||||
eprint('Generated keystore for {}'.format(self.beatName))
|
||||
eprint('\n'.join(out))
|
||||
else:
|
||||
raise Exception("Failed to enumerate keystore: {}".format(out))
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def beat_run(self):
|
||||
if self.debug:
|
||||
eprint("{}: {}".format(self.__class__.__name__, inspect.currentframe().f_code.co_name))
|
||||
|
||||
process = Popen(self.build_beat_command(['run', '-e']), stdout=PIPE)
|
||||
while True:
|
||||
output = process.stdout.readline()
|
||||
if (len(output) == 0) and (process.poll() is not None):
|
||||
break
|
||||
if output:
|
||||
print(output.decode().strip())
|
||||
else:
|
||||
time.sleep(0.5)
|
||||
process.poll()
|
||||
|
||||
###################################################################################################
|
||||
class LinuxBeatbox(Beatbox):
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def __init__(self, debug=False, ymlFileSpec=None, beatName=None):
|
||||
if PY3:
|
||||
super().__init__(debug=debug, ymlFileSpec=ymlFileSpec, beatName=beatName)
|
||||
else:
|
||||
super(LinuxBeatbox, self).__init__(debug=debug, ymlFileSpec=ymlFileSpec, beatName=beatName)
|
||||
|
||||
if not Which(self.beatExe, debug=self.debug):
|
||||
self.beatExe = self.beatExe.lower() if (self.beatExe is not None) else self.beatName.lower()
|
||||
|
||||
self.beatInstallDir = "/usr/share/{}".format(self.beatName)
|
||||
self.defaultKibanaDashboardDir = os.path.join(self.beatInstallDir, 'kibana')
|
||||
self.keystorePath = os.path.join(os.path.join(os.path.dirname(os.path.abspath(self.ymlFileSpec)), 'data'), "{}.keystore".format(self.beatName))
|
||||
|
||||
self.distro = None
|
||||
self.codename = None
|
||||
self.release = None
|
||||
|
||||
# determine the distro (e.g., ubuntu) and code name (e.g., bionic) if applicable
|
||||
|
||||
# check /etc/os-release values first
|
||||
if os.path.isfile('/etc/os-release'):
|
||||
osInfo = dict()
|
||||
|
||||
with open("/etc/os-release", 'r') as f:
|
||||
for line in f:
|
||||
try:
|
||||
k, v = line.rstrip().split("=")
|
||||
osInfo[k] = v.strip('"')
|
||||
except:
|
||||
pass
|
||||
|
||||
if ('NAME' in osInfo) and (len(osInfo['NAME']) > 0):
|
||||
distro = osInfo['NAME'].lower().split()[0]
|
||||
|
||||
if ('VERSION_CODENAME' in osInfo) and (len(osInfo['VERSION_CODENAME']) > 0):
|
||||
codename = osInfo['VERSION_CODENAME'].lower().split()[0]
|
||||
|
||||
if ('VERSION_ID' in osInfo) and (len(osInfo['VERSION_ID']) > 0):
|
||||
release = osInfo['VERSION_ID'].lower().split()[0]
|
||||
|
||||
# try lsb_release next
|
||||
if (self.distro is None):
|
||||
err, out = self.run_process(['lsb_release', '-is'], stderr=False)
|
||||
if (err == 0) and (len(out) > 0):
|
||||
self.distro = out[0].lower()
|
||||
|
||||
if (self.codename is None):
|
||||
err, out = self.run_process(['lsb_release', '-cs'], stderr=False)
|
||||
if (err == 0) and (len(out) > 0):
|
||||
self.codename = out[0].lower()
|
||||
|
||||
if (self.release is None):
|
||||
err, out = self.run_process(['lsb_release', '-rs'], stderr=False)
|
||||
if (err == 0) and (len(out) > 0):
|
||||
self.release = out[0].lower()
|
||||
|
||||
# try release-specific files
|
||||
if (self.distro is None):
|
||||
if os.path.isfile('/etc/centos-release'):
|
||||
distroFile = '/etc/centos-release'
|
||||
if os.path.isfile('/etc/redhat-release'):
|
||||
distroFile = '/etc/redhat-release'
|
||||
elif os.path.isfile('/etc/issue'):
|
||||
distroFile = '/etc/issue'
|
||||
else:
|
||||
distroFile = None
|
||||
if (distroFile is not None):
|
||||
with open(distroFile, 'r') as f:
|
||||
distroVals = f.read().lower().split()
|
||||
distroNums = [x for x in distroVals if x[0].isdigit()]
|
||||
self.distro = distroVals[0]
|
||||
if (self.release is None) and (len(distroNums) > 0):
|
||||
self.release = distroNums[0]
|
||||
|
||||
if (self.distro is None):
|
||||
self.distro = "linux"
|
||||
|
||||
if self.debug:
|
||||
eprint("distro: {}{}{}".format(self.distro,
|
||||
" {}".format(self.codename) if self.codename else "",
|
||||
" {}".format(self.release) if self.release else ""))
|
||||
|
||||
if not self.codename: self.codename = self.distro
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def configure_beat_yml(self):
|
||||
if PY3:
|
||||
super().configure_beat_yml()
|
||||
else:
|
||||
super(LinuxBeatbox, self).configure_beat_yml()
|
||||
|
||||
localModulePath = os.path.join(os.path.abspath(self.ymlFilePath), 'module')
|
||||
installedModulePath = os.path.join(self.beatInstallDir, 'module')
|
||||
if ((not os.path.exists(localModulePath)) and
|
||||
(os.path.isdir(installedModulePath)) and
|
||||
YesOrNo("Create symlink to module path {} as {}?".format(installedModulePath, localModulePath), default=True, acceptDefault=self.acceptDefaults)):
|
||||
os.symlink(installedModulePath, localModulePath)
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def configure_keystore(self):
|
||||
if PY3:
|
||||
super().configure_keystore()
|
||||
else:
|
||||
super(LinuxBeatbox, self).configure_keystore()
|
||||
|
||||
pass
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def beat_run(self):
|
||||
if PY3:
|
||||
super().beat_run()
|
||||
else:
|
||||
super(LinuxBeatbox, self).beat_run()
|
||||
|
||||
pass
|
||||
|
||||
###################################################################################################
|
||||
class WindowsBeatbox(Beatbox):
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def __init__(self, debug=False, ymlFileSpec=None, beatName=None):
|
||||
if PY3:
|
||||
super().__init__(debug=debug, ymlFileSpec=ymlFileSpec, beatName=beatName)
|
||||
else:
|
||||
super(WindowsBeatbox, self).__init__(debug=debug, ymlFileSpec=ymlFileSpec, beatName=beatName)
|
||||
|
||||
if not Which(self.beatExe, debug=self.debug):
|
||||
self.beatExe = self.beatExe + '.exe' if (self.beatExe is not None) else self.beatName + '.exe'
|
||||
|
||||
self.beatInstallDir = os.path.abspath(self.ymlFilePath)
|
||||
self.defaultKibanaDashboardDir = os.path.join(self.beatInstallDir, 'kibana')
|
||||
self.keystorePath = os.path.join(os.path.dirname(os.path.abspath(self.ymlFileSpec)), "{}.keystore".format(self.beatName))
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def configure_beat_yml(self):
|
||||
if PY3:
|
||||
super().configure_beat_yml()
|
||||
else:
|
||||
super(WindowsBeatbox, self).configure_beat_yml()
|
||||
|
||||
pass
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def configure_keystore(self):
|
||||
if PY3:
|
||||
super().configure_keystore()
|
||||
else:
|
||||
super(WindowsBeatbox, self).configure_keystore()
|
||||
|
||||
pass
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def beat_run(self):
|
||||
if PY3:
|
||||
super().beat_run()
|
||||
else:
|
||||
super(WindowsBeatbox, self).beat_run()
|
||||
|
||||
pass
|
||||
|
||||
###################################################################################################
|
||||
class MacBeatbox(Beatbox):
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def __init__(self, debug=False, ymlFileSpec=None, beatName=None):
|
||||
if PY3:
|
||||
super().__init__(debug=debug, ymlFileSpec=ymlFileSpec, beatName=beatName)
|
||||
else:
|
||||
super(MacBeatbox, self).__init__(debug=debug, ymlFileSpec=ymlFileSpec, beatName=beatName)
|
||||
|
||||
if not Which(self.beatExe, debug=self.debug):
|
||||
self.beatExe = self.beatExe.lower() if (self.beatExe is not None) else self.beatName.lower()
|
||||
|
||||
self.beatInstallDir = "/Library/Application Support/elastic/{}".format(self.beatName)
|
||||
self.defaultKibanaDashboardDir = os.path.join(self.beatInstallDir, 'kibana')
|
||||
self.keystorePath = os.path.join(os.path.join(os.path.dirname(os.path.abspath(self.ymlFileSpec)), 'data'), "{}.keystore".format(self.beatName))
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def configure_beat_yml(self):
|
||||
if PY3:
|
||||
super().configure_beat_yml()
|
||||
else:
|
||||
super(MacBeatbox, self).configure_beat_yml()
|
||||
|
||||
localModulePath = os.path.join(os.path.abspath(self.ymlFilePath), 'module')
|
||||
installedModulePath = os.path.join(self.beatInstallDir, 'module')
|
||||
if ((not os.path.exists(localModulePath)) and
|
||||
(os.path.isdir(installedModulePath)) and
|
||||
YesOrNo("Create symlink to module path {} as {}?".format(installedModulePath, localModulePath), default=True, acceptDefault=self.acceptDefaults)):
|
||||
os.symlink(installedModulePath, localModulePath)
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def configure_keystore(self):
|
||||
if PY3:
|
||||
super().configure_keystore()
|
||||
else:
|
||||
super(MacBeatbox, self).configure_keystore()
|
||||
|
||||
pass
|
||||
|
||||
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
def beat_run(self):
|
||||
if PY3:
|
||||
super().beat_run()
|
||||
else:
|
||||
super(MacBeatbox, self).beat_run()
|
||||
|
||||
pass
|
||||
65
Vagrant/resources/malcolm/scripts/beats/beat_config.py
Executable file
65
Vagrant/resources/malcolm/scripts/beats/beat_config.py
Executable file
@@ -0,0 +1,65 @@
|
||||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright (c) 2021 Battelle Energy Alliance, LLC. All rights reserved.
|
||||
|
||||
from __future__ import print_function
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import platform
|
||||
import sys
|
||||
from beat_common import *
|
||||
|
||||
###################################################################################################
|
||||
ScriptName = os.path.basename(__file__)
|
||||
|
||||
###################################################################################################
|
||||
# main
|
||||
def main():
|
||||
|
||||
# extract arguments from the command line
|
||||
# print (sys.argv[1:]);
|
||||
parser = argparse.ArgumentParser(description='Beat configure script', add_help=False, usage='{} <arguments>'.format(ScriptName))
|
||||
parser.add_argument('-v', '--verbose', dest='debug', type=str2bool, nargs='?', const=True, default=False, help="Verbose output")
|
||||
parser.add_argument('-b', '--beat', required=True, dest='beatName', metavar='<STR>', type=str, default=None, help='Beat name')
|
||||
parser.add_argument('-c', '--config-file', required=False, dest='configFile', metavar='<STR>', type=str, default=None, help='Beat YML file to configure')
|
||||
parser.add_argument('-d', '--defaults', dest='acceptDefault', type=str2bool, nargs='?', const=True, default=False, help="Accept defaults to prompts without user interaction")
|
||||
|
||||
try:
|
||||
parser.error = parser.exit
|
||||
args = parser.parse_args()
|
||||
except SystemExit:
|
||||
parser.print_help()
|
||||
exit(2)
|
||||
|
||||
if args.debug:
|
||||
eprint(os.path.join(ScriptPath, ScriptName))
|
||||
eprint("Arguments: {}".format(sys.argv[1:]))
|
||||
eprint("Arguments: {}".format(args))
|
||||
else:
|
||||
sys.tracebacklimit = 0
|
||||
|
||||
args.beatName = args.beatName.lower()
|
||||
if not args.beatName.endswith('beat'):
|
||||
args.beatName = args.beatName + 'beat'
|
||||
|
||||
if (args.configFile is None):
|
||||
args.configFile = args.beatName + '.yml'
|
||||
|
||||
installerPlatform = platform.system()
|
||||
if installerPlatform == PLATFORM_LINUX:
|
||||
Beatbox = LinuxBeatbox(debug=args.debug, ymlFileSpec=args.configFile, beatName=args.beatName)
|
||||
elif installerPlatform == PLATFORM_MAC:
|
||||
Beatbox = MacBeatbox(debug=args.debug, ymlFileSpec=args.configFile, beatName=args.beatName)
|
||||
elif installerPlatform == PLATFORM_WINDOWS:
|
||||
Beatbox = WindowsBeatbox(debug=args.debug, ymlFileSpec=args.configFile, beatName=args.beatName)
|
||||
|
||||
success = False
|
||||
if hasattr(Beatbox, 'configure_beat_yml'): success = Beatbox.configure_beat_yml()
|
||||
if hasattr(Beatbox, 'configure_keystore'): success = Beatbox.configure_keystore()
|
||||
|
||||
return success
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
63
Vagrant/resources/malcolm/scripts/beats/beat_run.py
Executable file
63
Vagrant/resources/malcolm/scripts/beats/beat_run.py
Executable file
@@ -0,0 +1,63 @@
|
||||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright (c) 2021 Battelle Energy Alliance, LLC. All rights reserved.
|
||||
|
||||
from __future__ import print_function
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import platform
|
||||
import sys
|
||||
from beat_common import *
|
||||
|
||||
###################################################################################################
|
||||
ScriptName = os.path.basename(__file__)
|
||||
|
||||
###################################################################################################
|
||||
# main
|
||||
def main():
|
||||
|
||||
# extract arguments from the command line
|
||||
# print (sys.argv[1:]);
|
||||
parser = argparse.ArgumentParser(description='Beat local execution script', add_help=False, usage='{} <arguments>'.format(ScriptName))
|
||||
parser.add_argument('-v', '--verbose', dest='debug', type=str2bool, nargs='?', const=True, default=False, help="Verbose output")
|
||||
parser.add_argument('-b', '--beat', required=True, dest='beatName', metavar='<STR>', type=str, default=None, help='Beat name')
|
||||
parser.add_argument('-c', '--config-file', required=False, dest='configFile', metavar='<STR>', type=str, default=None, help='Beat YML config file')
|
||||
|
||||
try:
|
||||
parser.error = parser.exit
|
||||
args = parser.parse_args()
|
||||
except SystemExit:
|
||||
parser.print_help()
|
||||
exit(2)
|
||||
|
||||
if args.debug:
|
||||
eprint(os.path.join(ScriptPath, ScriptName))
|
||||
eprint("Arguments: {}".format(sys.argv[1:]))
|
||||
eprint("Arguments: {}".format(args))
|
||||
else:
|
||||
sys.tracebacklimit = 0
|
||||
|
||||
args.beatName = args.beatName.lower()
|
||||
if not args.beatName.endswith('beat'):
|
||||
args.beatName = args.beatName + 'beat'
|
||||
|
||||
if (args.configFile is None):
|
||||
args.configFile = args.beatName + '.yml'
|
||||
|
||||
installerPlatform = platform.system()
|
||||
if installerPlatform == PLATFORM_LINUX:
|
||||
Beatbox = LinuxBeatbox(debug=args.debug, ymlFileSpec=args.configFile, beatName=args.beatName)
|
||||
elif installerPlatform == PLATFORM_MAC:
|
||||
Beatbox = MacBeatbox(debug=args.debug, ymlFileSpec=args.configFile, beatName=args.beatName)
|
||||
elif installerPlatform == PLATFORM_WINDOWS:
|
||||
Beatbox = WindowsBeatbox(debug=args.debug, ymlFileSpec=args.configFile, beatName=args.beatName)
|
||||
|
||||
success = False
|
||||
if hasattr(Beatbox, 'beat_run'): success = Beatbox.beat_run()
|
||||
|
||||
return success
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
89
Vagrant/resources/malcolm/scripts/beats/linux_vm_example/Vagrantfile
vendored
Normal file
89
Vagrant/resources/malcolm/scripts/beats/linux_vm_example/Vagrantfile
vendored
Normal file
@@ -0,0 +1,89 @@
|
||||
# -*- mode: ruby -*-
|
||||
# vi: set ft=ruby :
|
||||
|
||||
unless Vagrant.has_plugin?("vagrant-reload")
|
||||
raise 'vagrant-reload plugin is not installed!'
|
||||
end
|
||||
|
||||
# hack: https://github.com/hashicorp/vagrant/issues/8878#issuecomment-345112810
|
||||
class VagrantPlugins::ProviderVirtualBox::Action::Network
|
||||
def dhcp_server_matches_config?(dhcp_server, config)
|
||||
true
|
||||
end
|
||||
end
|
||||
|
||||
Vagrant.configure("2") do |config|
|
||||
|
||||
config.vm.box = "bento/ubuntu-20.04"
|
||||
|
||||
config.vm.network "private_network", type: "dhcp"
|
||||
|
||||
config.vm.synced_folder ".", "/vagrant", disabled: true
|
||||
|
||||
if Vagrant.has_plugin?("vagrant-vbguest")
|
||||
config.vbguest.auto_update = false
|
||||
end
|
||||
|
||||
config.vm.provider "virtualbox" do |vb|
|
||||
vb.customize ["modifyvm", :id, "--nictype1", "virtio" ]
|
||||
vb.customize ["modifyvm", :id, "--nicpromisc1", "allow-all"]
|
||||
vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
|
||||
vb.customize ["modifyvm", :id, "--natdnsproxy1", "on"]
|
||||
vb.customize ["modifyvm", :id, "--memory", 2048]
|
||||
vb.customize ["modifyvm", :id, "--cpus", 2]
|
||||
vb.customize ["modifyvm", :id, "--vram", 32]
|
||||
vb.customize ["modifyvm", :id, "--ioapic", "on"]
|
||||
vb.customize ["modifyvm", :id, "--nestedpaging", "on"]
|
||||
vb.customize ["modifyvm", :id, "--pae", "on"]
|
||||
vb.customize ["modifyvm", :id, "--hwvirtex", "on"]
|
||||
vb.customize ["modifyvm", :id, "--nested-hw-virt", "on"]
|
||||
end
|
||||
|
||||
config.vm.provision "shell", inline: <<-STEP1
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
export BEAT_VERSION=7.6.2
|
||||
apt-get update
|
||||
apt-get install -y auditd gnupg2 curl ca-certificates libcap2-bin libpcap0.8 python3-minimal python-is-python3
|
||||
curl -sSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -
|
||||
echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" >> /etc/apt/sources.list
|
||||
apt-get update
|
||||
for BEAT in auditbeat filebeat packetbeat metricbeat; do
|
||||
apt-get install -y $BEAT-oss=$BEAT_VERSION
|
||||
done;
|
||||
STEP1
|
||||
|
||||
config.vm.provision "file", source: "./audit.rules", destination: "/tmp/audit.rules"
|
||||
|
||||
config.vm.provision "file", source: "../beat_run.py", destination: "/tmp/beat_run.py"
|
||||
config.vm.provision "file", source: "../beat_config.py", destination: "/tmp/beat_config.py"
|
||||
config.vm.provision "file", source: "../beat_common.py", destination: "/tmp/beat_common.py"
|
||||
["auditbeat","filebeat","packetbeat","metricbeat"].to_enum.with_index(1).each do |beat, i|
|
||||
config.vm.provision "file", source: "./#{beat}.yml", destination: "/tmp/#{beat}.yml"
|
||||
end
|
||||
|
||||
config.vm.provision "shell", inline: <<-STEP2
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
|
||||
mv /tmp/beat*.py /usr/local/bin/
|
||||
chown root:root /usr/local/bin/beat*.py
|
||||
chmod 755 /usr/local/bin/beat_config.py /usr/local/bin/beat_run.py
|
||||
chmod 644 /usr/local/bin/beat_common.py
|
||||
|
||||
filebeat modules enable system
|
||||
|
||||
mv /tmp/audit.rules /etc/audit/rules.d/audit.rules
|
||||
find /etc/audit -type d -exec chmod 750 "{}" \\;
|
||||
find /etc/audit -type f -exec chmod 640 "{}" \\;
|
||||
|
||||
for BEAT in auditbeat filebeat packetbeat metricbeat; do
|
||||
mkdir -p /opt/$BEAT
|
||||
mv /tmp/$BEAT.yml /opt/$BEAT/
|
||||
chown -R root:root /opt/$BEAT
|
||||
chmod 700 /opt/$BEAT
|
||||
chmod 600 /opt/$BEAT/*
|
||||
done;
|
||||
STEP2
|
||||
|
||||
config.vm.provision :reload
|
||||
|
||||
end
|
||||
@@ -0,0 +1,146 @@
|
||||
## First rule - delete all
|
||||
-D
|
||||
|
||||
## Increase the buffers to survive stress events.
|
||||
## Make this bigger for busy systems
|
||||
-b 8192
|
||||
|
||||
## This determine how long to wait in burst of events
|
||||
--backlog_wait_time 0
|
||||
|
||||
## Set failure mode to syslog
|
||||
-f 1
|
||||
|
||||
# exclusions
|
||||
|
||||
-a always,exclude -F msgtype=AVC
|
||||
-a always,exclude -F msgtype=CRYPTO_KEY_USER
|
||||
-a always,exclude -F msgtype=CWD
|
||||
-a always,exclude -F msgtype=EOE
|
||||
|
||||
# commands
|
||||
|
||||
-a always,exit -F path=/bin/fusermount -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/bin/mount -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/bin/pmount -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/bin/pumount -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/bin/su -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-priv_change
|
||||
-a always,exit -F path=/bin/umount -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/etc/audisp/audisp-remote.conf -F perm=wa -k config_file_change
|
||||
-a always,exit -F path=/etc/audit/auditd.conf -F perm=wa -k config_file_change
|
||||
-a always,exit -F path=/etc/default/grub -F perm=wa -k config_file_change
|
||||
-a always,exit -F path=/etc/fstab -F perm=wa -k config_file_change
|
||||
-a always,exit -F path=/etc/hosts.deny -F perm=wa -k config_file_change
|
||||
-a always,exit -F path=/etc/login.defs -F perm=wa -k config_file_change
|
||||
-a always,exit -F path=/etc/profile -F perm=wa -k config_file_change
|
||||
-a always,exit -F path=/etc/sysctl.conf -F perm=wa -k config_file_change
|
||||
-a always,exit -F path=/sbin/apparmor_parser -F perm=x -F auid>=1000 -F auid!=4294967295 -k MAC-policy
|
||||
-a always,exit -F path=/sbin/pam_tally -F perm=wxa -F auid>=1000 -F auid!=4294967295 -k privileged-pam
|
||||
-a always,exit -F path=/sbin/pam_tally2 -F perm=wxa -F auid>=1000 -F auid!=4294967295 -k privileged-pam
|
||||
-a always,exit -F path=/sbin/unix_chkpwd -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-passwd
|
||||
-a always,exit -F path=/sbin/unix_update -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-unix-update
|
||||
-a always,exit -F path=/usr/bin/bsd-write -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/bin/chacl -F perm=x -F auid>=1000 -F auid!=4294967295 -k perm_chng
|
||||
-a always,exit -F path=/usr/bin/chage -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-passwd
|
||||
-a always,exit -F path=/usr/bin/chcon -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/bin/chfn -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/bin/chfn -F perm=x -F auid>=500 -F auid!=4294967295 -k privileged-priv_change
|
||||
-a always,exit -F path=/usr/bin/chsh -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-priv_change
|
||||
-a always,exit -F path=/usr/bin/crontab -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-cron
|
||||
-a always,exit -F path=/usr/bin/dotlock.mailutils -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/bin/expiry -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/bin/fusermount -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/bin/gpasswd -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-passwd
|
||||
-a always,exit -F path=/usr/bin/mount -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/bin/newgrp -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-priv_change
|
||||
-a always,exit -F path=/usr/bin/ntfs-3g -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/bin/passwd -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-passwd
|
||||
-a always,exit -F path=/usr/bin/pkexec -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/bin/pmount -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/bin/pumount -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/bin/setfacl -F perm=x -F auid>=1000 -F auid!=4294967295 -k perm_chng
|
||||
-a always,exit -F path=/usr/bin/ssh-agent -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-ssh
|
||||
-a always,exit -F path=/usr/bin/su -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-priv_change
|
||||
-a always,exit -F path=/usr/bin/sudo -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-priv_change
|
||||
-a always,exit -F path=/usr/bin/sudoedit -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-priv_change
|
||||
-a always,exit -F path=/usr/bin/umount -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/bin/wall -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/lib/dbus-1.0/dbus-daemon-launch-helper -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/lib/eject/dmcrypt-get-device -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/lib/openssh/ssh-keysign -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-ssh
|
||||
-a always,exit -F path=/usr/lib/policykit-1/polkit-agent-helper-1 -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/lib/x86_64-linux-gnu/utempter/utempter -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/lib/xorg/Xorg.wrap -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/sbin/addgroup -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/sbin/adduser -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/sbin/exim4 -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/sbin/groupadd -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/sbin/mount.cifs -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/sbin/netfilter-persistent -F perm=x -F auid>=1000 -F auid!=4294967295 -k nft_persistent_use
|
||||
-a always,exit -F path=/usr/sbin/nft -F perm=x -F auid>=1000 -F auid!=4294967295 -k nft_cmd_use
|
||||
-a always,exit -F path=/usr/sbin/pam_timestamp_check -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-pam
|
||||
-a always,exit -F path=/usr/sbin/postdrop -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-postfix
|
||||
-a always,exit -F path=/usr/sbin/postqueue -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-postfix
|
||||
-a always,exit -F path=/usr/sbin/semanage -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/sbin/setsebool -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/sbin/unix_chkpwd -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/sbin/useradd -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/sbin/userhelper -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
-a always,exit -F path=/usr/sbin/usermod -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged-usermod
|
||||
-a always,exit -F path=/usr/sbin/visudo -F perm=x -F auid>=1000 -F auid!=4294967295 -k privileged
|
||||
|
||||
# privileged files
|
||||
|
||||
-w /bin/kmod -p x -k modules
|
||||
-w /etc/apparmor.d/ -p wa -k MAC-policy
|
||||
-w /etc/apparmor/ -p wa -k MAC-policy
|
||||
-w /etc/group -p wa -k identity
|
||||
-w /etc/gshadow -p wa -k identity
|
||||
-w /etc/hosts -p wa -k system-locale
|
||||
-w /etc/issue -p wa -k system-locale
|
||||
-w /etc/issue.net -p wa -k system-locale
|
||||
-w /etc/localtime -p wa -k time-change
|
||||
-w /etc/network -p wa -k system-locale
|
||||
-w /etc/nftables.conf -p wa -k nft_config_file_change
|
||||
-w /etc/opasswd -p wa -k usergroup_modification
|
||||
-w /etc/passwd -p wa -k identity
|
||||
-w /etc/security/opasswd -p wa -k identity
|
||||
-w /etc/shadow -p wa -k identity
|
||||
-w /etc/sudoers -p wa -k sudoers
|
||||
-w /etc/sudoers.d/ -p wa -k sudoers
|
||||
-w /sbin/insmod -p x -k modules
|
||||
-w /sbin/modprobe -p x -k modules
|
||||
-w /sbin/rmmod -p x -k modules
|
||||
-w /var/log/btmp -p wa -k session
|
||||
-w /var/log/faillog -p wa -k logins
|
||||
-w /var/log/lastlog -p wa -k logins
|
||||
-w /var/log/sudo.log -p wa -k sudoaction
|
||||
-w /var/log/tallylog -p wa -k logins
|
||||
-w /var/log/wtmp -p wa -k session
|
||||
-w /var/run/faillock -p wa -k logins
|
||||
-w /var/run/utmp -p wa -k session
|
||||
|
||||
# syscalls
|
||||
|
||||
-a always,exit -F arch=b64 -S adjtimex -S settimeofday -k time-change
|
||||
-a always,exit -F arch=b64 -S chmod -S fchmod -S fchmodat -F auid>=1000 -F auid!=4294967295 -k perm_mod
|
||||
-a always,exit -F arch=b64 -S chown -S fchown -S fchownat -S lchown -F auid>=1000 -F auid!=4294967295 -k perm_mod
|
||||
-a always,exit -F arch=b64 -S clock_settime -k time-change
|
||||
-a always,exit -F arch=b64 -S creat -S open -S openat -S truncate -S ftruncate -F exit=-EACCES -F auid>=1000 -F auid!=4294967295 -k access
|
||||
-a always,exit -F arch=b64 -S open_by_handle_at -F exit=-EACCES -F auid>=1000 -F auid!=4294967295 -k access
|
||||
-a always,exit -F arch=b64 -S creat -S open -S openat -S truncate -S ftruncate -F exit=-EPERM -F auid>=1000 -F auid!=4294967295 -k access
|
||||
-a always,exit -F arch=b64 -S open_by_handle_at -F exit=-EPERM -F auid>=1000 -F auid!=4294967295 -k access
|
||||
-a always,exit -F arch=b64 -S execve -C gid!=egid -F key=execpriv
|
||||
-a always,exit -F arch=b64 -S execve -C uid!=euid -F key=execpriv
|
||||
-a always,exit -F arch=b64 -S init_module -S delete_module -S create_module -S finit_module -k modules
|
||||
-a always,exit -F arch=b64 -S mount -F auid>=1000 -F auid!=4294967295 -k mounts
|
||||
-a always,exit -F arch=b64 -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid>=1000 -F auid!=4294967295 -k perm_mod
|
||||
-a always,exit -F arch=b64 -S unlink -S unlinkat -S rename -S renameat -S rmdir -F auid>=1000 -F auid!=4294967295 -k delete
|
||||
-a always,exit -F dir=/etc/audit/rules.d/ -F perm=wa -k config_file_change
|
||||
-a always,exit -F dir=/etc/pam.d/ -F perm=wa -k config_file_change
|
||||
-a always,exit -F dir=/etc/profile.d/ -F perm=wa -k config_file_change
|
||||
-a always,exit -F dir=/etc/security/ -F perm=wa -k config_file_change
|
||||
-a exit,always -F arch=b64 -S sethostname -S setdomainname -k system-locale
|
||||
|
||||
# Make the configuration immutable -- reboot is required to change audit rules
|
||||
-e 2
|
||||
@@ -0,0 +1,154 @@
|
||||
# See https://www.elastic.co/guide/en/beats/auditbeat/current/auditbeat-reference-yml.html
|
||||
|
||||
# ==============================================================================
|
||||
|
||||
auditbeat.modules:
|
||||
|
||||
#------------------------------- auditd Module -------------------------------
|
||||
|
||||
- module: auditd
|
||||
socket_type: multicast
|
||||
resolve_ids: true
|
||||
failure_mode: log
|
||||
backlog_limit: 16384
|
||||
rate_limit: 0
|
||||
include_raw_message: false
|
||||
include_warnings: false
|
||||
backpressure_strategy: auto
|
||||
# audit_rule_files: [ '${path.config}/audit.rules.d/*.conf' ]
|
||||
# no rules specified, auditd will run and manage rules
|
||||
# see https://www.elastic.co/guide/en/beats/auditbeat/master/auditbeat-module-auditd.html
|
||||
|
||||
# don't forward some things that are always going to be happening to cut down on noise
|
||||
# and some other approved common stuff that would clutter the logs
|
||||
processors:
|
||||
- drop_event:
|
||||
when:
|
||||
and:
|
||||
- equals:
|
||||
auditd.message_type: 'syscall'
|
||||
- equals:
|
||||
auditd.summary.object.type: 'file'
|
||||
- or:
|
||||
- and:
|
||||
- or:
|
||||
- equals:
|
||||
auditd.data.syscall: 'open'
|
||||
- equals:
|
||||
auditd.data.syscall: 'openat'
|
||||
- regexp:
|
||||
auditd.summary.object.primary: '^/(proc/|etc/localtime|usr/lib/x86_64-linux-gnu/gconv/gconv-modules\.cache)'
|
||||
- or:
|
||||
- equals:
|
||||
auditd.summary.how: '/usr/share/auditbeat/bin/auditbeat'
|
||||
- and:
|
||||
- or:
|
||||
- equals:
|
||||
auditd.data.syscall: 'open'
|
||||
- equals:
|
||||
auditd.data.syscall: 'openat'
|
||||
- not:
|
||||
has_fields: ['auditd.summary.object.primary']
|
||||
- and:
|
||||
- equals:
|
||||
auditd.data.syscall: 'open'
|
||||
- regexp:
|
||||
auditd.summary.object.primary: '^/.+/__pycache__/$'
|
||||
- equals:
|
||||
auditd.summary.how: 'python3.8'
|
||||
|
||||
|
||||
- module: file_integrity
|
||||
paths:
|
||||
- /bin
|
||||
- /etc
|
||||
- /sbin
|
||||
- /usr/bin
|
||||
- /usr/local/bin
|
||||
- /usr/sbin
|
||||
recursive: true
|
||||
|
||||
# TODO: system module is apparently only available in the non-OSS basic license :-(
|
||||
|
||||
# - module: system
|
||||
# datasets:
|
||||
# - host # General host information, e.g. uptime, IPs
|
||||
# - user # User information
|
||||
# - login # Logins/logouts
|
||||
# - package # dpkg/rpm package manager logs
|
||||
# period: 1m
|
||||
# state.period: 12h
|
||||
# user.detect_password_changes: true
|
||||
|
||||
# - module: system
|
||||
# datasets:
|
||||
# - process # Started and stopped processes
|
||||
# - socket # Opened and closed sockets
|
||||
# period: 1s
|
||||
|
||||
# # drop noise
|
||||
# processors:
|
||||
# - drop_event:
|
||||
# when:
|
||||
# or:
|
||||
# - and:
|
||||
# - equals:
|
||||
# event.module: 'system'
|
||||
# - equals:
|
||||
# event.dataset: 'socket'
|
||||
# - equals:
|
||||
# destination.ip: '127.0.0.1'
|
||||
# - equals:
|
||||
# source.ip: '127.0.0.1'
|
||||
# - and:
|
||||
# - equals:
|
||||
# event.module: 'system'
|
||||
# - equals:
|
||||
# event.dataset: 'socket'
|
||||
# - equals:
|
||||
# destination.ip: "${BEAT_ES_HOST}"
|
||||
# - and:
|
||||
# - equals:
|
||||
# event.module: 'system'
|
||||
# - equals:
|
||||
# event.dataset: 'socket'
|
||||
# - equals:
|
||||
# destination.ip: "${BEAT_KIBANA_HOST}"
|
||||
# - and:
|
||||
# - equals:
|
||||
# event.module: 'system'
|
||||
# - equals:
|
||||
# event.dataset: 'process'
|
||||
# - or:
|
||||
# - equals:
|
||||
# process.executable: '/bin/sleep'
|
||||
# - equals:
|
||||
# process.executable: '/usr/bin/sort'
|
||||
# - equals:
|
||||
# process.executable: '/usr/bin/tail'
|
||||
# - equals:
|
||||
# process.executable: '/usr/bin/clear'
|
||||
# - equals:
|
||||
# process.executable: '/usr/bin/head'
|
||||
# - equals:
|
||||
# process.executable: '/bin/date'
|
||||
# - equals:
|
||||
# process.executable: '/bin/ls'
|
||||
# - equals:
|
||||
# process.executable: '/usr/bin/stat'
|
||||
# - equals:
|
||||
# process.executable: '/usr/bin/cut'
|
||||
# - equals:
|
||||
# process.executable: '/usr/bin/xargs'
|
||||
# - equals:
|
||||
# process.executable: '/usr/bin/tr'
|
||||
# - equals:
|
||||
# process.executable: '/bin/grep'
|
||||
# - equals:
|
||||
# process.executable: '/bin/sed'
|
||||
# - equals:
|
||||
# process.executable: '/bin/df'
|
||||
# - equals:
|
||||
# process.executable: '/usr/bin/du'
|
||||
# - equals:
|
||||
# process.executable: '/usr/bin/gawk'
|
||||
@@ -0,0 +1,14 @@
|
||||
# See https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-system.html
|
||||
# https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-reference-yml.html
|
||||
|
||||
# ==============================================================================
|
||||
|
||||
filebeat.modules:
|
||||
|
||||
#------------------------------- System Module -------------------------------
|
||||
|
||||
- module: system
|
||||
syslog:
|
||||
enabled: true
|
||||
auth:
|
||||
enabled: true
|
||||
@@ -0,0 +1,44 @@
|
||||
# See https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-reference-yml.html
|
||||
|
||||
# ==============================================================================
|
||||
|
||||
metricbeat.config.modules:
|
||||
path: ${path.config}/conf.d/*.yml
|
||||
reload.period: 10s
|
||||
reload.enabled: false
|
||||
|
||||
metricbeat.max_start_delay: 10s
|
||||
|
||||
metricbeat.modules:
|
||||
|
||||
#------------------------------- System Module -------------------------------
|
||||
|
||||
- module: system
|
||||
period: 30s
|
||||
metricsets:
|
||||
- cpu # CPU usage
|
||||
- load # CPU load averages
|
||||
- memory # Memory usage
|
||||
- network # Network IO
|
||||
- process # Per process metrics
|
||||
- process_summary # Process summary
|
||||
- uptime # System Uptime
|
||||
- diskio # Disk IO
|
||||
enabled: true
|
||||
processes: ['.*']
|
||||
process.include_top_n:
|
||||
enabled: true
|
||||
by_cpu: 10
|
||||
by_memory: 10
|
||||
|
||||
cpu.metrics: ["percentages"]
|
||||
core.metrics: ["percentages"]
|
||||
|
||||
- module: system
|
||||
period: 1m
|
||||
metricsets:
|
||||
- filesystem # File system usage for each mountpoint
|
||||
- fsstat # File system summary metrics
|
||||
processors:
|
||||
- drop_event.when.regexp:
|
||||
system.filesystem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib|boot)($|/)'
|
||||
@@ -0,0 +1,87 @@
|
||||
# See https://www.elastic.co/guide/en/beats/packetbeat/current/packetbeat-reference-yml.html
|
||||
|
||||
# ==============================================================================
|
||||
|
||||
#------------------------------- network device ------------------------------
|
||||
|
||||
packetbeat.interfaces.device: any
|
||||
packetbeat.interfaces.type: pcap
|
||||
packetbeat.interfaces.snaplen: 65535
|
||||
|
||||
#------------------------------- flows ---------------------------------------
|
||||
|
||||
packetbeat.flows:
|
||||
enabled: true
|
||||
timeout: 30s
|
||||
period: 10s
|
||||
|
||||
#------------------------------- transaction protocols -----------------------
|
||||
|
||||
packetbeat.protocols:
|
||||
- type: icmp
|
||||
enabled: true
|
||||
|
||||
- type: amqp
|
||||
enabled: true
|
||||
ports: [5672]
|
||||
|
||||
- type: cassandra
|
||||
enabled: true
|
||||
ports: [9042]
|
||||
|
||||
- type: dhcpv4
|
||||
enabled: true
|
||||
ports: [67, 68]
|
||||
|
||||
- type: dns
|
||||
enabled: true
|
||||
ports: [53]
|
||||
include_authorities: true
|
||||
include_additionals: true
|
||||
|
||||
- type: http
|
||||
enabled: true
|
||||
ports: [80, 8080, 8000, 5000, 8002]
|
||||
|
||||
- type: memcache
|
||||
enabled: true
|
||||
ports: [11211]
|
||||
|
||||
- type: mysql
|
||||
enabled: true
|
||||
ports: [3306,3307]
|
||||
|
||||
- type: pgsql
|
||||
enabled: true
|
||||
ports: [5432]
|
||||
|
||||
- type: redis
|
||||
enabled: true
|
||||
ports: [6379]
|
||||
|
||||
- type: thrift
|
||||
enabled: true
|
||||
ports: [9090]
|
||||
|
||||
- type: mongodb
|
||||
enabled: true
|
||||
ports: [27017]
|
||||
|
||||
- type: nfs
|
||||
enabled: true
|
||||
ports: [2049]
|
||||
|
||||
- type: tls
|
||||
enabled: true
|
||||
ports:
|
||||
- 443 # HTTPS
|
||||
- 993 # IMAPS
|
||||
- 995 # POP3S
|
||||
- 5223 # XMPP over SSL
|
||||
- 8883 # Secure MQTT
|
||||
- 9243 # Elasticsearch
|
||||
|
||||
#------------------------------- monitored processes -------------------------
|
||||
|
||||
packetbeat.procs.enabled: true
|
||||
packetbeat.ignore_outgoing: false
|
||||
@@ -0,0 +1,188 @@
|
||||
# configure a windows host to forward auditbeat and winlogbeat logs
|
||||
# to Malcolm (see https://github.com/idaholab/Malcolm/tree/master/scripts/beats)
|
||||
|
||||
$beatversion = "7.6.2"
|
||||
|
||||
################################################################################
|
||||
# Uninstall-Beat
|
||||
#
|
||||
# - Remove previous traces of this beat
|
||||
#
|
||||
function Uninstall-Beat {
|
||||
param( [string]$beat )
|
||||
|
||||
try {
|
||||
& "C:\\Program Files\\$beat\\uninstall-service-$beat.ps1"
|
||||
}
|
||||
catch {
|
||||
}
|
||||
remove-item "C:\\Program Files\\$beat" -Recurse -erroraction 'silentlycontinue';
|
||||
|
||||
}
|
||||
|
||||
################################################################################
|
||||
# Download-Beat
|
||||
#
|
||||
# - Download $beat-$beatversion-windows-x86_64.zip from artifacts.elastic.co
|
||||
# - Unzip to C:\Program Files\beat
|
||||
# - Download sample config for $beat from idaholab/Malcolm to C:\Program Files\beat
|
||||
#
|
||||
function Download-Beat {
|
||||
param( [string]$beat )
|
||||
|
||||
Invoke-WebRequest -Uri https://artifacts.elastic.co/downloads/beats/$beat/$beat-oss-$beatversion-windows-x86_64.zip -OutFile $beat-$beatversion-windows-x86_64.zip -UseBasicParsing
|
||||
Expand-Archive -LiteralPath $beat-$beatversion-windows-x86_64.zip -DestinationPath 'C:\\Program Files'
|
||||
Remove-Item $beat-$beatversion-windows-x86_64.zip
|
||||
Rename-Item "C:\\Program Files\\$beat-$beatversion-windows-x86_64" "C:\\Program Files\\$beat"
|
||||
((Get-Content -path "C:\\Program Files\\$beat\\install-service-$beat.ps1" -Raw) -replace 'ProgramData','Program Files') | Set-Content -Path "C:\\Program Files\\$beat\\install-service-$beat.ps1"
|
||||
((Get-Content -path "C:\\Program Files\\$beat\\install-service-$beat.ps1" -Raw) -replace ' -path',' --path') | Set-Content -Path "C:\\Program Files\\$beat\\install-service-$beat.ps1"
|
||||
|
||||
Invoke-WebRequest -UseBasicParsing -OutFile "C:\\Program Files\\$beat\\$beat.yml" -Uri https://raw.githubusercontent.com/idaholab/Malcolm/master/scripts/beats/windows_vm_example/$beat.yml
|
||||
(Get-Content "C:\\Program Files\\$beat\\$beat.yml") | Set-Content "C:\\Program Files\\$beat\\$beat.yml"
|
||||
}
|
||||
|
||||
################################################################################
|
||||
# Connectivity boilerplate to add to the sample .yml files downloaded from
|
||||
# idaholab/Malcolm
|
||||
#
|
||||
$beat_boilerplate = @'
|
||||
|
||||
#================================ General ======================================
|
||||
fields_under_root: true
|
||||
|
||||
#================================ Outputs ======================================
|
||||
|
||||
#-------------------------- Elasticsearch output -------------------------------
|
||||
output.elasticsearch:
|
||||
enabled: true
|
||||
hosts: ["${BEAT_ES_HOST}"]
|
||||
protocol: "${BEAT_ES_PROTOCOL}"
|
||||
username: "${BEAT_HTTP_USERNAME}"
|
||||
password: "${BEAT_HTTP_PASSWORD}"
|
||||
ssl.verification_mode: "${BEAT_ES_SSL_VERIFY}"
|
||||
|
||||
setup.template.enabled: true
|
||||
setup.template.overwrite: false
|
||||
setup.template.settings:
|
||||
index.number_of_shards: 1
|
||||
index.number_of_replicas: 0
|
||||
|
||||
#============================== Dashboards =====================================
|
||||
setup.dashboards.enabled: "${BEAT_KIBANA_DASHBOARDS_ENABLED}"
|
||||
setup.dashboards.directory: "${BEAT_KIBANA_DASHBOARDS_PATH}"
|
||||
|
||||
#============================== Kibana =====================================
|
||||
setup.kibana:
|
||||
host: "${BEAT_KIBANA_HOST}"
|
||||
protocol: "${BEAT_KIBANA_PROTOCOL}"
|
||||
username: "${BEAT_HTTP_USERNAME}"
|
||||
password: "${BEAT_HTTP_PASSWORD}"
|
||||
ssl.verification_mode: "${BEAT_KIBANA_SSL_VERIFY}"
|
||||
|
||||
#================================ Logging ======================================
|
||||
logging.metrics.enabled: false
|
||||
'@
|
||||
|
||||
################################################################################
|
||||
# Run-Beat-Command
|
||||
#
|
||||
# - Run C:\Program Files\$beat\$beat.exe with correct defaults for config paths
|
||||
# - specify beat, command array and (optionally) stdin string
|
||||
#
|
||||
function Run-Beat-Command {
|
||||
param( [string]$beat, [array]$command, [string]$stdin)
|
||||
|
||||
$exe = "C:\\Program Files\\$beat\\$beat.exe"
|
||||
$exe_config = '--path.home', "C:\\Program Files\\$beat", '--path.config', "C:\\Program Files\\$beat", '--path.data', "C:\\Program Files\\$beat", '--path.logs', "C:\\Program Files\\$beat\\logs", '-c', "C:\\Program Files\\$beat\\$beat.yml", '-E', "keystore.path='C:\\Program Files\\$beat\\$beat.keystore'"
|
||||
|
||||
if (!$stdin) {
|
||||
& $exe $exe_config $command
|
||||
} else {
|
||||
$stdin.Trim() | & $exe $exe_config $command
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
################################################################################
|
||||
# Configure config .yml and keystore for beat in "C:\\Program Files\\$beat"
|
||||
#
|
||||
function Configure-Beat {
|
||||
param( [string]$beat )
|
||||
|
||||
cd "C:\\Program Files\\$beat"
|
||||
|
||||
Run-Beat-Command $beat @("keystore","create","--force") $null
|
||||
|
||||
Add-Content -Path "C:\\Program Files\\$beat\\$beat.yml" -Value $beat_boilerplate
|
||||
|
||||
do {
|
||||
$es_host = Read-Host "Specify the Elasticsearch IP:port (e.g., 192.168.0.123:9200)"
|
||||
$es_host = $es_host.Trim()
|
||||
} while (!$es_host)
|
||||
|
||||
do {
|
||||
$kb_host = Read-Host "Specify the Kibana IP:port (e.g., 192.168.0.123:5601)"
|
||||
$kb_host = $kb_host.Trim()
|
||||
} while (!$kb_host)
|
||||
|
||||
do {
|
||||
$es_user = Read-Host "Specify the Elasticsearch/Kibana username"
|
||||
$es_user = $es_user.Trim()
|
||||
} while (!$es_user)
|
||||
|
||||
do {
|
||||
$es_pass = Read-Host "Specify the Elasticsearch/Kibana password" -AsSecureString
|
||||
$es_pass_confirm = Read-Host "Specify the Elasticsearch/Kibana password (again)" -AsSecureString
|
||||
$pwd1_text = [Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($es_pass))
|
||||
$pwd2_text = [Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($es_pass_confirm))
|
||||
} while ($pwd1_text -ne $pwd2_text)
|
||||
$es_pass = ([Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($es_pass))).Trim()
|
||||
|
||||
Run-Beat-Command $beat @("keystore","add","BEAT_ES_PROTOCOL","--stdin","--force") "https"
|
||||
Run-Beat-Command $beat @("keystore","add","BEAT_KIBANA_PROTOCOL","--stdin","--force") "https"
|
||||
Run-Beat-Command $beat @("keystore","add","BEAT_ES_SSL_VERIFY","--stdin","--force") "none"
|
||||
Run-Beat-Command $beat @("keystore","add","BEAT_KIBANA_SSL_VERIFY","--stdin","--force") "none"
|
||||
Run-Beat-Command $beat @("keystore","add","BEAT_KIBANA_DASHBOARDS_ENABLED","--stdin","--force") "true"
|
||||
Run-Beat-Command $beat @("keystore","add","BEAT_KIBANA_DASHBOARDS_PATH","--stdin","--force") "C:\\Program Files\\$beat\\kibana"
|
||||
Run-Beat-Command $beat @("keystore","add","BEAT_ES_HOST","--stdin","--force") "$es_host"
|
||||
Run-Beat-Command $beat @("keystore","add","BEAT_KIBANA_HOST","--stdin","--force") "$kb_host"
|
||||
Run-Beat-Command $beat @("keystore","add","BEAT_HTTP_USERNAME","--stdin","--force") "$es_user"
|
||||
Run-Beat-Command $beat @("keystore","add","BEAT_HTTP_PASSWORD","--stdin","--force") "$es_pass"
|
||||
|
||||
Run-Beat-Command $beat @("keystore","list") $null
|
||||
|
||||
$confirmation = Read-Host "Install $beat as a system service (y/n)"
|
||||
if ($confirmation -eq 'y') {
|
||||
& "C:\\Program Files\\$beat\\install-service-$beat.ps1"
|
||||
}
|
||||
}
|
||||
|
||||
################################################################################
|
||||
# Main
|
||||
#
|
||||
function Main {
|
||||
param( [array]$beats)
|
||||
$tempdir = New-TemporaryFile
|
||||
remove-item $tempdir;
|
||||
new-item -type directory -path $tempdir;
|
||||
cd $tempdir;
|
||||
|
||||
foreach ($beat in $beats) {
|
||||
cd $tempdir;
|
||||
|
||||
Uninstall-Beat $beat
|
||||
Download-Beat $beat
|
||||
Configure-Beat $beat
|
||||
}
|
||||
|
||||
cd $Env:Temp;
|
||||
remove-item $tempdir -Recurse;
|
||||
}
|
||||
|
||||
################################################################################
|
||||
#
|
||||
if ($args.count -eq 0) {
|
||||
Main @("auditbeat","winlogbeat")
|
||||
} else {
|
||||
Main $args
|
||||
}
|
||||
94
Vagrant/resources/malcolm/scripts/beats/windows_vm_example/Vagrantfile
vendored
Normal file
94
Vagrant/resources/malcolm/scripts/beats/windows_vm_example/Vagrantfile
vendored
Normal file
@@ -0,0 +1,94 @@
|
||||
# -*- mode: ruby -*-
|
||||
# vi: set ft=ruby :
|
||||
|
||||
unless Vagrant.has_plugin?("vagrant-reload")
|
||||
raise 'vagrant-reload plugin is not installed!'
|
||||
end
|
||||
|
||||
# hack: https://github.com/hashicorp/vagrant/issues/8878#issuecomment-345112810
|
||||
class VagrantPlugins::ProviderVirtualBox::Action::Network
|
||||
def dhcp_server_matches_config?(dhcp_server, config)
|
||||
true
|
||||
end
|
||||
end
|
||||
|
||||
Vagrant.configure("2") do |config|
|
||||
|
||||
config.vm.box = "StefanScherer/windows_10"
|
||||
|
||||
config.vm.network "private_network", type: "dhcp"
|
||||
|
||||
config.vm.synced_folder ".", "c:/vagrant_shared", disabled: true
|
||||
|
||||
if Vagrant.has_plugin?("vagrant-vbguest")
|
||||
config.vbguest.auto_update = false
|
||||
end
|
||||
|
||||
config.vm.communicator = "winrm"
|
||||
|
||||
config.winrm.username = "vagrant"
|
||||
config.winrm.password = "vagrant"
|
||||
|
||||
config.vm.guest = :windows
|
||||
config.windows.halt_timeout = 15
|
||||
|
||||
# port forward WinRM and RDP
|
||||
config.vm.network :forwarded_port, guest: 3389, host: 3389, id: "rdp", auto_correct: true
|
||||
config.vm.network :forwarded_port, guest: 5985, host: 5985, id: "winrm", auto_correct: true
|
||||
|
||||
config.vm.provider :virtualbox do |vb, override|
|
||||
vb.gui = true
|
||||
vb.customize ["modifyvm", :id, "--memory", 4096]
|
||||
vb.customize ["modifyvm", :id, "--cpus", 2]
|
||||
vb.customize ["modifyvm", :id, "--vram", 256]
|
||||
vb.customize ["modifyvm", :id, "--ioapic", "on"]
|
||||
vb.customize ["modifyvm", :id, "--nestedpaging", "on"]
|
||||
vb.customize ["modifyvm", :id, "--pae", "on"]
|
||||
vb.customize ["modifyvm", :id, "--hwvirtex", "on"]
|
||||
vb.customize ["modifyvm", :id, "--nested-hw-virt", "on"]
|
||||
vb.customize ["modifyvm", :id, "--graphicscontroller", "vboxsvga"]
|
||||
vb.customize ["modifyvm", :id, "--accelerate2dvideo", "on"]
|
||||
vb.customize ["modifyvm", :id, "--accelerate3d", "on"]
|
||||
vb.customize ["modifyvm", :id, "--clipboard", "bidirectional"]
|
||||
vb.customize ["setextradata", "global", "GUI/SuppressMessages", "all" ]
|
||||
vb.customize ["modifyvm", :id, "--usb", "on"]
|
||||
vb.customize ["modifyvm", :id, "--usbehci", "on"]
|
||||
vb.customize ["modifyvm", :id, "--audio", "pulse", "--audiocontroller", "hda"]
|
||||
end
|
||||
|
||||
config.vm.provision "shell", inline: <<-STEP1
|
||||
New-Item -Path 'HKLM:\\SOFTWARE\\Policies\\Microsoft\\Windows' -Name CloudContent | Out-Null
|
||||
New-ItemProperty -Path 'HKLM:\\SOFTWARE\\Policies\\Microsoft\\Windows\\CloudContent' -Name 'DisableWindowsConsumerFeatures' -PropertyType DWORD -Value '1' -Force | Out-Null
|
||||
New-Item -Path 'HKLM:\\SOFTWARE\\Policies\\Microsoft\\Windows\\' -Name 'Windows Search' | Out-Null
|
||||
New-ItemProperty -Path 'HKLM:\\SOFTWARE\\Policies\\Microsoft\\Windows\\Windows Search' -Name 'AllowCortana' -PropertyType DWORD -Value '0' -Force | Out-Null
|
||||
Set-ItemProperty 'HKLM:\\SOFTWARE\\Microsoft\\SQMClient\\Windows' CEIPEnable 0 | Out-Null
|
||||
schtasks /Change /TN 'Microsoft\\Windows\\Customer Experience Improvement Program\\UsbCeip' /Disable | Out-Null
|
||||
|
||||
taskkill /f /im OneDrive.exe
|
||||
C:/Windows/SysWOW64/OneDriveSetup.exe /uninstall
|
||||
STEP1
|
||||
config.vm.provision :reload
|
||||
|
||||
config.vm.provision "shell", inline: <<-STEP2
|
||||
Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
|
||||
choco install -y chocolateygui 7zip.install conemu firefox hackfont putty.install python sublimetext3 sysinternals
|
||||
|
||||
$beats = @("auditbeat","winlogbeat","packetbeat","metricbeat")
|
||||
foreach ($beat in $beats) {
|
||||
Invoke-WebRequest -Uri https://artifacts.elastic.co/downloads/beats/$beat/$beat-oss-7.6.2-windows-x86_64.zip -OutFile $beat-7.6.2-windows-x86_64.zip -UseBasicParsing
|
||||
Expand-Archive -LiteralPath $beat-7.6.2-windows-x86_64.zip -DestinationPath 'C:\\Program Files'
|
||||
Remove-Item $beat-7.6.2-windows-x86_64.zip
|
||||
Rename-Item "C:\\Program Files\\$beat-7.6.2-windows-x86_64" "C:\\Program Files\\$beat"
|
||||
((Get-Content -path "C:\\Program Files\\$beat\\install-service-$beat.ps1" -Raw) -replace 'ProgramData','Program Files') | Set-Content -Path "C:\\Program Files\\$beat\\install-service-$beat.ps1"
|
||||
((Get-Content -path "C:\\Program Files\\$beat\\install-service-$beat.ps1" -Raw) -replace ' -path',' --path') | Set-Content -Path "C:\\Program Files\\$beat\\install-service-$beat.ps1"
|
||||
}
|
||||
STEP2
|
||||
|
||||
["auditbeat","winlogbeat","packetbeat","metricbeat"].to_enum.with_index(1).each do |beat, i|
|
||||
config.vm.provision "file", source: "./#{beat}.yml", destination: "C:\\Program Files\\#{beat}\\#{beat}.yml"
|
||||
config.vm.provision "file", source: "../beat_run.py", destination: "C:\\Program Files\\#{beat}\\beat_run.py"
|
||||
config.vm.provision "file", source: "../beat_config.py", destination: "C:\\Program Files\\#{beat}\\beat_config.py"
|
||||
config.vm.provision "file", source: "../beat_common.py", destination: "C:\\Program Files\\#{beat}\\beat_common.py"
|
||||
end
|
||||
end
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
# See https://www.elastic.co/guide/en/beats/auditbeat/current/auditbeat-reference-yml.html
|
||||
|
||||
# Thanks to "The Windows File Auditing Logging Cheat Sheet" at
|
||||
# https://www.malwarearchaeology.com/cheat-sheets
|
||||
|
||||
# ==============================================================================
|
||||
|
||||
auditbeat.modules:
|
||||
|
||||
#------------------------------- file_integrity Module -----------------------
|
||||
|
||||
- module: file_integrity
|
||||
paths:
|
||||
- C:/Program Files
|
||||
- C:/Program Files/Internet Explorer
|
||||
- C:/Program Files/Common Files
|
||||
- C:/Program Files (x86)
|
||||
- C:/Program Files (x86) /Common Files
|
||||
- C:/ProgramData
|
||||
- C:/Windows
|
||||
- C:/Windows/System32
|
||||
- C:/Windows/System32/Drivers
|
||||
- C:/Windows/System32/Drivers/etc
|
||||
- C:/Windows/System32/Sysprep
|
||||
- C:/Windows/System32/wbem
|
||||
- C:/Windows/System32/WindowsPowerShell/v1.0
|
||||
- C:/Windows/Web
|
||||
- C:/Windows/SysWOW64
|
||||
- C:/Windows/SysWOW64/Drivers
|
||||
- C:/Windows/SysWOW64/wbem
|
||||
- C:/Windows/SysWOW64/WindowsPowerShell/v1.0
|
||||
recursive: false
|
||||
|
||||
- module: file_integrity
|
||||
paths:
|
||||
- C:/Boot
|
||||
- C:/Perflogs
|
||||
- C:/Users/All Users/Microsoft/Windows/Start Menu/Programs/Startup
|
||||
- C:/Users/Public
|
||||
# todo: wildcards handled?
|
||||
# - C:/Users/*/AppData/Local
|
||||
# - C:/Users/*/AppData/Local/Temp
|
||||
# - C:/Users/*/AppData/LocalLow
|
||||
# - C:/Users/*/AppData/Roaming
|
||||
- C:/Windows/Scripts
|
||||
- C:/Windows/System
|
||||
- C:/Windows/System32/GroupPolicy/Machine/Scripts/Startup
|
||||
- C:/Windows/System32/GroupPolicy/Machine/Scripts/Shutdown
|
||||
- C:/Windows/System32/GroupPolicy/User/Scripts/Logon
|
||||
- C:/Windows/System32/GroupPolicy/User/Scripts/Logoff
|
||||
- C:/Windows/System32/Repl
|
||||
recursive: true
|
||||
|
||||
# examples for exclusions if things are noisy
|
||||
# exclude_files:
|
||||
# - '(?i)\.blf$'
|
||||
# - '(?i)\.dat$'
|
||||
# - '(?i)\.lnk$'
|
||||
# - '(?i)\.log\w*$'
|
||||
# - '(?i)\.mum$'
|
||||
# - '(?i)\.regtrans-ms$'
|
||||
# - '(?i)\.swp$'
|
||||
# - '(?i)\.tmp$'
|
||||
# - '(?i)beat\.(lock|yml(\.new)?|db)$'
|
||||
# - '(?i)\\(assembly|CatRoot|CbsTemp|databases?|Deleted|diagnostics?|Log(File)?s?|Notifications?|Packages?|Prefetch|schemas?|servicing|Sessions?|SleepStudy|SoftwareDistribution|Tasks?|Temp|tracing|wbem|WinMetadata|WinSAT|WinSxS)\\?'
|
||||
# - '(?i)cache'
|
||||
|
||||
# TODO: system module is apparently only available in the non-OSS basic license :-(
|
||||
|
||||
# - module: system
|
||||
# datasets:
|
||||
# - host # General host information, e.g. uptime, IPs
|
||||
# period: 1m
|
||||
# state.period: 1h
|
||||
|
||||
# - module: system
|
||||
# datasets:
|
||||
# - process # Started and stopped processes
|
||||
# period: 1s
|
||||
@@ -0,0 +1,65 @@
|
||||
# See https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-reference-yml.html
|
||||
|
||||
# ==============================================================================
|
||||
|
||||
metricbeat.config.modules:
|
||||
path: ${path.config}/conf.d/*.yml
|
||||
reload.period: 10s
|
||||
reload.enabled: false
|
||||
|
||||
metricbeat.max_start_delay: 10s
|
||||
|
||||
metricbeat.modules:
|
||||
|
||||
#------------------------------- System Module -------------------------------
|
||||
|
||||
- module: system
|
||||
period: 30s
|
||||
metricsets:
|
||||
- cpu # CPU usage
|
||||
- memory # Memory usage
|
||||
- network # Network IO
|
||||
- process # Per process metrics
|
||||
- process_summary # Process summary
|
||||
- uptime # System Uptime
|
||||
- diskio # Disk IO
|
||||
enabled: true
|
||||
processes: ['.*']
|
||||
process.include_top_n:
|
||||
enabled: true
|
||||
by_cpu: 10
|
||||
by_memory: 10
|
||||
|
||||
cpu.metrics: ["percentages"]
|
||||
core.metrics: ["percentages"]
|
||||
|
||||
- module: system
|
||||
period: 1m
|
||||
metricsets:
|
||||
- filesystem # File system usage for each mountpoint
|
||||
- fsstat # File system summary metrics
|
||||
enabled: true
|
||||
|
||||
- module: windows
|
||||
metricsets: ["perfmon"]
|
||||
enabled: true
|
||||
period: 10s
|
||||
perfmon.ignore_non_existent_counters: false
|
||||
perfmon.group_measurements_by_instance: true
|
||||
perfmon.queries:
|
||||
- object: "Process"
|
||||
instance: ["svchost*", "conhost*"]
|
||||
counters:
|
||||
- name: "% Processor Time"
|
||||
field: time.processor.pct
|
||||
format: "float"
|
||||
perfmon.counters:
|
||||
- instance_label: processor.name
|
||||
instance_name: total
|
||||
measurement_label: processor.time.total.pct
|
||||
query: '\Processor Information(_Total)\% Processor Time'
|
||||
|
||||
- module: windows
|
||||
metricsets: ["service"]
|
||||
enabled: true
|
||||
period: 60s
|
||||
@@ -0,0 +1,90 @@
|
||||
# See https://www.elastic.co/guide/en/beats/packetbeat/current/packetbeat-reference-yml.html
|
||||
|
||||
# ==============================================================================
|
||||
|
||||
# packetbeat.exe requires Npcap (https://nmap.org/npcap/#download) to be installed
|
||||
|
||||
|
||||
#------------------------------- network device ------------------------------
|
||||
|
||||
packetbeat.interfaces.device: 0
|
||||
packetbeat.interfaces.type: pcap
|
||||
packetbeat.interfaces.snaplen: 65535
|
||||
|
||||
#------------------------------- flows ---------------------------------------
|
||||
|
||||
packetbeat.flows:
|
||||
enabled: true
|
||||
timeout: 30s
|
||||
period: 10s
|
||||
|
||||
#------------------------------- transaction protocols -----------------------
|
||||
|
||||
packetbeat.protocols:
|
||||
- type: icmp
|
||||
enabled: true
|
||||
|
||||
- type: amqp
|
||||
enabled: true
|
||||
ports: [5672]
|
||||
|
||||
- type: cassandra
|
||||
enabled: true
|
||||
ports: [9042]
|
||||
|
||||
- type: dhcpv4
|
||||
enabled: true
|
||||
ports: [67, 68]
|
||||
|
||||
- type: dns
|
||||
enabled: true
|
||||
ports: [53]
|
||||
include_authorities: true
|
||||
include_additionals: true
|
||||
|
||||
- type: http
|
||||
enabled: true
|
||||
ports: [80, 8080, 8000, 5000, 8002]
|
||||
|
||||
- type: memcache
|
||||
enabled: true
|
||||
ports: [11211]
|
||||
|
||||
- type: mysql
|
||||
enabled: true
|
||||
ports: [3306,3307]
|
||||
|
||||
- type: pgsql
|
||||
enabled: true
|
||||
ports: [5432]
|
||||
|
||||
- type: redis
|
||||
enabled: true
|
||||
ports: [6379]
|
||||
|
||||
- type: thrift
|
||||
enabled: true
|
||||
ports: [9090]
|
||||
|
||||
- type: mongodb
|
||||
enabled: true
|
||||
ports: [27017]
|
||||
|
||||
- type: nfs
|
||||
enabled: true
|
||||
ports: [2049]
|
||||
|
||||
- type: tls
|
||||
enabled: true
|
||||
ports:
|
||||
- 443 # HTTPS
|
||||
- 993 # IMAPS
|
||||
- 995 # POP3S
|
||||
- 5223 # XMPP over SSL
|
||||
- 8883 # Secure MQTT
|
||||
- 9243 # Elasticsearch
|
||||
|
||||
#------------------------------- monitored processes -------------------------
|
||||
|
||||
packetbeat.procs.enabled: true
|
||||
packetbeat.ignore_outgoing: false
|
||||
@@ -0,0 +1,43 @@
|
||||
# see https://www.elastic.co/guide/en/beats/winlogbeat/master/winlogbeat-reference-yml.html
|
||||
|
||||
# also see some of these excellent cheat sheets for Windows logging:
|
||||
# https://www.malwarearchaeology.com/cheat-sheets
|
||||
|
||||
# ==============================================================================
|
||||
|
||||
#------------------------------- event logs ----------------------------------
|
||||
|
||||
winlogbeat.event_logs:
|
||||
- name: AMSI/Operational
|
||||
- name: Application
|
||||
ignore_older: 72h
|
||||
- name: ForwardedEvents
|
||||
tags: ["forwarded"]
|
||||
- name: Internet Explorer
|
||||
- name: Microsoft-Windows-LSA/Operational
|
||||
- name: Microsoft-Windows-PowerShell/Admin
|
||||
- name: Microsoft-Windows-PowerShell/Operational
|
||||
- name: Microsoft-Windows-RemoteDesktopServices-RdpCoreTS/Operational
|
||||
- name: Microsoft-Windows-Windows Defender/Operational
|
||||
- name: Microsoft-Windows-Windows Defender/WHC
|
||||
- name: Microsoft-Windows-Windows Firewall With Advanced Security/Firewall
|
||||
- name: Microsoft-Windows-WMI-Activity/Operational
|
||||
- name: OpenSSH/Admin
|
||||
- name: OpenSSH/Operational
|
||||
|
||||
# TODO: the Security and Sysmon modules are apparently only available in the non-OSS basic license :-(
|
||||
|
||||
# - name: Security
|
||||
# processors:
|
||||
# - script:
|
||||
# lang: javascript
|
||||
# id: security
|
||||
# file: ${path.home}/module/security/config/winlogbeat-security.js
|
||||
# - name: System
|
||||
# - name: Windows PowerShell
|
||||
# - name: Microsoft-Windows-Sysmon/Operational
|
||||
# processors:
|
||||
# - script:
|
||||
# lang: javascript
|
||||
# id: sysmon
|
||||
# file: ${path.home}/module/sysmon/config/winlogbeat-sysmon.js
|
||||
Reference in New Issue
Block a user