Skip to content

Commit

Permalink
Force encoding for Windows 🤮
Browse files Browse the repository at this point in the history
Correct Typo in code
Update docs
  • Loading branch information
wagga40 committed Mar 29, 2024
1 parent 2133210 commit 08892e6
Show file tree
Hide file tree
Showing 4 changed files with 21 additions and 26 deletions.
27 changes: 11 additions & 16 deletions docs/Advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,32 +128,27 @@ Examples :

Some rules can be noisy or slow on specific datasets (check [here](https://github.com/wagga40/Zircolite/tree/master/rules/README.md)) so it is possible to skip them by using the `-R` or `--rulefilter` argument. This argument can be used multiple times.

The filter will apply on the rule title. Since there is a CRC32 in the rule title it is easier to use it. For example, to skip execution of the rule "Suspicious Eventlog Clear or Configuration Using Wevtutil - BFFA7F72" :
The filter will apply on the rule title. To avoid unexpected side-effect **comparison is case-sensitive**. For example, if you do not want to use all MSHTA related rules :

```shell
python3 zircolite.py --evtx logs/ --ruleset rules/rules_windows_sysmon.json -R BFFA7F72
```

You can also specify a string, to avoid unexpected side-effect **comparison is case-sensitive**. For example, if you do not want to use all MSHTA related rules and skip the execution of the rule "Suspicious Eventlog Clear or Configuration Using Wevtutil - BFFA7F72":

```shell
python3 zircolite.py --evtx logs/ --ruleset rules/rules_windows_sysmon.json \
-R BFFA7F72 -R MSHTA
python3 zircolite.py --evtx logs/ \
--ruleset rules/rules_windows_sysmon.json \
-R MSHTA
```
:information_source: As of version 2.2.0 of Zircolite, since the rulesets are directly generated from the official `sigmac` tool there is no more CRC32 in the rule title. Rule filtering is still available but you have to rely on other criteria.

### Limit the number of detected events

Sometimes, SIGMA rules can be very noisy (and generate a lot of false positives) but you still want to keep them in your rulesets. It is possible to filter rules that returns too mich detected events with the option `--limit <MAX_NUMBER>`. Please note that when using this option, the rules are not skipped the results are just ignored. But this is useful when forwarding events to Splunk.
Sometimes, SIGMA rules can be very noisy (and generate a lot of false positives) but you still want to keep them in your rulesets. It is possible to filter rules that returns too mich detected events with the option `--limit <MAX_NUMBER>`. **Please note that when using this option, the rules are not skipped the results are just ignored** but this is useful when forwarding events to Splunk.

## Forwarding detected events

Zircolite provide 2 ways to forward events to a collector :
Zircolite provide multiple ways to forward events to a collector :

- the HTTP forwarder : this is a very simple forwarder and pretty much a "toy" example and should be used when you have nothing else. An **example** server called is available in the [tools](../tools/zircolite_server/) directory
- the Splunk HEC Forwarder : it allows to forward all detected events to a Splunk instance using **HTTP Event Collector**.
- the HTTP forwarder : this is a very simple forwarder and pretty much a **"toy"** example and should be used when you have nothing else. An **example** server called is available in the [tools](../tools/zircolite_server/) directory
- the Splunk HEC Forwarder : it allows to forward all detected events to a Splunk instance using **HTTP Event Collector**
- the ELK ES client : it allows to forward all detected events to an ELK instance

For now, the forwarders are not asynchronous so it can slow Zircolite execution. There are two modes to forward the events :
There are two modes to forward the events :

- By default all events are forwarded after the detection process
- The argument `--stream` allow to forward events during the detection process
Expand Down Expand Up @@ -269,7 +264,7 @@ Then you just have to open `index.html` in your favorite browser and click on a
* After Python 3.8 install, you will need Nuitka : `pip3 install nuitka`
* In the root folder of Zircolite type : `python3 -m nuitka --onefile zircolite.py`

:warning: When packaging with PyInstaller some AV may not like your package.
:warning: When packaging with PyInstaller or Nuitka some AV may not like your package.

## Using With DFIR Orc

Expand Down
Binary file modified docs/Zircolite_manual.pdf
Binary file not shown.
10 changes: 5 additions & 5 deletions zircolite.py
Original file line number Diff line number Diff line change
Expand Up @@ -134,14 +134,14 @@ def __init__(self, logger=None, template=[], templateOutput=[], timeField=""):
self.templateOutput = templateOutput
self.timeField = timeField

def generateFromTemplate(self, templateFile, outpoutFilename, data):
def generateFromTemplate(self, templateFile, outputFilename, data):
"""Use Jinja2 to output data in a specific format"""
try:

tmpl = open(templateFile, "r", encoding="utf-8")
template = Template(tmpl.read())

with open(outpoutFilename, "a", encoding="utf-8") as tpl:
with open(outputFilename, "a", encoding="utf-8") as tpl:
tpl.write(template.render(data=data, timeField=self.timeField))
except Exception as e:
self.logger.error(
Expand Down Expand Up @@ -1391,7 +1391,7 @@ def run(self, file):
try:
data = ""
# We need to read the entire file to remove annoying newlines and fields with newlines (System.evtx Logs for example...)
with open(str(file), "r") as XMLFile:
with open(str(file), "r", encoding="utf-8") as XMLFile:
data = (
XMLFile.read()
.replace("\n", "")
Expand Down Expand Up @@ -1637,7 +1637,7 @@ def __init__(self, logger=None, config=None, listPipelineOnly=False):
def isYAML(self, filepath):
"""Test if the file is a YAML file"""
if filepath.suffix == ".yml" or filepath.suffix == ".yaml":
with open(filepath, "r") as file:
with open(filepath, "r", encoding="utf-8") as file:
content = file.read()
try:
yaml.safe_load(content)
Expand All @@ -1648,7 +1648,7 @@ def isYAML(self, filepath):
def isJSON(self, filepath):
"""Test if the file is a JSON file"""
if filepath.suffix == ".json":
with open(filepath, "r") as file:
with open(filepath, "r", encoding="utf-8") as file:
content = file.read()
try:
json.loads(content)
Expand Down
10 changes: 5 additions & 5 deletions zircolite_dev.py
Original file line number Diff line number Diff line change
Expand Up @@ -119,14 +119,14 @@ def __init__(self, logger=None, template=[], templateOutput=[], timeField=""):
self.templateOutput = templateOutput
self.timeField = timeField

def generateFromTemplate(self, templateFile, outpoutFilename, data):
def generateFromTemplate(self, templateFile, outputFilename, data):
""" Use Jinja2 to output data in a specific format """
try:

tmpl = open(templateFile, 'r', encoding='utf-8')
template = Template(tmpl.read())

with open(outpoutFilename, 'a', encoding='utf-8') as tpl:
with open(outputFilename, 'a', encoding='utf-8') as tpl:
tpl.write(template.render(data=data, timeField=self.timeField))
except Exception as e:
self.logger.error(f"{Fore.RED} [-] Template error, activate debug mode to check for errors{Fore.RESET}")
Expand Down Expand Up @@ -1022,7 +1022,7 @@ def run(self, file):
try:
data = ""
# We need to read the entire file to remove annoying newlines and fields with newlines (System.evtx Logs for example...)
with open(str(file), 'r') as XMLFile:
with open(str(file), 'r', encoding="utf-8") as XMLFile:
data = XMLFile.read().replace("\n","").replace("</Event>","</Event>\n").replace("<Event ","\n<Event ")
self.Logs2JSON(self.XMLLine2JSON, data, outputJSONFilename, isFile=False)
except Exception as e:
Expand Down Expand Up @@ -1201,7 +1201,7 @@ def __init__(self, logger=None, config=None, listPipelineOnly=False):
def isYAML(self, filepath):
""" Test if the file is a YAML file """
if (filepath.suffix == ".yml" or filepath.suffix == ".yaml"):
with open(filepath, 'r') as file:
with open(filepath, 'r', encoding="utf-8") as file:
content = file.read()
try:
yaml.safe_load(content)
Expand All @@ -1212,7 +1212,7 @@ def isYAML(self, filepath):
def isJSON(self, filepath):
""" Test if the file is a JSON file """
if (filepath.suffix == ".json"):
with open(filepath, 'r') as file:
with open(filepath, 'r', encoding="utf-8") as file:
content = file.read()
try:
json.loads(content)
Expand Down

0 comments on commit 08892e6

Please sign in to comment.