Skip to content

Commit

Permalink
Correct ORJSON related bug
Browse files Browse the repository at this point in the history
  • Loading branch information
wagga40 committed Jun 7, 2022
1 parent 188601f commit 621cd55
Show file tree
Hide file tree
Showing 4 changed files with 14 additions and 13 deletions.
17 changes: 8 additions & 9 deletions docs/Advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ The tool has been created to be used on very big datasets and there are a lot of

Except when `evtx_dump` is used, Zircolite only use one core. So if you have a lot of EVTX files and their total size is big, it is recommanded that you use a script to launch multiple Zircolite instances. On Linux or MacOS The easiest way is to use **GNU Parallel**.

ℹ️ on MacOS, please use GNU find (`brew install find` will install `gfind`)
:information_source: on MacOS, please use GNU find (`brew install find` will install `gfind`)

- **"DFIR Case mode" : One directory per computer/endpoint**

Expand Down Expand Up @@ -73,7 +73,7 @@ To speed up the detection process, you may want to use Zircolite on files matchi
- `-s` or `--select` : select files partly matching the provided a string (case insensitive)
- `-a` or `--avoid` : exclude files partly matching the provided a string (case insensitive)

ℹ️ When using the two arguments, the "select" argument is always applied first and then the "avoid" argument is applied. So, it is possible to exclude files from included files but not the opposite.
:information_source: When using the two arguments, the "select" argument is always applied first and then the "avoid" argument is applied. So, it is possible to exclude files from included files but not the opposite.

- Only use EVTX files that contains "sysmon" in their names

Expand Down Expand Up @@ -146,7 +146,7 @@ You can also specify a string, to avoid unexpected side-effect **comparison is c
```shell
python3 zircolite.py --evtx logs/ --ruleset rules/rules_windows_sysmon.json -R BFFA7F72 -R MSHTA
```
ℹ️ As of version 2.2.0 of Zircolite, since the rulesets are directly generated from the official `sigmac` tool there is no more CRC32 in the rule title. Rule filtering is still available but you have to rely on other criteria.
:information_source: As of version 2.2.0 of Zircolite, since the rulesets are directly generated from the official `sigmac` tool there is no more CRC32 in the rule title. Rule filtering is still available but you have to rely on other criteria.

#### Limit the number of detected events

Expand Down Expand Up @@ -202,9 +202,9 @@ python3 zircolite.py --evtx /sample.evtx --ruleset rules/rules_windows_sysmon.j
--eslogin "yourlogin" --espass "yourpass"
```

ℹ️ the `--eslogin` and `--espass` arguments are optional.
:information_source: the `--eslogin` and `--espass` arguments are optional.

⚠️ **Elastic is not handling logs the way Splunk does. Since Zircolite is flattening the field names in the JSON output some fields, especially when working with EVTX files, can have different types between Channels, logsources etc. So when Elastic uses automatic field mapping, mapping errors may prevent events insertion into Elastic.**
:warning: **Elastic is not handling logs the way Splunk does. Since Zircolite is flattening the field names in the JSON output some fields, especially when working with EVTX files, can have different types between Channels, logsources etc. So when Elastic uses automatic field mapping, mapping errors may prevent events insertion into Elastic.**

#### No local logs

Expand All @@ -214,8 +214,7 @@ When you forward detected events to an server, sometimes you don't want any log

Zircolite is able to forward all events and not just the detected events to Splunk, ELK or a custom HTTP Server. you just to use the `--forwardall` argument. Please note that this ability forward events as JSON and not specific `Windows` sourcetype.

⚠️ **Elastic is not handling logs the way Splunk does. Since Zircolite is flattening the field names in the JSON output some fields, especially when working with EVTX files, can have different types between Channels, logsources etc. So when Elastic uses automatic field mapping, mapping errors may prevent events insertion into Elastic.**

:warning: **Elastic is not handling logs the way Splunk does. Since Zircolite is flattening the field names in the JSON output some fields, especially when working with EVTX files, can have different types between Channels, logsources etc. So when Elastic uses automatic field mapping, mapping errors may prevent events insertion into Elastic.**

---

Expand Down Expand Up @@ -330,7 +329,7 @@ Basically, if you want to integrate Zircolite with **DFIR Orc** :
</wolf>
```

ℹ️ Please note that if you add this configuration to an existing one, you only need to keep the part between `<!-- BEGIN ... -->` and `<!-- /END ... -->` blocks.
:information_source: Please note that if you add this configuration to an existing one, you only need to keep the part between `<!-- BEGIN ... -->` and `<!-- /END ... -->` blocks.

- Put your custom or default mapping file `zircolite_win10_nuitka.exe ` (the default one is in the Zircolite repository `config` directory) `rules_windows_generic.json` (the default one is in the Zircolite repository `rules` directory) in the the `config` directory.

Expand Down Expand Up @@ -369,7 +368,7 @@ Basically, if you want to integrate Zircolite with **DFIR Orc** :
</archive>
</toolembed>
```
ℹ️ Please note that if you add this configuration to an existing one, you only need to keep the part between `<!-- BEGIN ... -->` and `<!-- /END ... -->` blocks.
:information_source: Please note that if you add this configuration to an existing one, you only need to keep the part between `<!-- BEGIN ... -->` and `<!-- /END ... -->` blocks.

- Now you need to generate the **DFIR Orc** binary by executing `.\configure.ps1` at the root of the repository
- The final output will be in the `output` directory
Binary file modified docs/Zircolite_manual.pdf
Binary file not shown.
6 changes: 4 additions & 2 deletions zircolite.py
Original file line number Diff line number Diff line change
Expand Up @@ -1001,7 +1001,9 @@ def runUsingBindings(self, file):
encoding="utf-8",
) as f:
for record in parser.records_json():
f.write(f'{json.dumps(json.loads(record["data"]))}\n')
f.write(
f'{json.dumps(json.loads(record["data"])).decode("utf-8")}\n'
)
except Exception as e:
self.logger.error(f"{Fore.RED} [-] {e}")

Expand Down Expand Up @@ -1093,7 +1095,7 @@ def Logs2JSON(self, func, file, outfile):
with open(outfile, "w", encoding="UTF-8") as fp:
for element in result:
if element is not None:
fp.write(json.dumps(element) + "\n")
fp.write(json.dumps(element).decode("utf-8") + "\n")

def run(self, file):
"""
Expand Down
4 changes: 2 additions & 2 deletions zircolite_dev.py
Original file line number Diff line number Diff line change
Expand Up @@ -717,7 +717,7 @@ def runUsingBindings(self, file):
parser = PyEvtxParser(str(filepath))
with open(f"{self.tmpDir}/{str(filename)}-{self.randString()}.json", "w", encoding="utf-8") as f:
for record in parser.records_json():
f.write(f'{json.dumps(json.loads(record["data"]))}\n')
f.write(f'{json.dumps(json.loads(record["data"])).decode("utf-8")}\n')
except Exception as e:
self.logger.error(f"{Fore.RED} [-] {e}")

Expand Down Expand Up @@ -799,7 +799,7 @@ def Logs2JSON(self, func, file, outfile):
with open(outfile, "w", encoding="UTF-8") as fp:
for element in result:
if element is not None:
fp.write(json.dumps(element) + '\n')
fp.write(json.dumps(element).decode("utf-8") + '\n')

def run(self, file):
"""
Expand Down

0 comments on commit 621cd55

Please sign in to comment.