Skip to content

Latest commit

 

History

History
135 lines (99 loc) · 4.68 KB

README.md

File metadata and controls

135 lines (99 loc) · 4.68 KB
Raw

LZappy87 - toids_remove License Languages CodeFactor.io issues - toids_remove LastCommit

stars - toids_remove forks - toids_remove

MISP IDS Tag Remover

CREATED BY: LZappy87

LAST VERSION: 2.0

CREATED ON: 03/02/2022

UPDATED ON: 07/04/2022

FILES USED

  • toids-remove.py (this script)
  • keys.py (the configuration file)

TESTED WITH

  • MISP 2.4.152
  • PyMISP 2.4.152
  • Python 3.8
  • VirusTotal APIv3
  • AbuseIPDB APIv2
  • Greynoise APIv3

LIBRARIES USED

  • contextlib (contextmanager)
  • sys
  • os
  • shutil
  • time
  • argparse
  • prettytable (PrettyTable)
  • pymisp (ExpandedPyMISP)
  • urllib3
  • requests
  • base64
  • requests
  • re

DESCRIPTION

This script it's used to disable the attribute 'to_ids' on MISP events, features removal of the IDS tag on old events or based on VirusTotal scan results. An idea developed from this article: https://www.vanimpe.eu/2019/09/24/tracking-false-positives-and-disabling-to_ids-in-misp/

USAGE & DEMO

Help Menu

help

Remove Mode (--mode rem)

rem

Reputation IDS Removal Mode (--mode reputation)

reputation

Sights Only Mode (--mode reputation --sightsonly True)

sightsonly

Sights IDS Removal Mode (--mode reputation --sightsrem True)

sightsrem

Demo (v 1.6, Reputation Mode)

toids_removev1.6.mp4

CHANGELOG

v 2.0 (07/04/2022):

  • Implemented sightsonly for Reputation mode: populate only sightings without removing IDS tags;
  • Implemented sightsrem for Reputation mode: remove IDS tags based on sightings\false positive percentage;
  • Removal of redundant code;
  • Overhaul of script messages (for better understanding).

v 1.8 (06/04/2022):

  • Testing sightings mode;
  • Further code optimization.

v 1.7 (29/03/2022):

  • Added sightings based on reputation results.

v 1.6 (27/03/2022):

  • Minor changes to the code;
  • Added further VTotal tags to maltag.

v 1.5 (25/03/2022):

  • Added Greynoise API to reputation mode;
  • Added tabled results at the end of the script;
  • Removed unnecessary script header informations.

v 1.4 (21/02/2022):

  • Added AbuseIPDB to the vt mode;
  • vt mode now is reputation mode;
  • remold mode is now rem mode;
  • Added minimal error handling for AbuseIPDB API.

v 1.3 (08/02/2022):

  • Implemented VirusTotal Mode (vt);
  • Implemented Remove Old Mode (remold);
  • Included arguments to launch the script;
  • Moved some variables to keys.py for better configuration;
  • Included the 'published=True' search constraint (this should speed up the queries);
  • Removed mintime and maxtime from keys.py, substituted with arguments --mintime --maxtime;
  • Overall revamp of the code.

v 1.2 (05/02/2022):

  • Preparing for VirusTotal API implementation

v 1.1 (04/02/2022):

  • Removed old search string (it was not getting all the attributes);
  • Added filtering based on event tags through build_complex_query (thanks Giuseppe for the idea);
  • Various code revamp (not necessary linked to the aforemended changes);
  • Moved misp_client_cert to keys.py;
  • Added: misp_excluded_tags (for tag exclusion), mintime and maxtime (for time reference regarding the query on MISP) on keys.py;
  • Added basic error handling;
  • Added the creation of a default 'keys.py' if not present.

v 1.0 (03/02/2022):

First release

TODO:

  • Add more API's (for domain\url);
  • Better error handling.