Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for new feeds to Dataplane parser #2102

Merged
3 commits merged into from
Sep 17, 2021
Merged

Conversation

monoidic
Copy link
Contributor

@monoidic monoidic commented Sep 7, 2021

This PR adds support for all of the currently available Dataplane feeds, i.e dns{rd{,any},version}, proto41, smtp{data,greet}, telnetlogin and vncrfb, in addition to the ones currently supported.
Some general code changes to the parser were also made.
Tests for the newly supported feeds were also added, along with a rework of the tests.

@codecov-commenter
Copy link

codecov-commenter commented Sep 7, 2021

Codecov Report

Merging #2102 (e14ec34) into develop (5328544) will decrease coverage by 0.00%.
The diff coverage is 90.56%.

@@             Coverage Diff             @@
##           develop    #2102      +/-   ##
===========================================
- Coverage    76.04%   76.04%   -0.01%     
===========================================
  Files          434      434              
  Lines        23218    23222       +4     
  Branches      3104     3108       +4     
===========================================
+ Hits         17657    17660       +3     
- Misses        4852     4853       +1     
  Partials       709      709              
Impacted Files Coverage Δ
intelmq/bin/intelmqctl.py 9.32% <0.00%> (-0.02%) ⬇️
intelmq/bots/collectors/http/collector_http.py 59.61% <ø> (ø)
...elmq/bots/collectors/http/collector_http_stream.py 40.00% <ø> (ø)
intelmq/bots/collectors/misp/collector.py 41.86% <ø> (ø)
...s/collectors/shadowserver/collector_reports_api.py 89.28% <ø> (ø)
intelmq/bots/experts/http/expert_status.py 80.00% <ø> (ø)
intelmq/bots/experts/jinja/expert.py 39.39% <ø> (ø)
intelmq/bots/experts/splunk_saved_search/expert.py 20.75% <ø> (ø)
intelmq/bots/experts/tuency/expert.py 86.00% <ø> (ø)
intelmq/bots/outputs/elasticsearch/output.py 78.04% <ø> (ø)
... and 14 more

@ghost ghost added this to the 3.1.0 milestone Sep 8, 2021
@ghost ghost added component: bots feature Indicates new feature requests or new features labels Sep 8, 2021
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would appreciate entries in intelmq/etc/feeds.yaml :)

Apart from that I found nothing to complain about ;)

@monoidic monoidic force-pushed the develop branch 2 times, most recently from b455cfa to fbc8276 Compare September 9, 2021 13:41
@monoidic
Copy link
Contributor Author

monoidic commented Sep 9, 2021

I added the documentation, plus a helper function called _convert_datetime to ensure extra.first_seen is converted to IntelMQ DateTime.

@monoidic monoidic requested a review from a user September 9, 2021 14:04
@ghost ghost merged commit f9ff4c1 into certtools:develop Sep 17, 2021
@ghost
Copy link

ghost commented Sep 17, 2021

Thank you!

@ghost ghost self-assigned this Sep 17, 2021
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: bots feature Indicates new feature requests or new features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants