Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for JSON containing multiple events #2545

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

Added support for JSON containing multiple events #2545

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion intelmq/bots/parsers/json/parser.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,19 @@
from intelmq.lib.bot import ParserBot
from intelmq.lib.message import MessageFactory
from intelmq.lib.utils import base64_decode
import json


class JSONParserBot(ParserBot):
"""Parse IntelMQ-JSON data"""
splitlines = False
multiple_events = False

def process(self):
report = self.receive_message()
if self.splitlines:
if self.multiple_events:
lines = [json.dumps(event) for event in json.loads(base64_decode(report['raw']))]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Converting the data forth and back appears to be inefficient.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, could imagine this. Any tips to do this a proper way?

Currently this PR is running in our production and works just fine, but I agree on the double JSON conversion isn't the most efficient way to do this.

elif self.splitlines:
lines = base64_decode(report['raw']).splitlines()
else:
lines = [base64_decode(report['raw'])]
Expand Down
Loading