Skip to content

2.0.2
Compare
Choose a tag to compare
released this 14 Oct 19:06
· 2726 commits to develop since this release
2.0.2
ff598e9

Install documentation:
https://github.com/certtools/intelmq/blob/2.0.2/docs/INSTALL.md
Upgrade documentation:
https://github.com/certtools/intelmq/blob/2.0.2/docs/UPGRADING.md

As always: read the NEWS file, upgrade according to the documentation
and have fun! If you get any errors, please report them here or in the
bug tracker.

Core

  • intelmq.lib.bot.CollectorBot: Support the deprecated parameter feed until version 2.2 as the documentation was not properly updated (#1445).
  • intelmq.lib.bot.Bot:
    • _dump_message: Wait for up to 60 seconds instead of 50 if the dump file is locked (the log message was said 60, but the code was for only 50).
  • intelmq.lib.upgrades.v202_fixes
    • Migration of deprecated parameter feed for Collectors.
    • Ripe expert parameter query_ripe_stat_ip was not correctly configured in v110_deprecations, now use query_ripe_stat_asn as default if it does not exist.
  • intelmq.lib.upgrades.v110_deprecations: Fix upgrade of ripe expert configuration.
  • intelmq.lib.bot_debugger:
    • Fix handling of empty messages generated by parser when user wanted to show the result by "--show-sent" flag.
    • Fix handling of sent messages for bots using the path_permissive paramter (#1453).
  • intelmq.lib.pipeline.Amqp:
    • use default SSL context for client purposes, fixes compatibility with python < 3.6 if TLS is used.
    • Reconnect once on sending messages if disconnect detected.

Bots

Collectors

  • intelmq.bots.collectors.api.collector_api:
    • Handle non-existing IO loop in shutdown.
    • Close socket on shutdown, fixes reloading.
    • Marked as non-threadable.
  • intelmq.bots.collectors.rt.collector_rt: Check for matching URLs if no attachment_regex is given.
  • intelmq.bots.collectors.stomp.collector_stomp: Handle disconnects by actively reconnecting.

Parsers

  • intelmq.bots.cymru.parser_cap_program: Fix parsing of the new $certname_$date.txt report format (#1443):
    • Support protocol ICMP.
    • Fix error message for unsupported protocols.
    • Support fields destination_port_numbers, port.
    • Support for all proxy types without ports.
    • Use Country Code of AS as source.geolocation.cc.
    • Support for 'scanner' and 'spam' categories.
    • Handle bogus lines with missing separator.
    • Fix bug preventing use of old format after using the new format.
    • Handle postfix (total_count:..) for destination port numbers.

Experts

  • intelmq.bots.experts.cymru_whois.expert: Add optional parameter overwrite, current behavior was True, default if not given is True now, will change to False in 3.0.0 (#1452, #1455).
  • intelmq.bots.experts.modify.expert: Add optional parameter overwrite, current behavior was True, default if not given is True now, will change to False in 3.0.0 (#1452, #1455).
  • intelmq.bots.experts.reverse_dns.expert: Add optional parameter overwrite, current behavior was True, default if not given is True now, will change to False in 3.0.0 (#1452, #1455).

Outputs

  • intelmq.bots.outputs.amqptopic.output: use default SSL context for client purposes, fixes compatibility with python < 3.6 if TLS is used.

Packaging

  • Rules:
    • Exclude intelmqsetup tool in packages
    • Include update-rfiprisk-data in packages

Tests

  • Tests for intelmq.lib.upgrades.v202_fixes.
  • Tests for intelmq.lib.upgrades.v110_deprecations.
  • Extended tests for intelmq.bots.parser.cymru.parser_cap_program.

Tools

  • intelmqctl:
    • More and more precise logging messages for botnet starting and restarting, enable and disable.
    • No error message for disabled bots on botnet reload.
    • Fix upgrade-conf is state file is empty or not existing.
    • Use arpgarse's store_true action for flags instead of store_const.
    • If the loading of the defaults configuration failed, a variable definition was missing and causing an exception (#1456).

Contrib

  • Check MK Statistics Cronjob:
    • Use statistics_* parameters.
    • Make file executable
    • Handle None values in *.temporary.* keys and treat them as 0.
  • systemd:
    • Add PIDFile parameter to service file.

Known issues

  • MongoDB authentication: compatibility on different MongoDB and pymongo versions (#1439)
  • ctl: shell colorizations are logged (#1436)
  • http stream collector: retry on regular connection problems? (#1435)
  • tests: capture logging with context manager (#1342)
  • Bots started with IntelMQ-Manager stop when the webserver is restarted. (#952)
  • n6 parser: mapping is modified within each run (#905)
  • reverse DNS: Only first record is used (#877)
  • Corrupt dump files when interrupted during writing (#870)
Assets 2
Loading