Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: add custom egress rules to docker-autoscaler security group #1222

Open
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

ikarlashov
Copy link

@ikarlashov ikarlashov commented Jan 6, 2025

Add custom egress rules to docker-autoscaler security group and remove condition to provision unused docker-machine security group.

Description

  1. By default, the module provisions a security group for Docker Autoscaler workers with egress rules that allow ALL traffic. Unlike ingress rules, egress rules are not customizable, which poses a significant security concern. This PR introduces the ability to customize egress rules for the Docker Autoscaler workers' security group by declaring a separate variable for docker-autoscaler egress rules.

  2. var.runner_worker_docker_autoscaler_asg becomes bulky and hard to read. Considering the complexity of the ingress rules structure, it makes sense to create a separate variable var.runner_worker_docker_autoscaler_ingress_rules for ingress rules. This way we will follow variable convention for existing security group rules variables, i.e. var.runner_worker_docker_machine_extra_egress_rules. In the result of the change: var.runner_worker_docker_autoscaler_asg.sg_ingresses is removed and its content should be moved to var.runner_worker_docker_autoscaler_ingress_rules.

  3. Additionally, PR removes the condition that provisions an unused security group intended solely for Docker Machine setup.

Migrations required

Yes

Move all docker-autoscaler ingress rules declaration from var.runner_worker_docker_autoscaler_asg.sg_ingresses to var.runner_worker_docker_autoscaler_ingress_rules.

Verification

runner_worker_docker_autoscaler_ingress_rules =  [
    {
      cidr_block      = "10.0.0.0/8"  # Example CIDR block for a private network in Amazon
      from_port        = 22
      protocol         = "tcp"
      to_port          = 22
      description      = "Allow SSH Ingress traffic for private network."
    }
]

runner_worker_docker_autoscaler_egress_rules = [
     {
      cidr_block    = "0.0.0.0/0"
      from_port     = 443
      protocol        = "tcp"
      to_port          = 443
      description   = "Allow HTTPS egress traffic."
    },
   {
    ipv6_cidr_block = "::/0"
    description         = "Allow HTTPS Egress everywhere"
    from_port           = 443
    protocol              = "tcp"
    to_port                =  443
   }
]

Copy link
Contributor

github-actions bot commented Jan 6, 2025

Hey @ikarlashov! 👋

Thank you for your contribution to the project. Please refer to the contribution rules for a quick overview of the process.

Make sure that this PR clearly explains:

  • the problem being solved
  • the best way a reviewer and you can test your changes

With submitting this PR you confirm that you hold the rights of the code added and agree that it will published under this LICENSE.

The following ChatOps commands are supported:

  • /help: notifies a maintainer to help you out

Simply add a comment with the command in the first line. If you need to pass more information, separate it with a blank line from the command.

This message was generated automatically. You are welcome to improve it.

@ikarlashov ikarlashov changed the title Add custom egress rules to docker-autoscaler security group feat: add custom egress rules to docker-autoscaler security group Jan 6, 2025
…Don't provision docker-machine security group when docker-autoscaler is used.

Signed-off-by: Yevgen Karlashov <[email protected]>
@ikarlashov ikarlashov force-pushed the custom_sg_docker_autoscaler branch from b2e2ef5 to bfa1b36 Compare January 6, 2025 19:47
@kayman-mk kayman-mk changed the title feat: add custom egress rules to docker-autoscaler security group feat!: add custom egress rules to docker-autoscaler security group Jan 10, 2025
Signed-off-by: Yevgen Karlashov <[email protected]>
Copy link
Collaborator

@kayman-mk kayman-mk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your work to improve this module. I noticed the major change, but I think we can go on as it is easy to handle for the users.

name = "${local.name_sg}-docker-autoscaler"
description = "Docker-autoscaler security group"

dynamic "egress" {
Copy link
Collaborator

@kayman-mk kayman-mk Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: can we move this to an aws_vpc_security_group_egress_rule resource? See aws_security_group

}
}

dynamic "ingress" {
Copy link
Collaborator

@kayman-mk kayman-mk Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: can we move this to an aws_vpc_security_group_ingress_rule resource? See aws_security_group

variables.tf Outdated
Comment on lines 765 to 777
default = [
{
cidr_blocks = ["0.0.0.0/0"]
ipv6_cidr_blocks = ["::/0"]
prefix_list_ids = null
from_port = 0
protocol = "-1"
security_groups = null
self = null
to_port = 0
description = "Allow all egress traffic for Docker-autoscaler runner workers."
}
]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: any specific reason for this default? As you already wrote, it might introduce a vulnerability.

suggestion: get rid of this default.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left it for lazy people who don't care about setting specific egresses and just want to have an ability to use the module right away. I.e. to pull docker images from Internet.
I can remove it, no problem.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe as a compromise solution we can set default to "Allow Egress to All for port 443 only"?

@@ -66,7 +66,7 @@ resource "aws_security_group_rule" "runner_ping_group" {

resource "aws_security_group" "docker_machine" {
# checkov:skip=CKV2_AWS_5:Security group is used within an template and assigned to the docker machines
count = contains(["docker+machine", "docker-autoscaler"], var.runner_worker.type) ? 1 : 0
count = var.runner_worker.type == "docker+machine" ? 1 : 0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: Good spot!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! :)

Signed-off-by: Yevgen Karlashov <[email protected]>
…r manager and Runner workers ASGs

Signed-off-by: Yevgen Karlashov <[email protected]>
…cker-autoscaler workers' ASG

Signed-off-by: Yevgen Karlashov <[email protected]>
Signed-off-by: Yevgen Karlashov <[email protected]>
Signed-off-by: Yevgen Karlashov <[email protected]>
Signed-off-by: Yevgen Karlashov <[email protected]>
@ikarlashov
Copy link
Author

It would be nice to allow traffic within runner-manager ASG and Docker-autoscaler workers ASG. Basically add this to security_groups.tf:

resource "aws_vpc_security_group_egress_rule" "runner_manager_to_docker_autoscaler_egress" {
  count = var.runner_worker.type == "docker-autoscaler" ? 1 : 0

  security_group_id            = aws_security_group.runner.id
  from_port                    = 0
  to_port                      = 0
  ip_protocol                  = "-1"
  description                  = "Allow ALL Egress traffic between Runner Manager and Docker-autoscaler workers security group"
  referenced_security_group_id = aws_security_group.docker_autoscaler[0].id
}

But if I add it, terraform doesn't detect any changes for runner's SG.

Signed-off-by: Yevgen Karlashov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants