Provider wants to replace kafka_acl that do not need to be replaced

[parnigot]

While developing a custom module to manage one of our kafka cluster I've discovered a strange behaviour on the resource kafka_acl - the providers sometimes tries to replace ACLs that do not need to be replaced.

I've reproduced the issue with the following simple manifest:

terraform {
  required_providers {
    kafka = {
      source  = "Mongey/kafka"
      version = "0.5.1"
    }
  }
}

provider "kafka" {
  bootstrap_servers = ["localhost:9092"]
  ca_cert           = "pki/ca.pem"
  client_cert       = "pki/client_admin.pem"
  client_key        = "pki/client_admin-key.pem"
  tls_enabled       = true
}

resource "kafka_topic" "test_topic" {
  name               = "test_topic"
  replication_factor = 1
  partitions         = 2
}

#
# Two ACLs almost identical - only the host changes 
#
resource "kafka_acl" "acl_1" {
  resource_name       = kafka_topic.test_topic.name
  resource_type       = "Topic"
  acl_permission_type = "Allow"
  acl_host            = "192.168.1.1"
  acl_principal       = "User:CN=user"
  acl_operation       = "Read"
}

resource "kafka_acl" "acl_2" {
  resource_name       = kafka_topic.test_topic.name
  resource_type       = "Topic"
  acl_permission_type = "Allow"
  acl_host            = "192.168.1.2"
  acl_principal       = "User:CN=user"
  acl_operation       = "Read"
}

The first terraform apply works well and I get the following output from terraform show:

resource "kafka_acl" "acl_1" {
    acl_host                     = "192.168.1.1"
    acl_operation                = "Read"
    acl_permission_type          = "Allow"
    acl_principal                = "User:CN=user"
    id                           = "User:CN=user|192.168.1.1|Read|Allow|Topic|test_topic|Literal"
    resource_name                = "test_topic"
    resource_pattern_type_filter = "Literal"
    resource_type                = "Topic"
}

# kafka_acl.acl_2:
resource "kafka_acl" "acl_2" {
    acl_host                     = "192.168.1.2"
    acl_operation                = "Read"
    acl_permission_type          = "Allow"
    acl_principal                = "User:CN=user"
    id                           = "User:CN=user|192.168.1.2|Read|Allow|Topic|test_topic|Literal"
    resource_name                = "test_topic"
    resource_pattern_type_filter = "Literal"
    resource_type                = "Topic"
}

# kafka_topic.test_topic:
resource "kafka_topic" "test_topic" {
    id                 = "test_topic"
    name               = "test_topic"
    partitions         = 2
    replication_factor = 1
}

and in Kafka the ACLs are also correct:

# kafka-acls.sh --bootstrap-server localhost:9092 --command-config /etc/kafka/admin.properties --list
Current ACLs for resource `ResourcePattern(resourceType=TOPIC, name=test_topic, patternType=LITERAL)`: 
 	(principal=User:CN=user, host=192.168.1.2, operation=READ, permissionType=ALLOW)
	(principal=User:CN=user, host=192.168.1.1, operation=READ, permissionType=ALLOW)

The problems starts if I run a terraform plan/terraform apply - even without changing the manifest terraform returns:

Terraform will perform the following actions:

  # kafka_acl.acl_2 must be replaced
-/+ resource "kafka_acl" "acl_2" {
      ~ acl_host                     = "192.168.1.1" -> "192.168.1.2" # forces replacement
      ~ id                           = "User:CN=user|192.168.1.2|Read|Allow|Topic|test_topic|Literal" -> (known after apply)
        # (6 unchanged attributes hidden)
    }

Plan: 1 to add, 0 to change, 1 to destroy.

It looks it mixes the two ACLs while reading the data from Kafka. In the log I get the following messages:

[WARN]  Provider "registry.terraform.io/mongey/kafka" produced an unexpected new value for kafka_acl.acl_2 during refresh.
      - .acl_host: was cty.StringVal("192.168.1.2"), but now cty.StringVal("192.168.1.1")

[smauermann]

I am running into the a similar issue with the following two ACLs:

resource "kafka_acl" "acl_1" {
  acl_host                     = "*"
  acl_operation                = "Read"
  acl_permission_type          = "Allow"
  acl_principal                = "User:user_name"
  id                           = "User:user_name|*|Read|Allow|Topic|topic_name|Literal"
  resource_name                = "topic_name"
  resource_pattern_type_filter = "Prefixed"
  resource_type                = "Topic"
}

resource "kafka_acl" "acl_2" {
  acl_host                     = "*"
  acl_operation                = "Read"
  acl_permission_type          = "Allow"
  acl_principal                = "User:user_name"
  id                           = "User:user_name|*|Read|Allow|Group|group_name|Prefixed"
  resource_name                = "group_name"
  resource_pattern_type_filter = "Prefixed"
  resource_type                = "Group"
}

Here topic_name and group_name are identical. Which leads to a constant diff in plans:

  # kafka_acl.acl_1 must be replaced
-/+ resource "kafka_acl" "acl_1" {
      ~ id                           = "User:user_name|*|Read|Allow|Topic|topic_name|Literal" -> (known after apply)
      ~ resource_pattern_type_filter = "Prefixed" -> "Literal" # forces replacement
        # (6 unchanged attributes hidden)
    }

It appears the Kafka provider is mixing up acl_1 and acl_2.

[parnigot]

Fixed for me in master after the merge of #286