From ced9dd2907ae8aef925d6a29c45646a8452d4d83 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Wed, 27 Mar 2024 12:10:32 +1100
Subject: [PATCH 01/54] Changelog.

---
 CHANGELOG/CHANGELOG-1.14.md | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/CHANGELOG/CHANGELOG-1.14.md b/CHANGELOG/CHANGELOG-1.14.md
index 7e981338f..0f9881bb7 100644
--- a/CHANGELOG/CHANGELOG-1.14.md
+++ b/CHANGELOG/CHANGELOG-1.14.md
@@ -21,6 +21,10 @@ When cutting a new release, update the `unreleased` heading to the tag being gen
 
 ## v1.14.0 - 2024-04-02
 
+## unreleased
+* [BUGFIX] [#1266](https://github.com/k8ssandra/k8ssandra-operator/issues/1217) MedusaConfigurations must now be namespace local to the K8ssandraCluster they are attached to. Additionally, ReplicatedSecrets should only pick up secrets from their local namespace to replicate.
+* [BUGFIX] [#1217](https://github.com/k8ssandra/k8ssandra-operator/issues/1217) Medusa storage secrets now use a ReplicatedSecret for synchronization, fixing an issue where changes to the secrets were not propagating. Additionally, fix a number of issues with local testing on ARM Macs.
+* [FEATURE] [#1260](https://github.com/k8ssandra/k8ssandra-operator/issues/1260) Update controller-gen to version 0.12.0.
 * [FEATURE] [#1242](https://github.com/k8ssandra/k8ssandra-operator/issues/1242) Allow for creation of replicated secrets with a prefix, so that we can distinguish between multiple secrets with the same origin but targeting different clusters.
 * [BUGFIX] [#1226](https://github.com/k8ssandra/k8ssandra-operator/issues/1226) Medusa purge cronjob should be created in the operator namespace
 * [BUGFIX] [#1141](https://github.com/k8ssandra/k8ssandra-operator/issues/1141) Use DC name override when naming secondary resources

From eff75008f9dead91eb664074342ec9504a62a929 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Wed, 3 Apr 2024 13:26:03 +1100
Subject: [PATCH 02/54] Fix changelog.

---
 CHANGELOG/CHANGELOG-1.14.md | 1 +
 1 file changed, 1 insertion(+)

diff --git a/CHANGELOG/CHANGELOG-1.14.md b/CHANGELOG/CHANGELOG-1.14.md
index 0f9881bb7..83dae6359 100644
--- a/CHANGELOG/CHANGELOG-1.14.md
+++ b/CHANGELOG/CHANGELOG-1.14.md
@@ -24,6 +24,7 @@ When cutting a new release, update the `unreleased` heading to the tag being gen
 ## unreleased
 * [BUGFIX] [#1266](https://github.com/k8ssandra/k8ssandra-operator/issues/1217) MedusaConfigurations must now be namespace local to the K8ssandraCluster they are attached to. Additionally, ReplicatedSecrets should only pick up secrets from their local namespace to replicate.
 * [BUGFIX] [#1217](https://github.com/k8ssandra/k8ssandra-operator/issues/1217) Medusa storage secrets now use a ReplicatedSecret for synchronization, fixing an issue where changes to the secrets were not propagating. Additionally, fix a number of issues with local testing on ARM Macs.
+* [BUGFIX] [#1253](https://github.com/k8ssandra/k8ssandra-operator/issues/1253) Medusa storage secrets are now labelled with a unique label.
 * [FEATURE] [#1260](https://github.com/k8ssandra/k8ssandra-operator/issues/1260) Update controller-gen to version 0.12.0.
 * [FEATURE] [#1242](https://github.com/k8ssandra/k8ssandra-operator/issues/1242) Allow for creation of replicated secrets with a prefix, so that we can distinguish between multiple secrets with the same origin but targeting different clusters.
 * [BUGFIX] [#1226](https://github.com/k8ssandra/k8ssandra-operator/issues/1226) Medusa purge cronjob should be created in the operator namespace

From 16cabdac8dfdb6e797275a94bd6b234df725ea82 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Wed, 3 Apr 2024 18:50:34 +1100
Subject: [PATCH 03/54] Micke's requested changes inc. unit test, changelog and
 conditional patching only.

---
 CHANGELOG/CHANGELOG-1.14.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/CHANGELOG/CHANGELOG-1.14.md b/CHANGELOG/CHANGELOG-1.14.md
index 83dae6359..bcc87a9ce 100644
--- a/CHANGELOG/CHANGELOG-1.14.md
+++ b/CHANGELOG/CHANGELOG-1.14.md
@@ -25,7 +25,7 @@ When cutting a new release, update the `unreleased` heading to the tag being gen
 * [BUGFIX] [#1266](https://github.com/k8ssandra/k8ssandra-operator/issues/1217) MedusaConfigurations must now be namespace local to the K8ssandraCluster they are attached to. Additionally, ReplicatedSecrets should only pick up secrets from their local namespace to replicate.
 * [BUGFIX] [#1217](https://github.com/k8ssandra/k8ssandra-operator/issues/1217) Medusa storage secrets now use a ReplicatedSecret for synchronization, fixing an issue where changes to the secrets were not propagating. Additionally, fix a number of issues with local testing on ARM Macs.
 * [BUGFIX] [#1253](https://github.com/k8ssandra/k8ssandra-operator/issues/1253) Medusa storage secrets are now labelled with a unique label.
-* [FEATURE] [#1260](https://github.com/k8ssandra/k8ssandra-operator/issues/1260) Update controller-gen to version 0.12.0.
+* [FEATURE] [#1260](https://github.com/k8ssandra/k8ssandra-operator/issues/1260) Update controller-gen to version 0.14.0.
 * [FEATURE] [#1242](https://github.com/k8ssandra/k8ssandra-operator/issues/1242) Allow for creation of replicated secrets with a prefix, so that we can distinguish between multiple secrets with the same origin but targeting different clusters.
 * [BUGFIX] [#1226](https://github.com/k8ssandra/k8ssandra-operator/issues/1226) Medusa purge cronjob should be created in the operator namespace
 * [BUGFIX] [#1141](https://github.com/k8ssandra/k8ssandra-operator/issues/1141) Use DC name override when naming secondary resources

From 29ef9f347085dd2cffe86dbade0e935013cc4d70 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Wed, 3 Apr 2024 12:01:10 +1100
Subject: [PATCH 04/54] Add deprecation warning for MedusaConfig non-local ns.

---
 controllers/k8ssandra/medusa_reconciler.go | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index a71629f43..31e0f76ed 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -246,8 +246,11 @@ func (r *K8ssandraClusterReconciler) mergeStorageProperties(
 		return result.Continue()
 	}
 	storageProperties := &medusaapi.MedusaConfiguration{}
+	// Deprecated: This code path can be removed at version 1.17, as MedusaConfigs should now always be namespace-local to the K8ssandraCluster referencing them.
 	configNamespace := utils.FirstNonEmptyString(medusaSpec.MedusaConfigurationRef.Namespace, namespace)
 	configKey := types.NamespacedName{Namespace: configNamespace, Name: medusaSpec.MedusaConfigurationRef.Name}
+	// End of block to be deprecated.
+
 	if err := remoteClient.Get(ctx, configKey, storageProperties); err != nil {
 		logger.Error(err, fmt.Sprintf("failed to get MedusaConfiguration %s", configKey))
 		return result.Error(err)

From a2ace04f51b7db00e76b2a5cdcf1ef8ffd0ffe3c Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Wed, 3 Apr 2024 12:02:41 +1100
Subject: [PATCH 05/54] Add webhook failure on resource creation for
 MedusaConfig non-local ns.

---
 .../v1alpha1/k8ssandracluster_webhook.go      | 40 ++++++++++++++-----
 1 file changed, 31 insertions(+), 9 deletions(-)

diff --git a/apis/k8ssandra/v1alpha1/k8ssandracluster_webhook.go b/apis/k8ssandra/v1alpha1/k8ssandracluster_webhook.go
index a0b0c2321..91b55d735 100644
--- a/apis/k8ssandra/v1alpha1/k8ssandracluster_webhook.go
+++ b/apis/k8ssandra/v1alpha1/k8ssandracluster_webhook.go
@@ -18,9 +18,10 @@ package v1alpha1
 
 import (
 	"fmt"
-	"k8s.io/apimachinery/pkg/util/validation"
 	"strings"
 
+	"k8s.io/apimachinery/pkg/util/validation"
+
 	"github.com/Masterminds/semver/v3"
 	"github.com/k8ssandra/k8ssandra-operator/pkg/clientcache"
 	"github.com/pkg/errors"
@@ -98,14 +99,8 @@ func (r *K8ssandraCluster) validateK8ssandraCluster() error {
 		}
 	}
 
-	// Verify the Medusa storage prefix is explicitly set
-	// only relevant if Medusa is enabled and the MedusaConfiguration object is referenced
-	if r.Spec.Medusa != nil {
-		if r.Spec.Medusa.MedusaConfigurationRef.Name != "" {
-			if r.Spec.Medusa.StorageProperties.Prefix == "" {
-				return ErrNoStoragePrefix
-			}
-		}
+	if err := r.ValidateMedusa(); err != nil {
+		return err
 	}
 
 	if err := r.validateStatefulsetNameSize(); err != nil {
@@ -201,3 +196,30 @@ func (r *K8ssandraCluster) ValidateDelete() error {
 	webhookLog.Info("validate K8ssandraCluster delete", "name", r.Name)
 	return nil
 }
+
+func (r *K8ssandraCluster) ValidateMedusa() error {
+	if r.Spec.Medusa == nil {
+		return nil
+	}
+
+	// Verify the Medusa storage prefix is explicitly set
+	// only relevant if Medusa is enabled and the MedusaConfiguration object is referenced
+	if r.Spec.Medusa.MedusaConfigurationRef.Name != "" {
+		if r.Spec.Medusa.StorageProperties.Prefix == "" {
+			return ErrNoStoragePrefix
+		}
+		// Verify that any referenced MedusaConfig is NS-local
+		if r.Spec.Medusa.MedusaConfigurationRef.Namespace != "" {
+			return errors.New("Medusa config must be namespace local")
+		}
+		if r.Spec.Medusa.MedusaConfigurationRef.APIVersion != "" ||
+			r.Spec.Medusa.MedusaConfigurationRef.Kind != "" ||
+			r.Spec.Medusa.MedusaConfigurationRef.FieldPath != "" ||
+			r.Spec.Medusa.MedusaConfigurationRef.ResourceVersion != "" ||
+			r.Spec.Medusa.MedusaConfigurationRef.UID != "" {
+			return errors.New("Medusa config invalid, invalid field used")
+		}
+	}
+
+	return nil
+}

From 98c371183e92fa4a4ab7045a68a0a879a8a1fc7a Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Wed, 3 Apr 2024 12:12:39 +1100
Subject: [PATCH 06/54] Unit test for this functionality.

---
 .../v1alpha1/k8ssandracluster_webhook_test.go          | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/apis/k8ssandra/v1alpha1/k8ssandracluster_webhook_test.go b/apis/k8ssandra/v1alpha1/k8ssandracluster_webhook_test.go
index aed36c9ed..eded2cb1b 100644
--- a/apis/k8ssandra/v1alpha1/k8ssandracluster_webhook_test.go
+++ b/apis/k8ssandra/v1alpha1/k8ssandracluster_webhook_test.go
@@ -148,6 +148,7 @@ func TestWebhook(t *testing.T) {
 	t.Run("StsNameTooLong", testStsNameTooLong)
 	t.Run("MedusaPrefixMissing", testMedusaPrefixMissing)
 	t.Run("InvalidDcName", testInvalidDcName)
+	t.Run("MedusaConfigNonLocalNamespace", testMedusaNonLocalNamespace)
 }
 
 func testContextValidation(t *testing.T) {
@@ -465,3 +466,12 @@ func testInvalidDcName(t *testing.T) {
 	required.Error(err)
 	required.Contains(err.Error(), "invalid DC name")
 }
+
+func testMedusaNonLocalNamespace(t *testing.T) {
+	required := require.New(t)
+	badCluster := createMinimalClusterObj("medusaconfig-nonlocal", "ns")
+	badCluster.Spec.Medusa.MedusaConfigurationRef.Namespace = "nonlocal-ns"
+	err := badCluster.validateK8ssandraCluster()
+	required.Error(err)
+	required.Contains(err.Error(), "Medusa config must be namespace local")
+}

From 6689fd4b4b0d7da8f2c6d7813684b877ce5deb0f Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Wed, 3 Apr 2024 12:16:42 +1100
Subject: [PATCH 07/54] Release notes, changelog.

---
 CHANGELOG/CHANGELOG-1.14.md |  7 ++-----
 CHANGELOG/RELEASE-NOTES.md  | 11 +++++++++++
 2 files changed, 13 insertions(+), 5 deletions(-)

diff --git a/CHANGELOG/CHANGELOG-1.14.md b/CHANGELOG/CHANGELOG-1.14.md
index bcc87a9ce..37c98b386 100644
--- a/CHANGELOG/CHANGELOG-1.14.md
+++ b/CHANGELOG/CHANGELOG-1.14.md
@@ -18,14 +18,11 @@ When cutting a new release, update the `unreleased` heading to the tag being gen
 * [FEATURE] [#1280](https://github.com/k8ssandra/k8ssandra-operator/issues/1280) Env variables DEFAULT_REGISTRY and IMAGE_PULL_SECRETS allow overriding the default imagePullSecrets as well as the default registry to use when deploying medusa/reaper/stargate.
 * [BUGFIX] [#1253](https://github.com/k8ssandra/k8ssandra-operator/issues/1253) Medusa storage secrets are now labelled with a unique label.
 * [BUGFIX] [#1240](https://github.com/k8ssandra/k8ssandra-operator/issues/1240) The PullSecretRef for medusa is ignored in the standalone deployment of medusa
-
-## v1.14.0 - 2024-04-02
-
-## unreleased
-* [BUGFIX] [#1266](https://github.com/k8ssandra/k8ssandra-operator/issues/1217) MedusaConfigurations must now be namespace local to the K8ssandraCluster they are attached to. Additionally, ReplicatedSecrets should only pick up secrets from their local namespace to replicate.
+* [BUGFIX] [#1266](https://github.com/k8ssandra/k8ssandra-operator/issues/1266) MedusaConfigurations must now be namespace local to the K8ssandraCluster they are attached to. Additionally, ReplicatedSecrets should only pick up secrets from their local namespace to replicate.
 * [BUGFIX] [#1217](https://github.com/k8ssandra/k8ssandra-operator/issues/1217) Medusa storage secrets now use a ReplicatedSecret for synchronization, fixing an issue where changes to the secrets were not propagating. Additionally, fix a number of issues with local testing on ARM Macs.
 * [BUGFIX] [#1253](https://github.com/k8ssandra/k8ssandra-operator/issues/1253) Medusa storage secrets are now labelled with a unique label.
 * [FEATURE] [#1260](https://github.com/k8ssandra/k8ssandra-operator/issues/1260) Update controller-gen to version 0.14.0.
+* [CHANGE] [#1266](https://github.com/k8ssandra/k8ssandra-operator/issues/1266) Deprecate the usage of non-namespace-local MedusaConfigurations within a K8ssandraCluster, and throw an error from the webhook when users attempt to use one.
 * [FEATURE] [#1242](https://github.com/k8ssandra/k8ssandra-operator/issues/1242) Allow for creation of replicated secrets with a prefix, so that we can distinguish between multiple secrets with the same origin but targeting different clusters.
 * [BUGFIX] [#1226](https://github.com/k8ssandra/k8ssandra-operator/issues/1226) Medusa purge cronjob should be created in the operator namespace
 * [BUGFIX] [#1141](https://github.com/k8ssandra/k8ssandra-operator/issues/1141) Use DC name override when naming secondary resources
diff --git a/CHANGELOG/RELEASE-NOTES.md b/CHANGELOG/RELEASE-NOTES.md
index 39f5f310b..4d234c9e8 100644
--- a/CHANGELOG/RELEASE-NOTES.md
+++ b/CHANGELOG/RELEASE-NOTES.md
@@ -1,5 +1,16 @@
 # k8ssandra-operator - Release Notes
 
+## v1.14.0
+
+### Deprecation of non-namespace-local MedusaConfigRef
+
+The previous version introduced functionality whereby a K8ssandraCluster could reference a MedusaConfiguration (via MedusaConfigRef) in a remote namespace within the same k8s cluster. This functionality is deprecated. Existing clusters will continue to reconcile, but new clusters (or updates to existing clusters) will be rejected at the webhook.
+
+To update an existing cluster, or create a new one, ensure that the `namespace` field is left unset in the `medusaConfigRef`, and ensure that the MedusaConfiguration you are referencing exists within the K8ssandraCluster's local namespace. 
+
+If this functionality is critical to your use case, please raise an issue on Github and describe why it is important to you.
+
+
 ## v1.12.0
 
 It is now possible to disable Reaper front end authentication by adding either `spec.reaper.uiUserSecretRef: {}` or `spec.reaper.uiUserSecretRef: ""`. 

From 8763987ca9fefa6bc032c5da2aed6e3196888e2d Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Wed, 3 Apr 2024 15:52:34 +1100
Subject: [PATCH 08/54] Scaffold out the planned changes and ensure deprecation
 comments are clear.

---
 controllers/k8ssandra/medusa_reconciler.go | 95 ++++++++++++----------
 1 file changed, 54 insertions(+), 41 deletions(-)

diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index 31e0f76ed..afffe43a2 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -196,7 +196,7 @@ func (r *K8ssandraClusterReconciler) reconcileMedusaSecrets(
 			return result.Error(err)
 		}
 
-		if err := r.reconcileBucketSecrets(ctx, r.ClientCache.GetLocalClient(), kc, logger); err != nil {
+		if err := r.reconcileRemoteBucketSecretsDeprecated(ctx, r.ClientCache.GetLocalClient(), kc, logger); err != nil {
 			logger.Error(err, "Failed to reconcile Medusa bucket secrets")
 			return result.Error(err)
 		}
@@ -271,63 +271,76 @@ func (r *K8ssandraClusterReconciler) mergeStorageProperties(
 	// we make a copy of that secret for each cluster/dc, and then point to it with a corev1.LocalObjectReference
 	// when we do the copy, we name the secret as <cluster-name>-<original-secret-name>
 	// here we need to update the reference to point to that copied secret
-	mergedProperties.StorageSecretRef.Name = fmt.Sprintf("%s-%s", desiredKc.Name, mergedProperties.StorageSecretRef.Name)
+	if desiredKc.Namespace != medusaSpec.MedusaConfigurationRef.Name {
+		// Deprecated: when we remove the ability to reference a non-namespace-local MedusaConfig in v 1.17,
+		// this if statement should be eliminated.
+		mergedProperties.StorageSecretRef.Name = fmt.Sprintf("%s-%s", desiredKc.Name, mergedProperties.StorageSecretRef.Name)
+	} else {
+		// this will be the only code branch after the deprecation ends.
+		mergedProperties.StorageSecretRef.Name = mergedProperties.StorageSecretRef.Name // Yes, I know this is assigned to itself.
+	}
 
 	// copy the merged properties back into the cluster
 	mergedProperties.DeepCopyInto(&desiredKc.Spec.Medusa.StorageProperties)
 	return result.Continue()
 }
 
-func (r *K8ssandraClusterReconciler) reconcileBucketSecrets(
+// Deprecated: This code path can be removed at version 1.17, as MedusaConfigs should now always be namespace-local to the K8ssandraCluster referencing them. At that point, we no longer
+// need this code, because it is mainly concerned with copying the bucket secrets into the K8ssandraCluster's namespace.
+func (r *K8ssandraClusterReconciler) reconcileRemoteBucketSecretsDeprecated(
 	ctx context.Context,
 	c client.Client,
 	kc *api.K8ssandraCluster,
 	logger logr.Logger,
 ) error {
+	if kc.Spec.Medusa.MedusaConfigurationRef.Namespace != kc.Namespace {
+		// This is the deprecated code path. Moving forward we will use a replicated secret with a prefix, but we will remove this code path after v1.17.
+		logger.Info("Reconciling Medusa bucket secrets")
+		medusaSpec := kc.Spec.Medusa
 
-	logger.Info("Reconciling Medusa bucket secrets")
-	medusaSpec := kc.Spec.Medusa
-
-	// there is nothing to reconcile if we're not using Medusa configuration reference
-	if medusaSpec == nil || medusaSpec.MedusaConfigurationRef.Name == "" {
-		logger.Info("MedusaConfigurationRef is not set, skipping bucket secret reconciliation")
-		return nil
-	}
-
-	// fetch the referenced configuration
-	medusaConfigName := medusaSpec.MedusaConfigurationRef.Name
-	medusaConfigNamespace := utils.FirstNonEmptyString(medusaSpec.MedusaConfigurationRef.Namespace, kc.Namespace)
-	medusaConfigKey := types.NamespacedName{Namespace: medusaConfigNamespace, Name: medusaConfigName}
-	medusaConfig := &medusaapi.MedusaConfiguration{}
-	if err := c.Get(ctx, medusaConfigKey, medusaConfig); err != nil {
-		logger.Error(err, fmt.Sprintf("could not get MedusaConfiguration %s/%s", medusaConfigNamespace, medusaConfigName))
-		return err
-	}
+		// there is nothing to reconcile if we're not using Medusa configuration reference
+		if medusaSpec == nil || medusaSpec.MedusaConfigurationRef.Name == "" {
+			logger.Info("MedusaConfigurationRef is not set, skipping bucket secret reconciliation")
+			return nil
+		}
 
-	// fetch the referenced medusa configuration's bucket secret
-	bucketSecretName := medusaConfig.Spec.StorageProperties.StorageSecretRef.Name
-	bucketSecret := &corev1.Secret{}
-	bucketSecretKey := types.NamespacedName{Namespace: medusaConfigNamespace, Name: bucketSecretName}
-	if err := c.Get(ctx, bucketSecretKey, bucketSecret); err != nil {
-		logger.Error(err, "could not get bucket Secret")
-		return err
-	}
+		// fetch the referenced configuration
+		medusaConfigName := medusaSpec.MedusaConfigurationRef.Name
+		medusaConfigNamespace := utils.FirstNonEmptyString(medusaSpec.MedusaConfigurationRef.Namespace, kc.Namespace)
+		medusaConfigKey := types.NamespacedName{Namespace: medusaConfigNamespace, Name: medusaConfigName}
+		medusaConfig := &medusaapi.MedusaConfiguration{}
+		if err := c.Get(ctx, medusaConfigKey, medusaConfig); err != nil {
+			logger.Error(err, fmt.Sprintf("could not get MedusaConfiguration %s/%s", medusaConfigNamespace, medusaConfigName))
+			return err
+		}
 
-	// write the secret into the namespace of the K8ssandraCluster
-	clusterBucketSecret := bucketSecret.DeepCopy()
-	clusterBucketSecret.ResourceVersion = ""
-	clusterBucketSecret.Name = fmt.Sprintf("%s-%s", kc.Name, bucketSecret.Name)
-	clusterBucketSecret.Namespace = kc.Namespace
-	labels.SetReplicatedBy(clusterBucketSecret, utils.GetKey(kc))
-	if err := c.Create(ctx, clusterBucketSecret); err != nil {
-		if !errors.IsAlreadyExists(err) {
-			logger.Error(err, fmt.Sprintf("failed to create cluster bucket secret %s", clusterBucketSecret))
+		// fetch the referenced medusa configuration's bucket secret
+		bucketSecretName := medusaConfig.Spec.StorageProperties.StorageSecretRef.Name
+		bucketSecret := &corev1.Secret{}
+		bucketSecretKey := types.NamespacedName{Namespace: medusaConfigNamespace, Name: bucketSecretName}
+		if err := c.Get(ctx, bucketSecretKey, bucketSecret); err != nil {
+			logger.Error(err, "could not get bucket Secret")
 			return err
 		}
-		// we already have the bucket secret, so continue to updating the cluster (it might have failed before)
-	}
 
-	return nil
+		// write the secret into the namespace of the K8ssandraCluster
+		clusterBucketSecret := bucketSecret.DeepCopy()
+		clusterBucketSecret.ResourceVersion = ""
+		clusterBucketSecret.Name = fmt.Sprintf("%s-%s", kc.Name, bucketSecret.Name)
+		clusterBucketSecret.Namespace = kc.Namespace
+		labels.SetReplicatedBy(clusterBucketSecret, utils.GetKey(kc))
+		if err := c.Create(ctx, clusterBucketSecret); err != nil {
+			if !errors.IsAlreadyExists(err) {
+				logger.Error(err, fmt.Sprintf("failed to create cluster bucket secret %s", clusterBucketSecret))
+				return err
+			}
+			// we already have the bucket secret, so continue to updating the cluster (it might have failed before)
+		}
+		return nil
+	} else {
+		// no-op, the bucket secret exists in the same namespace and doesn't need copying via a replicated secret.
+		return nil
+	}
 }
 
 func (r *K8ssandraClusterReconciler) getOperatorNamespace() string {

From 47564ea13ff0dc3b070003841c52de3d692c2467 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Fri, 5 Apr 2024 15:28:38 +1100
Subject: [PATCH 09/54] Juggle some ordering, move a constant.

---
 .../v1alpha1/medusaconfiguration_types.go      |  4 ++++
 controllers/k8ssandra/medusa_reconciler.go     | 18 +++++++++---------
 .../medusa/medusaconfiguration_controller.go   |  8 ++------
 .../medusaconfiguration_controller_test.go     |  2 +-
 4 files changed, 16 insertions(+), 16 deletions(-)

diff --git a/apis/medusa/v1alpha1/medusaconfiguration_types.go b/apis/medusa/v1alpha1/medusaconfiguration_types.go
index 94a72c1f5..6b6ed4e2c 100644
--- a/apis/medusa/v1alpha1/medusaconfiguration_types.go
+++ b/apis/medusa/v1alpha1/medusaconfiguration_types.go
@@ -20,6 +20,10 @@ import (
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 )
 
+const (
+	MedusaStorageSecretIdentifierLabel = "k8ssandra.io/medusa-storage-secret"
+)
+
 // MedusaConfigurationSpec defines the desired state of MedusaConfiguration
 type MedusaConfigurationSpec struct {
 	// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index afffe43a2..1dd4e4e85 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -293,17 +293,17 @@ func (r *K8ssandraClusterReconciler) reconcileRemoteBucketSecretsDeprecated(
 	kc *api.K8ssandraCluster,
 	logger logr.Logger,
 ) error {
-	if kc.Spec.Medusa.MedusaConfigurationRef.Namespace != kc.Namespace {
-		// This is the deprecated code path. Moving forward we will use a replicated secret with a prefix, but we will remove this code path after v1.17.
-		logger.Info("Reconciling Medusa bucket secrets")
-		medusaSpec := kc.Spec.Medusa
+	logger.Info("Reconciling Medusa bucket secrets")
+	medusaSpec := kc.Spec.Medusa
 
-		// there is nothing to reconcile if we're not using Medusa configuration reference
-		if medusaSpec == nil || medusaSpec.MedusaConfigurationRef.Name == "" {
-			logger.Info("MedusaConfigurationRef is not set, skipping bucket secret reconciliation")
-			return nil
-		}
+	// there is nothing to reconcile if we're not using Medusa configuration reference
+	if medusaSpec == nil || medusaSpec.MedusaConfigurationRef.Name == "" {
+		logger.Info("MedusaConfigurationRef is not set, skipping bucket secret reconciliation")
+		return nil
+	}
 
+	if kc.Spec.Medusa.MedusaConfigurationRef.Namespace != kc.Namespace {
+		// This is the deprecated code path. Moving forward we will use a replicated secret with a prefix, but we will remove this code path after v1.17.
 		// fetch the referenced configuration
 		medusaConfigName := medusaSpec.MedusaConfigurationRef.Name
 		medusaConfigNamespace := utils.FirstNonEmptyString(medusaSpec.MedusaConfigurationRef.Namespace, kc.Namespace)
diff --git a/controllers/medusa/medusaconfiguration_controller.go b/controllers/medusa/medusaconfiguration_controller.go
index 5421ea987..d0c1b7705 100644
--- a/controllers/medusa/medusaconfiguration_controller.go
+++ b/controllers/medusa/medusaconfiguration_controller.go
@@ -35,10 +35,6 @@ import (
 	medusav1alpha1 "github.com/k8ssandra/k8ssandra-operator/apis/medusa/v1alpha1"
 )
 
-const (
-	MedusaStorageSecretIdentifierLabel = "k8ssandra.io/medusa-storage-secret"
-)
-
 // MedusaConfigurationReconciler reconciles a MedusaConfiguration object
 type MedusaConfigurationReconciler struct {
 	*config.ReconcilerConfig
@@ -95,8 +91,8 @@ func (r *MedusaConfigurationReconciler) Reconcile(ctx context.Context, req ctrl.
 		if secret.Labels == nil {
 			secret.Labels = make(map[string]string)
 		}
-		if secret.Labels[MedusaStorageSecretIdentifierLabel] != utils.HashNameNamespace(secret.Name, secret.Namespace) {
-			secret.Labels[MedusaStorageSecretIdentifierLabel] = utils.HashNameNamespace(secret.Name, secret.Namespace)
+		if secret.Labels[medusav1alpha1.MedusaStorageSecretIdentifierLabel] != utils.HashNameNamespace(secret.Name, secret.Namespace) {
+			secret.Labels[medusav1alpha1.MedusaStorageSecretIdentifierLabel] = utils.HashNameNamespace(secret.Name, secret.Namespace)
 			if err = r.Client.Patch(ctx, secret, patch); err != nil {
 				logger.Error(err, "Failed to patch Medusa Bucket Secret with required label")
 				return ctrl.Result{}, err
diff --git a/controllers/medusa/medusaconfiguration_controller_test.go b/controllers/medusa/medusaconfiguration_controller_test.go
index a05c74644..db2a48046 100644
--- a/controllers/medusa/medusaconfiguration_controller_test.go
+++ b/controllers/medusa/medusaconfiguration_controller_test.go
@@ -69,7 +69,7 @@ func testMedusaConfigurationOk(t *testing.T, ctx context.Context, f *framework.F
 		updatedSecret := &corev1.Secret{}
 		require.NoError(f.Client.Get(ctx, types.NamespacedName{Name: "medusa-bucket-key", Namespace: namespace}, updatedSecret))
 		//Ensure that the unique label has been added to the secret.
-		if updatedSecret.Labels[MedusaStorageSecretIdentifierLabel] != utils.HashNameNamespace(updatedSecret.Name, updatedSecret.Namespace) {
+		if updatedSecret.Labels[api.MedusaStorageSecretIdentifierLabel] != utils.HashNameNamespace(updatedSecret.Name, updatedSecret.Namespace) {
 			return false
 		}
 		for _, condition := range updated.Status.Conditions {

From ed3b70af81007bc2a38ed3feed7d427c3fa478d8 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Fri, 5 Apr 2024 15:55:08 +1100
Subject: [PATCH 10/54] Switch out the current medusa bucket secret copying
 logic for a ReplicatedSecret.

---
 controllers/k8ssandra/medusa_reconciler.go | 64 ++++++++++++----------
 1 file changed, 35 insertions(+), 29 deletions(-)

diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index 1dd4e4e85..6ab6edbe2 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -9,6 +9,7 @@ import (
 	"github.com/go-logr/logr"
 	api "github.com/k8ssandra/k8ssandra-operator/apis/k8ssandra/v1alpha1"
 	medusaapi "github.com/k8ssandra/k8ssandra-operator/apis/medusa/v1alpha1"
+	replication "github.com/k8ssandra/k8ssandra-operator/apis/replication/v1alpha1"
 	cassandra "github.com/k8ssandra/k8ssandra-operator/pkg/cassandra"
 	"github.com/k8ssandra/k8ssandra-operator/pkg/labels"
 	medusa "github.com/k8ssandra/k8ssandra-operator/pkg/medusa"
@@ -18,9 +19,10 @@ import (
 	"github.com/k8ssandra/k8ssandra-operator/pkg/utils"
 	appsv1 "k8s.io/api/apps/v1"
 	corev1 "k8s.io/api/core/v1"
-	"k8s.io/apimachinery/pkg/api/errors"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/types"
 	"sigs.k8s.io/controller-runtime/pkg/client"
+	"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
 )
 
 const (
@@ -196,9 +198,13 @@ func (r *K8ssandraClusterReconciler) reconcileMedusaSecrets(
 			return result.Error(err)
 		}
 
-		if err := r.reconcileRemoteBucketSecretsDeprecated(ctx, r.ClientCache.GetLocalClient(), kc, logger); err != nil {
-			logger.Error(err, "Failed to reconcile Medusa bucket secrets")
-			return result.Error(err)
+		res := r.reconcileRemoteBucketSecretsDeprecated(ctx, r.ClientCache.GetLocalClient(), kc, logger)
+		switch {
+		case res.IsError():
+			logger.Error(res.GetError(), "Failed to reconcile Medusa bucket secrets")
+			return res
+		case res.IsRequeue():
+			return res
 		}
 	}
 
@@ -292,14 +298,14 @@ func (r *K8ssandraClusterReconciler) reconcileRemoteBucketSecretsDeprecated(
 	c client.Client,
 	kc *api.K8ssandraCluster,
 	logger logr.Logger,
-) error {
+) result.ReconcileResult {
 	logger.Info("Reconciling Medusa bucket secrets")
 	medusaSpec := kc.Spec.Medusa
 
 	// there is nothing to reconcile if we're not using Medusa configuration reference
 	if medusaSpec == nil || medusaSpec.MedusaConfigurationRef.Name == "" {
 		logger.Info("MedusaConfigurationRef is not set, skipping bucket secret reconciliation")
-		return nil
+		return result.Continue()
 	}
 
 	if kc.Spec.Medusa.MedusaConfigurationRef.Namespace != kc.Namespace {
@@ -311,35 +317,35 @@ func (r *K8ssandraClusterReconciler) reconcileRemoteBucketSecretsDeprecated(
 		medusaConfig := &medusaapi.MedusaConfiguration{}
 		if err := c.Get(ctx, medusaConfigKey, medusaConfig); err != nil {
 			logger.Error(err, fmt.Sprintf("could not get MedusaConfiguration %s/%s", medusaConfigNamespace, medusaConfigName))
-			return err
+			return result.Error(err)
 		}
 
-		// fetch the referenced medusa configuration's bucket secret
-		bucketSecretName := medusaConfig.Spec.StorageProperties.StorageSecretRef.Name
-		bucketSecret := &corev1.Secret{}
-		bucketSecretKey := types.NamespacedName{Namespace: medusaConfigNamespace, Name: bucketSecretName}
-		if err := c.Get(ctx, bucketSecretKey, bucketSecret); err != nil {
-			logger.Error(err, "could not get bucket Secret")
-			return err
+		//fmt.Sprintf("%s-%s", kc.Name, bucketSecret.Name)
+		repSecret := replication.ReplicatedSecret{
+			ObjectMeta: metav1.ObjectMeta{
+				Name:      kc.GetClusterIdHash(8) + "-" + medusaConfig.Spec.StorageProperties.StorageSecretRef.Name,
+				Namespace: medusaConfigNamespace,
+			},
+			Spec: replication.ReplicatedSecretSpec{
+				Selector: &metav1.LabelSelector{
+					MatchLabels: map[string]string{
+						medusaapi.MedusaStorageSecretIdentifierLabel: utils.HashNameNamespace(medusaConfig.Spec.StorageProperties.StorageSecretRef.Name, medusaConfigNamespace),
+					},
+					//TODO: we need to add a prefix to this secret so that it doesn't end up in conflict if referenced from multiple clusters.
+				},
+			},
 		}
-
-		// write the secret into the namespace of the K8ssandraCluster
-		clusterBucketSecret := bucketSecret.DeepCopy()
-		clusterBucketSecret.ResourceVersion = ""
-		clusterBucketSecret.Name = fmt.Sprintf("%s-%s", kc.Name, bucketSecret.Name)
-		clusterBucketSecret.Namespace = kc.Namespace
-		labels.SetReplicatedBy(clusterBucketSecret, utils.GetKey(kc))
-		if err := c.Create(ctx, clusterBucketSecret); err != nil {
-			if !errors.IsAlreadyExists(err) {
-				logger.Error(err, fmt.Sprintf("failed to create cluster bucket secret %s", clusterBucketSecret))
-				return err
-			}
-			// we already have the bucket secret, so continue to updating the cluster (it might have failed before)
+		if err := controllerutil.SetControllerReference(kc, &repSecret, r.Scheme); err != nil {
+			return result.Error(err)
 		}
-		return nil
+		// TODO: this should also have finalizer logic included in the k8ssandraCluster finalizer to remove the replicated secret if it is no longer being used.
+		// TODO: this should probably have a finalizer on it too so that the replicatedSecret cannot be deleted.
+
+		return reconciliation.ReconcileObject(ctx, c, r.DefaultDelay, repSecret)
+
 	} else {
 		// no-op, the bucket secret exists in the same namespace and doesn't need copying via a replicated secret.
-		return nil
+		return result.Continue()
 	}
 }
 

From 5d79ee30f96d14b1527a66742c9c8576a30cb62f Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Fri, 5 Apr 2024 16:24:46 +1100
Subject: [PATCH 11/54] Ensure labels are added to the ReplicatedSecret.

---
 controllers/k8ssandra/medusa_reconciler.go | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index 6ab6edbe2..9e4bf7335 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -8,6 +8,7 @@ import (
 	"github.com/adutra/goalesce"
 	"github.com/go-logr/logr"
 	api "github.com/k8ssandra/k8ssandra-operator/apis/k8ssandra/v1alpha1"
+	k8ssandraapi "github.com/k8ssandra/k8ssandra-operator/apis/k8ssandra/v1alpha1"
 	medusaapi "github.com/k8ssandra/k8ssandra-operator/apis/medusa/v1alpha1"
 	replication "github.com/k8ssandra/k8ssandra-operator/apis/replication/v1alpha1"
 	cassandra "github.com/k8ssandra/k8ssandra-operator/pkg/cassandra"
@@ -325,6 +326,10 @@ func (r *K8ssandraClusterReconciler) reconcileRemoteBucketSecretsDeprecated(
 			ObjectMeta: metav1.ObjectMeta{
 				Name:      kc.GetClusterIdHash(8) + "-" + medusaConfig.Spec.StorageProperties.StorageSecretRef.Name,
 				Namespace: medusaConfigNamespace,
+				Labels: map[string]string{
+					k8ssandraapi.K8ssandraClusterNameLabel:      kc.Name,
+					k8ssandraapi.K8ssandraClusterNamespaceLabel: kc.Namespace,
+				},
 			},
 			Spec: replication.ReplicatedSecretSpec{
 				Selector: &metav1.LabelSelector{

From 20e2949493bded595f9bb5792543f2189175903f Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Fri, 5 Apr 2024 16:27:33 +1100
Subject: [PATCH 12/54] Fix linter complaining at me.

---
 controllers/k8ssandra/medusa_reconciler.go | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index 9e4bf7335..b90e5bc41 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -282,10 +282,7 @@ func (r *K8ssandraClusterReconciler) mergeStorageProperties(
 		// Deprecated: when we remove the ability to reference a non-namespace-local MedusaConfig in v 1.17,
 		// this if statement should be eliminated.
 		mergedProperties.StorageSecretRef.Name = fmt.Sprintf("%s-%s", desiredKc.Name, mergedProperties.StorageSecretRef.Name)
-	} else {
-		// this will be the only code branch after the deprecation ends.
-		mergedProperties.StorageSecretRef.Name = mergedProperties.StorageSecretRef.Name // Yes, I know this is assigned to itself.
-	}
+	} // imagine an else here which just contains `mergedProperties.StorageSecretRef.Name = mergedProperties.StorageSecretRef.Name`, since this is a no-op.
 
 	// copy the merged properties back into the cluster
 	mergedProperties.DeepCopyInto(&desiredKc.Spec.Medusa.StorageProperties)

From 0e8838925453a9d7d28ca1911a641dd6e5fc86f6 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Fri, 5 Apr 2024 17:12:55 +1100
Subject: [PATCH 13/54] Fix failing envtests.

---
 controllers/k8ssandra/medusa_reconciler_test.go | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/controllers/k8ssandra/medusa_reconciler_test.go b/controllers/k8ssandra/medusa_reconciler_test.go
index fe5d94c64..25740587a 100644
--- a/controllers/k8ssandra/medusa_reconciler_test.go
+++ b/controllers/k8ssandra/medusa_reconciler_test.go
@@ -77,8 +77,7 @@ func medusaTemplateWithoutConfigRef() *medusaapi.MedusaClusterTemplate {
 
 func medusaTemplateWithConfigRef(configRefName, namespace string) *medusaapi.MedusaClusterTemplate {
 	configRef := &corev1.ObjectReference{
-		Name:      configRefName,
-		Namespace: namespace,
+		Name: configRefName,
 	}
 	return medusaTemplate(configRef)
 }
@@ -623,8 +622,7 @@ func createMultiDcClusterWithReplicatedSecrets(t *testing.T, ctx context.Context
 			},
 			Medusa: &medusaapi.MedusaClusterTemplate{
 				MedusaConfigurationRef: corev1.ObjectReference{
-					Namespace: namespace,
-					Name:      originalConfigName,
+					Name: originalConfigName,
 				},
 				StorageProperties: medusaapi.Storage{
 					Prefix: "some-prefix",

From dd5e24cf094aa8fc37f10813627c6374ba29e4b3 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Fri, 5 Apr 2024 17:45:18 +1100
Subject: [PATCH 14/54] Put an eventually in place since this update is failing
 on some optimistic locking issue.

---
 controllers/k8ssandra/add_dc_test.go | 49 ++++++++++++++++------------
 1 file changed, 28 insertions(+), 21 deletions(-)

diff --git a/controllers/k8ssandra/add_dc_test.go b/controllers/k8ssandra/add_dc_test.go
index 77cedad40..71daec2b0 100644
--- a/controllers/k8ssandra/add_dc_test.go
+++ b/controllers/k8ssandra/add_dc_test.go
@@ -744,32 +744,39 @@ func addStargateAndReaperToCluster(ctx context.Context, t *testing.T, f *framewo
 }
 
 func addDcToCluster(ctx context.Context, t *testing.T, f *framework.Framework, kc *api.K8ssandraCluster, dcKey framework.ClusterKey) {
-	t.Logf("add %s to cluster", dcKey.Name)
+	require.Eventually(t, func() bool {
+		t.Logf("add %s to cluster", dcKey.Name)
 
-	key := utils.GetKey(kc)
-	err := f.Client.Get(ctx, key, kc)
-	require.NoError(t, err, "failed to get K8ssandraCluster")
+		key := utils.GetKey(kc)
+		err := f.Client.Get(ctx, key, kc)
+		require.NoError(t, err, "failed to get K8ssandraCluster")
 
-	kc.Spec.Cassandra.Datacenters = append(kc.Spec.Cassandra.Datacenters, api.CassandraDatacenterTemplate{
-		Meta: api.EmbeddedObjectMeta{
-			Name:      dcKey.Name,
-			Namespace: dcKey.Namespace,
-		},
-		K8sContext: dcKey.K8sContext,
-		Size:       3,
-		DatacenterOptions: api.DatacenterOptions{
-			ServerVersion: "4.0.1",
-			StorageConfig: &cassdcapi.StorageConfig{
-				CassandraDataVolumeClaimSpec: &corev1.PersistentVolumeClaimSpec{
-					StorageClassName: &defaultStorageClass,
+		kc.Spec.Cassandra.Datacenters = append(kc.Spec.Cassandra.Datacenters, api.CassandraDatacenterTemplate{
+			Meta: api.EmbeddedObjectMeta{
+				Name:      dcKey.Name,
+				Namespace: dcKey.Namespace,
+			},
+			K8sContext: dcKey.K8sContext,
+			Size:       3,
+			DatacenterOptions: api.DatacenterOptions{
+				ServerVersion: "4.0.1",
+				StorageConfig: &cassdcapi.StorageConfig{
+					CassandraDataVolumeClaimSpec: &corev1.PersistentVolumeClaimSpec{
+						StorageClassName: &defaultStorageClass,
+					},
 				},
 			},
-		},
-	})
-	annotations.AddAnnotation(kc, api.DcReplicationAnnotation, fmt.Sprintf(`{"%s": {"ks1": 3, "ks2": 3}}`, dcKey.Name))
+		})
+		annotations.AddAnnotation(kc, api.DcReplicationAnnotation, fmt.Sprintf(`{"%s": {"ks1": 3, "ks2": 3}}`, dcKey.Name))
+
+		err = f.Client.Update(ctx, kc)
+		if err != nil {
+			t.Logf("failed to add %s to cluster: %v", dcKey.Name, err)
+			return false
+		}
+		return err == nil
+	}, timeout, interval)
 
-	err = f.Client.Update(ctx, kc)
-	require.NoError(t, err, "failed to add dc to K8ssandraCluster")
 }
 
 func verifyReplicationOfSystemKeyspacesUpdated(t *testing.T, mockMgmtApi *testutils.FakeManagementApiFacade, replication, updatedReplication map[string]int) {

From f3eb52a2037b5252fd06ca44bdfc81ed0cb7f83f Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Fri, 5 Apr 2024 18:04:14 +1100
Subject: [PATCH 15/54] Fix webhook test.

---
 .../v1alpha1/k8ssandracluster_webhook_test.go          | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/apis/k8ssandra/v1alpha1/k8ssandracluster_webhook_test.go b/apis/k8ssandra/v1alpha1/k8ssandracluster_webhook_test.go
index eded2cb1b..46c0c656a 100644
--- a/apis/k8ssandra/v1alpha1/k8ssandracluster_webhook_test.go
+++ b/apis/k8ssandra/v1alpha1/k8ssandracluster_webhook_test.go
@@ -470,7 +470,15 @@ func testInvalidDcName(t *testing.T) {
 func testMedusaNonLocalNamespace(t *testing.T) {
 	required := require.New(t)
 	badCluster := createMinimalClusterObj("medusaconfig-nonlocal", "ns")
-	badCluster.Spec.Medusa.MedusaConfigurationRef.Namespace = "nonlocal-ns"
+	badCluster.Spec.Medusa = &medusaapi.MedusaClusterTemplate{
+		MedusaConfigurationRef: corev1.ObjectReference{
+			Namespace: "nonlocal-ns",
+			Name:      "medusa-config",
+		},
+		StorageProperties: medusaapi.Storage{
+			Prefix: "some-prefix",
+		},
+	}
 	err := badCluster.validateK8ssandraCluster()
 	required.Error(err)
 	required.Contains(err.Error(), "Medusa config must be namespace local")

From 20b784e1eaf11dcc60506f30ab2b8adffcc97d1e Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 8 Apr 2024 10:37:56 +1000
Subject: [PATCH 16/54] Update changelog for correct version.

---
 CHANGELOG/CHANGELOG-1.15.md | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/CHANGELOG/CHANGELOG-1.15.md b/CHANGELOG/CHANGELOG-1.15.md
index 58ab84827..082195ce5 100644
--- a/CHANGELOG/CHANGELOG-1.15.md
+++ b/CHANGELOG/CHANGELOG-1.15.md
@@ -14,3 +14,6 @@ Changelog for the K8ssandra Operator, new PRs should update the `unreleased` sec
 When cutting a new release, update the `unreleased` heading to the tag being generated and date, like `## vX.Y.Z - YYYY-MM-DD` and create a new placeholder section for  `unreleased` entries.
 
 ## unreleased
+* [BUGFIX] [#1253](https://github.com/k8ssandra/k8ssandra-operator/issues/1253) Medusa storage secrets are now labelled with a unique label.
+* [FEATURE] [#1260](https://github.com/k8ssandra/k8ssandra-operator/issues/1260) Update controller-gen to version 0.14.0.
+* [CHANGE] [#1266](https://github.com/k8ssandra/k8ssandra-operator/issues/1266) Deprecate the usage of non-namespace-local MedusaConfigurations within a K8ssandraCluster, and throw an error from the webhook when users attempt to use one.
\ No newline at end of file

From 90c4ce1318e5e8f25e966657987c3bbd0f2c2b4b Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 8 Apr 2024 11:02:56 +1000
Subject: [PATCH 17/54] Fix multi-dc-cluster-scoped e2e test.

---
 test/testdata/fixtures/multi-dc-cluster-scope/k8ssandra.yaml | 1 -
 1 file changed, 1 deletion(-)

diff --git a/test/testdata/fixtures/multi-dc-cluster-scope/k8ssandra.yaml b/test/testdata/fixtures/multi-dc-cluster-scope/k8ssandra.yaml
index ef54eb8fc..43106ac60 100644
--- a/test/testdata/fixtures/multi-dc-cluster-scope/k8ssandra.yaml
+++ b/test/testdata/fixtures/multi-dc-cluster-scope/k8ssandra.yaml
@@ -7,7 +7,6 @@ spec:
   medusa:
     medusaConfigurationRef:
       name: global-medusa-config
-      namespace: test-0
     storageProperties:
       prefix: test
   cassandra:

From 3ca2fa90b217280946eacd2d5ad57f223da8cd84 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 8 Apr 2024 12:37:44 +1000
Subject: [PATCH 18/54] Fix another e2e test.

---
 test/e2e/medusa_test.go | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test/e2e/medusa_test.go b/test/e2e/medusa_test.go
index 1c2134dce..188b2af2e 100644
--- a/test/e2e/medusa_test.go
+++ b/test/e2e/medusa_test.go
@@ -125,7 +125,7 @@ func checkBucketKeyPresent(t *testing.T, f *framework.E2eFramework, ctx context.
 	require := require.New(t)
 
 	// work out the name of the replicated bucket key. should be "clusterName-<original-bucket-key-name>"
-	localBucketKeyName := fmt.Sprintf("%s-%s", kc.Name, globalBucketSecretName)
+	localBucketKeyName := fmt.Sprintf(globalBucketSecretName)
 
 	// Check that the bucket key has been replicated to the current namespace
 	bucketKey := &corev1.Secret{}

From 6296040ed037d7ad93c483d0a85f205ef232c0c1 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 8 Apr 2024 13:15:38 +1000
Subject: [PATCH 19/54] Add purpose annotation.

---
 apis/k8ssandra/v1alpha1/constants.go       | 3 +++
 controllers/k8ssandra/medusa_reconciler.go | 3 +++
 2 files changed, 6 insertions(+)

diff --git a/apis/k8ssandra/v1alpha1/constants.go b/apis/k8ssandra/v1alpha1/constants.go
index 68b99364d..128e714a5 100644
--- a/apis/k8ssandra/v1alpha1/constants.go
+++ b/apis/k8ssandra/v1alpha1/constants.go
@@ -61,6 +61,9 @@ const (
 	DatacenterLabel = "k8ssandra.io/datacenter"
 	// Forces refresh of secrets which relate to roles and authn in Cassandra.
 	RefreshAnnotation = "k8ssandra.io/refresh"
+
+	// Annotation to indicate the purpose of a given resource.
+	PurposeAnnotation = "k8ssandra.io/purpose"
 )
 
 var (
diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index b90e5bc41..26830197c 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -327,6 +327,9 @@ func (r *K8ssandraClusterReconciler) reconcileRemoteBucketSecretsDeprecated(
 					k8ssandraapi.K8ssandraClusterNameLabel:      kc.Name,
 					k8ssandraapi.K8ssandraClusterNamespaceLabel: kc.Namespace,
 				},
+				Annotations: map[string]string{
+					k8ssandraapi.PurposeAnnotation: "This replicated secret is designed for the old codepath in the Medusa reconciler within the k8ssandra cluster controller. In this deprecated path, a MedusaConfig could reside in a different namespace to the K8ssandraCluster. This ReplicatedSecret ensures that the bucket secret is copied into the K8ssandraCluster's namespace. This codepath will be removed in v1.17.",
+				},
 			},
 			Spec: replication.ReplicatedSecretSpec{
 				Selector: &metav1.LabelSelector{

From 26a0c8bec37d00f3f71d3e3769321d7b8b807b83 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 8 Apr 2024 13:30:10 +1000
Subject: [PATCH 20/54] Fix target definition in ReplicatedSecret.

---
 controllers/k8ssandra/medusa_reconciler.go | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index 26830197c..3439b1efb 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -318,7 +318,6 @@ func (r *K8ssandraClusterReconciler) reconcileRemoteBucketSecretsDeprecated(
 			return result.Error(err)
 		}
 
-		//fmt.Sprintf("%s-%s", kc.Name, bucketSecret.Name)
 		repSecret := replication.ReplicatedSecret{
 			ObjectMeta: metav1.ObjectMeta{
 				Name:      kc.GetClusterIdHash(8) + "-" + medusaConfig.Spec.StorageProperties.StorageSecretRef.Name,
@@ -334,9 +333,16 @@ func (r *K8ssandraClusterReconciler) reconcileRemoteBucketSecretsDeprecated(
 			Spec: replication.ReplicatedSecretSpec{
 				Selector: &metav1.LabelSelector{
 					MatchLabels: map[string]string{
-						medusaapi.MedusaStorageSecretIdentifierLabel: utils.HashNameNamespace(medusaConfig.Spec.StorageProperties.StorageSecretRef.Name, medusaConfigNamespace),
+						medusaapi.MedusaStorageSecretIdentifierLabel: utils.HashNameNamespace(
+							medusaConfig.Spec.StorageProperties.StorageSecretRef.Name,
+							medusaConfigNamespace),
+					},
+				},
+				ReplicationTargets: []replication.ReplicationTarget{
+					{
+						Namespace:    kc.Namespace,
+						TargetPrefix: kc.Name + "-",
 					},
-					//TODO: we need to add a prefix to this secret so that it doesn't end up in conflict if referenced from multiple clusters.
 				},
 			},
 		}

From 7bb6db94f0ea390a018cdb39d02d8d0137e4bd9a Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 8 Apr 2024 14:03:39 +1000
Subject: [PATCH 21/54] Create new DropLabels field on ReplicatedSecret to
 ensure we don't end up looping infinitely.

---
 .../v1alpha1/replicatedsecret_types.go        |  3 +++
 .../v1alpha1/zz_generated.deepcopy.go         |  9 ++++++-
 .../crds/k8ssandra-operator-crds.yaml         |  7 ++++++
 ...cation.k8ssandra.io_replicatedsecrets.yaml |  7 ++++++
 controllers/k8ssandra/medusa_reconciler.go    |  3 +++
 controllers/replication/secret_controller.go  | 24 +++++++++++++++++--
 .../replication/secret_controller_test.go     | 20 ++++++++++++----
 7 files changed, 65 insertions(+), 8 deletions(-)

diff --git a/apis/replication/v1alpha1/replicatedsecret_types.go b/apis/replication/v1alpha1/replicatedsecret_types.go
index fcc546354..f6568eef0 100644
--- a/apis/replication/v1alpha1/replicatedsecret_types.go
+++ b/apis/replication/v1alpha1/replicatedsecret_types.go
@@ -56,6 +56,9 @@ type ReplicationTarget struct {
 	// as the original secret.
 	// +optional
 	TargetPrefix string `json:"targetPrefix,omitempty"`
+
+	// DropLabels defines the labels to be dropped from the secret before replication, this is sometimes neccessary to avoid infinite replication.
+	DropLabels []string `json:"dropLabels,omitempty"`
 }
 
 // ReplicatedSecretStatus defines the observed state of ReplicatedSecret
diff --git a/apis/replication/v1alpha1/zz_generated.deepcopy.go b/apis/replication/v1alpha1/zz_generated.deepcopy.go
index 34a0a59f4..efe3ff994 100644
--- a/apis/replication/v1alpha1/zz_generated.deepcopy.go
+++ b/apis/replication/v1alpha1/zz_generated.deepcopy.go
@@ -95,7 +95,9 @@ func (in *ReplicatedSecretSpec) DeepCopyInto(out *ReplicatedSecretSpec) {
 	if in.ReplicationTargets != nil {
 		in, out := &in.ReplicationTargets, &out.ReplicationTargets
 		*out = make([]ReplicationTarget, len(*in))
-		copy(*out, *in)
+		for i := range *in {
+			(*in)[i].DeepCopyInto(&(*out)[i])
+		}
 	}
 }
 
@@ -153,6 +155,11 @@ func (in *ReplicationCondition) DeepCopy() *ReplicationCondition {
 // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
 func (in *ReplicationTarget) DeepCopyInto(out *ReplicationTarget) {
 	*out = *in
+	if in.DropLabels != nil {
+		in, out := &in.DropLabels, &out.DropLabels
+		*out = make([]string, len(*in))
+		copy(*out, *in)
+	}
 }
 
 // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ReplicationTarget.
diff --git a/charts/k8ssandra-operator/crds/k8ssandra-operator-crds.yaml b/charts/k8ssandra-operator/crds/k8ssandra-operator-crds.yaml
index 4bbcb7537..e552183fb 100644
--- a/charts/k8ssandra-operator/crds/k8ssandra-operator-crds.yaml
+++ b/charts/k8ssandra-operator/crds/k8ssandra-operator-crds.yaml
@@ -33387,6 +33387,13 @@ spec:
                   the secrets are replicated to. If empty, no clusters are targeted
                 items:
                   properties:
+                    dropLabels:
+                      description: DropLabels defines the labels to be dropped from
+                        the secret before replication, this is sometimes neccessary
+                        to avoid infinite replication.
+                      items:
+                        type: string
+                      type: array
                     k8sContextName:
                       description: K8sContextName defines the target cluster name
                         as set in the ClientConfig. If left empty, current cluster
diff --git a/config/crd/bases/replication.k8ssandra.io_replicatedsecrets.yaml b/config/crd/bases/replication.k8ssandra.io_replicatedsecrets.yaml
index 2b38cdc31..a3f729c31 100644
--- a/config/crd/bases/replication.k8ssandra.io_replicatedsecrets.yaml
+++ b/config/crd/bases/replication.k8ssandra.io_replicatedsecrets.yaml
@@ -44,6 +44,13 @@ spec:
                   the secrets are replicated to. If empty, no clusters are targeted
                 items:
                   properties:
+                    dropLabels:
+                      description: DropLabels defines the labels to be dropped from
+                        the secret before replication, this is sometimes neccessary
+                        to avoid infinite replication.
+                      items:
+                        type: string
+                      type: array
                     k8sContextName:
                       description: K8sContextName defines the target cluster name
                         as set in the ClientConfig. If left empty, current cluster
diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index 3439b1efb..d94bf9ea4 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -342,6 +342,9 @@ func (r *K8ssandraClusterReconciler) reconcileRemoteBucketSecretsDeprecated(
 					{
 						Namespace:    kc.Namespace,
 						TargetPrefix: kc.Name + "-",
+						DropLabels: []string{
+							medusaapi.MedusaStorageSecretIdentifierLabel,
+						},
 					},
 				},
 			},
diff --git a/controllers/replication/secret_controller.go b/controllers/replication/secret_controller.go
index 3902e3ef9..543e3c288 100644
--- a/controllers/replication/secret_controller.go
+++ b/controllers/replication/secret_controller.go
@@ -235,6 +235,7 @@ func (s *SecretSyncController) Reconcile(ctx context.Context, req ctrl.Request)
 					copiedSecret.ResourceVersion = ""
 					copiedSecret.OwnerReferences = []metav1.OwnerReference{}
 					copiedSecret.Name = getPrefixedSecretName(target.TargetPrefix, sec.Name)
+					stripDroppedLabels(copiedSecret, target.DropLabels)
 					if err = remoteClient.Create(ctx, copiedSecret); err != nil {
 						logger.Error(err, "Failed to sync secret to target cluster", "Secret", copiedSecret.Name, "TargetContext", target)
 						break TargetSecrets
@@ -253,7 +254,7 @@ func (s *SecretSyncController) Reconcile(ctx context.Context, req ctrl.Request)
 
 			if requiresUpdate(sec, fetchedSecret) {
 				logger.Info("Modifying secret in target cluster", "Secret", sec.Name, "TargetContext", target)
-				syncSecrets(sec, fetchedSecret)
+				syncSecrets(sec, fetchedSecret, target)
 				copiedSecret := fetchedSecret.DeepCopy()
 				copiedSecret.Name = getPrefixedSecretName(target.TargetPrefix, sec.Name)
 				if err = remoteClient.Update(ctx, copiedSecret); err != nil {
@@ -315,7 +316,7 @@ func requiresUpdate(source, dest client.Object) bool {
 	return false
 }
 
-func syncSecrets(src, dest *corev1.Secret) {
+func syncSecrets(src, dest *corev1.Secret, target api.ReplicationTarget) {
 	origMeta := dest.ObjectMeta
 	src.DeepCopyInto(dest)
 	dest.ObjectMeta = origMeta
@@ -342,6 +343,7 @@ func syncSecrets(src, dest *corev1.Secret) {
 			dest.Labels[k] = v
 		}
 	}
+	stripDroppedLabels(dest, target.DropLabels)
 }
 
 // filterValue verifies the annotation is not something datacenter specific
@@ -441,3 +443,21 @@ func (s *SecretSyncController) initializeCache() error {
 func getPrefixedSecretName(prefix string, secretName string) string {
 	return fmt.Sprintf("%s%s", prefix, secretName)
 }
+
+// stripDroppedLabels removes the labels from the secret that are defined in the dropLabels field.
+func stripDroppedLabels(secret *corev1.Secret, dropLabels []string) {
+	for key, _ := range secret.Labels {
+		if contains(key, dropLabels) {
+			delete(secret.Labels, key)
+		}
+	}
+}
+
+func contains(s string, arr []string) bool {
+	for _, i := range arr {
+		if i == s {
+			return true
+		}
+	}
+	return false
+}
diff --git a/controllers/replication/secret_controller_test.go b/controllers/replication/secret_controller_test.go
index f8bb06d8a..8429ad3d0 100644
--- a/controllers/replication/secret_controller_test.go
+++ b/controllers/replication/secret_controller_test.go
@@ -396,6 +396,7 @@ func TestSyncSecrets(t *testing.T) {
 			Namespace: "b",
 			Labels: map[string]string{
 				"label1": "value1",
+				"dropMe": "true",
 			},
 			Annotations: map[string]string{
 				coreapi.ResourceHashAnnotation: "12345678",
@@ -406,10 +407,15 @@ func TestSyncSecrets(t *testing.T) {
 		},
 	}
 
-	syncSecrets(orig, dest)
+	target := api.ReplicationTarget{
+		DropLabels: []string{"dropMe"},
+	}
+
+	syncSecrets(orig, dest, target)
 
 	assert.Equal(orig.GetAnnotations(), dest.GetAnnotations())
-	assert.Equal(orig.GetLabels(), dest.GetLabels())
+	assert.Equal(orig.GetLabels()["label1"], dest.GetLabels())
+	assert.Equal(orig.GetLabels()["dropMe"], "")
 
 	assert.Equal(orig.Data, dest.Data)
 
@@ -417,7 +423,7 @@ func TestSyncSecrets(t *testing.T) {
 
 	dest.GetAnnotations()[coreapi.ResourceHashAnnotation] = "9876555"
 
-	syncSecrets(orig, dest)
+	syncSecrets(orig, dest, target)
 
 	// Verify additional orphan annotation was not removed
 	assert.Contains(dest.GetLabels(), secret.OrphanResourceAnnotation)
@@ -463,7 +469,11 @@ func TestRequiresUpdate(t *testing.T) {
 	// Secrets don't match
 	assert.True(requiresUpdate(orig, dest))
 
-	syncSecrets(orig, dest)
+	target := api.ReplicationTarget{
+		DropLabels: []string{"dropMe"},
+	}
+
+	syncSecrets(orig, dest, target)
 
 	assert.False(requiresUpdate(orig, dest))
 
@@ -471,7 +481,7 @@ func TestRequiresUpdate(t *testing.T) {
 	dest.Data["secondKey"] = []byte("thisValWillBeGone")
 	assert.True(requiresUpdate(orig, dest))
 
-	syncSecrets(orig, dest)
+	syncSecrets(orig, dest, target)
 	assert.False(requiresUpdate(orig, dest))
 }
 

From 76e138a6b3521972eeb11ef0224deb9e5a53093c Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 8 Apr 2024 14:04:07 +1000
Subject: [PATCH 22/54] Make linter happy.

---
 controllers/replication/secret_controller.go      |  2 +-
 controllers/replication/secret_controller_test.go | 15 ++++++---------
 test/e2e/medusa_test.go                           |  2 +-
 3 files changed, 8 insertions(+), 11 deletions(-)

diff --git a/controllers/replication/secret_controller.go b/controllers/replication/secret_controller.go
index 543e3c288..e8eb25f4f 100644
--- a/controllers/replication/secret_controller.go
+++ b/controllers/replication/secret_controller.go
@@ -446,7 +446,7 @@ func getPrefixedSecretName(prefix string, secretName string) string {
 
 // stripDroppedLabels removes the labels from the secret that are defined in the dropLabels field.
 func stripDroppedLabels(secret *corev1.Secret, dropLabels []string) {
-	for key, _ := range secret.Labels {
+	for key := range secret.Labels {
 		if contains(key, dropLabels) {
 			delete(secret.Labels, key)
 		}
diff --git a/controllers/replication/secret_controller_test.go b/controllers/replication/secret_controller_test.go
index 8429ad3d0..c2e2f959a 100644
--- a/controllers/replication/secret_controller_test.go
+++ b/controllers/replication/secret_controller_test.go
@@ -396,7 +396,7 @@ func TestSyncSecrets(t *testing.T) {
 			Namespace: "b",
 			Labels: map[string]string{
 				"label1": "value1",
-				"dropMe": "true",
+				"dropMe": "false",
 			},
 			Annotations: map[string]string{
 				coreapi.ResourceHashAnnotation: "12345678",
@@ -407,15 +407,13 @@ func TestSyncSecrets(t *testing.T) {
 		},
 	}
 
-	target := api.ReplicationTarget{
-		DropLabels: []string{"dropMe"},
-	}
+	target := api.ReplicationTarget{}
 
 	syncSecrets(orig, dest, target)
 
 	assert.Equal(orig.GetAnnotations(), dest.GetAnnotations())
-	assert.Equal(orig.GetLabels()["label1"], dest.GetLabels())
-	assert.Equal(orig.GetLabels()["dropMe"], "")
+	assert.Equal(orig.GetLabels()["label1"], dest.GetLabels()["label1"])
+	assert.Equal(orig.GetLabels()["dropMe"], dest.GetLabels()["dropMe"])
 
 	assert.Equal(orig.Data, dest.Data)
 
@@ -429,9 +427,8 @@ func TestSyncSecrets(t *testing.T) {
 	assert.Contains(dest.GetLabels(), secret.OrphanResourceAnnotation)
 
 	// Verify original labels and their values are set
-	for k, v := range orig.GetLabels() {
-		assert.Equal(v, dest.GetLabels()[k])
-	}
+
+	assert.Equal(orig.GetLabels()["label1"], dest.GetLabels()["label1"])
 
 	// Verify original annotations and their values are set (modified hash annotation is overwritten)
 	for k, v := range orig.GetAnnotations() {
diff --git a/test/e2e/medusa_test.go b/test/e2e/medusa_test.go
index 188b2af2e..4ffaf5539 100644
--- a/test/e2e/medusa_test.go
+++ b/test/e2e/medusa_test.go
@@ -125,7 +125,7 @@ func checkBucketKeyPresent(t *testing.T, f *framework.E2eFramework, ctx context.
 	require := require.New(t)
 
 	// work out the name of the replicated bucket key. should be "clusterName-<original-bucket-key-name>"
-	localBucketKeyName := fmt.Sprintf(globalBucketSecretName)
+	localBucketKeyName := "test-" + globalBucketSecretName
 
 	// Check that the bucket key has been replicated to the current namespace
 	bucketKey := &corev1.Secret{}

From 809c5efbfdadce32a98b6f1684ebcefa2d63161c Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 8 Apr 2024 16:35:09 +1000
Subject: [PATCH 23/54] Allow labels to be added when reconciling a
 ReplicatedSecret.

---
 apis/replication/v1alpha1/replicatedsecret_types.go       | 3 +++
 apis/replication/v1alpha1/zz_generated.deepcopy.go        | 7 +++++++
 .../k8ssandra-operator/crds/k8ssandra-operator-crds.yaml  | 5 +++++
 .../bases/replication.k8ssandra.io_replicatedsecrets.yaml | 5 +++++
 controllers/k8ssandra/medusa_reconciler.go                | 4 ++++
 controllers/replication/secret_controller.go              | 8 ++++++++
 controllers/replication/secret_controller_test.go         | 2 ++
 7 files changed, 34 insertions(+)

diff --git a/apis/replication/v1alpha1/replicatedsecret_types.go b/apis/replication/v1alpha1/replicatedsecret_types.go
index f6568eef0..41a6b9e40 100644
--- a/apis/replication/v1alpha1/replicatedsecret_types.go
+++ b/apis/replication/v1alpha1/replicatedsecret_types.go
@@ -59,6 +59,9 @@ type ReplicationTarget struct {
 
 	// DropLabels defines the labels to be dropped from the secret before replication, this is sometimes neccessary to avoid infinite replication.
 	DropLabels []string `json:"dropLabels,omitempty"`
+
+	// AddLabels adds labels to the target secret.
+	AddLabels map[string]string `json:"addLabels,omitempty"`
 }
 
 // ReplicatedSecretStatus defines the observed state of ReplicatedSecret
diff --git a/apis/replication/v1alpha1/zz_generated.deepcopy.go b/apis/replication/v1alpha1/zz_generated.deepcopy.go
index efe3ff994..5a2eb9494 100644
--- a/apis/replication/v1alpha1/zz_generated.deepcopy.go
+++ b/apis/replication/v1alpha1/zz_generated.deepcopy.go
@@ -160,6 +160,13 @@ func (in *ReplicationTarget) DeepCopyInto(out *ReplicationTarget) {
 		*out = make([]string, len(*in))
 		copy(*out, *in)
 	}
+	if in.AddLabels != nil {
+		in, out := &in.AddLabels, &out.AddLabels
+		*out = make(map[string]string, len(*in))
+		for key, val := range *in {
+			(*out)[key] = val
+		}
+	}
 }
 
 // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ReplicationTarget.
diff --git a/charts/k8ssandra-operator/crds/k8ssandra-operator-crds.yaml b/charts/k8ssandra-operator/crds/k8ssandra-operator-crds.yaml
index e552183fb..cbb79f047 100644
--- a/charts/k8ssandra-operator/crds/k8ssandra-operator-crds.yaml
+++ b/charts/k8ssandra-operator/crds/k8ssandra-operator-crds.yaml
@@ -33387,6 +33387,11 @@ spec:
                   the secrets are replicated to. If empty, no clusters are targeted
                 items:
                   properties:
+                    addLabels:
+                      additionalProperties:
+                        type: string
+                      description: AddLabels adds labels to the target secret.
+                      type: object
                     dropLabels:
                       description: DropLabels defines the labels to be dropped from
                         the secret before replication, this is sometimes neccessary
diff --git a/config/crd/bases/replication.k8ssandra.io_replicatedsecrets.yaml b/config/crd/bases/replication.k8ssandra.io_replicatedsecrets.yaml
index a3f729c31..857e7696c 100644
--- a/config/crd/bases/replication.k8ssandra.io_replicatedsecrets.yaml
+++ b/config/crd/bases/replication.k8ssandra.io_replicatedsecrets.yaml
@@ -44,6 +44,11 @@ spec:
                   the secrets are replicated to. If empty, no clusters are targeted
                 items:
                   properties:
+                    addLabels:
+                      additionalProperties:
+                        type: string
+                      description: AddLabels adds labels to the target secret.
+                      type: object
                     dropLabels:
                       description: DropLabels defines the labels to be dropped from
                         the secret before replication, this is sometimes neccessary
diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index d94bf9ea4..fac78feab 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -345,6 +345,10 @@ func (r *K8ssandraClusterReconciler) reconcileRemoteBucketSecretsDeprecated(
 						DropLabels: []string{
 							medusaapi.MedusaStorageSecretIdentifierLabel,
 						},
+						AddLabels: map[string]string{
+							k8ssandraapi.K8ssandraClusterNameLabel:      kc.GetLabels()[k8ssandraapi.K8ssandraClusterNameLabel],
+							k8ssandraapi.K8ssandraClusterNamespaceLabel: kc.GetLabels()[k8ssandraapi.K8ssandraClusterNamespaceLabel],
+						},
 					},
 				},
 			},
diff --git a/controllers/replication/secret_controller.go b/controllers/replication/secret_controller.go
index e8eb25f4f..690fec7a5 100644
--- a/controllers/replication/secret_controller.go
+++ b/controllers/replication/secret_controller.go
@@ -236,6 +236,7 @@ func (s *SecretSyncController) Reconcile(ctx context.Context, req ctrl.Request)
 					copiedSecret.OwnerReferences = []metav1.OwnerReference{}
 					copiedSecret.Name = getPrefixedSecretName(target.TargetPrefix, sec.Name)
 					stripDroppedLabels(copiedSecret, target.DropLabels)
+					addAdditionalLabels(copiedSecret, target.AddLabels)
 					if err = remoteClient.Create(ctx, copiedSecret); err != nil {
 						logger.Error(err, "Failed to sync secret to target cluster", "Secret", copiedSecret.Name, "TargetContext", target)
 						break TargetSecrets
@@ -344,6 +345,7 @@ func syncSecrets(src, dest *corev1.Secret, target api.ReplicationTarget) {
 		}
 	}
 	stripDroppedLabels(dest, target.DropLabels)
+	addAdditionalLabels(dest, target.AddLabels)
 }
 
 // filterValue verifies the annotation is not something datacenter specific
@@ -461,3 +463,9 @@ func contains(s string, arr []string) bool {
 	}
 	return false
 }
+
+func addAdditionalLabels(secret *corev1.Secret, addLabels map[string]string) {
+	for k, v := range addLabels {
+		secret.Labels[k] = v
+	}
+}
diff --git a/controllers/replication/secret_controller_test.go b/controllers/replication/secret_controller_test.go
index c2e2f959a..e9cd9c77a 100644
--- a/controllers/replication/secret_controller_test.go
+++ b/controllers/replication/secret_controller_test.go
@@ -468,6 +468,7 @@ func TestRequiresUpdate(t *testing.T) {
 
 	target := api.ReplicationTarget{
 		DropLabels: []string{"dropMe"},
+		AddLabels:  map[string]string{"added": "true"},
 	}
 
 	syncSecrets(orig, dest, target)
@@ -477,6 +478,7 @@ func TestRequiresUpdate(t *testing.T) {
 	// Modify target data without fixing the hash annotation, this should cause update requirement
 	dest.Data["secondKey"] = []byte("thisValWillBeGone")
 	assert.True(requiresUpdate(orig, dest))
+	assert.Equal("true", dest.GetLabels()["added"])
 
 	syncSecrets(orig, dest, target)
 	assert.False(requiresUpdate(orig, dest))

From 012faf3cc871d07ea74fea59123626aab3e5ad39 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 8 Apr 2024 16:44:18 +1000
Subject: [PATCH 24/54] Additional label required for replication.

---
 controllers/k8ssandra/medusa_reconciler.go | 1 +
 1 file changed, 1 insertion(+)

diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index fac78feab..bb231b2fa 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -348,6 +348,7 @@ func (r *K8ssandraClusterReconciler) reconcileRemoteBucketSecretsDeprecated(
 						AddLabels: map[string]string{
 							k8ssandraapi.K8ssandraClusterNameLabel:      kc.GetLabels()[k8ssandraapi.K8ssandraClusterNameLabel],
 							k8ssandraapi.K8ssandraClusterNamespaceLabel: kc.GetLabels()[k8ssandraapi.K8ssandraClusterNamespaceLabel],
+							k8ssandraapi.ReplicatedByLabel:              k8ssandraapi.ReplicatedByLabelValue,
 						},
 					},
 				},

From cf37dfa5c26d22c5511f676e5333d633f7657429 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 8 Apr 2024 17:58:40 +1000
Subject: [PATCH 25/54] Fix ldflags variables for date and commit.

---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 2b5ffca2f..63bca0faf 100644
--- a/Makefile
+++ b/Makefile
@@ -177,7 +177,7 @@ DATE  ?= $(shell date)
 COMMIT ?= $(shell git rev-parse --short HEAD)
 
 build: generate fmt vet ## Build manager binary.
-	go build --ldflags "-X \"main.version=${VERSION}\" -X \"main.date=${DATE}\" -X \"main.commit=${COMMIT}\"" -o bin/manager main.go
+	go build --ldflags "-X \"main.version=${VERSION}\" -X \"main.date=$(DATE)\" -X \"main.commit=$(COMMIT)\"" -o bin/manager main.go
 
 run: manifests generate fmt vet ## Run a controller from your host.
 	go run ./main.go

From cfdd5083f3588628dadd5b241ff8822401457532 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 8 Apr 2024 18:12:46 +1000
Subject: [PATCH 26/54] Ensure the cluster name/namespace labels are applied
 correctly.

---
 controllers/k8ssandra/medusa_reconciler.go | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index bb231b2fa..8edafa8cd 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -346,8 +346,8 @@ func (r *K8ssandraClusterReconciler) reconcileRemoteBucketSecretsDeprecated(
 							medusaapi.MedusaStorageSecretIdentifierLabel,
 						},
 						AddLabels: map[string]string{
-							k8ssandraapi.K8ssandraClusterNameLabel:      kc.GetLabels()[k8ssandraapi.K8ssandraClusterNameLabel],
-							k8ssandraapi.K8ssandraClusterNamespaceLabel: kc.GetLabels()[k8ssandraapi.K8ssandraClusterNamespaceLabel],
+							k8ssandraapi.K8ssandraClusterNameLabel:      kc.Name,
+							k8ssandraapi.K8ssandraClusterNamespaceLabel: kc.Namespace,
 							k8ssandraapi.ReplicatedByLabel:              k8ssandraapi.ReplicatedByLabelValue,
 						},
 					},

From 67a379dd4446a821ee7ec6e5fab1759dbd22d8ed Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Tue, 9 Apr 2024 11:23:48 +1000
Subject: [PATCH 27/54] more trying to get ldflags working.

---
 Makefile | 1 +
 main.go  | 6 +++---
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
index 63bca0faf..95d090217 100644
--- a/Makefile
+++ b/Makefile
@@ -177,6 +177,7 @@ DATE  ?= $(shell date)
 COMMIT ?= $(shell git rev-parse --short HEAD)
 
 build: generate fmt vet ## Build manager binary.
+	echo "\"main.version=${VERSION}\" -X \"main.date=$(DATE)\" -X \"main.commit=$(COMMIT)\"" 
 	go build --ldflags "-X \"main.version=${VERSION}\" -X \"main.date=$(DATE)\" -X \"main.commit=$(COMMIT)\"" -o bin/manager main.go
 
 run: manifests generate fmt vet ## Run a controller from your host.
diff --git a/main.go b/main.go
index cd2557523..4b5321d24 100644
--- a/main.go
+++ b/main.go
@@ -68,9 +68,9 @@ import (
 )
 
 var (
-	version        = "dev"
-	commit         = "n/a"
-	date           = "n/a"
+	version        string
+	commit         string
+	date           string
 	versionMessage = "#######################" +
 		fmt.Sprintf("#### version %s commit %s date %s ####", version, commit, date) +
 		"#######################"

From 63eb086fadee44c403418cd33d0ae9356610e359 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Wed, 10 Apr 2024 12:23:01 +1000
Subject: [PATCH 28/54] More informative error message for a failing call to
 .Get().

---
 controllers/k8ssandra/medusa_reconciler.go | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index 8edafa8cd..3a358079a 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -259,7 +259,7 @@ func (r *K8ssandraClusterReconciler) mergeStorageProperties(
 	// End of block to be deprecated.
 
 	if err := remoteClient.Get(ctx, configKey, storageProperties); err != nil {
-		logger.Error(err, fmt.Sprintf("failed to get MedusaConfiguration %s", configKey))
+		logger.Error(err, "failed to get MedusaConfiguration", "MedusaConfigKey", configKey, "K8ssandraCluster", desiredKc)
 		return result.Error(err)
 	}
 	// check if the StorageProperties from the cluster have the prefix field set

From 34b0897000393e96c593b0fb061048ec7297383f Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Wed, 10 Apr 2024 12:42:24 +1000
Subject: [PATCH 29/54] Still trying to fix -ldflags.

---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 95d090217..049ec4330 100644
--- a/Makefile
+++ b/Makefile
@@ -178,7 +178,7 @@ COMMIT ?= $(shell git rev-parse --short HEAD)
 
 build: generate fmt vet ## Build manager binary.
 	echo "\"main.version=${VERSION}\" -X \"main.date=$(DATE)\" -X \"main.commit=$(COMMIT)\"" 
-	go build --ldflags "-X \"main.version=${VERSION}\" -X \"main.date=$(DATE)\" -X \"main.commit=$(COMMIT)\"" -o bin/manager main.go
+	go build -ldflags "-X \"main.version=${VERSION}\" -X \"main.date=$(DATE)\" -X \"main.commit=$(COMMIT)\"" -o bin/manager main.go
 
 run: manifests generate fmt vet ## Run a controller from your host.
 	go run ./main.go

From 61f305f65380507e2c1ec419e71436742c9e0fb8 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Wed, 10 Apr 2024 12:46:58 +1000
Subject: [PATCH 30/54] More informative error logging again.

---
 controllers/k8ssandra/medusa_reconciler.go | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index 3a358079a..1f69e7f20 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -259,7 +259,7 @@ func (r *K8ssandraClusterReconciler) mergeStorageProperties(
 	// End of block to be deprecated.
 
 	if err := remoteClient.Get(ctx, configKey, storageProperties); err != nil {
-		logger.Error(err, "failed to get MedusaConfiguration", "MedusaConfigKey", configKey, "K8ssandraCluster", desiredKc)
+		logger.Error(err, "failed to get MedusaConfiguration", "MedusaConfigKey", configKey, "K8ssandraCluster", desiredKc, "called with Namespace", namespace)
 		return result.Error(err)
 	}
 	// check if the StorageProperties from the cluster have the prefix field set

From f848502d9e1cf7b4a2ae4cea1235775e5b0da7d7 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Wed, 10 Apr 2024 12:48:50 +1000
Subject: [PATCH 31/54] Straighten out what namespaces are which in
 MedusaReconciler.

---
 controllers/k8ssandra/medusa_reconciler.go | 17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)

diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index 1f69e7f20..9c0603056 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -39,12 +39,12 @@ func (r *K8ssandraClusterReconciler) reconcileMedusa(
 	logger logr.Logger,
 ) result.ReconcileResult {
 	kc := desiredKc.DeepCopy()
-	namespace := utils.FirstNonEmptyString(dcConfig.Meta.Namespace, kc.Namespace)
-	logger.Info("Medusa reconcile for " + dcConfig.CassDcName() + " on namespace " + namespace)
+	dcNamespace := utils.FirstNonEmptyString(dcConfig.Meta.Namespace, kc.Namespace)
+	logger.Info("Medusa reconcile for " + dcConfig.CassDcName() + " on namespace " + dcNamespace)
 	if kc.Spec.Medusa != nil {
 		logger.Info("Medusa is enabled")
 
-		mergeResult := r.mergeStorageProperties(ctx, r.Client, namespace, kc.Spec.Medusa, logger, kc)
+		mergeResult := r.mergeStorageProperties(ctx, r.Client, kc.Spec.Medusa, logger, kc)
 		medusaSpec := kc.Spec.Medusa
 		if mergeResult.IsError() {
 			return result.Error(mergeResult.GetError())
@@ -68,7 +68,7 @@ func (r *K8ssandraClusterReconciler) reconcileMedusa(
 				return result.Error(fmt.Errorf("medusa storage secret is not defined for storage provider %s", medusaSpec.StorageProperties.StorageProvider))
 			}
 		}
-		if res := r.reconcileMedusaConfigMap(ctx, remoteClient, kc, dcConfig, logger, namespace); res.Completed() {
+		if res := r.reconcileMedusaConfigMap(ctx, remoteClient, kc, dcConfig, logger, dcNamespace); res.Completed() {
 			return res
 		}
 
@@ -86,7 +86,7 @@ func (r *K8ssandraClusterReconciler) reconcileMedusa(
 		}
 
 		// Create the Medusa standalone pod
-		desiredMedusaStandalone := medusa.StandaloneMedusaDeployment(*medusaContainer, kc.SanitizedName(), dcConfig.SanitizedName(), namespace, logger, kc.Spec.Medusa.ContainerImage)
+		desiredMedusaStandalone := medusa.StandaloneMedusaDeployment(*medusaContainer, kc.SanitizedName(), dcConfig.SanitizedName(), dcNamespace, logger, kc.Spec.Medusa.ContainerImage)
 
 		// Add the volumes previously computed to the Medusa standalone pod
 		for _, volume := range volumes {
@@ -117,7 +117,7 @@ func (r *K8ssandraClusterReconciler) reconcileMedusa(
 		}
 
 		// Create and reconcile the Medusa service for the standalone deployment
-		medusaService := medusa.StandaloneMedusaService(dcConfig, medusaSpec, kc.SanitizedName(), namespace, logger)
+		medusaService := medusa.StandaloneMedusaService(dcConfig, medusaSpec, kc.SanitizedName(), dcNamespace, logger)
 		medusaService.SetLabels(labels.CleanedUpByLabels(kcKey))
 		recRes = reconciliation.ReconcileObject(ctx, remoteClient, r.DefaultDelay, *medusaService)
 		switch {
@@ -243,7 +243,6 @@ func (r *K8ssandraClusterReconciler) reconcileMedusaConfigMap(
 func (r *K8ssandraClusterReconciler) mergeStorageProperties(
 	ctx context.Context,
 	remoteClient client.Client,
-	namespace string,
 	medusaSpec *medusaapi.MedusaClusterTemplate,
 	logger logr.Logger,
 	desiredKc *api.K8ssandraCluster,
@@ -254,12 +253,12 @@ func (r *K8ssandraClusterReconciler) mergeStorageProperties(
 	}
 	storageProperties := &medusaapi.MedusaConfiguration{}
 	// Deprecated: This code path can be removed at version 1.17, as MedusaConfigs should now always be namespace-local to the K8ssandraCluster referencing them.
-	configNamespace := utils.FirstNonEmptyString(medusaSpec.MedusaConfigurationRef.Namespace, namespace)
+	configNamespace := utils.FirstNonEmptyString(medusaSpec.MedusaConfigurationRef.Namespace, desiredKc.Namespace)
 	configKey := types.NamespacedName{Namespace: configNamespace, Name: medusaSpec.MedusaConfigurationRef.Name}
 	// End of block to be deprecated.
 
 	if err := remoteClient.Get(ctx, configKey, storageProperties); err != nil {
-		logger.Error(err, "failed to get MedusaConfiguration", "MedusaConfigKey", configKey, "K8ssandraCluster", desiredKc, "called with Namespace", namespace)
+		logger.Error(err, "failed to get MedusaConfiguration", "MedusaConfigKey", configKey, "K8ssandraCluster", desiredKc)
 		return result.Error(err)
 	}
 	// check if the StorageProperties from the cluster have the prefix field set

From 8814026d941cdd9fff2932db987d728b8d7af4bd Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Wed, 10 Apr 2024 15:07:33 +1000
Subject: [PATCH 32/54] We want to ensure that the secret gets replicated to
 both the KlusterNamespace (with a prefix) and also the DC1 namespace.

---
 test/e2e/cluster_scope_test.go | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/test/e2e/cluster_scope_test.go b/test/e2e/cluster_scope_test.go
index d4c5e0159..7cdd913e2 100644
--- a/test/e2e/cluster_scope_test.go
+++ b/test/e2e/cluster_scope_test.go
@@ -23,7 +23,8 @@ func multiDcMultiCluster(t *testing.T, ctx context.Context, klusterNamespace str
 
 	dc1Key := framework.ClusterKey{K8sContext: f.DataPlaneContexts[0], NamespacedName: types.NamespacedName{Namespace: dc1Namespace, Name: "dc1"}}
 	checkDatacenterReady(t, ctx, dc1Key, f)
-	checkBucketKeyPresent(t, f, ctx, dc1Namespace, dc1Key.K8sContext, k8ssandra)
+	checkBucketKeyPresent(t, f, ctx, klusterNamespace, dc1Key.K8sContext, k8ssandra)
+	checkBucketKeyPresent(t, f, ctx, dc1Key.Namespace, dc1Key.K8sContext, k8ssandra)
 
 	t.Log("check k8ssandra cluster status")
 	require.Eventually(func() bool {

From 48f85b853d9b403925d2a3e9c319b6ca2856cae1 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Thu, 11 Apr 2024 12:01:02 +1000
Subject: [PATCH 33/54] Fix hash method.

---
 pkg/utils/hash.go | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/pkg/utils/hash.go b/pkg/utils/hash.go
index 4ddbba793..4023b0b71 100644
--- a/pkg/utils/hash.go
+++ b/pkg/utils/hash.go
@@ -36,6 +36,7 @@ func DeepHashObject(hasher hash.Hash, objectToWrite interface{}) {
 }
 
 func HashNameNamespace(name, namespace string) string {
-	bytes := []byte(namespace + name)
-	return string(sha256.New().Sum(bytes)[:hashLength])
+	h := sha256.New()
+	h.Write([]byte(namespace + name))
+	return string(h.Sum(nil))[:hashLength]
 }

From 19962adec8498ae14764f3e64d701c4c1ecf326b Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Thu, 11 Apr 2024 12:04:12 +1000
Subject: [PATCH 34/54] Add owner reference so that ReplicatedSecret gets
 cleaned up.

---
 controllers/k8ssandra/medusa_reconciler.go | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index 9c0603056..b0a437149 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -356,6 +356,9 @@ func (r *K8ssandraClusterReconciler) reconcileRemoteBucketSecretsDeprecated(
 		if err := controllerutil.SetControllerReference(kc, &repSecret, r.Scheme); err != nil {
 			return result.Error(err)
 		}
+		if err := controllerutil.SetOwnerReference(kc, &repSecret, r.Scheme); err != nil {
+			return result.Error(err)
+		}
 		// TODO: this should also have finalizer logic included in the k8ssandraCluster finalizer to remove the replicated secret if it is no longer being used.
 		// TODO: this should probably have a finalizer on it too so that the replicatedSecret cannot be deleted.
 

From 85100ecbca43d6ee0471f91f6320019605e35f6e Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Thu, 11 Apr 2024 12:20:11 +1000
Subject: [PATCH 35/54] Eliminate redundant if statement, add more deprecation
 info in MedusaConfig controller.

---
 controllers/k8ssandra/medusa_reconciler.go    | 103 +++++++++---------
 .../medusa/medusaconfiguration_controller.go  |   2 +
 2 files changed, 51 insertions(+), 54 deletions(-)

diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index b0a437149..5cb2db045 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -305,69 +305,64 @@ func (r *K8ssandraClusterReconciler) reconcileRemoteBucketSecretsDeprecated(
 		return result.Continue()
 	}
 
-	if kc.Spec.Medusa.MedusaConfigurationRef.Namespace != kc.Namespace {
-		// This is the deprecated code path. Moving forward we will use a replicated secret with a prefix, but we will remove this code path after v1.17.
-		// fetch the referenced configuration
-		medusaConfigName := medusaSpec.MedusaConfigurationRef.Name
-		medusaConfigNamespace := utils.FirstNonEmptyString(medusaSpec.MedusaConfigurationRef.Namespace, kc.Namespace)
-		medusaConfigKey := types.NamespacedName{Namespace: medusaConfigNamespace, Name: medusaConfigName}
-		medusaConfig := &medusaapi.MedusaConfiguration{}
-		if err := c.Get(ctx, medusaConfigKey, medusaConfig); err != nil {
-			logger.Error(err, fmt.Sprintf("could not get MedusaConfiguration %s/%s", medusaConfigNamespace, medusaConfigName))
-			return result.Error(err)
-		}
+	// This is the deprecated code path. Moving forward we will use a replicated secret with a prefix, but we will remove this code path after v1.17.
+	// fetch the referenced configuration
+	medusaConfigName := medusaSpec.MedusaConfigurationRef.Name
+	medusaConfigNamespace := utils.FirstNonEmptyString(medusaSpec.MedusaConfigurationRef.Namespace, kc.Namespace)
+	medusaConfigKey := types.NamespacedName{Namespace: medusaConfigNamespace, Name: medusaConfigName}
+	medusaConfig := &medusaapi.MedusaConfiguration{}
+	if err := c.Get(ctx, medusaConfigKey, medusaConfig); err != nil {
+		logger.Error(err, fmt.Sprintf("could not get MedusaConfiguration %s/%s", medusaConfigNamespace, medusaConfigName))
+		return result.Error(err)
+	}
 
-		repSecret := replication.ReplicatedSecret{
-			ObjectMeta: metav1.ObjectMeta{
-				Name:      kc.GetClusterIdHash(8) + "-" + medusaConfig.Spec.StorageProperties.StorageSecretRef.Name,
-				Namespace: medusaConfigNamespace,
-				Labels: map[string]string{
-					k8ssandraapi.K8ssandraClusterNameLabel:      kc.Name,
-					k8ssandraapi.K8ssandraClusterNamespaceLabel: kc.Namespace,
-				},
-				Annotations: map[string]string{
-					k8ssandraapi.PurposeAnnotation: "This replicated secret is designed for the old codepath in the Medusa reconciler within the k8ssandra cluster controller. In this deprecated path, a MedusaConfig could reside in a different namespace to the K8ssandraCluster. This ReplicatedSecret ensures that the bucket secret is copied into the K8ssandraCluster's namespace. This codepath will be removed in v1.17.",
+	repSecret := replication.ReplicatedSecret{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:      kc.GetClusterIdHash(8) + "-" + medusaConfig.Spec.StorageProperties.StorageSecretRef.Name,
+			Namespace: medusaConfigNamespace,
+			Labels: map[string]string{
+				k8ssandraapi.K8ssandraClusterNameLabel:      kc.Name,
+				k8ssandraapi.K8ssandraClusterNamespaceLabel: kc.Namespace,
+			},
+			Annotations: map[string]string{
+				k8ssandraapi.PurposeAnnotation: "This replicated secret is designed for the old codepath in the Medusa reconciler within the k8ssandra cluster controller. In this deprecated path, a MedusaConfig could reside in a different namespace to the K8ssandraCluster. This ReplicatedSecret ensures that the bucket secret is copied into the K8ssandraCluster's namespace. This codepath will be removed in v1.17.",
+			},
+		},
+		Spec: replication.ReplicatedSecretSpec{
+			Selector: &metav1.LabelSelector{
+				MatchLabels: map[string]string{
+					medusaapi.MedusaStorageSecretIdentifierLabel: utils.HashNameNamespace(
+						medusaConfig.Spec.StorageProperties.StorageSecretRef.Name,
+						medusaConfigNamespace),
 				},
 			},
-			Spec: replication.ReplicatedSecretSpec{
-				Selector: &metav1.LabelSelector{
-					MatchLabels: map[string]string{
-						medusaapi.MedusaStorageSecretIdentifierLabel: utils.HashNameNamespace(
-							medusaConfig.Spec.StorageProperties.StorageSecretRef.Name,
-							medusaConfigNamespace),
+			ReplicationTargets: []replication.ReplicationTarget{
+				{
+					Namespace:    kc.Namespace,
+					TargetPrefix: kc.Name + "-",
+					DropLabels: []string{
+						medusaapi.MedusaStorageSecretIdentifierLabel,
 					},
-				},
-				ReplicationTargets: []replication.ReplicationTarget{
-					{
-						Namespace:    kc.Namespace,
-						TargetPrefix: kc.Name + "-",
-						DropLabels: []string{
-							medusaapi.MedusaStorageSecretIdentifierLabel,
-						},
-						AddLabels: map[string]string{
-							k8ssandraapi.K8ssandraClusterNameLabel:      kc.Name,
-							k8ssandraapi.K8ssandraClusterNamespaceLabel: kc.Namespace,
-							k8ssandraapi.ReplicatedByLabel:              k8ssandraapi.ReplicatedByLabelValue,
-						},
+					AddLabels: map[string]string{
+						k8ssandraapi.K8ssandraClusterNameLabel:      kc.Name,
+						k8ssandraapi.K8ssandraClusterNamespaceLabel: kc.Namespace,
+						k8ssandraapi.ReplicatedByLabel:              k8ssandraapi.ReplicatedByLabelValue,
 					},
 				},
 			},
-		}
-		if err := controllerutil.SetControllerReference(kc, &repSecret, r.Scheme); err != nil {
-			return result.Error(err)
-		}
-		if err := controllerutil.SetOwnerReference(kc, &repSecret, r.Scheme); err != nil {
-			return result.Error(err)
-		}
-		// TODO: this should also have finalizer logic included in the k8ssandraCluster finalizer to remove the replicated secret if it is no longer being used.
-		// TODO: this should probably have a finalizer on it too so that the replicatedSecret cannot be deleted.
+		},
+	}
+	if err := controllerutil.SetControllerReference(kc, &repSecret, r.Scheme); err != nil {
+		return result.Error(err)
+	}
+	if err := controllerutil.SetOwnerReference(kc, &repSecret, r.Scheme); err != nil {
+		return result.Error(err)
+	}
+	// TODO: this should also have finalizer logic included in the k8ssandraCluster finalizer to remove the replicated secret if it is no longer being used.
+	// TODO: this should probably have a finalizer on it too so that the replicatedSecret cannot be deleted.
 
-		return reconciliation.ReconcileObject(ctx, c, r.DefaultDelay, repSecret)
+	return reconciliation.ReconcileObject(ctx, c, r.DefaultDelay, repSecret)
 
-	} else {
-		// no-op, the bucket secret exists in the same namespace and doesn't need copying via a replicated secret.
-		return result.Continue()
-	}
 }
 
 func (r *K8ssandraClusterReconciler) getOperatorNamespace() string {
diff --git a/controllers/medusa/medusaconfiguration_controller.go b/controllers/medusa/medusaconfiguration_controller.go
index d0c1b7705..e5219c988 100644
--- a/controllers/medusa/medusaconfiguration_controller.go
+++ b/controllers/medusa/medusaconfiguration_controller.go
@@ -91,6 +91,7 @@ func (r *MedusaConfigurationReconciler) Reconcile(ctx context.Context, req ctrl.
 		if secret.Labels == nil {
 			secret.Labels = make(map[string]string)
 		}
+		// Deprecated: Eventually this should be removed in favour of adding replication labels to this resource. For now, we'll create the replica in any event.
 		if secret.Labels[medusav1alpha1.MedusaStorageSecretIdentifierLabel] != utils.HashNameNamespace(secret.Name, secret.Namespace) {
 			secret.Labels[medusav1alpha1.MedusaStorageSecretIdentifierLabel] = utils.HashNameNamespace(secret.Name, secret.Namespace)
 			if err = r.Client.Patch(ctx, secret, patch); err != nil {
@@ -98,6 +99,7 @@ func (r *MedusaConfigurationReconciler) Reconcile(ctx context.Context, req ctrl.
 				return ctrl.Result{}, err
 			}
 		}
+		// End deprecation
 	}
 
 	configuration.Status.SetCondition(medusav1alpha1.ControlStatusReady, metav1.ConditionTrue)

From ea039525327cd7cd7f19df380039094a0fb3a0d8 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Thu, 11 Apr 2024 12:23:09 +1000
Subject: [PATCH 36/54] Fix hash properly this time.

---
 pkg/utils/hash.go | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/pkg/utils/hash.go b/pkg/utils/hash.go
index 4023b0b71..02aaf3aad 100644
--- a/pkg/utils/hash.go
+++ b/pkg/utils/hash.go
@@ -3,6 +3,7 @@ package utils
 import (
 	"crypto/sha256"
 	"encoding/base64"
+	"fmt"
 	"hash"
 
 	"github.com/davecgh/go-spew/spew"
@@ -38,5 +39,5 @@ func DeepHashObject(hasher hash.Hash, objectToWrite interface{}) {
 func HashNameNamespace(name, namespace string) string {
 	h := sha256.New()
 	h.Write([]byte(namespace + name))
-	return string(h.Sum(nil))[:hashLength]
+	return fmt.Sprintf("%x", h.Sum(nil))[:hashLength]
 }

From 77fe9d5fd112c44e30fd0f4242856972e31d7d13 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Thu, 11 Apr 2024 14:02:47 +1000
Subject: [PATCH 37/54] Need a longer timeout on Medusa restore, since this
 takes in excess of 20 min at times.

---
 test/e2e/medusa_test.go | 2 +-
 test/e2e/suite_test.go  | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/test/e2e/medusa_test.go b/test/e2e/medusa_test.go
index 4ffaf5539..14752f432 100644
--- a/test/e2e/medusa_test.go
+++ b/test/e2e/medusa_test.go
@@ -286,7 +286,7 @@ func verifyRestoreJobFinished(t *testing.T, ctx context.Context, f *framework.E2
 		}
 
 		return !restore.Status.FinishTime.IsZero()
-	}, polling.medusaRestoreDone.timeout*2, polling.medusaRestoreDone.interval, "restore didn't finish within timeout")
+	}, polling.medusaRestoreDone.timeout, polling.medusaRestoreDone.interval, "restore didn't finish within timeout")
 
 	require.Eventually(func() bool {
 		dc := &cassdcapi.CassandraDatacenter{}
diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go
index d98a8206d..edde50001 100644
--- a/test/e2e/suite_test.go
+++ b/test/e2e/suite_test.go
@@ -712,7 +712,7 @@ func applyPollingDefaults() {
 	polling.medusaBackupDone.timeout = 10 * time.Minute
 	polling.medusaBackupDone.interval = 15 * time.Second
 
-	polling.medusaRestoreDone.timeout = 10 * time.Minute
+	polling.medusaRestoreDone.timeout = 25 * time.Minute
 	polling.medusaRestoreDone.interval = 15 * time.Second
 
 	polling.datacenterUpdating.timeout = 1 * time.Minute

From 43768933d93e21986bc718dc60ccc512fa086fc1 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Fri, 12 Apr 2024 10:17:17 +1000
Subject: [PATCH 38/54] Increase timeout.

---
 test/e2e/suite_test.go | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go
index edde50001..d98a8206d 100644
--- a/test/e2e/suite_test.go
+++ b/test/e2e/suite_test.go
@@ -712,7 +712,7 @@ func applyPollingDefaults() {
 	polling.medusaBackupDone.timeout = 10 * time.Minute
 	polling.medusaBackupDone.interval = 15 * time.Second
 
-	polling.medusaRestoreDone.timeout = 25 * time.Minute
+	polling.medusaRestoreDone.timeout = 10 * time.Minute
 	polling.medusaRestoreDone.interval = 15 * time.Second
 
 	polling.datacenterUpdating.timeout = 1 * time.Minute

From 45cc714fbe9e84ef1490698a4754822aa2ea2c2d Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Fri, 12 Apr 2024 13:23:37 +1000
Subject: [PATCH 39/54] Remove unused argument.

---
 apis/k8ssandra/v1alpha1/k8ssandracluster_types.go | 2 +-
 controllers/k8ssandra/medusa_reconciler.go        | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/apis/k8ssandra/v1alpha1/k8ssandracluster_types.go b/apis/k8ssandra/v1alpha1/k8ssandracluster_types.go
index fcf3205b0..4006e6402 100644
--- a/apis/k8ssandra/v1alpha1/k8ssandracluster_types.go
+++ b/apis/k8ssandra/v1alpha1/k8ssandracluster_types.go
@@ -543,6 +543,6 @@ func (kc *K8ssandraCluster) DefaultNumTokens(serverVersion *semver.Version) floa
 // this is to be used to name resources that are cluster specific in preference of concatenations
 // of the namespaced name, as the latter are becoming too long and causing issues due to DNS name length
 // constraints within k8s
-func (kc *K8ssandraCluster) GetClusterIdHash(nchars int) string {
+func (kc *K8ssandraCluster) GetClusterIdHash() string {
 	return utils.HashNameNamespace(kc.Name, kc.Namespace)
 }
diff --git a/controllers/k8ssandra/medusa_reconciler.go b/controllers/k8ssandra/medusa_reconciler.go
index 5cb2db045..b23bb0cc0 100644
--- a/controllers/k8ssandra/medusa_reconciler.go
+++ b/controllers/k8ssandra/medusa_reconciler.go
@@ -318,7 +318,7 @@ func (r *K8ssandraClusterReconciler) reconcileRemoteBucketSecretsDeprecated(
 
 	repSecret := replication.ReplicatedSecret{
 		ObjectMeta: metav1.ObjectMeta{
-			Name:      kc.GetClusterIdHash(8) + "-" + medusaConfig.Spec.StorageProperties.StorageSecretRef.Name,
+			Name:      kc.GetClusterIdHash() + "-" + medusaConfig.Spec.StorageProperties.StorageSecretRef.Name,
 			Namespace: medusaConfigNamespace,
 			Labels: map[string]string{
 				k8ssandraapi.K8ssandraClusterNameLabel:      kc.Name,

From 3509c9fb4eb37dc447b71963d98871069264851b Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Fri, 12 Apr 2024 15:02:25 +1000
Subject: [PATCH 40/54] Small refactor.

---
 controllers/replication/secret_controller.go | 31 ++++++++++----------
 1 file changed, 15 insertions(+), 16 deletions(-)

diff --git a/controllers/replication/secret_controller.go b/controllers/replication/secret_controller.go
index 690fec7a5..e52ee5b20 100644
--- a/controllers/replication/secret_controller.go
+++ b/controllers/replication/secret_controller.go
@@ -235,8 +235,7 @@ func (s *SecretSyncController) Reconcile(ctx context.Context, req ctrl.Request)
 					copiedSecret.ResourceVersion = ""
 					copiedSecret.OwnerReferences = []metav1.OwnerReference{}
 					copiedSecret.Name = getPrefixedSecretName(target.TargetPrefix, sec.Name)
-					stripDroppedLabels(copiedSecret, target.DropLabels)
-					addAdditionalLabels(copiedSecret, target.AddLabels)
+					copiedSecret.Labels = calculateTargetLabels(copiedSecret.Labels, target)
 					if err = remoteClient.Create(ctx, copiedSecret); err != nil {
 						logger.Error(err, "Failed to sync secret to target cluster", "Secret", copiedSecret.Name, "TargetContext", target)
 						break TargetSecrets
@@ -344,11 +343,14 @@ func syncSecrets(src, dest *corev1.Secret, target api.ReplicationTarget) {
 			dest.Labels[k] = v
 		}
 	}
-	stripDroppedLabels(dest, target.DropLabels)
-	addAdditionalLabels(dest, target.AddLabels)
+	// TODO: it would be nice at some point in future to remove the DC specific hardcoded stuff and
+	// filter DC specific stuff by setting dropLabels in the replicatedsecret resource.
+	dest.Labels = calculateTargetLabels(dest.Labels, target)
 }
 
 // filterValue verifies the annotation is not something datacenter specific
+// TODO: it would be nice at some point in future to remove the DC specific hardcoded stuff and
+// filter DC specific stuff by setting dropLabels in the replicatedsecret resource.
 func filterValue(key string) bool {
 	return strings.HasPrefix(key, "cassandra.datastax.com/")
 }
@@ -446,15 +448,6 @@ func getPrefixedSecretName(prefix string, secretName string) string {
 	return fmt.Sprintf("%s%s", prefix, secretName)
 }
 
-// stripDroppedLabels removes the labels from the secret that are defined in the dropLabels field.
-func stripDroppedLabels(secret *corev1.Secret, dropLabels []string) {
-	for key := range secret.Labels {
-		if contains(key, dropLabels) {
-			delete(secret.Labels, key)
-		}
-	}
-}
-
 func contains(s string, arr []string) bool {
 	for _, i := range arr {
 		if i == s {
@@ -464,8 +457,14 @@ func contains(s string, arr []string) bool {
 	return false
 }
 
-func addAdditionalLabels(secret *corev1.Secret, addLabels map[string]string) {
-	for k, v := range addLabels {
-		secret.Labels[k] = v
+func calculateTargetLabels(originalLabels map[string]string, target api.ReplicationTarget) map[string]string {
+	for k, v := range target.AddLabels {
+		originalLabels[k] = v
+	}
+	for key := range originalLabels {
+		if contains(key, target.DropLabels) {
+			delete(originalLabels, key)
+		}
 	}
+	return originalLabels
 }

From 05b7a894e51ad28e00b2c72c631bb671d82366e4 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Fri, 12 Apr 2024 15:31:04 +1000
Subject: [PATCH 41/54] Get a test going for the broken cleanup issues.

---
 .../replication/secret_controller_test.go     | 149 ++++++++++++++++++
 1 file changed, 149 insertions(+)

diff --git a/controllers/replication/secret_controller_test.go b/controllers/replication/secret_controller_test.go
index e9cd9c77a..ea68287e3 100644
--- a/controllers/replication/secret_controller_test.go
+++ b/controllers/replication/secret_controller_test.go
@@ -11,6 +11,7 @@ import (
 	"github.com/k8ssandra/k8ssandra-operator/pkg/config"
 	"github.com/k8ssandra/k8ssandra-operator/pkg/secret"
 	"k8s.io/apimachinery/pkg/runtime"
+	"k8s.io/apimachinery/pkg/util/rand"
 	"sigs.k8s.io/controller-runtime/pkg/cluster"
 	"sigs.k8s.io/controller-runtime/pkg/manager"
 
@@ -65,6 +66,7 @@ func TestSecretController(t *testing.T) {
 	t.Run("MultiClusterSyncSecretsTest", testEnv.ControllerTest(ctx, copySecretsFromClusterToCluster))
 	t.Run("VerifyFinalizerInMultiCluster", testEnv.ControllerTest(ctx, verifySecretIsDeleted))
 	t.Run("TargetSecretsPrefixTest", testEnv.ControllerTest(ctx, prefixedSecret))
+	t.Run("VerifySecretIsDeletedComplicated", testEnv.ControllerTest(ctx, verifySecretIsDeletedComplicated))
 }
 
 // copySecretsFromClusterToCluster Tests:
@@ -238,6 +240,153 @@ func verifySecretIsDeleted(t *testing.T, ctx context.Context, f *framework.Frame
 	}, timeout, interval)
 }
 
+// verifySecretIsDeleted checks that the finalizer functionality works in a very complicated use case.
+//   - Create secret and ReplicatedSecret with dropped labels, added labels and a different prefix.
+//   - Verify it is copied to another cluster, the local namespace-local cluster, and a namespace in the local cluster too
+//   - Delete ReplicatedSecret from main cluster
+//   - Verify the secrets are deleted from the remote cluster (but not local)
+func verifySecretIsDeletedComplicated(t *testing.T, ctx context.Context, f *framework.Framework, namespace string) {
+	require := require.New(t)
+	remoteClient := testEnv.Clients[f.DataPlaneContexts[targetCopyToCluster]]
+	localClient := f.Client
+	localNamespaceLocalCluster := "ns-" + framework.CleanupForKubernetes(rand.String(9))
+	remoteNamespaceLocalCluster := "ns-" + framework.CleanupForKubernetes(rand.String(9))
+	remoteNamespaceRemoteCluster := "ns-" + framework.CleanupForKubernetes(rand.String(9))
+
+	require.NoError(remoteClient.Create(ctx, &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: remoteNamespaceRemoteCluster}}))
+	require.NoError(localClient.Create(ctx, &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: localNamespaceLocalCluster}}))
+	require.NoError(localClient.Create(ctx, &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: remoteNamespaceLocalCluster}}))
+
+	sec := corev1.Secret{
+		ObjectMeta: metav1.ObjectMeta{
+			Namespace: localNamespaceLocalCluster,
+			Name:      "original-secret",
+			Labels: map[string]string{
+				"pickme":     "true",
+				"dropme":     "true",
+				"alwayshere": "true",
+			},
+		},
+		Type: "Opaque",
+		StringData: map[string]string{
+			"key": "value",
+		},
+	}
+	repSecret := api.ReplicatedSecret{
+		ObjectMeta: metav1.ObjectMeta{
+			Namespace: localNamespaceLocalCluster,
+			Name:      "replicated-secret",
+		},
+		Spec: api.ReplicatedSecretSpec{
+			Selector: &metav1.LabelSelector{
+				MatchLabels: map[string]string{
+					"pickme": "true",
+				},
+			},
+			ReplicationTargets: []api.ReplicationTarget{
+				{
+					//Local cluster local namespace
+					TargetPrefix: "targetprefix-",
+					Namespace:    localNamespaceLocalCluster,
+					DropLabels:   []string{"dropme", "pickme"},
+					AddLabels: map[string]string{
+						"addMe": "true",
+					},
+				},
+				{
+					//Local cluster remote namespace
+					Namespace:    remoteNamespaceLocalCluster,
+					TargetPrefix: "targetprefix-",
+					DropLabels:   []string{"dropme"},
+					AddLabels: map[string]string{
+						"addMe": "true",
+					},
+				},
+				{ //remote cluster remote namespace
+					K8sContextName: f.DataPlaneContexts[1],
+					Namespace:      remoteNamespaceRemoteCluster,
+					TargetPrefix:   "targetprefix-",
+					DropLabels:     []string{"dropme"},
+					AddLabels: map[string]string{
+						"addMe": "true",
+					},
+				},
+			},
+		},
+	}
+
+	require.NoError(localClient.Create(ctx, &sec), "failed to create original secret to main cluster")
+	require.NoError(localClient.Create(ctx, &repSecret), "failed to create replicatedsecret to main cluster")
+
+	t.Log("check that the secret was copied to local namespace local cluster")
+	require.Eventually(func() bool {
+		secret := &corev1.Secret{}
+		if err := localClient.Get(ctx, types.NamespacedName{Name: "targetprefix-original-secret", Namespace: localNamespaceLocalCluster}, secret); err != nil {
+			return false
+		}
+		return secret.Labels["alwayshere"] == "true" && secret.Labels["dropme"] == "" && secret.Labels["addMe"] == "true" && secret.Labels["pickme"] == ""
+	}, timeout, interval)
+
+	t.Log("check that the secret was copied to remote namespace local cluster")
+	require.Eventually(func() bool {
+		secret := &corev1.Secret{}
+		if err := localClient.Get(ctx, types.NamespacedName{Name: "targetprefix-original-secret", Namespace: remoteNamespaceLocalCluster}, secret); err != nil {
+			return false
+		}
+		return secret.Labels["alwayshere"] == "true" && secret.Labels["dropme"] == "" && secret.Labels["addMe"] == "true" && secret.Labels["pickme"] == ""
+	}, timeout, interval)
+
+	t.Log("check that the secret was copied to remote namespace remote cluster")
+	require.Eventually(func() bool {
+		secret := &corev1.Secret{}
+		if err := remoteClient.Get(ctx, types.NamespacedName{Name: "targetprefix-original-secret", Namespace: remoteNamespaceRemoteCluster}, secret); err != nil {
+			return false
+		}
+		return secret.Labels["alwayshere"] == "true" && secret.Labels["dropme"] == "" && secret.Labels["addMe"] == "true" && secret.Labels["pickme"] == ""
+	}, timeout, interval)
+
+	t.Log("delete the replicated secret")
+	err := f.Client.Delete(ctx, &repSecret)
+	require.NoError(err, "failed to delete replicated secret from main cluster")
+
+	t.Log("verify the replicated secrets are gone from the local cluster local namespace")
+	require.Eventually(func() bool {
+		remoteSecret := &corev1.Secret{}
+		if err := localClient.Get(context.TODO(), types.NamespacedName{Name: "targetprefix-original-secret", Namespace: localNamespaceLocalCluster}, remoteSecret); err != nil {
+			return errors.IsNotFound(err)
+		}
+		return false
+	}, timeout, interval)
+
+	t.Log("verify the replicated secrets are gone from the local cluster remote namespace")
+	require.Eventually(func() bool {
+		remoteSecret := &corev1.Secret{}
+		if err := localClient.Get(context.TODO(), types.NamespacedName{Name: "targetprefix-original-secret", Namespace: remoteNamespaceLocalCluster}, remoteSecret); err != nil {
+			return errors.IsNotFound(err)
+		}
+		return false
+	}, timeout, interval)
+
+	t.Log("verify the replicated secrets are gone from the remote cluster remote namespace")
+	require.Eventually(func() bool {
+		remoteSecret := &corev1.Secret{}
+
+		if err := remoteClient.Get(context.TODO(), types.NamespacedName{Name: "targetprefix-original-secret", Namespace: remoteNamespaceRemoteCluster}, remoteSecret); err != nil {
+			return errors.IsNotFound(err)
+		}
+		return false
+	}, timeout, interval)
+
+	t.Log("verify the original secret is NOT GONE from the local cluster local namespace")
+	require.Eventually(func() bool {
+		remoteSecret := &corev1.Secret{}
+		if err := localClient.Get(context.TODO(), types.NamespacedName{Name: "original-secret", Namespace: localNamespaceLocalCluster}, remoteSecret); err != nil {
+			return false
+		}
+		return true
+	}, timeout, interval)
+}
+
 // wrongClusterIgnoreCopy tests that the secrets are not copied if written to the target cluster instead of local cluster
 func wrongClusterIgnoreCopy(t *testing.T, ctx context.Context, f *framework.Framework, namespace string) {
 	require := require.New(t)

From 28080a7c9e2d290f667ed6be40e8b4c7ff058bd7 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Fri, 12 Apr 2024 17:24:34 +1000
Subject: [PATCH 42/54] QOL tweaks.

---
 .../replication/secret_controller_test.go        | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/controllers/replication/secret_controller_test.go b/controllers/replication/secret_controller_test.go
index ea68287e3..afb3439c0 100644
--- a/controllers/replication/secret_controller_test.go
+++ b/controllers/replication/secret_controller_test.go
@@ -30,7 +30,7 @@ import (
 )
 
 const (
-	timeout  = time.Second * 5
+	timeout  = time.Minute * 5
 	interval = time.Millisecond * 500
 )
 
@@ -62,10 +62,10 @@ func TestSecretController(t *testing.T) {
 	defer cancel()
 
 	// Secret controller tests
-	t.Run("SingleClusterDoNothingToSecretsTest", testEnv.ControllerTest(ctx, wrongClusterIgnoreCopy))
-	t.Run("MultiClusterSyncSecretsTest", testEnv.ControllerTest(ctx, copySecretsFromClusterToCluster))
-	t.Run("VerifyFinalizerInMultiCluster", testEnv.ControllerTest(ctx, verifySecretIsDeleted))
-	t.Run("TargetSecretsPrefixTest", testEnv.ControllerTest(ctx, prefixedSecret))
+	// t.Run("SingleClusterDoNothingToSecretsTest", testEnv.ControllerTest(ctx, wrongClusterIgnoreCopy))
+	// t.Run("MultiClusterSyncSecretsTest", testEnv.ControllerTest(ctx, copySecretsFromClusterToCluster))
+	// t.Run("VerifyFinalizerInMultiCluster", testEnv.ControllerTest(ctx, verifySecretIsDeleted))
+	// t.Run("TargetSecretsPrefixTest", testEnv.ControllerTest(ctx, prefixedSecret))
 	t.Run("VerifySecretIsDeletedComplicated", testEnv.ControllerTest(ctx, verifySecretIsDeletedComplicated))
 }
 
@@ -249,9 +249,9 @@ func verifySecretIsDeletedComplicated(t *testing.T, ctx context.Context, f *fram
 	require := require.New(t)
 	remoteClient := testEnv.Clients[f.DataPlaneContexts[targetCopyToCluster]]
 	localClient := f.Client
-	localNamespaceLocalCluster := "ns-" + framework.CleanupForKubernetes(rand.String(9))
-	remoteNamespaceLocalCluster := "ns-" + framework.CleanupForKubernetes(rand.String(9))
-	remoteNamespaceRemoteCluster := "ns-" + framework.CleanupForKubernetes(rand.String(9))
+	localNamespaceLocalCluster := "ns-ll" + framework.CleanupForKubernetes(rand.String(9))
+	remoteNamespaceLocalCluster := "ns-rl" + framework.CleanupForKubernetes(rand.String(10))
+	remoteNamespaceRemoteCluster := "ns-rr" + framework.CleanupForKubernetes(rand.String(11))
 
 	require.NoError(remoteClient.Create(ctx, &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: remoteNamespaceRemoteCluster}}))
 	require.NoError(localClient.Create(ctx, &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: localNamespaceLocalCluster}}))

From b29b800d048cf09b99e8d0bc5e17e9c94932e58c Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 15 Apr 2024 11:18:51 +1000
Subject: [PATCH 43/54] Restore some envtests I'd commented out.

---
 controllers/replication/secret_controller_test.go | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/controllers/replication/secret_controller_test.go b/controllers/replication/secret_controller_test.go
index afb3439c0..1f765b07b 100644
--- a/controllers/replication/secret_controller_test.go
+++ b/controllers/replication/secret_controller_test.go
@@ -62,10 +62,10 @@ func TestSecretController(t *testing.T) {
 	defer cancel()
 
 	// Secret controller tests
-	// t.Run("SingleClusterDoNothingToSecretsTest", testEnv.ControllerTest(ctx, wrongClusterIgnoreCopy))
-	// t.Run("MultiClusterSyncSecretsTest", testEnv.ControllerTest(ctx, copySecretsFromClusterToCluster))
-	// t.Run("VerifyFinalizerInMultiCluster", testEnv.ControllerTest(ctx, verifySecretIsDeleted))
-	// t.Run("TargetSecretsPrefixTest", testEnv.ControllerTest(ctx, prefixedSecret))
+	t.Run("SingleClusterDoNothingToSecretsTest", testEnv.ControllerTest(ctx, wrongClusterIgnoreCopy))
+	t.Run("MultiClusterSyncSecretsTest", testEnv.ControllerTest(ctx, copySecretsFromClusterToCluster))
+	t.Run("VerifyFinalizerInMultiCluster", testEnv.ControllerTest(ctx, verifySecretIsDeleted))
+	t.Run("TargetSecretsPrefixTest", testEnv.ControllerTest(ctx, prefixedSecret))
 	t.Run("VerifySecretIsDeletedComplicated", testEnv.ControllerTest(ctx, verifySecretIsDeletedComplicated))
 }
 

From 4b48bc3e807393e387442500d9399411e9984a29 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 15 Apr 2024 12:30:26 +1000
Subject: [PATCH 44/54] Test now failing in the right place...

---
 controllers/replication/secret_controller_test.go | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/controllers/replication/secret_controller_test.go b/controllers/replication/secret_controller_test.go
index 1f765b07b..b1c40870e 100644
--- a/controllers/replication/secret_controller_test.go
+++ b/controllers/replication/secret_controller_test.go
@@ -250,8 +250,8 @@ func verifySecretIsDeletedComplicated(t *testing.T, ctx context.Context, f *fram
 	remoteClient := testEnv.Clients[f.DataPlaneContexts[targetCopyToCluster]]
 	localClient := f.Client
 	localNamespaceLocalCluster := "ns-ll" + framework.CleanupForKubernetes(rand.String(9))
-	remoteNamespaceLocalCluster := "ns-rl" + framework.CleanupForKubernetes(rand.String(10))
-	remoteNamespaceRemoteCluster := "ns-rr" + framework.CleanupForKubernetes(rand.String(11))
+	remoteNamespaceLocalCluster := "ns-rl" + framework.CleanupForKubernetes(rand.String(9))
+	remoteNamespaceRemoteCluster := "ns-rr" + framework.CleanupForKubernetes(rand.String(9))
 
 	require.NoError(remoteClient.Create(ctx, &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: remoteNamespaceRemoteCluster}}))
 	require.NoError(localClient.Create(ctx, &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: localNamespaceLocalCluster}}))
@@ -333,7 +333,7 @@ func verifySecretIsDeletedComplicated(t *testing.T, ctx context.Context, f *fram
 		if err := localClient.Get(ctx, types.NamespacedName{Name: "targetprefix-original-secret", Namespace: remoteNamespaceLocalCluster}, secret); err != nil {
 			return false
 		}
-		return secret.Labels["alwayshere"] == "true" && secret.Labels["dropme"] == "" && secret.Labels["addMe"] == "true" && secret.Labels["pickme"] == ""
+		return secret.Labels["alwayshere"] == "true" && secret.Labels["dropme"] == "" && secret.Labels["addMe"] == "true" && secret.Labels["pickme"] == "true"
 	}, timeout, interval)
 
 	t.Log("check that the secret was copied to remote namespace remote cluster")
@@ -342,7 +342,7 @@ func verifySecretIsDeletedComplicated(t *testing.T, ctx context.Context, f *fram
 		if err := remoteClient.Get(ctx, types.NamespacedName{Name: "targetprefix-original-secret", Namespace: remoteNamespaceRemoteCluster}, secret); err != nil {
 			return false
 		}
-		return secret.Labels["alwayshere"] == "true" && secret.Labels["dropme"] == "" && secret.Labels["addMe"] == "true" && secret.Labels["pickme"] == ""
+		return secret.Labels["alwayshere"] == "true" && secret.Labels["dropme"] == "" && secret.Labels["addMe"] == "true" && secret.Labels["pickme"] == "true"
 	}, timeout, interval)
 
 	t.Log("delete the replicated secret")

From b0fe006a7fe7ad9fdd0e5b7744f89aaabd2b94a8 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 15 Apr 2024 12:37:55 +1000
Subject: [PATCH 45/54] Fix issue where original secrets get deleted, and
 target secrets don't.

---
 controllers/replication/secret_controller.go | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/controllers/replication/secret_controller.go b/controllers/replication/secret_controller.go
index e52ee5b20..078b0a979 100644
--- a/controllers/replication/secret_controller.go
+++ b/controllers/replication/secret_controller.go
@@ -81,14 +81,14 @@ func (s *SecretSyncController) Reconcile(ctx context.Context, req ctrl.Request)
 					logger.Error(err, "Failed to delete the replicated secret, defined labels are invalid", "ReplicatedSecret", req.NamespacedName)
 					return reconcile.Result{}, err
 				}
-
+				// These are the secrets which are currently being replicated by THIS ReplicatedSecret in the local namespace.
 				secrets, err := s.fetchAllMatchingSecrets(ctx, selector)
 				if err != nil {
 					logger.Error(err, "Failed to fetch the replicated secrets to cleanup", "ReplicatedSecret", req.NamespacedName)
 					return reconcile.Result{}, err
 				}
 
-				secretsToDelete := make([]*corev1.Secret, 0, len(secrets))
+				sourceSecretsToMapToTargets := make([]*corev1.Secret, 0, len(secrets))
 
 				s.selectorMutex.RLock()
 
@@ -118,24 +118,24 @@ func (s *SecretSyncController) Reconcile(ctx context.Context, req ctrl.Request)
 						}
 					}
 					logger.Info("Preparing to delete secret", "key", key)
-					secretsToDelete = append(secretsToDelete, &sec)
+					sourceSecretsToMapToTargets = append(sourceSecretsToMapToTargets, &sec)
 				}
 
 				s.selectorMutex.RUnlock()
 
 				for _, target := range rsec.Spec.ReplicationTargets {
 					logger.Info("Deleting secrets for ReplicationTarget", "Target", target)
-
 					// Only replicate to clusters that are in the ReplicatedSecret's context
 					remoteClient, err := s.ClientCache.GetRemoteClient(target.K8sContextName)
 					if err != nil {
 						logger.Error(err, "Failed to fetch remote client for managed cluster", "ReplicatedSecret", req.NamespacedName, "TargetContext", target)
 						return ctrl.Result{}, err
 					}
-					for _, deleteKey := range secretsToDelete {
-						logger.Info("Deleting secret", "key", client.ObjectKey{Namespace: deleteKey.Namespace, Name: deleteKey.Name},
+					for _, origSecret := range sourceSecretsToMapToTargets {
+						deleteObject := &corev1.Secret{ObjectMeta: metav1.ObjectMeta{Name: getPrefixedSecretName(target.TargetPrefix, origSecret.Name), Namespace: target.Namespace}}
+						logger.Info("Deleting secrets for", "objectMeta", deleteObject.ObjectMeta,
 							"Cluster", target.K8sContextName)
-						err = remoteClient.Delete(ctx, deleteKey)
+						err = remoteClient.Delete(ctx, deleteObject)
 						if err != nil && !errors.IsNotFound(err) {
 							logger.Error(err, "Failed to remove secrets from target cluster", "ReplicatedSecret", req.NamespacedName, "TargetContext", target)
 							return ctrl.Result{}, err

From ea0ac6a939f312c9984b8a5b6f21c7aa293e9b9e Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 15 Apr 2024 13:17:39 +1000
Subject: [PATCH 46/54] Fix a bug where a ReplicatedSecret was going
 cluster-wide. They should only be replicating secrets from their local
 namespace.

---
 controllers/replication/secret_controller.go | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/controllers/replication/secret_controller.go b/controllers/replication/secret_controller.go
index 078b0a979..8777a28e2 100644
--- a/controllers/replication/secret_controller.go
+++ b/controllers/replication/secret_controller.go
@@ -82,7 +82,7 @@ func (s *SecretSyncController) Reconcile(ctx context.Context, req ctrl.Request)
 					return reconcile.Result{}, err
 				}
 				// These are the secrets which are currently being replicated by THIS ReplicatedSecret in the local namespace.
-				secrets, err := s.fetchAllMatchingSecrets(ctx, selector)
+				secrets, err := s.fetchAllMatchingSecrets(ctx, selector, req.Namespace)
 				if err != nil {
 					logger.Error(err, "Failed to fetch the replicated secrets to cleanup", "ReplicatedSecret", req.NamespacedName)
 					return reconcile.Result{}, err
@@ -117,7 +117,7 @@ func (s *SecretSyncController) Reconcile(ctx context.Context, req ctrl.Request)
 							continue SecretsToCheck
 						}
 					}
-					logger.Info("Preparing to delete secret", "key", key)
+					logger.Info("Preparing to delete secrets downstream from", "key", key)
 					sourceSecretsToMapToTargets = append(sourceSecretsToMapToTargets, &sec)
 				}
 
@@ -137,7 +137,7 @@ func (s *SecretSyncController) Reconcile(ctx context.Context, req ctrl.Request)
 							"Cluster", target.K8sContextName)
 						err = remoteClient.Delete(ctx, deleteObject)
 						if err != nil && !errors.IsNotFound(err) {
-							logger.Error(err, "Failed to remove secrets from target cluster", "ReplicatedSecret", req.NamespacedName, "TargetContext", target)
+							logger.Error(err, "Failed to remove secrets from target cluster", "ReplicatedSecret", req.NamespacedName, "TargetContext", target, "targetSecret", deleteObject.ObjectMeta)
 							return ctrl.Result{}, err
 						}
 					}
@@ -177,7 +177,7 @@ func (s *SecretSyncController) Reconcile(ctx context.Context, req ctrl.Request)
 	s.selectorMutex.Unlock()
 
 	// Fetch all the secrets that match the ReplicatedSecret's rules
-	secrets, err := s.fetchAllMatchingSecrets(ctx, selector)
+	secrets, err := s.fetchAllMatchingSecrets(ctx, selector, req.Namespace)
 	if err != nil {
 		logger.Error(err, "Failed to fetch linked secrets", "ReplicatedSecret", req.NamespacedName)
 		return reconcile.Result{Requeue: true}, err
@@ -367,10 +367,11 @@ func (s *SecretSyncController) verifyHashAnnotation(ctx context.Context, sec *co
 	return nil
 }
 
-func (s *SecretSyncController) fetchAllMatchingSecrets(ctx context.Context, selector labels.Selector) ([]corev1.Secret, error) {
+func (s *SecretSyncController) fetchAllMatchingSecrets(ctx context.Context, selector labels.Selector, namespace string) ([]corev1.Secret, error) {
 	secrets := &corev1.SecretList{}
 	listOption := client.ListOptions{
 		LabelSelector: selector,
+		Namespace:     namespace,
 	}
 	err := s.ClientCache.GetLocalClient().List(ctx, secrets, &listOption)
 	if err != nil {

From bf43fb0603271a2861d9c51c6a77623752010a69 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 15 Apr 2024 13:34:13 +1000
Subject: [PATCH 47/54] Small rebase error fixed.

---
 CHANGELOG/CHANGELOG-1.14.md | 1 -
 1 file changed, 1 deletion(-)

diff --git a/CHANGELOG/CHANGELOG-1.14.md b/CHANGELOG/CHANGELOG-1.14.md
index 37c98b386..7ee94963a 100644
--- a/CHANGELOG/CHANGELOG-1.14.md
+++ b/CHANGELOG/CHANGELOG-1.14.md
@@ -22,7 +22,6 @@ When cutting a new release, update the `unreleased` heading to the tag being gen
 * [BUGFIX] [#1217](https://github.com/k8ssandra/k8ssandra-operator/issues/1217) Medusa storage secrets now use a ReplicatedSecret for synchronization, fixing an issue where changes to the secrets were not propagating. Additionally, fix a number of issues with local testing on ARM Macs.
 * [BUGFIX] [#1253](https://github.com/k8ssandra/k8ssandra-operator/issues/1253) Medusa storage secrets are now labelled with a unique label.
 * [FEATURE] [#1260](https://github.com/k8ssandra/k8ssandra-operator/issues/1260) Update controller-gen to version 0.14.0.
-* [CHANGE] [#1266](https://github.com/k8ssandra/k8ssandra-operator/issues/1266) Deprecate the usage of non-namespace-local MedusaConfigurations within a K8ssandraCluster, and throw an error from the webhook when users attempt to use one.
 * [FEATURE] [#1242](https://github.com/k8ssandra/k8ssandra-operator/issues/1242) Allow for creation of replicated secrets with a prefix, so that we can distinguish between multiple secrets with the same origin but targeting different clusters.
 * [BUGFIX] [#1226](https://github.com/k8ssandra/k8ssandra-operator/issues/1226) Medusa purge cronjob should be created in the operator namespace
 * [BUGFIX] [#1141](https://github.com/k8ssandra/k8ssandra-operator/issues/1141) Use DC name override when naming secondary resources

From a0c23c797b284a59a4d9b1d71f240905bb9d1511 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 15 Apr 2024 13:58:14 +1000
Subject: [PATCH 48/54] Ensure that the original secret isn't cleaned up if it
 was never replicated due to having no target prefix.

---
 controllers/replication/secret_controller.go | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/controllers/replication/secret_controller.go b/controllers/replication/secret_controller.go
index 8777a28e2..46f8c65b5 100644
--- a/controllers/replication/secret_controller.go
+++ b/controllers/replication/secret_controller.go
@@ -133,6 +133,10 @@ func (s *SecretSyncController) Reconcile(ctx context.Context, req ctrl.Request)
 					}
 					for _, origSecret := range sourceSecretsToMapToTargets {
 						deleteObject := &corev1.Secret{ObjectMeta: metav1.ObjectMeta{Name: getPrefixedSecretName(target.TargetPrefix, origSecret.Name), Namespace: target.Namespace}}
+						if origSecret.Namespace == target.Namespace && origSecret.Name == deleteObject.Name {
+							// Target is the same secret as the original - bail.
+							continue
+						}
 						logger.Info("Deleting secrets for", "objectMeta", deleteObject.ObjectMeta,
 							"Cluster", target.K8sContextName)
 						err = remoteClient.Delete(ctx, deleteObject)

From d9918bc839eb1995486c7db0cdfec2dcc526c078 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 15 Apr 2024 14:33:55 +1000
Subject: [PATCH 49/54] Add another TODO, cleanup here is still going to be
 broken under some circumstances.

---
 controllers/replication/secret_controller.go | 1 +
 1 file changed, 1 insertion(+)

diff --git a/controllers/replication/secret_controller.go b/controllers/replication/secret_controller.go
index 46f8c65b5..eaaae2184 100644
--- a/controllers/replication/secret_controller.go
+++ b/controllers/replication/secret_controller.go
@@ -135,6 +135,7 @@ func (s *SecretSyncController) Reconcile(ctx context.Context, req ctrl.Request)
 						deleteObject := &corev1.Secret{ObjectMeta: metav1.ObjectMeta{Name: getPrefixedSecretName(target.TargetPrefix, origSecret.Name), Namespace: target.Namespace}}
 						if origSecret.Namespace == target.Namespace && origSecret.Name == deleteObject.Name {
 							// Target is the same secret as the original - bail.
+							// TODO: Note that this will cause secrets to not be cleaned up if they are in a remote cluster.
 							continue
 						}
 						logger.Info("Deleting secrets for", "objectMeta", deleteObject.ObjectMeta,

From d58cc83f8a0e6d84f1da5e0d240bba7b72543886 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 15 Apr 2024 15:08:28 +1000
Subject: [PATCH 50/54] Fix issue where sometimes we'd try to cleanup a secret
 with no namespace.

---
 controllers/replication/secret_controller.go | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/controllers/replication/secret_controller.go b/controllers/replication/secret_controller.go
index eaaae2184..ac9649e76 100644
--- a/controllers/replication/secret_controller.go
+++ b/controllers/replication/secret_controller.go
@@ -132,7 +132,8 @@ func (s *SecretSyncController) Reconcile(ctx context.Context, req ctrl.Request)
 						return ctrl.Result{}, err
 					}
 					for _, origSecret := range sourceSecretsToMapToTargets {
-						deleteObject := &corev1.Secret{ObjectMeta: metav1.ObjectMeta{Name: getPrefixedSecretName(target.TargetPrefix, origSecret.Name), Namespace: target.Namespace}}
+						targetNamespace := utils.FirstNonEmptyString(target.Namespace, origSecret.Namespace)
+						deleteObject := &corev1.Secret{ObjectMeta: metav1.ObjectMeta{Name: getPrefixedSecretName(target.TargetPrefix, origSecret.Name), Namespace: targetNamespace}}
 						if origSecret.Namespace == target.Namespace && origSecret.Name == deleteObject.Name {
 							// Target is the same secret as the original - bail.
 							// TODO: Note that this will cause secrets to not be cleaned up if they are in a remote cluster.

From 0417bafcdaf56bfc6034c3a39893edf32e0e18f1 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Mon, 15 Apr 2024 16:47:59 +1000
Subject: [PATCH 51/54] Back out namespace scoping change. This needs to be in
 a separate PR.

---
 controllers/replication/secret_controller.go      | 7 +++----
 controllers/replication/secret_controller_test.go | 4 ++--
 2 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/controllers/replication/secret_controller.go b/controllers/replication/secret_controller.go
index ac9649e76..697d744ca 100644
--- a/controllers/replication/secret_controller.go
+++ b/controllers/replication/secret_controller.go
@@ -82,7 +82,7 @@ func (s *SecretSyncController) Reconcile(ctx context.Context, req ctrl.Request)
 					return reconcile.Result{}, err
 				}
 				// These are the secrets which are currently being replicated by THIS ReplicatedSecret in the local namespace.
-				secrets, err := s.fetchAllMatchingSecrets(ctx, selector, req.Namespace)
+				secrets, err := s.fetchAllMatchingSecrets(ctx, selector)
 				if err != nil {
 					logger.Error(err, "Failed to fetch the replicated secrets to cleanup", "ReplicatedSecret", req.NamespacedName)
 					return reconcile.Result{}, err
@@ -183,7 +183,7 @@ func (s *SecretSyncController) Reconcile(ctx context.Context, req ctrl.Request)
 	s.selectorMutex.Unlock()
 
 	// Fetch all the secrets that match the ReplicatedSecret's rules
-	secrets, err := s.fetchAllMatchingSecrets(ctx, selector, req.Namespace)
+	secrets, err := s.fetchAllMatchingSecrets(ctx, selector)
 	if err != nil {
 		logger.Error(err, "Failed to fetch linked secrets", "ReplicatedSecret", req.NamespacedName)
 		return reconcile.Result{Requeue: true}, err
@@ -373,11 +373,10 @@ func (s *SecretSyncController) verifyHashAnnotation(ctx context.Context, sec *co
 	return nil
 }
 
-func (s *SecretSyncController) fetchAllMatchingSecrets(ctx context.Context, selector labels.Selector, namespace string) ([]corev1.Secret, error) {
+func (s *SecretSyncController) fetchAllMatchingSecrets(ctx context.Context, selector labels.Selector) ([]corev1.Secret, error) {
 	secrets := &corev1.SecretList{}
 	listOption := client.ListOptions{
 		LabelSelector: selector,
-		Namespace:     namespace,
 	}
 	err := s.ClientCache.GetLocalClient().List(ctx, secrets, &listOption)
 	if err != nil {
diff --git a/controllers/replication/secret_controller_test.go b/controllers/replication/secret_controller_test.go
index b1c40870e..53aa27611 100644
--- a/controllers/replication/secret_controller_test.go
+++ b/controllers/replication/secret_controller_test.go
@@ -297,7 +297,7 @@ func verifySecretIsDeletedComplicated(t *testing.T, ctx context.Context, f *fram
 					//Local cluster remote namespace
 					Namespace:    remoteNamespaceLocalCluster,
 					TargetPrefix: "targetprefix-",
-					DropLabels:   []string{"dropme"},
+					DropLabels:   []string{"dropme", "pickme"},
 					AddLabels: map[string]string{
 						"addMe": "true",
 					},
@@ -405,7 +405,7 @@ func wrongClusterIgnoreCopy(t *testing.T, ctx context.Context, f *framework.Fram
 	for _, s := range generatedSecrets {
 		require.Never(func() bool {
 			return verifySecretCopied(t, ctx, f.DataPlaneContexts[targetCopyToCluster], s, nil)
-		}, timeout, interval)
+		}, 3, interval)
 	}
 }
 

From 064641301d1388f8f0e0e437aff5e1adc3069e72 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Tue, 16 Apr 2024 10:44:13 +1000
Subject: [PATCH 52/54] Line tests back up with current functionality.

---
 controllers/replication/secret_controller_test.go | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/controllers/replication/secret_controller_test.go b/controllers/replication/secret_controller_test.go
index 53aa27611..65a34019e 100644
--- a/controllers/replication/secret_controller_test.go
+++ b/controllers/replication/secret_controller_test.go
@@ -333,7 +333,7 @@ func verifySecretIsDeletedComplicated(t *testing.T, ctx context.Context, f *fram
 		if err := localClient.Get(ctx, types.NamespacedName{Name: "targetprefix-original-secret", Namespace: remoteNamespaceLocalCluster}, secret); err != nil {
 			return false
 		}
-		return secret.Labels["alwayshere"] == "true" && secret.Labels["dropme"] == "" && secret.Labels["addMe"] == "true" && secret.Labels["pickme"] == "true"
+		return secret.Labels["alwayshere"] == "true" && secret.Labels["dropme"] == "" && secret.Labels["addMe"] == "true" && secret.Labels["pickme"] == ""
 	}, timeout, interval)
 
 	t.Log("check that the secret was copied to remote namespace remote cluster")

From 8b097cfd736eeffd98d060a1c5b1d3f42edf8d50 Mon Sep 17 00:00:00 2001
From: Miles Garnsey <11435896+Miles-Garnsey@users.noreply.github.com>
Date: Tue, 16 Apr 2024 16:12:31 +1000
Subject: [PATCH 53/54] Update CHANGELOG/RELEASE-NOTES.md

Co-authored-by: Alexander Dejanovski <alex.dejanovski@datastax.com>
---
 CHANGELOG/RELEASE-NOTES.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/CHANGELOG/RELEASE-NOTES.md b/CHANGELOG/RELEASE-NOTES.md
index 4d234c9e8..1bb01fdf1 100644
--- a/CHANGELOG/RELEASE-NOTES.md
+++ b/CHANGELOG/RELEASE-NOTES.md
@@ -1,6 +1,6 @@
 # k8ssandra-operator - Release Notes
 
-## v1.14.0
+## v1.15.0
 
 ### Deprecation of non-namespace-local MedusaConfigRef
 

From 4e73cb4dbab0d5273e66768779df81f29600c97b Mon Sep 17 00:00:00 2001
From: Miles Garnsey <miles.garnsey@datastax.com>
Date: Tue, 16 Apr 2024 16:16:40 +1000
Subject: [PATCH 54/54] Ensure changes in right CHANGELOG file.

---
 CHANGELOG/CHANGELOG-1.14.md | 7 +++----
 CHANGELOG/CHANGELOG-1.15.md | 5 +++--
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/CHANGELOG/CHANGELOG-1.14.md b/CHANGELOG/CHANGELOG-1.14.md
index 7ee94963a..7e981338f 100644
--- a/CHANGELOG/CHANGELOG-1.14.md
+++ b/CHANGELOG/CHANGELOG-1.14.md
@@ -18,10 +18,9 @@ When cutting a new release, update the `unreleased` heading to the tag being gen
 * [FEATURE] [#1280](https://github.com/k8ssandra/k8ssandra-operator/issues/1280) Env variables DEFAULT_REGISTRY and IMAGE_PULL_SECRETS allow overriding the default imagePullSecrets as well as the default registry to use when deploying medusa/reaper/stargate.
 * [BUGFIX] [#1253](https://github.com/k8ssandra/k8ssandra-operator/issues/1253) Medusa storage secrets are now labelled with a unique label.
 * [BUGFIX] [#1240](https://github.com/k8ssandra/k8ssandra-operator/issues/1240) The PullSecretRef for medusa is ignored in the standalone deployment of medusa
-* [BUGFIX] [#1266](https://github.com/k8ssandra/k8ssandra-operator/issues/1266) MedusaConfigurations must now be namespace local to the K8ssandraCluster they are attached to. Additionally, ReplicatedSecrets should only pick up secrets from their local namespace to replicate.
-* [BUGFIX] [#1217](https://github.com/k8ssandra/k8ssandra-operator/issues/1217) Medusa storage secrets now use a ReplicatedSecret for synchronization, fixing an issue where changes to the secrets were not propagating. Additionally, fix a number of issues with local testing on ARM Macs.
-* [BUGFIX] [#1253](https://github.com/k8ssandra/k8ssandra-operator/issues/1253) Medusa storage secrets are now labelled with a unique label.
-* [FEATURE] [#1260](https://github.com/k8ssandra/k8ssandra-operator/issues/1260) Update controller-gen to version 0.14.0.
+
+## v1.14.0 - 2024-04-02
+
 * [FEATURE] [#1242](https://github.com/k8ssandra/k8ssandra-operator/issues/1242) Allow for creation of replicated secrets with a prefix, so that we can distinguish between multiple secrets with the same origin but targeting different clusters.
 * [BUGFIX] [#1226](https://github.com/k8ssandra/k8ssandra-operator/issues/1226) Medusa purge cronjob should be created in the operator namespace
 * [BUGFIX] [#1141](https://github.com/k8ssandra/k8ssandra-operator/issues/1141) Use DC name override when naming secondary resources
diff --git a/CHANGELOG/CHANGELOG-1.15.md b/CHANGELOG/CHANGELOG-1.15.md
index 082195ce5..d3dbf8287 100644
--- a/CHANGELOG/CHANGELOG-1.15.md
+++ b/CHANGELOG/CHANGELOG-1.15.md
@@ -14,6 +14,7 @@ Changelog for the K8ssandra Operator, new PRs should update the `unreleased` sec
 When cutting a new release, update the `unreleased` heading to the tag being generated and date, like `## vX.Y.Z - YYYY-MM-DD` and create a new placeholder section for  `unreleased` entries.
 
 ## unreleased
+* [BUGFIX] [#1266](https://github.com/k8ssandra/k8ssandra-operator/issues/1266) MedusaConfigurations must now be namespace local to the K8ssandraCluster they are attached to, a webhook error will be thrown otherwise (for new clusters only). Additionally, ReplicatedSecrets should only pick up secrets from their local namespace to replicate.
+* [BUGFIX] [#1217](https://github.com/k8ssandra/k8ssandra-operator/issues/1217) Medusa storage secrets now use a ReplicatedSecret for synchronization, fixing an issue where changes to the secrets were not propagating. Additionally, fix a number of issues with local testing on ARM Macs.
 * [BUGFIX] [#1253](https://github.com/k8ssandra/k8ssandra-operator/issues/1253) Medusa storage secrets are now labelled with a unique label.
-* [FEATURE] [#1260](https://github.com/k8ssandra/k8ssandra-operator/issues/1260) Update controller-gen to version 0.14.0.
-* [CHANGE] [#1266](https://github.com/k8ssandra/k8ssandra-operator/issues/1266) Deprecate the usage of non-namespace-local MedusaConfigurations within a K8ssandraCluster, and throw an error from the webhook when users attempt to use one.
\ No newline at end of file
+* [FEATURE] [#1260](https://github.com/k8ssandra/k8ssandra-operator/issues/1260) Update controller-gen to version 0.14.0.
\ No newline at end of file