[SPARK-57131] Provide configuration option to specify ConfigMap name for dynamic configs

[csviri]

What changes were proposed in this pull request?

This PR makes the dynamic-config ConfigMap name an explicit, configurable property and enforces it in the reconciler:

1. Adds a new config option spark.kubernetes.operator.dynamicConfig.name (default: spark-kubernetes-operator-dynamic-configuration) in SparkOperatorConf.
2. In SparkOperatorConfigMapReconciler#reconcile, short-circuits with UpdateControl.noUpdate() when the incoming ConfigMap's name does not match the configured value — so only the operator's intended ConfigMap can mutate
   SparkOperatorConfManager state and trigger a watched-namespace update.
3. Documents the new property in docs/config_properties.md.
4. Adds unit tests covering the new check (positive, negative, and custom-name path).

Why are the changes needed?

Previously the reconciler trusted any ConfigMap that matched the label selector configured via spark.kubernetes.operator.dynamicConfig.selector in the operator namespace. A user with create configmap permission in the
operator namespace (but no broader operator privileges) could create a ConfigMap that matches the selector to:

• Override spark.kubernetes.operator.watchedNamespaces and force the operator to start watching/reconciling namespaces it should not have access to.
• More generally, override any dynamic-overridable operator property at runtime.

Pinning the reconciler to a specific, configurable ConfigMap name closes this privilege-escalation path: a rogue ConfigMap with the right labels but a different name is now ignored.

Does this PR introduce any user-facing change?

Yes (configuration). A new property is added:

Key	Default
spark.kubernetes.operator.dynamicConfig.name	spark-kubernetes-operator-dynamic-configuration

Operators that previously relied on a differently-named ConfigMap matching the dynamic-config selector must either rename their ConfigMap to the default value or set spark.kubernetes.operator.dynamicConfig.name to the
existing name. The default matches the name used by the project's Helm chart, so standard deployments are unaffected.

How was this patch tested?

Added unit tests in SparkOperatorConfigMapReconcilerTest:

• reconcileSkipsConfigMapWithNonMatchingName — a ConfigMap with a non-matching name is reconciled; verifies neither namespaceUpdater nor watchedNamespacesGetter are invoked and the rogue data does not leak into
  SparkOperatorConfManager.configOverrides.
• reconcileRefreshesConfigForMatchingName — a ConfigMap with the configured name triggers the refresh path and the namespace-update callbacks; verifies the override is applied.
• reconcileHonorsCustomConfiguredConfigMapName — when dynamicConfig.name is set to a custom value, the default-named ConfigMap is rejected and the custom-named one is processed.
• The existing integration test (sanityTest, using EnableKubeAPIServer) was updated to use the matching ConfigMap name so it continues to exercise the end-to-end refresh path.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 4.7)

[dongjoon-hyun]

Thank you, @csviri .

Could you clarify in the documentation that this makes spark.kubernetes.operator.dynamicConfig.selector obsolete? When we allow only one name exactly, why do we make a redundant and inefficient API call with selector. Technically, this PR proposes to make spark.kubernetes.operator.dynamicConfig.selector have no meaning.