[BUG] Modifying a static parameter caused the pod to lose with the primary role.

[weicao]

Describe the bug
After modifying a static parameter like MySQL's "performance_schema", the operation was successful, but both replicas in the cluster became secondary.

Using KubeBlocks v1.0.0-beta.30.

To Reproduce
Steps to reproduce the behavior:

1. Below is an example configuration for deploying a MySQL cluster with 2 nodes (1 primary, 1 replicas) in semi-synchronous mode. Deploy the cluster using the following YAML manifest:

kubectl apply -f - <<EOF
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
  name: example-mysql-cluster
  namespace: demo
spec:
  clusterDef: mysql
  topology: semisync
  terminationPolicy: Delete
  componentSpecs:
    - name: mysql
      serviceVersion: 8.0.35
      replicas: 2
      resources:
        limits:
          cpu: '0.5'
          memory: 0.5Gi
        requests:
          cpu: '0.5'
          memory: 0.5Gi
      volumeClaimTemplates:
        - name: data
          spec:
            storageClassName: ""
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
EOF

2. After the cluster is deployed, verify its status using the following command:

$ kubectl get cluster example-mysql-cluster -n demo -w
NAME                     CLUSTER-DEFINITION   TERMINATION-POLICY   STATUS    AGE
example-mysql-cluster   mysql                Delete               Creating   12s
example-mysql-cluster   mysql                Delete               Running   46s

3. Create a Reconfigure OpsRequest. Apply the following OpsRequest YAML to update the 'performance_schema':

kubectl apply -f - <<EOF
apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: mysql-reconfigure-static
  namespace: demo
spec:
  clusterName: example-mysql-cluster
  force: false
  reconfigures:
  - componentName: mysql
    configurations:
    - keys:
      - key: my.cnf
        parameters:
        - key: performance_schema
          value: 'ON'
      name: mysql-replication-config
  preConditionDeadlineSeconds: 0
  type: Reconfiguring
EOF

4. Check the status of the OpsRequest，until succeeded.

kubectl get ops mysql-reconfigure-static -n demo -w

Example output:

mysql-reconfigure-static   Reconfiguring   example-mysql-cluster   Running   -/-        9s
mysql-reconfigure-static   Reconfiguring   example-mysql-cluster   Running   1/2        28s
mysql-reconfigure-static   Reconfiguring   example-mysql-cluster   Succeed   2/2        57s
mysql-reconfigure-static   Reconfiguring   example-mysql-cluster   Succeed   2/2        57s

5. Verify roles

$ kubectl get pods -n demo -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.metadata.labels.kubeblocks\.io/role}{"\n"}{end}'
example-mysql-cluster-mysql-0	secondary
example-mysql-cluster-mysql-1	secondary

6. Verify paramter value

In both pods, the value of "performance_schema" changed to "ON"

[weicao]

v1.0.0-beta.32 is fine.

[weicao]

In v1.0.0-beta.32:

1. Create a Reconfigure OpsRequest. Apply the following OpsRequest YAML to update the 'performance_schema':

kubectl apply -f - <<EOF
apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: mysql-reconfigure-static
  namespace: demo
spec:
  clusterName: example-mysql-cluster
  force: false
  reconfigures:
  - componentName: mysql
    parameters:
    - key: performance_schema
      value: 'ON'
  preConditionDeadlineSeconds: 0
  type: Reconfiguring
EOF

2. Verify parameter update. It seems it did not take effect.

mysql> SHOW VARIABLES LIKE 'performance_schema';
+--------------------+-------+
| Variable_name | Value |
+--------------------+-------+
| performance_schema | OFF |
+--------------------+-------+
1 row in set (0.00 sec)

[xuriwuyun]

In version 1.0.0-beta.32, the pod does not restart for the static parameter 'performance_schema', so it does not take effect. This is unexpected.

[sophon-zt]

related bug: #9012

[JashBook]

in KB v1.0.0-beta.33 still does not take effect

1. create cluster

kubectl apply -f - <<EOF
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
  name: mysql-cluster
  namespace: demo
spec:
  clusterDef: mysql
  topology: semisync
  terminationPolicy: Delete
  componentSpecs:
    - name: mysql
      serviceVersion: 8.0.35
      replicas: 2
      resources:
        limits:
          cpu: '0.5'
          memory: 0.5Gi
        requests:
          cpu: '0.5'
          memory: 0.5Gi
      volumeClaimTemplates:
        - name: data
          spec:
            storageClassName: ""
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
EOF

2. config

kubectl apply -f - <<EOF
apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: mysql-reconfigure-static2
  namespace: demo
spec:
  clusterName: mysql-cluster
  force: false
  reconfigures:
  - componentName: mysql
    parameters:
    - key: performance_schema
      value: 'ON'
  preConditionDeadlineSeconds: 0
  type: Reconfiguring
EOF

kubectl get ops -n demo 
NAME                        TYPE            CLUSTER         STATUS    PROGRESS   AGE
mysql-reconfigure-static2   Reconfiguring   mysql-cluster   Succeed   -/-        4m51s

3. see error

kubectl exec -it mysql-cluster-mysql-0 -n demo -- bash
Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init)
bash-4.4# mysql -h127.0.0.1 -uroot -P3306 -p'3Ug32V1yZ0'
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 22
Server version: 8.0.35 MySQL Community Server - GPL

Copyright (c) 2000, 2023, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> SHOW VARIABLES LIKE 'performance_schema';
+--------------------+-------+
| Variable_name      | Value |
+--------------------+-------+
| performance_schema | OFF   |
+--------------------+-------+
1 row in set (0.00 sec)

mysql> 

kubectl exec -it mysql-cluster-mysql-1 -n demo -- bash
Defaulted container "mysql" out of: mysql, mysql-exporter, kbagent, config-manager, init-data (init), init-xtrabackup (init), init-jemalloc (init), init-syncer (init), init-kbagent (init), kbagent-worker (init)
bash-4.4# mysql -h127.0.0.1 -uroot -P3306 -p'3Ug32V1yZ0'
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 15
Server version: 8.0.35 MySQL Community Server - GPL

Copyright (c) 2000, 2023, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> SHOW VARIABLES LIKE 'performance_schema';
+--------------------+-------+
| Variable_name      | Value |
+--------------------+-------+
| performance_schema | OFF   |
+--------------------+-------+
1 row in set (0.00 sec)

mysql>