> ## Documentation Index
> Fetch the complete documentation index at: https://cockroachlabs.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Cluster Upgrades

export const InternalLink = ({version, path = "", children, ...props}) => {
  let detectedVersion = version || "stable";
  if (typeof window !== 'undefined' && !version) {
    const match = window.location.pathname.match(/\/docs\/([^/]+)/);
    if (match) {
      detectedVersion = match[1];
    }
  }
  const normalizedPath = path.startsWith("/") ? path.slice(1) : path;
  return <a href={`/docs/${detectedVersion}/${normalizedPath}`} {...props}>
      {children}
    </a>;
};

<Note>
  This article assumes you have already <InternalLink path="deploy-cockroachdb-with-kubernetes">deployed CockroachDB on a single Kubernetes cluster</InternalLink>.
</Note>

We strongly recommend that you regularly upgrade your CockroachDB version in order to pick up bug fixes, performance improvements, and new features.

The upgrade process on Kubernetes is a [staged update](https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set#staging-an-update) in which the Docker image is applied to the pods one at a time, with each pod being stopped and restarted in turn. This is to ensure that the cluster remains available during the upgrade.

<Note>
  All `kubectl` steps should be performed in the <InternalLink path="deploy-cockroachdb-with-kubernetes#install-the-operator">namespace where you installed the Operator</InternalLink>. By default, this is `cockroach-operator-system`.
</Note>

<Tip>
  If you <InternalLink path="deploy-cockroachdb-with-kubernetes-openshift">deployed CockroachDB on Red Hat OpenShift</InternalLink>, substitute `kubectl` with `oc` in the following commands.
</Tip>

1. Verify that you can upgrade.

   To upgrade to a new major version, you must first be on a production release of the previous version. The release does not need to be the latest production release of the previous version, but it must be a production <InternalLink version="releases" path="index">release</InternalLink> and not a testing release (alpha/beta).

   Therefore, in order to upgrade to v24.1, you must be on a production release of v23.2.

   1. If you are upgrading to v24.1 from a production release earlier than v23.2, or from a testing release (alpha/beta), first <InternalLink path="upgrade-cockroachdb-kubernetes">upgrade to a production release of v23.2</InternalLink>. Be sure to complete all the steps.
   2. Then return to this page and perform a second upgrade to v24.1.
   3. If you are upgrading from a production release of v23.2, or from any earlier v24.1 patch release, you do not have to go through intermediate releases; continue to step 2.
2. Verify the overall health of your cluster using the <InternalLink path="ui-overview">DB Console</InternalLink>. On the **Overview**:
   * Under **Node Status**, make sure all nodes that should be live are listed as such. If any nodes are unexpectedly listed as suspect or dead, identify why the nodes are offline and either restart them or <InternalLink path="scale-cockroachdb-kubernetes#remove-nodes">decommission</InternalLink> them before beginning your upgrade. If there are dead and non-decommissioned nodes in your cluster, it will not be possible to finalize the upgrade (either automatically or manually).
   * Under **Replication Status**, make sure there are 0 under-replicated and unavailable ranges. Otherwise, performing a rolling upgrade increases the risk that ranges will lose a majority of their replicas and cause cluster unavailability. Therefore, it's important to <InternalLink path="cluster-setup-troubleshooting#replication-issues">identify and resolve the cause of range under-replication and/or unavailability</InternalLink> before beginning your upgrade.
   * In the **Node List**:
     * Make sure all nodes are on the same version. If not all nodes are on the same version, upgrade them to the cluster's highest current version first, and then start this process over.
     * Make sure capacity and memory usage are reasonable for each node. Nodes must be able to tolerate some increase in case the new version uses more resources for your workload. Also go to **Metrics > Dashboard: Hardware** and make sure CPU percent is reasonable across the cluster. If there's not enough headroom on any of these metrics, consider <InternalLink path="scale-cockroachdb-kubernetes#add-nodes">adding nodes</InternalLink> to your cluster before beginning your upgrade.
3. Review the <InternalLink version="releases" path="v24.1">backward-incompatible changes in v24.1</InternalLink> and <InternalLink version="releases" path="v24.1">deprecated features</InternalLink>. If any affect your deployment, make the necessary changes before starting the rolling upgrade to v24.1.
4. Change the desired Docker image in the custom resource:

   ```text theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
   image:
     name: cockroachdb/cockroach:v24.1.26
   ```
5. Apply the new settings to the cluster:

   ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
   $ kubectl apply -f example.yaml
   ```

   The Operator will perform the staged update.

<Note>
  The Operator automatically sets the `cluster.preserve_downgrade_option` <InternalLink path="cluster-settings">cluster setting</InternalLink> to the version you are upgrading from. This disables auto-finalization of the upgrade so that you can monitor the stability and performance of the upgraded cluster before manually finalizing the upgrade. This will enable certain features and performance improvements introduced in v24.1.

  Note that after finalization, it will no longer be possible to perform a downgrade to v23.2. In the event of a catastrophic failure or corruption, the only option will be to start a new cluster using the previous binary and then restore from a <InternalLink path="take-full-and-incremental-backups">backup</InternalLink> created prior to performing the upgrade.

  Finalization only applies when performing a major version upgrade (for example, from v23.2.x to v24.1). Patch version upgrades (for example, within the v24.1.x series) can always be downgraded.
</Note>

6. To check the status of the rolling upgrade, run `kubectl get pods`. The pods are restarted one at a time with the new image.
7. Verify that all pods have been upgraded by running:

   ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
   $ kubectl get pods \
   -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.containers[0].image}{"\n"}'
   ```

   You can also check the CockroachDB version of each node in the <InternalLink path="ui-cluster-overview-page#node-details">DB Console</InternalLink>.
8. Monitor the stability and performance of your cluster until you are comfortable with the upgrade (generally at least a day).

   If you decide to roll back the upgrade, revert the image name in the custom resource and apply the new value.

<Note>
  This is only possible when performing a major version upgrade (for example, from v23.2.x to v24.1). Patch version upgrades (for example, within the v24.1.x series) are auto-finalized.
</Note>

To finalize the upgrade, re-enable auto-finalization:

1. Start the CockroachDB <InternalLink path="cockroach-sql">built-in SQL client</InternalLink>. For example, if you followed the steps in <InternalLink path="deploy-cockroachdb-with-kubernetes#step-3-use-the-built-in-sql-client">Deploy CockroachDB with Kubernetes</InternalLink> to launch a secure client pod, get a shell into the `cockroachdb-client-secure` pod:

   ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
   $ kubectl exec -it cockroachdb-client-secure \-- ./cockroach sql \
   --certs-dir=/cockroach/cockroach-certs \
   --host={cluster-name}-public
   ```

2. Re-enable auto-finalization:

   ```sql theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
   > RESET CLUSTER SETTING cluster.preserve_downgrade_option;
   ```

   After the upgrade to v24.1 is finalized, you may notice an increase in <InternalLink path="architecture/storage-layer#compaction">compaction</InternalLink> activity due to a background migration within the storage engine. To observe the migration's progress, check the **Compactions** section of the <InternalLink path="ui-storage-dashboard">Storage Dashboard</InternalLink> in the DB Console or monitor the `storage.marked-for-compaction-files` <InternalLink path="metrics">time-series metric</InternalLink>. When the metric's value nears or reaches `0`, the migration is complete and compaction activity will return to normal levels. By default, the <InternalLink path="architecture/storage-layer">storage engine</InternalLink> uses a compaction concurrency of 3. If you have sufficient IOPS and CPU headroom, you can consider increasing this setting via the `COCKROACH_COMPACTION_CONCURRENCY` environment variable. This may help to reshape the LSM more quickly in <InternalLink path="architecture/storage-layer#inverted-lsms">inverted LSM</InternalLink> scenarios; and it can lead to increased overall performance for some workloads. Cockroach Labs **strongly recommends** testing your workload against non-default values of this setting.

3. Exit the SQL shell and pod:

   ```sql theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
   > \q
   ```

4. Verify that you can upgrade.

   To upgrade to a new major version, you must first be on a production release of the previous version. The release does not need to be the latest production release of the previous version, but it must be a production <InternalLink version="releases" path="index">release</InternalLink> and not a testing release (alpha/beta).

   Therefore, in order to upgrade to v24.1, you must be on a production release of v23.2.

   1. If you are upgrading to v24.1 from a production release earlier than v23.2, or from a testing release (alpha/beta), first <InternalLink path="upgrade-cockroachdb-kubernetes?filters=manual">upgrade to a production release of v23.2</InternalLink>. Be sure to complete all the steps.
   2. Then return to this page and perform a second upgrade to v24.1.
   3. If you are upgrading from any production release of v23.2, or from any earlier v24.1 patch release, you do not have to go through intermediate releases; continue to step 2.

5. Verify the overall health of your cluster using the <InternalLink path="ui-overview">DB Console</InternalLink>. On the **Overview**:
   * Under **Node Status**, make sure all nodes that should be live are listed as such. If any nodes are unexpectedly listed as suspect or dead, identify why the nodes are offline and either restart them or <InternalLink path="scale-cockroachdb-kubernetes?filters=manual#remove-nodes">decommission</InternalLink> them before beginning your upgrade. If there are dead and non-decommissioned nodes in your cluster, it will not be possible to finalize the upgrade (either automatically or manually).
   * Under **Replication Status**, make sure there are 0 under-replicated and unavailable ranges. Otherwise, performing a rolling upgrade increases the risk that ranges will lose a majority of their replicas and cause cluster unavailability. Therefore, it's important to <InternalLink path="cluster-setup-troubleshooting#replication-issues">identify and resolve the cause of range under-replication and/or unavailability</InternalLink> before beginning your upgrade.
   * In the **Node List**:
     * Make sure all nodes are on the same version. If not all nodes are on the same version, upgrade them to the cluster's highest current version first, and then start this process over.
     * Make sure capacity and memory usage are reasonable for each node. Nodes must be able to tolerate some increase in case the new version uses more resources for your workload. Also go to **Metrics > Dashboard: Hardware** and make sure CPU percent is reasonable across the cluster. If there's not enough headroom on any of these metrics, consider <InternalLink path="scale-cockroachdb-kubernetes?filters=manual#add-nodes">adding nodes</InternalLink> to your cluster before beginning your upgrade.

6. Review the <InternalLink version="releases" path="v24.1">backward-incompatible changes in v24.1</InternalLink> and <InternalLink version="releases" path="v24.1">deprecated features</InternalLink>. If any affect your deployment, make the necessary changes before starting the rolling upgrade to v24.1.

7. Decide how the upgrade will be finalized.

   By default, after all nodes are running the new version, the upgrade process will be **auto-finalized**. This will enable certain <InternalLink path="take-full-and-incremental-backups">features and performance improvements introduced in v24.1. After finalization, however, it will no longer be possible to perform a downgrade to v23.2. In the event of a catastrophic failure or corruption, the only option is to start a new cluster using the old binary and then restore from a \[backup</InternalLink> created prior to the upgrade. For this reason, **we recommend disabling auto-finalization** so you can monitor the stability and performance of the upgraded cluster before finalizing the upgrade, but note that you will need to follow all of the subsequent directions, including the manual finalization in a later step.

   Finalization only applies when performing a major version upgrade (for example, from v23.2.x to v24.1). Patch version upgrades (for example, within the v24.1.x series) can always be downgraded.

   1. Start the CockroachDB <InternalLink path="cockroach-sql">built-in SQL client</InternalLink>. For example, if you followed the steps in <InternalLink path="deploy-cockroachdb-with-kubernetes?filters=manual#step-3-use-the-built-in-sql-client">Deploy CockroachDB with Kubernetes</InternalLink> to launch a secure client pod, get a shell into the `cockroachdb-client-secure` pod:

      ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
      $ kubectl exec -it cockroachdb-client-secure \-- ./cockroach sql \
      --certs-dir=/cockroach-certs \
      --host=cockroachdb-public
      ```
   2. Set the `cluster.preserve_downgrade_option` <InternalLink path="cluster-settings">cluster setting</InternalLink> to the version you are upgrading from:

      ```sql theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
      > SET CLUSTER SETTING cluster.preserve_downgrade_option = '["23.2"]';
      ```
   3. Exit the SQL shell and delete the temporary pod:

      ```sql theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
      > \q
      ```

8. Add a [partition](https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set#staging-an-update) to the update strategy defined in the StatefulSet. Only the pods numbered greater than or equal to the partition value will be updated. For a cluster with 3 pods (e.g., `cockroachdb-0`, `cockroachdb-1`, `cockroachdb-2`) the partition value should be 2:

   ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
   $ kubectl patch statefulset cockroachdb \
   -p='{"spec":{"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"partition":2}}}}'
   ```

   ```text theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
   statefulset.apps/cockroachdb patched
   ```

9. Kick off the upgrade process by changing the Docker image used in the CockroachDB StatefulSet:

   ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
   $ kubectl patch statefulset cockroachdb \
   --type='json' \
   -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/image", "value":"cockroachdb/cockroach:v24.1.26"}]'
   ```

   ```text theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
   statefulset.apps/cockroachdb patched
   ```

10. Check the status of your cluster's pods. You should see one of them being restarted:

    ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
    $ kubectl get pods
    ```

    ```text theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
    NAME            READY     STATUS        RESTARTS   AGE
    cockroachdb-0   1/1       Running       0          2m
    cockroachdb-1   1/1       Running       0          2m
    cockroachdb-2   0/1       Terminating   0          1m
    ...
    ```

11. After the pod has been restarted with the new image, start the CockroachDB <InternalLink path="cockroach-sql">built-in SQL client</InternalLink>:

    ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
    $ kubectl exec -it cockroachdb-client-secure \-- ./cockroach sql \
    --certs-dir=/cockroach-certs \
    --host=cockroachdb-public
    ```

12. Run the following SQL query to verify that the number of underreplicated ranges is zero:

    ```sql theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
    SELECT sum((metrics->>'ranges.underreplicated')::DECIMAL)::INT AS ranges_underreplicated FROM crdb_internal.kv_store_status;
    ```

    ```text theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
      ranges_underreplicated
    --------------------------
                           0
    (1 row)
    ```

    This indicates that it is safe to proceed to the next pod.

13. Exit the SQL shell:

    ```sql theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
    > \q
    ```

14. Decrement the partition value by 1 to allow the next pod in the cluster to update:

    ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
    $ kubectl patch statefulset cockroachdb \
    -p='{"spec":{"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"partition":1}}}}'
    ```

    ```text theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
    statefulset.apps/cockroachdb patched
    ```

15. Repeat steps 4-8 until all pods have been restarted and are running the new image (the final partition value should be `0`).

16. Check the image of each pod to confirm that all have been upgraded:

    ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
    $ kubectl get pods \
    -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.containers[0].image}{"\n"}'
    ```

    ```text theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
    cockroachdb-0   cockroachdb/cockroach:v24.1.26
    cockroachdb-1   cockroachdb/cockroach:v24.1.26
    cockroachdb-2   cockroachdb/cockroach:v24.1.26
    ...
    ```

    You can also check the CockroachDB version of each node in the <InternalLink path="ui-cluster-overview-page#node-details">DB Console</InternalLink>.

17. If you disabled auto-finalization earlier, monitor the stability and performance of your cluster until you are comfortable with the upgrade (generally at least a day).

    If you decide to roll back the upgrade, repeat the rolling restart procedure with the old binary.

    This is only possible when performing a major version upgrade (for example, from v23.2.x to v24.1). Patch version upgrades (for example, within the v24.1.x series) are auto-finalized.

    To finalize the upgrade, re-enable auto-finalization:

    1. Start the CockroachDB <InternalLink path="cockroach-sql">built-in SQL client</InternalLink>:

       ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
       $ kubectl exec -it cockroachdb-client-secure \
       -- ./cockroach sql \
       --certs-dir=/cockroach-certs \
       --host=cockroachdb-public
       ```
    2. Re-enable auto-finalization:

       ```sql theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
       > RESET CLUSTER SETTING cluster.preserve_downgrade_option;
       ```

       After the upgrade to v24.1 is finalized, you may notice an increase in compaction activity due to a background migration within the storage engine. To observe the migration's progress, check the **Compactions** section of the <InternalLink path="ui-storage-dashboard">Storage Dashboard</InternalLink> in the DB Console or monitor the `storage.marked-for-compaction-files` <InternalLink path="metrics">time-series metric</InternalLink>. When the metric's value nears or reaches `0`, the migration is complete and compaction activity will return to normal levels.
    3. Exit the SQL shell and delete the temporary pod:

       ```sql theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
       > \q
       ```

18. Verify that you can upgrade.

    To upgrade to a new major version, you must first be on a production release of the previous version. The release does not need to be the latest production release of the previous version, but it must be a production <InternalLink version="releases" path="index">release</InternalLink> and not a testing release (alpha/beta).

    Therefore, in order to upgrade to v24.1, you must be on a production release of v23.2.

    1. If you are upgrading to v24.1 from a production release earlier than v23.2, or from a testing release (alpha/beta), first <InternalLink path="upgrade-cockroachdb-kubernetes?filters=helm">upgrade to a production release of v23.2</InternalLink>. Be sure to complete all the steps.
    2. Then return to this page and perform a second upgrade to v24.1.
    3. If you are upgrading from any production release of v23.2, or from any earlier v24.1 patch release, you do not have to go through intermediate releases; continue to step 2.

19. Verify the overall health of your cluster using the <InternalLink path="ui-overview">DB Console</InternalLink>. On the **Overview**:
    * Under **Node Status**, make sure all nodes that should be live are listed as such. If any nodes are unexpectedly listed as suspect or dead, identify why the nodes are offline and either restart them or <InternalLink path="scale-cockroachdb-kubernetes?filters=helm#remove-nodes">decommission</InternalLink> them before beginning your upgrade. If there are dead and non-decommissioned nodes in your cluster, it will not be possible to finalize the upgrade (either automatically or manually).
    * Under **Replication Status**, make sure there are 0 under-replicated and unavailable ranges. Otherwise, performing a rolling upgrade increases the risk that ranges will lose a majority of their replicas and cause cluster unavailability. Therefore, it's important to <InternalLink path="cluster-setup-troubleshooting#replication-issues">identify and resolve the cause of range under-replication and/or unavailability</InternalLink> before beginning your upgrade.
    * In the **Node List**:
      * Make sure all nodes are on the same version. If not all nodes are on the same version, upgrade them to the cluster's highest current version first, and then start this process over.
      * Make sure capacity and memory usage are reasonable for each node. Nodes must be able to tolerate some increase in case the new version uses more resources for your workload. Also go to **Metrics > Dashboard: Hardware** and make sure CPU percent is reasonable across the cluster. If there's not enough headroom on any of these metrics, consider <InternalLink path="scale-cockroachdb-kubernetes?filters=helm#add-nodes">adding nodes</InternalLink> to your cluster before beginning your upgrade.

20. Review the <InternalLink version="releases" path="v24.1">backward-incompatible changes in v24.1</InternalLink> and <InternalLink version="releases" path="v24.1">deprecated features</InternalLink>. If any affect your deployment, make the necessary changes before starting the rolling upgrade to v24.1.

21. Decide how the upgrade will be finalized.

    By default, after all nodes are running the new version, the upgrade process will be **auto-finalized**. This will enable certain <InternalLink path="take-full-and-incremental-backups">features and performance improvements introduced in v24.1. After finalization, however, it will no longer be possible to perform a downgrade to v23.2. In the event of a catastrophic failure or corruption, the only option is to start a new cluster using the old binary and then restore from a \[backup</InternalLink> created prior to the upgrade. For this reason, **we recommend disabling auto-finalization** so you can monitor the stability and performance of the upgraded cluster before finalizing the upgrade, but note that you will need to follow all of the subsequent directions, including the manual finalization in a later step.

    Finalization only applies when performing a major version upgrade (for example, from v23.2.x to v24.1). Patch version upgrades (for example, within the v24.1.x series) can always be downgraded.

    1. Get a shell into the pod with the `cockroach` binary created earlier and start the CockroachDB <InternalLink path="cockroach-sql">built-in SQL client</InternalLink>:

       ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
       $ kubectl exec -it cockroachdb-client-secure \
       -- ./cockroach sql \
       --certs-dir=/cockroach-certs \
       --host=my-release-cockroachdb-public
       ```
    2. Set the `cluster.preserve_downgrade_option` <InternalLink path="cluster-settings">cluster setting</InternalLink> to the version you are upgrading from:

       ```sql theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
       > SET CLUSTER SETTING cluster.preserve_downgrade_option = '["23.2"]';
       ```
    3. Exit the SQL shell and delete the temporary pod:

       ```sql theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
       > \q
       ```

22. Add a [partition](https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set#staging-an-update) to the update strategy defined in the StatefulSet. Only the pods numbered greater than or equal to the partition value will be updated. For a cluster with 3 pods (e.g., `cockroachdb-0`, `cockroachdb-1`, `cockroachdb-2`) the partition value should be 2:

    ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
    $ helm upgrade \
    my-release \
    cockroachdb/cockroachdb \
    --set statefulset.updateStrategy.rollingUpdate.partition=2
    ```

23. Kick off the upgrade process by changing the Docker image used in the CockroachDB StatefulSet:

<Note>
  For Helm, you must remove the cluster initialization job from when the cluster was created before the cluster version can be changed.
</Note>

```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
$ kubectl delete job my-release-cockroachdb-init
```

```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
$ helm upgrade \
my-release \
cockroachdb/cockroachdb \
--set image.tag=v24.1.26 \
--reuse-values
```

7. Check the status of your cluster's pods. You should see one of them being restarted:

   ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
   $ kubectl get pods
   ```

   ```text theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
   NAME                                READY     STATUS              RESTARTS   AGE
   my-release-cockroachdb-0            1/1       Running             0          2m
   my-release-cockroachdb-1            1/1       Running             0          3m
   my-release-cockroachdb-2            0/1       ContainerCreating   0          25s
   my-release-cockroachdb-init-nwjkh   0/1       ContainerCreating   0          6s
   ...
   ```

<Note>
  Ignore the pod for cluster initialization. It is re-created as a byproduct of the StatefulSet configuration but does not impact your existing cluster.
</Note>

8. After the pod has been restarted with the new image, start the CockroachDB <InternalLink path="cockroach-sql">built-in SQL client</InternalLink>:

   ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
   $ kubectl exec -it cockroachdb-client-secure \
   -- ./cockroach sql \
   --certs-dir=/cockroach-certs \
   --host=my-release-cockroachdb-public
   ```
9. Run the following SQL query to verify that the number of underreplicated ranges is zero:

   ```sql theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
   SELECT sum((metrics->>'ranges.underreplicated')::DECIMAL)::INT AS ranges_underreplicated FROM crdb_internal.kv_store_status;
   ```

   ```text theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
     ranges_underreplicated
   --------------------------
                          0
   (1 row)
   ```

   This indicates that it is safe to proceed to the next pod.
10. Exit the SQL shell:

    ```sql theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
    > \q
    ```
11. Decrement the partition value by 1 to allow the next pod in the cluster to update:

    ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
    $ helm upgrade \
    my-release \
    cockroachdb/cockroachdb \
    --set statefulset.updateStrategy.rollingUpdate.partition=1 \
    ```
12. Repeat steps 4-8 until all pods have been restarted and are running the new image (the final partition value should be `0`).
13. Check the image of each pod to confirm that all have been upgraded:

    ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
    $ kubectl get pods \
    -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.containers[0].image}{"\n"}'
    ```

    ```text theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
    my-release-cockroachdb-0    cockroachdb/cockroach:v24.1.26
    my-release-cockroachdb-1    cockroachdb/cockroach:v24.1.26
    my-release-cockroachdb-2    cockroachdb/cockroach:v24.1.26
    ...
    ```

    You can also check the CockroachDB version of each node in the <InternalLink path="ui-cluster-overview-page#node-details">DB Console</InternalLink>.
14. If you disabled auto-finalization earlier, monitor the stability and performance of your cluster until you are comfortable with the upgrade (generally at least a day).

    If you decide to roll back the upgrade, repeat the rolling restart procedure with the old binary.

    This is only possible when performing a major version upgrade (for example, from v23.2.x to v24.1). Patch version upgrades (for example, within the v24.1.x series) are auto-finalized.

    To finalize the upgrade, re-enable auto-finalization:

    1. Get a shell into the pod with the `cockroach` binary created earlier and start the CockroachDB <InternalLink path="cockroach-sql">built-in SQL client</InternalLink>:

       ```shell theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
       $ kubectl exec -it cockroachdb-client-secure \
       -- ./cockroach sql \
       --certs-dir=/cockroach-certs \
       --host=my-release-cockroachdb-public
       ```
    2. Re-enable auto-finalization:

       ```sql theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
       > RESET CLUSTER SETTING cluster.preserve_downgrade_option;
       ```

       After the upgrade to v24.1 is finalized, you may notice an increase in compaction activity due to a background migration within the storage engine. To observe the migration's progress, check the **Compactions** section of the <InternalLink path="ui-storage-dashboard">Storage Dashboard</InternalLink> in the DB Console or monitor the `storage.marked-for-compaction-files` <InternalLink path="metrics">time-series metric</InternalLink>. When the metric's value nears or reaches `0`, the migration is complete and compaction activity will return to normal levels.
    3. Exit the SQL shell and delete the temporary pod:

       ```sql theme={"theme":{"light":"catppuccin-mocha","dark":"catppuccin-mocha"}}
       > \q
       ```