avro\_schema\_prefix | Schema prefix name | Provide a namespace for the schema of a table in addition to the default, the table name. This allows multiple databases or clusters to share the same schema registry when the same table name is present in multiple databases. Example: CREATE CHANGEFEED FOR foo WITH format=avro, confluent\_schema\_registry='registry\_url', avro\_schema\_prefix='super' will register subjects as superfoo-key and superfoo-value with the namespace super. | |
| compression | gzip, zstd | Compress changefeed data files written to a . For compression options when using a Kafka sink, see . |
confluent\_schema\_registry | Schema Registry address | The Schema Registry address is required to use avro. Use the timeout={duration} query parameter (duration string) in your Confluent Schema Registry URI to change the default timeout for contacting the schema registry. By default, the timeout is 30 seconds. To connect to Confluent Cloud, use the following URL structure: 'https://{API\_KEY\_ID}:{API\_SECRET\_URL\_ENCODED}@{CONFLUENT\_REGISTRY\_URL}:443'. See the tutorial for further detail. Use the changefeed.schema\_registry.retry\_count metric to measure the number of request retries performed when sending requests to the schema registry. For more detail on monitoring changefeeds, refer to . | |
cursor | | Emit any changes after the given timestamp. cursor does not output the current state of the table first. When cursor is not specified, the changefeed starts by doing an initial scan of all the watched rows and emits the current value, then moves to emitting any changes that happen after the scan. The changefeed will encounter an error if you specify a timestamp that is before the configured garbage collection window for the target table. (Refer to .) With default garbage collection settings, this means you cannot create a changefeed that starts more than in the past. You can use cursor to start-a-new-changefeed-where-a-previous-changefeed-ended. Example: cursor='1536242855577149065.0000000000' | |
diff | N/A | Publish a with each message, which includes the value of the row before the update was applied. Changefeeds must use the diff option with the default wrapped envelope to emit the before field. | |
New in v24.2:encode\_json\_value\_null\_as\_object | N/A | Emit JSON NULL values as {"\_\_crdb\_json\_null\_\_": true} to distinguish these values from SQL NULL values. Refer to the page for an example. Note: When this option is enabled, if the changefeed encounters the literal value {"\_\_crdb\_json\_null\_\_": true} in JSON, it will have the same representation as a JSON NULL value and a warning will be printed to the . | |
end\_time | | Indicate the timestamp up to which the changefeed will emit all events and then complete with a successful status. Provide a future timestamp to end\_time in number of nanoseconds since the Unix epoch. For example, end\_time="1655402400000000000". You cannot use end\_time and initial\_scan = 'only' simultaneously. | |
envelope | wrapped / bare / key\_only / row | wrapped the default envelope structure for changefeed messages containing an array of the primary key, a top-level field for the type of message, and the current state of the row (or null for deleted rows). bare removes the after key from the changefeed message and stores any metadata in a crdb field. When used with avro format, record will replace the after key. Note: This is the default envelope format for . For an example, refer to . key\_only emits only the key and no value, which is faster if you only need to know the key of the changed row. This envelope option is only supported for or sinkless changefeeds. row emits the row without any additional metadata fields in the message. This envelope option is only supported in or sinkless changefeeds. row does not support avro format. Refer to for more detail on message format. Default: envelope=wrapped. Default for : envelope=bare. | |
execution\_locality | Key-value pairs | Restricts the execution of a changefeed to nodes that match the defined locality filter requirements, e.g., WITH execution\_locality = 'region=us-west-1a,cloud=aws'. See for usage and reference detail. | |
| format | json / avro / csv / parquet | Format of the emitted message. avro: For mappings of CockroachDB types to Avro types, and detail on . Note:confluent\_schema\_registry is required with format=avro. csv: You cannot combine format=csv with the diff or resolved options. Changefeeds use the same CSV format as the statement. Refer to for details using these options to create a changefeed as an alternative to EXPORT. Note:initial\_scan = 'only' is required with format=csv. parquet: Cloud storage is the only supported sink. The topic\_in\_value option is not compatible with parquet format. Default: format=json. |
full\_table\_name | N/A | Use fully qualified table name in topics, subjects, schemas, and record output instead of the default table name. This can prevent unintended behavior when the same table name is present in multiple databases. Note: This option cannot modify existing table names used as topics, subjects, etc., as part of an statement. To modify a topic, subject, etc., to use a fully qualified table name, create a new changefeed with this option. Example: CREATE CHANGEFEED FOR foo... WITH full\_table\_name will create the topic name defaultdb.public.foo instead of foo. | |
gc\_protect\_expires\_after | Duration string | Automatically expires protected timestamp records that are older than the defined duration. In the case where a changefeed job remains paused, gc\_protect\_expires\_after will trigger the underlying protected timestamp record to expire and cancel the changefeed job to prevent accumulation of protected data. Refer to for more detail on protecting changefeed data. | |
ignore\_disable\_changefeed\_replication | | When set to true, the changefeed will emit events even if CDC filtering for TTL jobs is configured using the disable\_changefeed\_replication, sql.ttl.changefeed\_replication.disabled, or the ttl\_disable\_changefeed\_replication. Refer to Filter changefeeds for tables using TTL for usage details. | |
initial\_scan | yes/no/only | Control whether or not an initial scan will occur at the start time of a changefeed. Only one initial\_scan option (yes, no, or only) can be used. If none of these are set, an initial scan will occur if there is no cursor, and will not occur if there is one. This preserves the behavior from previous releases. With initial\_scan = 'only' set, the changefeed job will end with a successful status (succeeded) after the initial scan completes. You cannot specify yes, no, only simultaneously. If used in conjunction with cursor, an initial scan will be performed at the cursor timestamp. If no cursor is specified, the initial scan is performed at now(). Although the syntax from previous versions is still supported, you cannot combine the previous and current syntax. Default: initial\_scan = 'yes' | |
kafka\_sink\_config | | Set fields to configure the required level of message acknowledgement from the Kafka server, the version of the server, and batching parameters for Kafka sinks. Set the message file compression type. See for more detail on configuring all the available fields for this option. Example: CREATE CHANGEFEED FOR table INTO 'kafka://localhost:9092' WITH kafka\_sink\_config='{"Flush": {"MaxMessages": 1, "Frequency": "1s"}, "RequiredAcks": "ONE"}' | |
key\_column | 'column' | Override the key used in . This changes the key hashed to determine downstream partitions. In sinks that support partitioning by message, CockroachDB uses the 32-bit FNV-1a hashing algorithm to determine which partition to send to. Note:key\_column does not preserve ordering of messages from CockroachDB to the downstream sink, therefore you must also include the unordered option in your changefeed creation statement. It does not affect per-key or the output of key\_in\_value. See the Define a key to determine the changefeed sink partition example. | |
key\_in\_value | N/A | Add a primary key array to the emitted message. This makes the of a deleted row recoverable in sinks where each message has a value but not a key (most have a key and value in each message). key\_in\_value is automatically used for , , and . | |
lagging\_ranges\_threshold | Duration string | Set a duration from the present that determines the length of time a range is considered to be lagging behind, which will then track in the metric. Note that ranges undergoing an initial scan for longer than the threshold duration are considered to be lagging. Starting a changefeed with an initial scan on a large table will likely increment the metric for each range in the table. As ranges complete the initial scan, the number of ranges lagging behind will decrease. Default:3m | |
lagging\_ranges\_polling\_interval | Duration string | Set the interval rate for when lagging ranges are checked and the lagging\_ranges metric is updated. Polling adds latency to the lagging\_ranges metric being updated. For example, if a range falls behind by 3 minutes, the metric may not update until an additional minute afterward. Default:1m | |
metrics\_label | | Define a metrics label to which the metrics for one or multiple changefeeds increment. All changefeeds also have their metrics aggregated. The maximum length of a label is 128 bytes. There is a limit of 1024 unique labels. WITH metrics\_label=label\_name For more detail on usage and considerations, see . | |
min\_checkpoint\_frequency | Duration string | Controls how often a node’s changefeed will flush their progress to the . A node’s changefeed aggregator will wait at least the specified duration between sending progress updates for the ranges it is watching to the coordinator. This can help you control the flush frequency of higher latency sinks to achieve better throughput. However, more frequent checkpointing can increase CPU usage. If this is set to 0s, a node will flush messages as long as the high-water mark has increased for the ranges that particular node is processing. If a changefeed is resumed, then min\_checkpoint\_frequency is the amount of time that changefeed will need to catch up. That is, it could emit during this time. Note:resolved messages will not be emitted more frequently than the configured min\_checkpoint\_frequency (but may be emitted less frequently). If you require resolved messages more frequently than 30s, you must configure min\_checkpoint\_frequency to at least the desired resolved message frequency. For more details, refer to . Default:30s | |
mvcc\_timestamp | N/A | Include the timestamp for each emitted row in a changefeed. With the mvcc\_timestamp option, each emitted row will always contain its MVCC timestamp, even during the changefeed’s initial backfill. | |
on\_error | pause / fail | Use on\_error=pause to pause the changefeed when encountering non-retryable errors. on\_error=pause will pause the changefeed instead of sending it into a terminal failure state. Note: Retryable errors will continue to be retried with this option specified. Use with protect\_data\_from\_gc\_on\_pause to protect changes from . If a changefeed with on\_error=pause is running when a watched table is , the changefeed will pause but will not be able to resume reads from that table. Using to drop the table from the changefeed and then will work, but you cannot add the same table to the changefeed again. Instead, you will need to create a new changefeed for that table. Default: on\_error=fail | |
protect\_data\_from\_gc\_on\_pause | N/A | This option is deprecated as of v23.2 and will be removed in a future release. When a , ensure that the data needed to is not garbage collected. If protect\_data\_from\_gc\_on\_pause is unset, pausing the changefeed will release the existing protected timestamp records. It is also important to note that pausing and adding protect\_data\_from\_gc\_on\_pause to a changefeed will not protect data if the window has already passed. Use with on\_error=pause to protect changes from garbage collection when encountering non-retryable errors. Refer to for more detail on protecting changefeed data. Note: If you use this option, changefeeds that are left paused for long periods of time can prevent garbage collection. Use with the gc\_protect\_expires\_after option to set a limit for protected data and for how long a changefeed will remain paused. | |
pubsub\_sink\_config | | Set fields to configure sink batching and retries. The schema is as follows: { "Flush": { "Messages": ..., "Bytes": ..., "Frequency": ..., }, "Retry": {"Max": ..., "Backoff": ..., } }. Note that if either Messages or Bytes are nonzero, then a non-zero value for Frequency must be provided. Refer to for more details on using this option. | |
| | resolved | Duration string | Emit in a format dependent on the connected sink. Resolved timestamps do not emit until the changefeed job’s progress has been checkpointed. Set a minimum amount of time that the changefeed’s high-water mark (overall resolved timestamp) must advance by before another resolved timestamp is emitted. Example: resolved='10s'. This option will only emit a resolved timestamp if the timestamp has advanced (and by at least the optional duration, if set). If a duration is unspecified, all resolved timestamps are emitted as the high-water mark advances. Note: If you set resolved lower than 30s, then you must also set min\_checkpoint\_frequency to at minimum the same value as resolved, because resolved messages may be emitted less frequently than min\_checkpoint\_frequency, but cannot be emitted more frequently. Refer to for more detail. |
| schema_change_events | default / column_changes | The type of schema change event that triggers the behavior specified by the schema_change_policy option:default: Include all events for columns that have a non-NULL or are , and all events.column_changes: Include all schema change events that add or remove any column.
Default: schema_change_events=default |
| schema_change_policy | backfill / nobackfill / stop | The behavior to take when an event specified by the schema_change_events option occurs:backfill: When are finished, output all watched rows using the new schema.nobackfill: For , perform no logical backfills. The changefeed will not emit any messages about the schema change. However, if the schema change is executed inside an while the is off, CockroachDB will emit all rows in the table to the changefeed once the transaction commits. To avoid this behavior, set autocommit_before_ddl to on.stop: For , wait for all data preceding the schema change to be resolved before exiting with an error indicating the timestamp at which the schema change occurred. An error: schema change occurred at <timestamp> will display in the cockroach.log file.
Default: schema_change_policy=backfill |
split\_column\_families | N/A | Use this option to create a changefeed on a table with multiple . The changefeed will emit messages for each of the table’s column families. See for more usage detail. | |
topic\_in\_value | | Set to include the topic in each emitted row update. This option is automatically set for . Note:topic\_in\_value is not compatible with changefeeds running in parquet format. | |
unordered | N/A | Run a changefeed to without specifying a region. You must include the unordered option with key\_column in your changefeed creation statement. You cannot use unordered with resolved, because resolved timestamps may not be correct in unordered mode. | |
| updated | N/A | Include updated timestamps with each row. If a cursor is provided, the “updated” timestamps will match the timestamps of the emitted rows, and there is no initial scan. If a cursor is not provided, the changefeed will perform an initial scan (as of the time the changefeed was created), and the “updated” timestamp for each change record emitted in the initial scan will be the timestamp of the initial scan. Similarly, when a , the “updated” timestamp is set to the first timestamp for when the new schema is valid. |
virtual\_columns | STRING | Changefeeds omit from emitted by default. To maintain the behavior of previous CockroachDB versions where the changefeed would emit values for virtual computed columns, set virtual\_columns = "null" when you start a changefeed. You may also define virtual\_columns = "omitted", though this is already the default behavior for v22.1+. If you do not set "omitted" on a table with virtual computed columns when you create a changefeed, you will receive a warning that changefeeds will filter out virtual computed values. Default:"omitted" | |
webhook\_auth\_header | | Pass a value (password, token etc.) to the HTTP Authorization header with a webhook request for a “Basic” HTTP authentication scheme. Example: With a username of “user” and password of “pwd”, add a colon between “user:pwd” and then base64 encode, which results in “dXNlcjpwd2Q=”. WITH webhook\_auth\_header='Basic dXNlcjpwd2Q='. | |
webhook\_client\_timeout | | If a response is not recorded from the sink within this timeframe, it will error and retry to connect. Note this must be a positive value. Default:"3s" | |
webhook\_sink\_config | | Set fields to configure sink batching and retries. The schema is as follows: { "Flush": { "Messages": ..., "Bytes": ..., "Frequency": ..., }, "Retry": {"Max": ..., "Backoff": ..., } }. Note that if either Messages or Bytes are nonzero, then a non-zero value for Frequency must be provided. See for more details on using this option. | |
