diff --git a/docs/configuring.md b/docs/configuring.md new file mode 100644 index 00000000..18f36c7d --- /dev/null +++ b/docs/configuring.md @@ -0,0 +1,226 @@ +# Configuring Spock + +Add the following `postgresql.conf` settings on each node in your Spock +replication scenario before creating the Spock extension: + +```sql + wal_level = 'logical' + max_worker_processes = 10 # one per database needed on provider node + # one per node needed on subscriber node + max_replication_slots = 10 # one per node needed on provider node + max_wal_senders = 10 # one per node needed on provider node + shared_preload_libraries = 'spock' + track_commit_timestamp = on # needed for conflict resolution +``` + +After modifying the parameters and restarting the Postgres server +with your OS-specific restart command, connect with psql and create the Spock +extension: + +```sql +[pg18]$ sudo -u postgres psql -U postgres -p 5432 +psql (18.1) +Type "help" for help. + +postgres=# CREATE EXTENSION spock; +CREATE EXTENSION +``` + +You will also need to modify your +[`pg_hba.conf` file](https://www.postgresql.org/docs/18/auth-pg-hba-conf.html) +to allow logical replication connections from localhost and between nodes. +Logical replication connections are treated by `pg_hba.conf` as regular +connections to the provider database. + +After modifying the pg_hba.conf file on each node, restart the server to apply +the changes. + + +## Advanced Configuration Options for Spock + +You can use the following configuration parameters (GUCs) on the psql +command line with a `SET` statement or in a Postgres configuration file +(like `postgresql.conf`) to configure a Spock installation. + +### `spock.allow_ddl_from_functions` + +`spock.allow_ddl_from_functions` enables spock to automatically replicate +DDL statements that are called within functions to also be automatically +replicated. This can be turned `off` if these functions are expected to run +on every node. When this is set to `off`, statements replicated from +functions adhere to the same rule previously described for +`include_ddl_repset`. If a table possesses a defined primary key, it will be +added into the `default` replication set; alternatively, they will be added +to the `default_insert_only` replication set. + +### `spock.batch_inserts` + +`spock.batch_inserts` tells Spock to use batch insert mechanism if +possible. The batch mechanism uses Postgres internal batch insert mode which +is also used by `COPY` command. + +### `spock.channel_counters` + +`spock.channel_counters` is a boolean value (the default is `true`) that +enables or disables the Spock channel statistic information collection. This +option can only be set when the postmaster starts. + +### `spock.check_all_uc_indexes` + +!!! info + + This feature is experimental and `OFF` by default. Use this feature at + your own risk. + +This parameter enhances conflict resolution during `INSERT` operations in +scenarios where a row exists that meets unique constraints defined on the +table (rather than primary keys or replication identity). + +If this GUC is `enabled`, Spock will continue to check unique constraint +indexes, after checking the primary key / replica identity index. Only one +conflict will be resolved, using Last-Write-Wins logic. This includes the +primary key / replica identity index. If a second conflict occurs, an +exception is recorded in the `spock.exception_log` table. + +Partial unique constraints are supported, but nullable unique constraints +are not. Deferable constraints are not supported, are filtered out and are +not checked, and therefore may still cause an exception during the final +commit. + +Unique constraint indexes are checked in OID order, after the primary key. +It is therefore possible for the resolution of a unique constraint to change +the primary key value. + +Child tables / foreign keys are not checked or handled. Therefore, if +resolving a unique constraint changes the primary key value, you can +inadvertently create orphaned foreign key records. + +### `spock.conflict_resolution` + +`spock.conflict_resolution` sets the resolution method for any detected +conflicts between local data and incoming changes. Possible values are: + +* `error` - the replication will stop on error if conflict is detected and + manual action is needed for resolving +* `apply_remote` - always apply the change that's conflicting with local + data +* `keep_local` - keep the local version of the data and ignore the + conflicting change that is coming from the remote node +* `last_update_wins` - the version of data with newest commit timestamp + will be kept (this can be either local or remote version) + +To enable conflict resolution, the `track_commit_timestamp` setting must be +enabled. + +### `spock.deny_all_ddl` + +`spock.deny_all_ddl` is a boolean value (the default is `false`) that +enables or disables the execution of DDL statements within a Spock +configured cluster. This option can be set at postmaster startup, with the +SIGHUP mechanism, or on the command line with SQL if you're a superuser. + +### `spock.enable_ddl_replication` + +`spock.enable_ddl_replication` enables +[automatic replication](managing/spock_autoddl.md) of ddl statements through +the `default` replication set. + +### `spock.exception_behaviour` + +Use this GUC to specify the commit behavior of Postgres when it encounters +an ERROR within a transaction: + +* `transdiscard` (the default) - Set `spock.exception_behaviour` to + `transdiscard` to instruct the server to discard a transaction if any + operation within that transaction returns an `ERROR`. +* `discard` - Set `spock.exception_behaviour` to `discard` to continue + processing (and replicating) without interrupting server use. The server + will commit any operation (any portion of a transaction) that does not + return an `ERROR` statement. +* `sub_disable` - Set `spock.exception_behaviour` to `sub_disable` to + disable the subscription for the node on which the exception was reported. + Transactions for the disabled node will be added to a queue that is + written to the WAL log file; when the subscription is enabled, replication + will resume with the transaction that caused the exception, followed by + the other queued transactions. You can use the + `spock.alter_sub_skip_lsn` function to skip the transaction that caused + the exception and resume processing with the next transaction in the + queue. + +Note that the value you choose for `spock.exception_behaviour` could +potentially result in a large WAL log if transactions are allowed to +accumulate. + +### `spock.exception_logging` + +Use this GUC to specify which operations/transactions are written to the +exception log table: + +* `all` (the default) - Set `spock.exception_logging` to `all` to instruct + the server to record all transactions that contain one or more failed + operations in the `spock.exception_log` table. Note that this setting will + log all operations that are part of a transaction if one operation fails. +* `discard` - Add a row to the `spock.exception_log` table for any + discarded operation; successful transactions are not logged. +* `none` - Instructs the server to not log any operation or transactions to + the exception log table. + +### `spock.exception_replay_queue_size` - DEPRECATED + +This parameter will not be supported after version 5.X. + +When Spock encounters a replication exception, it attempts to resolve the +exception by entering exception-handling mode, based on the value of +`spock.exception_behaviour`. Spock then writes any transaction up to a +default size of `4MB` to memory, and the apply worker replays the +transaction from memory. This provides a massive speed and performance +increase in the handling of the vast majority of exceptions. The memory +size is configurable with `spock.exception_replay_queue_size`. + +Now, Spock performs as specified by the +[`spock.exception_behaviour`](#spock-exception_behaviour) parameter. + +### `spock.extra_connection_options` + +You can use the `spock.extra_connection_options` parameter in the +`postgresql.conf` file to assign connection options that apply to all +connections made by Spock. This can be a useful place to set up custom +keepalive options, etc. + +`spock` defaults to enabling TCP keepalives to ensure that it notices when +the upstream server disappears unexpectedly. To disable them add +`keepalives = 0` to `spock.extra_connection_options`. + +### `spock.include_ddl_repset` + +`spock.include_ddl_repset` enables spock to automatically add tables to +replication sets at the time they are created on each node. Tables with +primary keys will be added to the `default` replication set, and tables +without primary keys will be added to the `default_insert_only` replication +set. Altering a table to add or remove a primary key will make the correct +adjustment to which replication set the table is part of. Setting a table to +unlogged will remove it from replication. Detaching a partition will not +remove it from replication. + +### `spock.save_resolutions` + +`spock.save_resolutions` is a boolean value (the default is `false`) that +logs all conflict resolutions to the `spock.resolutions` table. This option +can only be set when the postmaster starts. + +### `spock.stats_max_entries` + +`spock.stats_max_entries` specifies the maximum number of entries that can +be stored into the Spock channel statistic. This option can only be set when +the postmaster starts. The parameter accepts values from `-1` to `INT_MAX` +(the default is `-1`). + +### `spock.temp_directory` + + `spock.temp_directory` defines the system path where temporary files + needed for schema synchronization are written. This path needs to exist + and be writable by the user running Postgres. The default is `empty`, + which tells Spock to use the default temporary directory based on + environment or operating system settings. + + diff --git a/docs/index.md b/docs/index.md index 38fc8609..73e36979 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,6 +1,6 @@ # The Spock Extension -The Spock extension provides multi-master (active-active) replication for pgEdge Distributed and Enterprise Postgres, versions 15, 16, and 17. The extension leverages the pgLogical open-source project as a solid foundation to build upon for this enterprise-class extension. +The Spock extension provides multi-master (active-active) replication for pgEdge Distributed and Enterprise Postgres, versions 15, 16, 17, and 18. The extension leverages the pgLogical open-source project as a solid foundation to build upon for this enterprise-class extension. !!! info "Welcome to Spock - Live Long and Prosper!" diff --git a/docs/install_spock.md b/docs/install_spock.md index 01c2bc85..92e0248b 100644 --- a/docs/install_spock.md +++ b/docs/install_spock.md @@ -18,11 +18,11 @@ For detailed information about using Spock to create a two-node cluster, visit [here](two_node_cluster.md). -## Installing Spock with pgEdge Distributed or Enterprise Postgres +## Installing Spock with pgEdge Enterprise Postgres -The latest Spock extension is automatically installed and created with any -[pgEdge Distributed or Enterprise Postgres](https://docs.pgedge.com/) -installation. A pgEdge deployment provides quick and easy access to: +The latest Spock extension is automatically installed with any +[pgEdge Enterprise Postgres](https://docs.pgedge.com/enterprise) installation. +A pgEdge deployment provides quick and easy access to: * the latest minor version of your preferred Postgres version. * Spock [functions and procedures](spock_functions/index.md). @@ -33,7 +33,8 @@ installation. A pgEdge deployment provides quick and easy access to: * [LOLOR](https://docs.pgedge.com/lolor/v1-2-2/) large object support. * management tools for your active-active distributed cluster. -### Building the Extension from Source + +## Building the Extension from Source Spock is available as an [open-source project](https://github.com/pgEdge/spock) that you can build @@ -71,47 +72,75 @@ other Postgres extension: `make-install` -Then, update your `postgresql.conf` file, setting: -```bash -shared_preload_libraries = 'spock' -track_commit_timestamp = on # needed for conflict resolution -``` +## Configuring Postgres for Spock Replication -Then, connect to the database with psql, and use the `CREATE EXTENSION` -command to create the Spock extension: - - `CREATE EXTENSION spock;` +After installing Postgres on each node that will host a spock instance, +use [initdb](https://www.postgresql.org/docs/18/app-initdb.html) to initialize +a Postgres cluster. After initializing the cluster, you can connect +with psql, and query the server for file locations and cluster details: +```sql +[sdouglas@lima-pg3 pg18]$ sudo -u postgres psql -U postgres -p 5432 +psql (18.1) +Type "help" for help. + +postgres=# SHOW data_directory; +SHOW config_file; +SHOW hba_file; + + data_directory +------------------------ + /var/lib/pgsql/18/data +(1 row) + + config_file +---------------------------------------- + /var/lib/pgsql/18/data/postgresql.conf +(1 row) + + hba_file +------------------------------------ + /var/lib/pgsql/18/data/pg_hba.conf +(1 row) +``` -### Basic Configuration and Usage +Then, use your choice of editor to update the `postgresql.conf` file, adding +required Postgres parameters to the end of the file: -!!! info +```sql +wal_level = logical +max_worker_processes = 10 +max_replication_slots = 10 +max_wal_senders = 10 +shared_preload_libraries = 'spock' +track_commit_timestamp = on +``` - pgEdge Distributed and Enterprise Postgres simplify installation and - management of a distributed multi-master active-active replication - cluster. When deployed with pgEdge Postgres, replication clusters are - automatically configured and deployed with Spock, ACE, Snowflake, and - more. +After modifying the Postgres parameters, use your OS-specific command to +restart the Postgres server: -If you're not deploying a cluster with pgEdge Postgres, you will need to -configure your server to support logical decoding: +```bash +[sdouglas@lima-pg3 pg18]$ sudo systemctl restart postgresql-18 +[sdouglas@lima-pg3 pg18]$ sudo systemctl status postgresql-18 +● postgresql-18.service - PostgreSQL 18 database server + Loaded: loaded (/usr/lib/systemd/system/postgresql-18.service; enabled; preset: disabled) + Active: active (running) since Wed 2026-02-11 11:44:11 EST; 7s ago +``` +Then, connect to the psql command line, and create the spock extension: + ```sql - wal_level = 'logical' - max_worker_processes = 10 # one per database needed on provider node - # one per node needed on subscriber node - max_replication_slots = 10 # one per node needed on provider node - max_wal_senders = 10 # one per node needed on provider node - shared_preload_libraries = 'spock' - track_commit_timestamp = on # needed for conflict resolution +postgres=# CREATE EXTENSION spock; +CREATE EXTENSION +postgres=# ``` -Also, you will need to configure your `pg_hba.conf` file to allow logical -replication connections from localhost. Logical replication connections are -treated by `pg_hba.conf` as regular connections to the provider database. +### Creating a Replication Scenario -Then, use the `spock.node_create` command to create the provider node: +After configuring the nodes, connect to the psql command line of the first +provider node, and use the `spock.node_create` command to create the provider +node: ```sql SELECT spock.node_create( @@ -148,192 +177,3 @@ replication on the subscriber node: SELECT spock.sub_wait_for_sync('subscription1'); ``` - -### Advanced Configuration Options for Spock - -You can use the following configuration parameters (GUCs) on the psql -command line with a `SET` statement or in a Postgres configuration file -(like `postgresql.conf`) to configure a Spock installation. - -#### `spock.allow_ddl_from_functions` - -`spock.allow_ddl_from_functions` enables spock to automatically replicate -DDL statements that are called within functions to also be automatically -replicated. This can be turned `off` if these functions are expected to run -on every node. When this is set to `off`, statements replicated from -functions adhere to the same rule previously described for -`include_ddl_repset`. If a table possesses a defined primary key, it will be -added into the `default` replication set; alternatively, they will be added -to the `default_insert_only` replication set. - -#### `spock.batch_inserts` - -`spock.batch_inserts` tells Spock to use batch insert mechanism if -possible. The batch mechanism uses Postgres internal batch insert mode which -is also used by `COPY` command. - -#### `spock.channel_counters` - -`spock.channel_counters` is a boolean value (the default is `true`) that -enables or disables the Spock channel statistic information collection. This -option can only be set when the postmaster starts. - -#### `spock.check_all_uc_indexes` - -!!! info - - This feature is experimental and `OFF` by default. Use this feature at - your own risk. - -This parameter enhances conflict resolution during `INSERT` operations in -scenarios where a row exists that meets unique constraints defined on the -table (rather than primary keys or replication identity). - -If this GUC is `enabled`, Spock will continue to check unique constraint -indexes, after checking the primary key / replica identity index. Only one -conflict will be resolved, using Last-Write-Wins logic. This includes the -primary key / replica identity index. If a second conflict occurs, an -exception is recorded in the `spock.exception_log` table. - -Partial unique constraints are supported, but nullable unique constraints -are not. Deferable constraints are not supported, are filtered out and are -not checked, and therefore may still cause an exception during the final -commit. - -Unique constraint indexes are checked in OID order, after the primary key. -It is therefore possible for the resolution of a unique constraint to change -the primary key value. - -Child tables / foreign keys are not checked or handled. Therefore, if -resolving a unique constraint changes the primary key value, you can -inadvertently create orphaned foreign key records. - -#### `spock.conflict_resolution` - -`spock.conflict_resolution` sets the resolution method for any detected -conflicts between local data and incoming changes. Possible values are: - -* `error` - the replication will stop on error if conflict is detected and - manual action is needed for resolving -* `apply_remote` - always apply the change that's conflicting with local - data -* `keep_local` - keep the local version of the data and ignore the - conflicting change that is coming from the remote node -* `last_update_wins` - the version of data with newest commit timestamp - will be kept (this can be either local or remote version) - -To enable conflict resolution, the `track_commit_timestamp` setting must be -enabled. - -#### `spock.deny_all_ddl` - -`spock.deny_all_ddl` is a boolean value (the default is `false`) that -enables or disables the execution of DDL statements within a Spock -configured cluster. This option can be set at postmaster startup, with the -SIGHUP mechanism, or on the command line with SQL if you're a superuser. - -#### `spock.enable_ddl_replication` - -`spock.enable_ddl_replication` enables -[automatic replication](managing/spock_autoddl.md) of ddl statements through -the `default` replication set. - -#### `spock.exception_behaviour` - -Use this GUC to specify the commit behavior of Postgres when it encounters -an ERROR within a transaction: - -* `transdiscard` (the default) - Set `spock.exception_behaviour` to - `transdiscard` to instruct the server to discard a transaction if any - operation within that transaction returns an `ERROR`. -* `discard` - Set `spock.exception_behaviour` to `discard` to continue - processing (and replicating) without interrupting server use. The server - will commit any operation (any portion of a transaction) that does not - return an `ERROR` statement. -* `sub_disable` - Set `spock.exception_behaviour` to `sub_disable` to - disable the subscription for the node on which the exception was reported. - Transactions for the disabled node will be added to a queue that is - written to the WAL log file; when the subscription is enabled, replication - will resume with the transaction that caused the exception, followed by - the other queued transactions. You can use the - `spock.alter_sub_skip_lsn` function to skip the transaction that caused - the exception and resume processing with the next transaction in the - queue. - -Note that the value you choose for `spock.exception_behaviour` could -potentially result in a large WAL log if transactions are allowed to -accumulate. - -#### `spock.exception_logging` - -Use this GUC to specify which operations/transactions are written to the -exception log table: - -* `all` (the default) - Set `spock.exception_logging` to `all` to instruct - the server to record all transactions that contain one or more failed - operations in the `spock.exception_log` table. Note that this setting will - log all operations that are part of a transaction if one operation fails. -* `discard` - Add a row to the `spock.exception_log` table for any - discarded operation; successful transactions are not logged. -* `none` - Instructs the server to not log any operation or transactions to - the exception log table. - -#### `spock.exception_replay_queue_size` - DEPRECATED - -This parameter will not be supported after version 5.X. - -When Spock encounters a replication exception, it attempts to resolve the -exception by entering exception-handling mode, based on the value of -`spock.exception_behaviour`. Spock then writes any transaction up to a -default size of `4MB` to memory, and the apply worker replays the -transaction from memory. This provides a massive speed and performance -increase in the handling of the vast majority of exceptions. The memory -size is configurable with `spock.exception_replay_queue_size`. - -Now, Spock performs as specified by the -[`spock.exception_behaviour`](#spock-exception_behaviour) parameter. - -#### `spock.extra_connection_options` - -You can use the `spock.extra_connection_options` parameter in the -`postgresql.conf` file to assign connection options that apply to all -connections made by Spock. This can be a useful place to set up custom -keepalive options, etc. - -`spock` defaults to enabling TCP keepalives to ensure that it notices when -the upstream server disappears unexpectedly. To disable them add -`keepalives = 0` to `spock.extra_connection_options`. - -#### `spock.include_ddl_repset` - -`spock.include_ddl_repset` enables spock to automatically add tables to -replication sets at the time they are created on each node. Tables with -primary keys will be added to the `default` replication set, and tables -without primary keys will be added to the `default_insert_only` replication -set. Altering a table to add or remove a primary key will make the correct -adjustment to which replication set the table is part of. Setting a table to -unlogged will remove it from replication. Detaching a partition will not -remove it from replication. - -#### `spock.save_resolutions` - -`spock.save_resolutions` is a boolean value (the default is `false`) that -logs all conflict resolutions to the `spock.resolutions table`. This option -can only be set when the postmaster starts. - -#### `spock.stats_max_entries` - -`spock.stats_max_entries` specifies the maximum number of entries that can -be stored into the Spock channel statistic. This option can only be set when -the postmaster starts. The parameter accepts values from `-1` to `INT_MAX` -(the default is `-1`). - -#### `spock.temp_directory` - - `spock.temp_directory` defines the system path where temporary files - needed for schema synchronization are written. This path needs to exist - and be writable by the user running Postgres. The default is `empty`, - which tells Spock to use the default temporary directory based on - environment or operating system settings. - - diff --git a/docs/spock_functions/functions/index.md b/docs/spock_functions/functions/index.md deleted file mode 100644 index 977dfb2e..00000000 --- a/docs/spock_functions/functions/index.md +++ /dev/null @@ -1,5 +0,0 @@ -# Spock Functions - -You can access the reference pages in this section via online help; at the terminal window, enter: - -`command_name --help` \ No newline at end of file diff --git a/docs/spock_functions/functions/spock_gen_slot_name.md b/docs/spock_functions/functions/spock_gen_slot_name.md new file mode 100644 index 00000000..aa24f9f0 --- /dev/null +++ b/docs/spock_functions/functions/spock_gen_slot_name.md @@ -0,0 +1,50 @@ +## NAME + +spock.spock_gen_slot_name() + +### SYNOPSIS + +spock.spock_gen_slot_name(dbname name, provider_node name, +subscription name) + +### RETURNS + +The generated replication slot name as a name type. + +### DESCRIPTION + +Generates a standardized replication slot name for a Spock subscription. + +This function creates a consistent naming convention for replication slots +based on the database name, provider node name, and subscription name. The +generated name follows Spock's internal naming scheme to ensure uniqueness +and traceability. + +This function that performs a calculation without accessing the +database or modifying any data. It can be used to predict what slot name +Spock will generate for a given subscription configuration. + +### ARGUMENTS + +dbname + + The name of the database. + +provider_node + + The name of the provider node. + +subscription + + The name of the subscription. + +### EXAMPLE + +The following example executes the function; the database name is postgres, +the node name is n1, and the subscription name is sub_n2n1: + + postgres=# SELECT spock.spock_gen_slot_name('postgres', 'n1', 'sub_n2n1'); + spock_gen_slot_name + -------------------------- + spk_postgres_n1_sub_n2n1 + (1 row) diff --git a/docs/spock_functions/functions/spock_get_country.md b/docs/spock_functions/functions/spock_get_country.md new file mode 100644 index 00000000..1e3a0a2f --- /dev/null +++ b/docs/spock_functions/functions/spock_get_country.md @@ -0,0 +1,47 @@ +## NAME + +spock.get_country() + +### SYNOPSIS + +spock.get_country () + +### RETURNS + +The country code configured for the node as text, or `??` if not set. + +### DESCRIPTION + +Returns the spock.country property that was set during node creation. You can +modify the property after configuration; modification requires a server +restart to apply the change. + +This function retrieves the value of the spock.country configuration +parameter for the current session. The country code is typically used in +multi-region or geo-distributed replication topologies to identify the +geographic location of nodes and apply region-specific routing or filtering +rules. + +This function does not modify any data. + +### ARGUMENTS + +This function takes no arguments. + +### EXAMPLE + +If the spock.country parameter has not been set, the function returns: + + postgres=# SELECT spock.get_country(); + get_country + ------------- + ?? + (1 row) + +If spock.country is set to `US`, the function returns: + + postgres=# SELECT spock.get_country(); + get_country + ------------- + US + (1 row) \ No newline at end of file diff --git a/docs/spock_functions/functions/spock_max_proto_version.md b/docs/spock_functions/functions/spock_max_proto_version.md new file mode 100644 index 00000000..8b3122db --- /dev/null +++ b/docs/spock_functions/functions/spock_max_proto_version.md @@ -0,0 +1,48 @@ +## NAME + +spock.spock_max_proto_version() + +### SYNOPSIS + +spock.spock_max_proto_version() + +### RETURNS + +The maximum spock protocol version supported by the installed Spock extension +as an integer. The protocol version determines compatibility between Spock +versions. + +### DESCRIPTION + +Returns the maximum protocol version supported by the Spock extension. + +This function queries the Spock extension and returns the highest protocol +version number it can use for replication communication. The protocol +version determines the features and capabilities available for replication +between nodes. + +When establishing replication connections between nodes running different +Spock versions, the nodes negotiate to use the lower of the two maximum +protocol versions. This ensures compatibility between nodes running +different Spock releases. + +The protocol version is returned as an integer value. Higher numbers +indicate newer protocol versions with additional features. + +This is a read-only query function that does not modify data. + +### ARGUMENTS + +This function takes no arguments. + +### EXAMPLE + +The following command shows that the current version of Spock uses protocol +version 4: + + postgres=# SELECT spock.spock_max_proto_version(); + spock_max_proto_version + ------------------------- + 4 + (1 row) + diff --git a/docs/spock_functions/functions/spock_min_proto_version.md b/docs/spock_functions/functions/spock_min_proto_version.md new file mode 100644 index 00000000..211f06e6 --- /dev/null +++ b/docs/spock_functions/functions/spock_min_proto_version.md @@ -0,0 +1,49 @@ +## NAME + +spock.spock_min_proto_version() + +### SYNOPSIS + +spock.spock_min_proto_version() + +### RETURNS + +The minimum Spock protocol version supported by the installed Spock extension +as an integer. The protocol version determines compatibility between Spock +versions. + +### DESCRIPTION + +Returns the minimum protocol version supported by the Spock extension. + +This function queries the Spock extension and returns the lowest protocol +version number it can use for replication communication. The protocol +version determines the features and capabilities available for replication +between nodes. + +When establishing replication connections between nodes running different +Spock versions, the nodes negotiate to use a protocol version that falls +within the supported range of both nodes. The minimum protocol version +defines the lower bound of compatibility - nodes supporting only protocol +versions below this minimum cannot replicate with this Spock installation. + +The protocol version is returned as an integer value. This minimum version +ensures backward compatibility with older Spock releases while still +allowing the use of newer protocol features when both nodes support them. + +This is a read-only query function that does not modify data. + +### ARGUMENTS + +This function takes no arguments. + +### EXAMPLE + +The following command shows that the version of Spock in use is using protocol +version 3: + + postgres=# SELECT spock.spock_min_proto_version(); + spock_min_proto_version + ------------------------- + 3 + (1 row) diff --git a/docs/spock_functions/functions/spock_node_add_interface.md b/docs/spock_functions/functions/spock_node_add_interface.md index ff3197d9..d8ee6aff 100644 --- a/docs/spock_functions/functions/spock_node_add_interface.md +++ b/docs/spock_functions/functions/spock_node_add_interface.md @@ -5,22 +5,57 @@ ### SYNOPSIS -`spock.node_add_interface (node_name name, interface_name name, dsn text)` +spock.node_add_interface (node_name name, interface_name name, dsn text) + +### RETURNS + +The OID of the newly created interface. Returns NULL if any argument is NULL. ### DESCRIPTION -Add an additional interface to a spock node. - -When a node is created, the interface is also created using the dsn specified in the create_node command, and with the same name as the node. This interface allows you to add alternative interfaces with different connection strings to an existing node. +Adds a new network interface definition to an existing Spock node. + +Interfaces allow a single node to be reachable through multiple connection +endpoints. This is commonly used in environments where nodes are accessible +through different hostnames, IP addresses, private networks, public networks, +or load balancers. + +The interface definition consists of a name and a PostgreSQL DSN string that +describes how other nodes should connect to this node. + +This function writes metadata into the Spock catalogs but does not modify +PostgreSQL server configuration or networking settings. + +This command must be executed by a superuser. + +### ARGUMENTS + +node_name + + The name of an existing Spock node. + +interface_name + + A unique name for this interface on the node. + +dsn + + A PostgreSQL connection string that other nodes will use to connect to + this node. The user in this string should equal the OS user. This + connection string should be reachable from outside and match the one used + later in the sub-create command. Example: host=10.1.2.5 port=5432 + user=rocky + +### EXAMPLE -### EXAMPLE +The following example adds an interface named private_net that uses the +connection string defined in the last argument of the function call +('host=10.0.0.10 port=5432 dbname=postgres'): -`spock.node_add_interface ('n1', 'n1_2', 'host=10.1.2.5 user=rocky')` + postgres=# SELECT spock.node_add_interface('n1', 'private_net', + 'host=10.0.0.10 port=5432 dbname=postgres'); -### POSITIONAL ARGUMENTS - node_name - The name of the node. Should reference the node already created in this database. Example: n1 - interface_name - The interface name to add to the node. The interface created by default matches the node name, add a new interface with a unique name. Example: n1_2 - dsn - The additional connection string to the node. The user in this string should equal the OS user. This connection string should be reachable from outside and match the one used later in the sub-create command. Example: host=10.1.2.5 port= 5432 user=rocky + node_add_interface + -------------------- + 1239112588 + (1 row) diff --git a/docs/spock_functions/functions/spock_node_create.md b/docs/spock_functions/functions/spock_node_create.md index 1e7c4211..e0a23bbe 100644 --- a/docs/spock_functions/functions/spock_node_create.md +++ b/docs/spock_functions/functions/spock_node_create.md @@ -1,21 +1,67 @@ ## NAME -`spock.node_create()` +spock.node_create() ### SYNOPSIS -`spock.node_create (node_name name, dsn text, location text, country text, info jsonb)` - +spock.node_create (node_name name, dsn text, location text, +country text, info jsonb) + +### RETURNS + +The OID of the newly created Spock node. + ### DESCRIPTION -Create a spock node. +Creates a new Spock node definition. + +A node represents a PostgreSQL instance that participates in Spock +replication. The node definition includes the connection information that +other nodes will use to communicate with this node. + +Optional metadata fields such as location, country, and info can be provided +to describe the node for organizational or management purposes. These values +are stored in the Spock catalogs but are not required for replication to +function. + +This function writes metadata into the Spock catalogs but does not modify +Postgres server configuration or networking settings. + +This command must be executed by a superuser. + +### ARGUMENTS + +node_name + + A unique name for the Spock node. + +dsn + + A PostgreSQL connection string that other nodes will use to + connect to this node. + +location + + Optional descriptive text indicating the physical or logical + location of the node. The default is NULL. + +country + + Optional country code or name associated with the node. The default is + NULL. + +info + + Optional JSONB field for storing arbitrary metadata about + the node. The default is NULL. + +### EXAMPLE -### EXAMPLE +In the following example, Spock creates a node named n3, with a connection +to a database named 'inventory': -`spock.node_create ('n1', 'host=10.1.2.5 user=rocky dbname=demo')` - -### POSITIONAL ARGUMENTS - node_name - The name of the node. Only one node is allowed per database, and each node in a cluster must have a unique name. To use the Snowflake extension, use the convention n1,n2, etc. Example: n1 - dsn - The connection string to the node. The user in this string should equal the OS user. This connection string should be reachable from outside and match the one used later in the sub-create command. Example: host=10.1.2.5 port= 5432 user=rocky dbname=demo + inventory=# SELECT spock.node_create('n3','host=10.0.0.12 port=5432 dbname=inventory'); + node_create + ------------- + 9057 + (1 row) diff --git a/docs/spock_functions/functions/spock_node_drop.md b/docs/spock_functions/functions/spock_node_drop.md index 8c5dff87..9ed9a2ef 100644 --- a/docs/spock_functions/functions/spock_node_drop.md +++ b/docs/spock_functions/functions/spock_node_drop.md @@ -1,20 +1,53 @@ ## NAME -`spock.node_drop()` +spock.node_drop() ### SYNOPSIS -`spock.node_drop (node_name name, ifexists bool)` - +spock.node_drop(node_name name, ifexists boolean) + +### RETURNS + + - true if the node was dropped successfully. + + - false if the node did not exist and ifexists was set to true. + + - ERROR if the call has invalid parameters, if the invoker has insufficient + privileges, or the node cannot be removed due to existing dependencies. + ### DESCRIPTION - Drop a spock node. -### EXAMPLE +Removes an existing Spock node from the cluster metadata. + +This function deletes the node definition and all associated metadata from the +Spock catalogs. It does not remove any PostgreSQL data directory or stop +the PostgreSQL server; it only removes Spock’s logical representation of the +node. + +If ifexists is set to false (default), an error is raised when the specified +node does not exist. If ifexists is true, the function returns false +instead of raising an error. + +This command must be executed by a superuser and modifies Spock catalog +tables. + +### ARGUMENTS + +node_name + + The name of the existing Spock node to remove. + +ifexists + + If true, do not raise an error when the node does not exist; return + false instead. Default is false. + +### EXAMPLE + +The following function call drops a node named 'n3': -`spock.node_drop ('n1')` - -### POSITIONAL ARGUMENTS - node_name - The name of the node. Example: n1 - ifexists - `ifexists` specifies the Spock extension behavior with regards to error messages. If `true`, an error is not thrown when the specified node does not exist. The default is `false`. + inventory=# SELECT spock.node_drop('n3'); + node_drop + ----------- + t + (1 row) diff --git a/docs/spock_functions/functions/spock_node_drop_interface.md b/docs/spock_functions/functions/spock_node_drop_interface.md index 58b8569a..3bef8944 100644 --- a/docs/spock_functions/functions/spock_node_drop_interface.md +++ b/docs/spock_functions/functions/spock_node_drop_interface.md @@ -1,21 +1,53 @@ ## NAME -`spock.node_drop_interface()` +spock.node_drop_interface() ### SYNOPSIS -`spock.node_drop_interface (node_name name, interface_name name)` - +spock.node_drop_interface (node_name name, interface_name name) + +### RETURNS + + - true if the interface was dropped successfully. + + - false if the interface does not exist. + + - ERROR if the call has invalid parameters, insufficient privileges, or + the interface cannot be removed. + ### DESCRIPTION -Drop an interface from a spock node. +Removes an existing network interface definition from a Spock node. + +This function deletes the interface definition from the Spock catalogs. It +does not modify PostgreSQL server configuration or networking settings; it +only removes Spock's logical representation of the interface. + +Other nodes that were using this interface to connect will no longer be able +to use it. Ensure no active subscriptions are relying on this interface +before removing it. + +This function writes metadata into the Spock catalogs but does not modify +PostgreSQL server configuration or networking settings. + +This command must be executed by a superuser. + +### ARGUMENTS + +node_name + + The name of an existing Spock node. + +interface_name + + The name of the interface to remove from the node. ### EXAMPLE -`spock.node_drop_interface (n1 n1_2 demo)` - -### POSITIONAL ARGUMENTS - node_name - The name of the node. Example: n1 - interface_name - The interface name (the named DSN created with `spock node-add-interface`) to remove from the node. Example: n1_2 +The following example drops an interface named 'private_net': + + inventory=# SELECT spock.node_drop_interface('n3', 'private_net'); + node_drop_interface + --------------------- + t + (1 row) diff --git a/docs/spock_functions/functions/spock_node_info.md b/docs/spock_functions/functions/spock_node_info.md new file mode 100644 index 00000000..7a234839 --- /dev/null +++ b/docs/spock_functions/functions/spock_node_info.md @@ -0,0 +1,48 @@ +## NAME + +spock.node_info() + +### SYNOPSIS + +spock.node_info() + +### RETURNS + +A record containing information about the local Spock node: + + - node_id is the OID of the node. + - node_name is the name of the node. + - sysid is the system identifier. + - dbname is the database name. + - replication_sets is the available replication sets. + - location is the node location (if set). + - country is the node country (if set). + - info is additional metadata stored in JSONB format (if set). + +### DESCRIPTION + +Returns information about the local Spock node. + +This function queries the Spock catalogs and returns metadata about the +current node, including its identifier, name, database information, and any +optional descriptive fields that were set during node creation. + +This is a read-only query function that does not modify data. + +### ARGUMENTS + +This function takes no arguments. + +### EXAMPLE + + postgres=# SELECT * FROM spock.node_info(); + + -[ RECORD 1 ]----+-------------------------------------- + node_id | 49708 + node_name | n1 + sysid | 7600444661598442547 + dbname | postgres + replication_sets | "default",default_insert_only,ddl_sql + location | + country | + info | diff --git a/docs/spock_functions/functions/spock_replicate_ddl.md b/docs/spock_functions/functions/spock_replicate_ddl.md index 8f1b94ea..e41ccd9c 100644 --- a/docs/spock_functions/functions/spock_replicate_ddl.md +++ b/docs/spock_functions/functions/spock_replicate_ddl.md @@ -1,17 +1,89 @@ ## NAME -`spock.replicate_ddl()` +spock.replicate_ddl() ### SYNOPSIS -`spock.replicate_ddl(command text, repsets text[])` - +spock.replicate_ddl (command text[], replication_sets text[], search_path text, +role text) + +### RETURNS + + - true if the DDL command was successfully executed and queued for + replication. + + - false if the command execution fails or replication queueing fails. + +When called with a text array, returns a set of boolean values, one for +each command in the array. + ### DESCRIPTION -Execute locally and then send the specified command to the replication queue for execution on subscribers which are subscribed to one of the specified `repsets`. - +Executes DDL commands locally and queues them for replication to subscribers. + +This function runs the specified DDL command on the local node first, then +adds it to the replication queue for execution on all subscriber nodes that +are subscribed to one of the specified replication sets. + +The function accepts either a single DDL command as text or an array of DDL +commands. When provided with an array, each command is executed and queued +sequentially. + +The search_path and role parameters allow you to control the execution +context for the DDL command on both the provider and subscriber nodes. + +This function writes to the replication queue and modifies the database +schema locally before propagating changes. + +This command must be executed by a user with sufficient privileges to execute +the DDL command. + ### ARGUMENTS - command - The DDL query to execute. - repsets - The array of replication sets which this command should be associated with, default "{ddl_sql}". + +command + + A DDL command as text, or an array of DDL commands to execute and + replicate. + +replication_sets + + An array of replication set names. Only subscribers subscribed to one + of these sets will receive and execute the DDL. Default is '{ddl_sql}'. + +search_path + + The schema search path to use when executing the DDL command. Default + is the current session's search_path setting. + +role + + The role (user) under which to execute the DDL command. Default is the + current user. + +### EXAMPLES + + The following command creates a table and instructs Spock to replicate the + DDL to other nodes: + + postgres=# SELECT spock.replicate_ddl('CREATE TABLE users (id SERIAL PRIMARY KEY, + name TEXT)'); + -[ RECORD 1 ]-+-- + replicate_ddl | t + + The following command alters the table adding a column, and instructs Spock + to replicate the DDL to other nodes: + + postgres=# SELECT spock.replicate_ddl('ALTER TABLE users ADD COLUMN email TEXT', + '{default,ddl_sql}'); + -[ RECORD 1 ]-+-- + replicate_ddl | t + + The following command creates a table and an index on the tables, and + instructs Spock to replicate DDL to other nodes: + + postgres=# SELECT spock.replicate_ddl(ARRAY['CREATE TABLE orders (id SERIAL)', + 'CREATE INDEX ON orders(id)']); + -[ RECORD 1 ]-+-- + replicate_ddl | t + -[ RECORD 2 ]-+-- + replicate_ddl | t \ No newline at end of file diff --git a/docs/spock_functions/functions/spock_repset_add_all_seqs.md b/docs/spock_functions/functions/spock_repset_add_all_seqs.md index 17527432..cb50d17c 100644 --- a/docs/spock_functions/functions/spock_repset_add_all_seqs.md +++ b/docs/spock_functions/functions/spock_repset_add_all_seqs.md @@ -1,21 +1,69 @@ ## NAME -`spock.repset_add_all_seqs()` +spock.repset_add_all_seqs() ### SYNOPSIS -`spock.repset_add_all_seqs(set_name name, schema_names text[], sync_data boolean)` - +spock.repset_add_all_seqs (set_name name, schema_names text[], +synchronize_data boolean) + +### RETURNS + + - true if all sequences were successfully added to the replication set. + + - ERROR if the call has invalid parameters, insufficient privileges, or + the operation fails. + ### DESCRIPTION -Adds all sequences from the given schemas. This command only adds existing sequences; if you create additional sequences, add those sequences to your repset with the `spock.repset_add_seq` function. +Adds all existing sequences from the specified schemas to a replication set. + +This function registers all sequence objects found in the given schemas with +the specified replication set. Only sequences that exist at the time of +execution are added; sequences created afterward must be added separately +using spock.repset_add_seq(). + +The synchronize_data parameter controls whether sequence values are +immediately synchronized across nodes. When set to true, the current value of +each sequence is propagated to subscribers. + +This function writes metadata into the Spock catalogs to track which +sequences are part of the replication set. + +This command must be executed by a superuser. + +**Warning:** If you're deploying a multi-master replication scenario, we +recommend that you not add sequences to a replication set. Instead, use +[Snowflake Sequences](https://github.com/pgEdge/snowflake) to manage +sequences in a distributed environment and avoid conflicts. ### ARGUMENTS - set_name - The name of the existing replication set. - schema_names - An array of names name of existing schemas from which tables should be added. - sync_data - If true, the sequence value will be synchronized immediately; the default is false. - - *Warning:* If you're deploying a multi-master replication scenario, we recommend that you not add sequences to a replication set. Instead, use the [Snowflake Sequences](https://github.com/pgEdge/snowflake-sequences) to manage sequences. + +set_name + + The name of an existing replication set. + +schema_names + + An array of schema names from which all sequences will be added. + +synchronize_data + + If true, synchronize the current value of each sequence immediately. + Default is false. + +### EXAMPLE + +You can specify one or more schemas in the call to repset_add_all_seqs. +The first command adds the sequences in the 'public' schema to the 'default' +replication set: + + postgres=# SELECT spock.repset_add_all_seqs('default', ARRAY['public']); + -[ RECORD 1 ]-------+-- + repset_add_all_seqs | t + +The following command adds all sequences in the schemas named 'public' and 'alice' to the 'default' replication set: + + postgres=# SELECT spock.repset_add_all_seqs('default', ARRAY['public', 'alice'],true); + -[ RECORD 1 ]-------+-- + repset_add_all_seqs | t \ No newline at end of file diff --git a/docs/spock_functions/functions/spock_repset_add_all_tables.md b/docs/spock_functions/functions/spock_repset_add_all_tables.md index a61ed11d..fb196805 100644 --- a/docs/spock_functions/functions/spock_repset_add_all_tables.md +++ b/docs/spock_functions/functions/spock_repset_add_all_tables.md @@ -1,24 +1,61 @@ ## NAME -`spock.repset_add_all_tables()` +spock.repset_add_all_tables() ### SYNOPSIS -`spock.repset_add_table (set_name name, schema_names text[], sync_data boolean)` - +spock.repset_add_all_tables (set_name name, schema_names text[], +synchronize_data boolean) + +### RETURNS + + - true if all tables were successfully added to the replication set. + + - false if the call has invalid parameters, insufficient privileges, or + the operation fails. + ### DESCRIPTION -Adds all tables in given schemas. Only existing tables are added; any table you create in future will not be added automatically. +Adds all existing tables from the specified schemas to a replication set. -### EXAMPLE +This function registers all table objects found in the given schemas with the +specified replication set. Only tables that exist at the time of execution +are added; tables created afterward must be added separately using +spock.repset_add_table(). + +The synchronize_data parameter controls whether existing table data is +immediately synchronized to subscribers. When set to true, a full table copy +is initiated for each table on all subscribers subscribed to the replication +set. + +This function writes metadata into the Spock catalogs to track which tables +are part of the replication set. + +This command must be executed by a superuser. -`spock.repset_add_table ('demo_repset', 'public')` - ### ARGUMENTS - set_name - The name of the existing replication set. - schema_names - An array of names of existing schemas from which tables should be added. - sync_data. - If true, the table data is synchronized on all subscribers which are subscribed to given replication set; the default is false. + +set_name + + The name of an existing replication set. + +schema_names + + An array of schema names from which all tables will be added. + +synchronize_data + + If true, synchronize existing table data to all subscribers + immediately. Default is false. + +### EXAMPLE + +Specify the names of one or more schemas in an array to add all of the +tables in the schemas to the specified replication set; the following +command adds all of the tables in the public schema to the default +replication set: + + postgres=# SELECT spock.repset_add_all_tables('default', ARRAY['public']); + -[ RECORD 1 ]---------+-- + repset_add_all_tables | t diff --git a/docs/spock_functions/functions/spock_repset_add_partition.md b/docs/spock_functions/functions/spock_repset_add_partition.md index 48a12fcd..dd2bfda3 100644 --- a/docs/spock_functions/functions/spock_repset_add_partition.md +++ b/docs/spock_functions/functions/spock_repset_add_partition.md @@ -2,28 +2,73 @@ `spock.repset-add-partition ()` -## SYNOPSIS - -`spock.repset-add-partition (PARENT_TABLE DB )` - -## DESCRIPTION - -Add a partition to the same replication set that the parent table is a part of. - -## EXAMPLE - -`spock.repset-add-partition (mytable demo --partition=mytable_202012)` - -## POSITIONAL ARGUMENTS - PARENT_TABLE - The name of the parent table. Example: mytable - DB - The name of the database. Example: demo - -## FLAGS - -p, --partition=PARTITION - The name of the partition. If none is provided, it will add all unreplicated partitions to the replication set. Example: mytable_202012 - - -r, --row_filter=ROW_FILTER - The row filtering expression. Example: my_id = 1001 - +### SYNOPSIS + +spock.repset_add_partition ( + parent regclass, + partition regclass, + row_filter text) + +### RETURNS + + - 0 if the parent table is not a partitioned table or if all partitions + are already in the replication sets. + + - A positive integer indicating the count of partitions successfully + added to replication sets. + +### DESCRIPTION + +Adds a partition of a table to the same replication set(s) as its parent +table. + +This function is used when working with partitioned tables to ensure that +new or existing partitions are properly included in replication. If the +partition argument is omitted, Spock automatically discovers and adds all +partitions of the specified parent table. + +An optional row_filter can be supplied to limit which rows from the +partition are replicated. + +This function writes metadata into the Spock catalogs and does not modify +PostgreSQL configuration. + +This command must be executed by a superuser. + +### ARGUMENTS + +parent + + The parent partitioned table, specified as a regclass. + +partition + + A specific partition table to add. If omitted, all + partitions of the parent are added. The default is NULL. + +row_filter + + An optional SQL expression used to filter which rows + from the partition are replicated. The default is NULL. + +### EXAMPLE + +Add all four partitions of a parent table (named 'public.sales_parent') to +the replication set to which the parent belongs: + + SELECT spock.repset_add_partition('public.sales_parent'); + -[ RECORD 1 ]--------+-- + repset_add_partition | 4 + +Add a specific partition (named public.sales_2026_q1) to the replication +set to which the parent table belongs: + + postgres=# SELECT spock.repset_add_partition( + 'public.sales_parent', + 'public.sales_2026_q1', + 'region = ''US''' + ); + -[ RECORD 1 ]--------+-- + repset_add_partition | 1 + + diff --git a/docs/spock_functions/functions/spock_repset_add_seq.md b/docs/spock_functions/functions/spock_repset_add_seq.md index 31360e26..e36ff5e5 100644 --- a/docs/spock_functions/functions/spock_repset_add_seq.md +++ b/docs/spock_functions/functions/spock_repset_add_seq.md @@ -4,18 +4,67 @@ ### SYNOPSIS -`spock.repset_add_seq(set_name name, relation regclass, sync_data boolean)` - +spock.repset_add_seq ( + set_name name, + relation regclass, + synchronize_data boolean) + +### RETURNS + + - true if the sequence was successfully added to the replication set. + - false if the sequence was already a member of the replication set. + ### DESCRIPTION -Add a sequence to a replication set. +Adds a sequence to an existing Spock replication set. + +Once added, changes to the sequence value are replicated to subscribers +that are subscribed to the replication set. This ensures that nextval +operations remain consistent across nodes in a multi-master environment. + +If synchronize_data is true, the current sequence value is immediately +synchronized to all subscribers. + +This function updates metadata stored in the Spock catalogs and does not +modify PostgreSQL configuration. + +This command must be executed by a superuser. ### ARGUMENTS - set_name - The name of the existing replication set. - relation - The name or OID of the sequence to be added to the set. - sync_data - If true, the sequence value will be synchronized immediately; the default is false. - - *Warning:* If you're deploying a multi-master replication scenario, we recommend that you not add sequences to a replication set. Instead, use the [Snowflake Sequences](https://github.com/pgEdge/snowflake-sequences) to manage sequences. + +set_name + + The name of an existing replication set. + +relation + + The sequence to add, specified as a regclass + (for example, 'public.my_sequence'). + +synchronize_data + + If true, the current sequence value is synchronized to + all subscribers. The default is false. + +### EXAMPLE + +Add a sequence (named public.sales_order_no_seq) to the 'default' replication +set: + + postgres=# SELECT spock.repset_add_seq('default', 'public.sales_order_no_seq'); + repset_add_seq + ---------------- + t + (1 row) + +Add a sequence (named public.sales_invoice_no_seq) and synchronize its current value: + + postgres=# SELECT spock.repset_add_seq( + 'default', + 'public.sales_invoice_no_seq', + synchronize_data := true + ); + repset_add_seq + ---------------- + t + (1 row) diff --git a/docs/spock_functions/functions/spock_repset_add_table.md b/docs/spock_functions/functions/spock_repset_add_table.md index 33b2fdea..346150a2 100644 --- a/docs/spock_functions/functions/spock_repset_add_table.md +++ b/docs/spock_functions/functions/spock_repset_add_table.md @@ -4,27 +4,99 @@ ### SYNOPSIS -`spock.repset_add_table (set_name name, relation regclass, sync_data boolean, columns text[], row_filter text)` - +spock.repset_add_table ( + set_name name, + relation regclass, + synchronize_data boolean, + columns text[], + row_filter text, + include_partitions boolean +) + +### RETURNS + + - true if the table was successfully added to the replication set. + - false if the table was already a member of the replication set. + ### DESCRIPTION -Add a table or tables to a replication set. +Adds a specific table to an existing Spock replication set. -### EXAMPLE +This function allows fine-grained control over what data is replicated by +optionally specifying: + + - A subset of columns to replicate. + - A row filter to restrict which rows are replicated. + - Whether partitions of a partitioned table should also be included. + +If synchronize_data is set to true, existing table data is copied to all +subscribers that are subscribed to this replication set. Otherwise, only +future changes are replicated. + +When used with partitioned tables, include_partitions controls whether +child partitions are automatically included in the replication set. + +This function writes metadata into the Spock catalogs and does not modify +PostgreSQL configuration. + +This command must be executed by a superuser. -`spock.repset_add_table ('demo_repset', 'public.my_table')` - ### ARGUMENTS - set_name - The name of the existing replication set. - relation - The name or OID of the table to be added to the set. - sync_data - If true, the table data is synchronized on all subscribers which are subscribed to given replication set; the default is false. - columns - A list of columns to replicate. Normally when all columns should be replicated, this will be set to NULL (the default). - row_filter - A row filtering expression; the default is NULL (no filtering). - - **WARNING: Use caution when synchronizing data with a valid row filter.** - Using `sync_data=true` with a valid `row_filter` is usually a one_time operation for a table. Executing it again with a modified `row_filter` won't synchronize data to subscriber. You may need to call `spock.alter_sub_resync_table()` to fix it. + +set_name + + The name of an existing replication set. + +relation + + The table to add, specified as a regclass (for example, + 'public.mytable'). + +synchronize_data + + If true, existing table data is synchronized to all + subscribers. Default is false. + +columns + + An optional list of column names to replicate. If NULL, + all columns are replicated. + +row_filter + + An optional SQL WHERE clause used to filter which rows + are replicated. + +include_partitions + + If true and the table is partitioned, all partitions are + included automatically. Default is true. + +### EXAMPLE + +The following command adds a table named public.accounts to the demo_repset +replication set with full replication: + + SELECT spock.repset_add_table('demo_repset', 'public.accounts'); + +The following command adds a table named public.accounts to the demo_repset +replication set, and synchronizes existing data: + + SELECT spock.repset_add_table('demo_repset', + 'public.accounts', + synchronize_data := true); + +The following command adds only specific columns (id and balance) from the +public.accounts table to the demo_repset replication set, and filters the +rows: + + SELECT spock.repset_add_table('demo_repset', + 'public.accounts', + columns := ARRAY['id','balance'], + row_filter := 'balance > 0'); + +**WARNING: Use caution when synchronizing data with a valid row filter.** +Using `sync_data=true` with a valid `row_filter` is usually a one_time +operation for a table. Executing it again with a modified `row_filter` +won't synchronize data to subscriber. You may need to call +`spock.alter_sub_resync_table()` to fix it. diff --git a/docs/spock_functions/functions/spock_repset_alter.md b/docs/spock_functions/functions/spock_repset_alter.md index 3986ada3..f0200c51 100644 --- a/docs/spock_functions/functions/spock_repset_alter.md +++ b/docs/spock_functions/functions/spock_repset_alter.md @@ -4,24 +4,80 @@ ### SYNOPSIS -`spock.repset_alter (set_name name, replicate_inserts boolean, replicate_updates boolean, replicate_deletes boolean, replicate_truncate boolean)` - +spock.repset_alter ( + set_name name, + replicate_insert boolean, + replicate_update boolean, + replicate_delete boolean, + replicate_truncate boolean +) + +### RETURNS + +The OID of the altered replication set. + ### DESCRIPTION -Alter a replication set. +Modifies the replication behavior of an existing Spock replication set. + +Each replication set defines which types of table operations are replicated to +subscribers. This function allows you to change those behaviors after the set +has been created. + +Any argument left as NULL will retain its current setting. Only the +specified operation types are modified. + +This function updates metadata in the Spock catalogs and does not change +PostgreSQL server configuration. + +This command must be executed by a superuser. + +### ARGUMENTS + +set_name + + The name of an existing replication set. + +replicate_insert + + If true, INSERT operations are replicated; the default is NULL. + +replicate_update + + If true, UPDATE operations are replicated; the default is NULL. + +replicate_delete + + If true, DELETE operations are replicated; the default is NULL. + +replicate_truncate + + If true, TRUNCATE operations are replicated; the default is NULL. ### EXAMPLE -`spock.repset_alter ('demo_repset', 'demo', 'replicate_truncate=False')` - -### ARGUMENTS - set_name - The name of the set, must be unique. - replicate_insert - Specifies if INSERT statements are replicated; the default is true. - replicate_update - Specifies if UPDATE statements are replicated; the default is true. - replicate_delete - Specifies if DELETE statements are replicated; the default is true. - replicate_truncate - Specifies if TRUNCATE statements are replicated; the default is true. +The following command enables all operations in the demo_repset replication +set: + + postgres=# SELECT spock.repset_alter('demo_repset', + replicate_insert := true, + replicate_update := true, + replicate_delete := true, + replicate_truncate := true); + repset_alter + -------------- + 3356172311 + (1 row) + +The following command disables DELETE and TRUNCATE replication in the +audit_only replication_set: + + postgres=# SELECT spock.repset_alter('audit_only', + replicate_delete := false, + replicate_truncate := false); + repset_alter + -------------- + 3744767607 + (1 row) + + diff --git a/docs/spock_functions/functions/spock_repset_create.md b/docs/spock_functions/functions/spock_repset_create.md index f2e99a41..ff90d900 100644 --- a/docs/spock_functions/functions/spock_repset_create.md +++ b/docs/spock_functions/functions/spock_repset_create.md @@ -4,24 +4,76 @@ ### SYNOPSIS -`spock.repset_create (set_name name, replicate_insert bool, replicate_update bool, replicate_delete bool, replicate_truncate bool)` - +spock.repset_create ( + set_name name, + replicate_insert boolean = true, + replicate_update boolean = true, + replicate_delete boolean = true, + replicate_truncate boolean = true +) + +### RETURNS + +The OID of the newly created replication set. + ### DESCRIPTION -Create a replication set. +Creates a new Spock replication set. -### EXAMPLE +A replication set defines which types of table operations are replicated to +subscribers. Tables can later be added to this replication set with other +Spock functions. + +By default, all data modification operations are replicated; INSERT, UPDATE, +DELETE, and TRUNCATE. + +You can selectively disable specific operation types when creating the set. + +This function writes metadata into the Spock catalogs but does not alter any +PostgreSQL server configuration. + +This command must be executed by a superuser. -`spock.repset_create ('demo_repset')` - ### ARGUMENTS - set_name - The name of the set, must be unique. - replicate_insert - Specifies if INSERT statements are replicated; the default is true. - replicate_update - Specifies if UPDATE statements are replicated; the default is true. - replicate_delete - Specifies if DELETE statements are replicated; the default is true. - replicate_truncate - Specifies if TRUNCATE statements are replicated; the default is true. + +set_name + + The unique name for the replication set. + +replicate_insert + + If true (the default), INSERT operations are replicated. + +replicate_update + + If true (the default), UPDATE operations are replicated. + +replicate_delete + + If true (the default), DELETE operations are replicated. + +replicate_truncate + + If true (the default), TRUNCATE operations are replicated. + +### EXAMPLE + +The following example creates a replication set named demo_repset; all +operations are replicated to the set: + + postgres=# SELECT spock.repset_create('demo_repset'); + repset_create + --------------- + 3356172311 + (1 row) + +The following command creates a replication set named audit_only; only +INSERT and UPDATE statements are replicated: + + postgres=# SELECT spock.repset_create('audit_only', + replicate_delete := false, + replicate_truncate := false); + repset_create + --------------- + 3744767607 + (1 row) diff --git a/docs/spock_functions/functions/spock_repset_drop.md b/docs/spock_functions/functions/spock_repset_drop.md index 66a048d6..ae5a3175 100644 --- a/docs/spock_functions/functions/spock_repset_drop.md +++ b/docs/spock_functions/functions/spock_repset_drop.md @@ -1,17 +1,61 @@ -## NAME +## NAME -`spock.repset_drop()` +spock.repset_drop() ### SYNOPSIS -`spock.repset_drop (set_name text)` - + +spock.repset_drop (set_name name, ifexists boolean) + +### RETURNS + + - true if the replication set was successfully dropped. + + - false if the replication set does not exist and ifexists is true. + + - ERROR if the replication set does not exist and ifexists is false. + ### DESCRIPTION - Drop a replication set. -### EXAMPLE +Drops an existing replication set. + +This function removes a replication set from the Spock configuration. When +a replication set is dropped, all table, sequence, and DDL memberships +associated with it are removed. However, the actual tables and sequences +themselves remain in the database - only their association with the +replication set is deleted. + +Dropping a replication set does not affect existing subscriptions that +reference it, but those subscriptions will stop receiving changes for +objects that were in the dropped replication set. Subscriptions can be +modified to remove the dropped replication set using +spock.sub_remove_repset(). + +The ifexists parameter controls error-handling behavior for the function; +when set to true, the function returns false if the replication set does not +exist instead of raising an error. This is useful in situations where the +replication set may or may not be present. + +This function modifies the Spock catalogs but does not modify any user data +or the Postgres server configuration. + +### ARGUMENTS + +set_name + + The name of the replication set to drop. + +ifexists + + If true, the function returns false instead of raising an error + when the replication set does not exist. Defaults to false. + +### EXAMPLE + +The following command drops a replication set named demo_repset; it does not +raise an error if the set does not exist: -`spock.repset_drop ('demo_repset')` - -### POSITIONAL ARGUMENTS - set_name - The name of the existing replication set. + postgres=# SELECT spock.repset_drop('demo_repset', true); + repset_drop + ------------- + t + (1 row) diff --git a/docs/spock_functions/functions/spock_repset_list_tables.md b/docs/spock_functions/functions/spock_repset_list_tables.md deleted file mode 100644 index 8f141dba..00000000 --- a/docs/spock_functions/functions/spock_repset_list_tables.md +++ /dev/null @@ -1,19 +0,0 @@ -# `pgedge spock repset-list-tables` - -```text - -SYNOPSIS - ./pgedge spock repset-list-tables SCHEMA DB - -DESCRIPTION - List all tables in all replication sets. - -Example: spock repset-list-tables '*' demo - -POSITIONAL ARGUMENTS - SCHEMA - The name of the schema to list tables from. To list tables matching a pattern use single quotes and * as a wildcard. Examples: '*', mytable, 'my*' - DB - The name of the database. Example: demo - -``` diff --git a/docs/spock_functions/functions/spock_repset_remove_partition.md b/docs/spock_functions/functions/spock_repset_remove_partition.md index 66b4e158..88293845 100644 --- a/docs/spock_functions/functions/spock_repset_remove_partition.md +++ b/docs/spock_functions/functions/spock_repset_remove_partition.md @@ -1,26 +1,54 @@ ## NAME -`spock repset-remove-partition ()` - -## SYNOPSIS - -`spock repset-remove-partition (PARENT_TABLE DB )` - -## DESCRIPTION - -Remove a partition from the replication set that the parent table is a part of. - -## EXAMPLE - -`spock.repset-remove-partition (mytable demo --partition=mytable_202012)` - -## POSITIONAL ARGUMENTS - PARENT_TABLE - The name of the parent table. Example: mytable - DB - The name of the database. Example: demo - -## FLAGS - -p, --partition=PARTITION - The name of the partition. If none is provided, it will remove all replicated partitions from the replication set. Example: mytable_202012 - +spock.repset_remove_partition() + +### SYNOPSIS + +spock.repset_remove_partition (parent regclass, partition regclass) + +### RETURNS + +The number of partitions removed from replication sets as an integer. + +### DESCRIPTION + +Removes a partition or all partitions of a partitioned table from the +replication sets that include the parent table. + +This function removes partition tables from replication without affecting +the parent table's replication membership. When a specific partition is +provided, only that partition is removed. When the partition parameter is +NULL, all partitions of the parent table are removed from replication. + +This is useful when you want to exclude specific partitions from +replication while keeping the parent table and other partitions replicated. +For example, you might want to replicate recent partitions but not +historical data stored in older partitions. + +Removing a partition from replication does not drop the partition table +itself - it only removes the partition from the replication sets. The +partition remains in the database and can still be accessed locally. + +The function returns the count of partitions that were actually removed +from replication sets. If no partitions were removed (because they were not +in any replication sets), it returns 0. + +This function modifies the Spock catalogs but does not modify any user data +or PostgreSQL server configuration. + +### ARGUMENTS + +parent + + The parent partitioned table whose partition(s) should be removed + from replication. + +partition + + The specific partition to remove from replication. If NULL, all + partitions of the parent table are removed. Defaults to NULL. + +### EXAMPLE + + SELECT spock.repset_remove_partition('public.mytable', + 'public.mytable_202012'); diff --git a/docs/spock_functions/functions/spock_repset_remove_seq.md b/docs/spock_functions/functions/spock_repset_remove_seq.md index 0607bcc1..a0f8555b 100644 --- a/docs/spock_functions/functions/spock_repset_remove_seq.md +++ b/docs/spock_functions/functions/spock_repset_remove_seq.md @@ -4,18 +4,44 @@ ### SYNOPSIS -`spock.repset_remove_seq (set_name name, relation regclass)` - +spock.repset_remove_seq ( + set_name name, + relation regclass) + +### RETURNS + + - true if the sequence was successfully removed from the replication set. + - false if the sequence was not a member of the replication set. + ### DESCRIPTION -Remove a sequence from a replication set. +Removes a sequence from an existing Spock replication set. + +After removal, changes to the sequence value are no longer replicated to +subscribers of the replication set. + +This function updates metadata stored in the Spock catalogs and does not +modify PostgreSQL configuration. + +This command must be executed by a superuser. + +### ARGUMENTS + +set_name + + The name of an existing replication set. + +relation + + The sequence to remove, specified as a regclass + (for example, 'public.my_sequence'). + +### EXAMPLE -### EXAMPLE +Remove a sequence (public.orders_id_seq) from a replication set (demo_repset): -`spock.repset_remove_sequence ('demo_repset', 'public.mysequence')` - -### POSITIONAL ARGUMENTS - set_name - The name of the existing replication set. - relation - The name or OID of the sequence to be removed from the set. + postgres=# SELECT spock.repset_remove_seq('demo_repset', 'public.orders_id_seq'); + repset_remove_seq + ------------------- + t + (1 row) \ No newline at end of file diff --git a/docs/spock_functions/functions/spock_repset_remove_table.md b/docs/spock_functions/functions/spock_repset_remove_table.md index 8313c147..7c0a34f5 100644 --- a/docs/spock_functions/functions/spock_repset_remove_table.md +++ b/docs/spock_functions/functions/spock_repset_remove_table.md @@ -4,18 +4,62 @@ ### SYNOPSIS -`spock.repset_remove_table (set_name name, relation regclass)` - +spock.repset_remove_table ( + set_name name, + relation regclass, + include_partitions boolean) + +### RETURNS + + - true if the table was successfully removed from the replication set. + - false if the table was not a member of the replication set. + ### DESCRIPTION -Remove a table from a replication set. +Removes a table from an existing Spock replication set. After removal, +changes to this table will no longer be replicated to subscribers that +are subscribed to the replication set. + +If the table is partitioned, the include_partitions argument controls +whether all child partitions are also removed from the replication set. + +This function updates metadata stored in the Spock catalogs and does not +modify the PostgreSQL configuration. + +This command must be executed by a superuser. + +### ARGUMENTS + +set_name + + The name of an existing replication set. + +relation + + The table to remove, specified as a regclass + (for example, 'public.mytable'). + +include_partitions + + If true and the table is partitioned, all partitions are + also removed from the replication set. Default is true. + +### EXAMPLE + +Remove a table (public.users) from the default replication set: + + postgres=# SELECT spock.repset_remove_table('default', 'public.users'); + repset_remove_table + --------------------- + t + (1 row) -### EXAMPLE +Remove a partitioned table named public.sales_parent from the default +replication set without removing its partitions: -`spock.repset_remove_table ('demo_repset', 'public.mytable')` - -### POSITIONAL ARGUMENTS - set_name - The name of the existing replication set. - relation - The name or OID of the table to be removed from the set. + postgres=# SELECT spock.repset_remove_table('default', + 'public.users', + include_partitions := false); + --------------------- + t + (1 row) diff --git a/docs/spock_functions/functions/spock_repset_show_table.md b/docs/spock_functions/functions/spock_repset_show_table.md new file mode 100644 index 00000000..91fec152 --- /dev/null +++ b/docs/spock_functions/functions/spock_repset_show_table.md @@ -0,0 +1,74 @@ +## NAME + +spock.repset_show_table() + +### SYNOPSIS + +spock.repset_show_table (relation regclass, repsets text[]) + +### RETURNS + +A record containing detailed information about how a table is configured +for replication: + + - relid is the OID of the table relation. + - nspname is the schema name of the table. + - relname is the table name. + - att_list is an array of column names included in replication. + - has_row_filter is true if a row filter is applied to the table. + - relkind is the relation kind ('r' for regular table, 'p' for + partitioned table). + - relispartition is true if the table is a partition of a partitioned + table. + +### DESCRIPTION + +Returns detailed replication configuration information for a specific table +within the specified replication sets. + +This function queries how a table is configured for replication across one +or more replication sets. It shows which columns are included in +replication, whether row filtering is applied, and metadata about the table +type and partition status. + +The att_list field shows the specific columns that will be replicated. If +all columns are replicated, this will contain all column names. If column +filtering was configured when the table was added to a replication set, +only the specified columns appear in this list. + +The has_row_filter field indicates whether a row filter expression is +applied to the table, which would limit which rows are replicated based on +custom criteria. + +The repsets parameter allows you to check the table's configuration across +multiple replication sets. The function returns information based on how +the table is configured in those sets. + +This is a read-only query function that does not modify any data. + +### ARGUMENTS + +relation + + The table to query information about. + +repsets + + An array of replication set names to check the table's + configuration within. + +### EXAMPLE + +The following command shows detailed information about the public.invoices +table: + + postgres=# SELECT * FROM spock.repset_show_table('public.invoices', + ARRAY['default', 'ddl_sql']); + -[ RECORD 1 ]--+----------------------------------------- + relid | 17241 + nspname | public + relname | invoices + att_list | {order_id,customer_id,order_date,amount} + has_row_filter | f + relkind | r + relispartition | f diff --git a/docs/spock_functions/functions/spock_seq_sync.md b/docs/spock_functions/functions/spock_seq_sync.md index 734ecb72..36ab2bab 100644 --- a/docs/spock_functions/functions/spock_seq_sync.md +++ b/docs/spock_functions/functions/spock_seq_sync.md @@ -1,19 +1,54 @@ ## NAME -`spock.seq_sync()` +spock.sync_seq() ### SYNOPSIS -`spock.seq_sync(relation regclass)` - +spock.sync_seq(relation regclass) + +### RETURNS + +- true if the sequence state was successfully synchronized. + ### DESCRIPTION -Push the sequence state to all subscribers. Unlike the subscription and table synchronization functions, this function should be run only on the provider. It forces an update of the tracked sequence state that will be consumed by all subscribers (replication set filtering still applies) when they replicate the transaction in which this function has been executed. +Synchronizes the current state of a sequence to all subscribers. + +This function captures the current value of a sequence on the provider node +and pushes it into the replication stream so that subscribers can update +their local copy of the sequence to match. This ensures sequence values +remain coordinated across nodes in a replication topology. + +Unlike subscription and table synchronization functions which run on the +subscriber, this function must be executed on the provider node. The +sequence state update is embedded in the transaction where this function is +called, and subscribers will apply the sequence update when they replicate +that transaction. + +Replication set filtering still applies - only subscribers whose +subscriptions include the replication set containing this sequence will +receive the update. -### EXAMPLE +This is particularly useful after making direct changes to a sequence value +or when you need to ensure subscribers have the latest sequence state +before performing operations that depend on sequence coordination. + +This function modifies the replication stream by inserting a sequence +synchronization event. -`spock.seq_sync ('public.my_sequence')` - ### ARGUMENTS - relation - The name of an existing sequence, optionally qualified. + +relation + + The name of the sequence to synchronize, optionally + schema-qualified. + +### EXAMPLE + +The following command synchronizes a sequence named public.sales_order_no_seq: + + postgres=# SELECT spock.sync_seq('public.sales_order_no_seq'); + sync_seq + ---------- + t + (1 row) diff --git a/docs/spock_functions/functions/spock_sub_add_repset.md b/docs/spock_functions/functions/spock_sub_add_repset.md index f58e8003..0bf3017d 100644 --- a/docs/spock_functions/functions/spock_sub_add_repset.md +++ b/docs/spock_functions/functions/spock_sub_add_repset.md @@ -1,21 +1,47 @@ ## NAME -`spock.sub_add_repset()` +spock.sub_add_repset() ### SYNOPSIS -`spock.sub_add_repset (subscription_name name, replication_set name)` - +spock.sub_add_repset (subscription_name name, replication_set name) + +### RETURNS + + - true if the replication set was successfully added. + + - false if the operation fails. + ### DESCRIPTION -Adds one replication set into a subscriber. Does not synchronize, only activates consumption of events. +Adds a replication set to an existing subscription. + +This function modifies a subscription to begin receiving changes from an +additional replication set on the provider node. The subscription will start +consuming events from the newly added replication set immediately. + +This operation does not perform data synchronization. It only activates the +consumption of future events from the replication set. If you need to +synchronize existing data from the replication set, you must handle that +separately. + +This function writes metadata into the Spock catalogs. + +Returns NULL if any argument is NULL. + +This command must be executed by a superuser. + +### ARGUMENTS + +subscription_name + + The name of an existing subscription. + +replication_set + + The name of the replication set to add to the subscription. ### EXAMPLE -`spock.sub_add_repset ('sub_n2n1', 'demo_repset')` - -### POSITIONAL ARGUMENTS - `subscription_name` - The name of the existing subscription. - `replication_set` - The name of replication set to add. + + SELECT spock.sub_add_repset('sub_n2_n1', 'custom_repset'); diff --git a/docs/spock_functions/functions/spock_sub_alter_interface.md b/docs/spock_functions/functions/spock_sub_alter_interface.md index 29c19a2e..637b9e3c 100644 --- a/docs/spock_functions/functions/spock_sub_alter_interface.md +++ b/docs/spock_functions/functions/spock_sub_alter_interface.md @@ -1,21 +1,46 @@ ## NAME -`spock.sub_alter_interface()` +spock.sub_alter_interface() ### SYNOPSIS - -`spock.sub_alter_interface (subscription_name name, interface_name name)` - + +spock.sub_alter_interface (subscription_name name, interface_name name) + +### RETURNS + + - true if the interface was successfully changed. + + - false if the operation fails. + ### DESCRIPTION -Alter the subscription to use a different interface when connecting to the provider node. +Changes the network interface used by a subscription to connect to its +provider node. + +This function modifies an existing subscription to use a different interface +when establishing the replication connection to the provider. The specified +interface must already exist on the provider node. + +This is useful for switching between different network paths, such as moving +from a public network to a private network connection, or switching to a +different hostname or IP address for the same provider node. + +The subscription connection will be restarted using the new interface. + +This function writes metadata into the Spock catalogs. + +This command must be executed by a superuser. + +### ARGUMENTS + +subscription_name + + The name of an existing subscription. + +interface_name + + The name of an existing interface on the provider node. ### EXAMPLE -`spock.sub_alter_interface ('sub_n2n1', 'n1_2 demo')` - -### POSITIONAL ARGUMENTS - subscription_name - The name of an existing subscription. - interface_name - The name of an existing interface of the current provider node. + SELECT spock.sub_alter_interface('sub_n2_n1', 'private_net'); diff --git a/docs/spock_functions/functions/spock_sub_alter_skiplsn.md b/docs/spock_functions/functions/spock_sub_alter_skiplsn.md new file mode 100644 index 00000000..d5b3a0cb --- /dev/null +++ b/docs/spock_functions/functions/spock_sub_alter_skiplsn.md @@ -0,0 +1,50 @@ +## NAME + +NAME + +spock.sub_alter_skiplsn() + +### SYNOPSIS + +spock.sub_alter_skiplsn (subscription_name name, lsn pg_lsn) + +### RETURNS + + - true if the subscription LSN was successfully advanced. + + - false if the subscription does not exist. + +### DESCRIPTION + +Advances the replication position of an existing Spock subscription to the +specified Log Sequence Number (LSN). + +This function is typically used as a recovery or repair operation when a +subscription is unable to proceed due to a problematic or corrupted change in +the replication stream. By skipping forward to a known good LSN, replication +can resume without replaying the offending transaction. + +This operation does not replay or repair the skipped data. Any changes +between the old LSN and the specified LSN will be permanently ignored by the +subscriber. This should be used with caution and only when the administrator +understands the data implications. + +This function writes metadata into the Spock catalogs but does not modify +PostgreSQL server configuration. + +This command must be executed by a superuser. + +### ARGUMENTS + +subscription_name + + The name of an existing Spock subscription. + +lsn + + The Log Sequence Number to which the subscription should + advance. + +## EXAMPLE + + SELECT spock.sub_alter_skiplsn('sub_n1_to_n2', '0/16B6A50'); \ No newline at end of file diff --git a/docs/spock_functions/functions/spock_sub_create.md b/docs/spock_functions/functions/spock_sub_create.md index 9ee18dd8..a72306d9 100644 --- a/docs/spock_functions/functions/spock_sub_create.md +++ b/docs/spock_functions/functions/spock_sub_create.md @@ -1,38 +1,102 @@ ## NAME -`spock.sub_create()` +spock.sub_create() ### SYNOPSIS -`spock.sub_create (subscription_name name, provider_dsn text, repsets text[], sync_structure boolean, - sync_data boolean, forward_origins text[], apply_delay interval)` - +spock.sub_create (subscription_name name, provider_dsn text, +replication_sets text[], synchronize_structure boolean, +synchronize_data boolean, forward_origins text[], apply_delay interval, +force_text_transfer boolean, enabled boolean, skip_schema text[]) + +### RETURNS + +The OID of the newly created subscription. + ### DESCRIPTION -Creates a subscription from current node to the provider node. The command does not wait for completion before returning to the caller. +Creates a subscription from the current node to a provider node. + +This function establishes a replication subscription that allows the current +(subscriber) node to receive changes from the specified provider node. The +subscription defines which replication sets to subscribe to and how to +handle initial synchronization. -The `subscription_name` is used as `application_name` by the replication connection. This means that it's visible in the `pg_stat_replication` monitoring view. It can also be used in `synchronous_standby_names` when Spock is used as part of a synchronous replication scenario. +The command returns immediately without waiting for the subscription to +complete its initialization. Use spock.sub_wait_for_sync() to wait for the +subscription to finish synchronizing and begin replicating. -Use `spock.sub_wait_for_sync(subscription_name)` to wait for the subscription to asynchronously start replicating and complete any needed schema and/or data sync. +The subscription_name is used as the application_name for the replication +connection, making it visible in pg_stat_replication. This can be useful for +monitoring and for configuring synchronous_standby_names in synchronous +replication scenarios. -### EXAMPLE +This function writes metadata into the Spock catalogs and initiates a +replication connection to the provider. + +This command must be executed by a superuser. -`spock.sub_create ('sub_n2n1', 'host=10.1.2.5 port=5432 user=rocky dbname=demo')` - ### ARGUMENTS - subscription_name - The name of the subscription. Each subscription in a cluster must have a unique name. The name is used as application_name by the replication connection. This means that the name is visible in the pg_stat_replication monitoring view. - provider_dsn - The connection string to a provider. - repsets - An array of existing replication sets to subscribe to; the default is {default,default_insert_only,ddl_sql}. - sync_structure - Specifies if Spock should synchronize the structure from provider to the subscriber; the default is false. - sync_data - Specifies if Spock should synchronize data from provider to the subscriber, the default is true. - forward_origins - An array of origin names to forward; currently the only supported values are an empty array (don't forward any changes that didn't originate on provider node, useful for two-way replication between the nodes), or {all} which means replicate all changes no matter what is their origin. The default is {all}. - apply_delay - How much to delay replication; the default is 0 seconds. - force_text_transfer - Force the provider to replicate all columns using a text representation (which is slower, but may be used to change the type of a replicated column on the subscriber). The default is false. + +subscription_name + + A unique name for the subscription. Must be unique across the cluster. + This name appears in pg_stat_replication as application_name. + +provider_dsn + + A PostgreSQL connection string specifying how to connect to the + provider node. + +replication_sets + + An array of replication set names to subscribe to. Accepted values + are default, default_insert_only, ddl_sql. + +synchronize_structure + + If true, synchronize table and schema structure from the provider + before starting replication. Default is false. + +synchronize_data + + If true, perform an initial data copy for all tables in the subscribed + replication sets. Default is false. + +forward_origins + + Controls which changes to replicate based on their origin. Use '{}' to + only replicate changes originating on the provider (useful for + bidirectional replication). Use {all} to replicate all changes + regardless of origin. Default is '{}'. + +apply_delay + + An interval specifying how long to delay applying changes from the + provider. Useful for delayed standby scenarios. Default is '0'. + +force_text_transfer + + If true, replicate all columns using text representation. This is + slower but allows type conversions on the subscriber. Default is false. + +enabled + + If true, the subscription is enabled immediately. If false, the + subscription is created but not activated. Default is true. + +skip_schema + + An array of schema names to exclude from structure and data + synchronization. Default is '{}' (no schemas skipped). + +### EXAMPLE + + SELECT spock.sub_create('sub_n2_n1', + 'host=127.0.0.1 port=5432 dbname=postgres'); + + SELECT spock.sub_create('sub_n2_n1', + 'host=10.0.0.10 port=5432 dbname=postgres', + replication_sets := '{default}', + synchronize_structure := true, + synchronize_data := true); diff --git a/docs/spock_functions/functions/spock_sub_disable.md b/docs/spock_functions/functions/spock_sub_disable.md index 988ec2af..e4747e31 100644 --- a/docs/spock_functions/functions/spock_sub_disable.md +++ b/docs/spock_functions/functions/spock_sub_disable.md @@ -1,21 +1,52 @@ -## NAME +## NAME -`spock.sub_disable ()` +spock.sub_disable() ### SYNOPSIS -`spock.sub_disable (subscription_name name, immediate boolean)` - +spock.sub_disable (subscription_name name, immediate boolean) + +### RETURNS + + - true if the subscription was successfully disabled. + + - false if the operation fails. + ### DESCRIPTION - Disable a subscription by putting it on hold and disconnect from provider. -### EXAMPLE +Disables a subscription and disconnects from the provider node. + +This function pauses an active subscription by disconnecting the replication +connection to the provider and marking the subscription as disabled. While +disabled, the subscription will not receive any changes from the provider. + +The immediate parameter controls when the subscription is stopped. If set to +true, the subscription is terminated immediately. If false (default), the +subscription continues processing until the end of the current transaction +before stopping. + +A disabled subscription can be re-enabled later using spock.sub_enable() +without losing its position in the replication stream. + +This function writes metadata into the Spock catalogs. + +Returns NULL if any argument is NULL. + +This command must be executed by a superuser. -`spock sub_disable 'sub_n2n1'` - ### ARGUMENTS - subscription_name - The name of the existing subscription. - immediate - If true, the subscription is stopped immediately, otherwise it will be only stopped at the end of current transaction; the default is false. - + +subscription_name + + The name of the existing subscription to disable. + +immediate + + If true, stop the subscription immediately. If false, stop at the end + of the current transaction. Default is false. + +### EXAMPLE + + SELECT spock.sub_disable('sub_n2_n1'); + + SELECT spock.sub_disable('sub_n2_n1', true); diff --git a/docs/spock_functions/functions/spock_sub_drop.md b/docs/spock_functions/functions/spock_sub_drop.md index 31f7740e..75c7fd74 100644 --- a/docs/spock_functions/functions/spock_sub_drop.md +++ b/docs/spock_functions/functions/spock_sub_drop.md @@ -1,21 +1,44 @@ ## NAME -`spock.sub_drop ()` +spock.sub_drop() ### SYNOPSIS -`spock.sub_drop (subscription_name name, ifexists bool)` - +spock.sub_drop (subscription_name name, ifexists boolean) + +### RETURNS + +The OID of the dropped subscription. + ### DESCRIPTION -Disconnects the subscription and removes it from the catalog. +Disconnects and removes a subscription from the cluster. + +This function terminates the replication connection for the specified +subscription and removes all associated metadata from the Spock catalogs. It +does not affect data that has already been replicated to the subscriber. + +If ifexists is set to false (default), an error is raised when the specified +subscription does not exist. If ifexists is true, the function returns +successfully without error. + +Returns NULL if any argument is NULL. + +This command must be executed by a superuser. + +### ARGUMENTS + +subscription_name + + The name of the existing subscription to remove. + +ifexists + + If true, do not raise an error when the subscription does not exist. + Default is false. ### EXAMPLE -`spock.sub_drop ('sub_n2n1')` - -### POSITIONAL ARGUMENTS - subscription_name - The name of the existing subscription. - ifexists - If true, an error is not thrown when subscription does not exist; the default is false. + SELECT spock.sub_drop('sub_n2_n1'); + + SELECT spock.sub_drop('sub_n2_n1', true); diff --git a/docs/spock_functions/functions/spock_sub_enable.md b/docs/spock_functions/functions/spock_sub_enable.md index b3113ce6..5b7e62df 100644 --- a/docs/spock_functions/functions/spock_sub_enable.md +++ b/docs/spock_functions/functions/spock_sub_enable.md @@ -1,21 +1,47 @@ ## NAME -`spock.sub_enable()` +spock.sub_enable() ### SYNOPSIS -`spock.sub_enable (subscription_name name, immediate boolean)` +spock.sub_enable (subscription_name name, immediate boolean) + +### RETURNS + + - true if the subscription was successfully enabled. + - false if the operation fails. ### DESCRIPTION -Enable a subscription. +Enables a previously disabled subscription and reconnects to the provider +node. + +This function activates a disabled subscription by re-establishing the +replication connection to the provider and resuming replication. The +subscription will continue receiving changes from where it left off when it +was disabled. -### Example +The immediate parameter controls when the subscription is started. If set to +true, the subscription is activated immediately. If false (default), the +subscription is activated at the end of the current transaction. + +This function writes metadata into the Spock catalogs. + +This command must be executed by a superuser. -`spock sub_enable ('sub_n2n1')` - ### ARGUMENTS - subscription_name - The name of the existing subscription. - immediate - If true, the subscription is started immediately, otherwise it will be only started at the end of current transaction; the default is false. + +subscription_name + + The name of the existing subscription to enable. + +immediate + + If true, start the subscription immediately. If false, start at the + end of the current transaction. Default is false. + +### EXAMPLE + + SELECT spock.sub_enable('sub_n2_n1'); + + SELECT spock.sub_enable('sub_n2_n1', true); diff --git a/docs/spock_functions/functions/spock_sub_remove_repset.md b/docs/spock_functions/functions/spock_sub_remove_repset.md index 0416ef54..b6c51899 100644 --- a/docs/spock_functions/functions/spock_sub_remove_repset.md +++ b/docs/spock_functions/functions/spock_sub_remove_repset.md @@ -1,25 +1,43 @@ -## NAME +## NAME -`spock.sub_remove_repset()` +spock.sub_remove_repset() ### SYNOPSIS -`spock.sub_remove_repset (subscription_name name, replication_set name)` - +spock.sub_remove_repset (subscription_name name, replication_set name) + +### RETURNS + + - true if the replication set was successfully removed. + + - false if the operation fails. + ### DESCRIPTION - -Remove a replication set from a subscription. -There is also a `postgresql.conf` parameter, `spock.extra_connection_options`, that you can use to assign connection options that apply to all connections made by spock. This can be a useful place to set up custom keepalive options, etc. +Removes a replication set from an existing subscription. + +This function modifies a subscription to stop receiving changes from the +specified replication set on the provider node. The subscription will +immediately stop consuming events from the removed replication set. + +Only the replication set association is removed. This does not affect data +that has already been replicated, nor does it drop any tables or other +database objects. + +This function writes metadata into the Spock catalogs. + +This command must be executed by a superuser. + +### ARGUMENTS + +subscription_name + + The name of an existing subscription. + +replication_set -spock defaults to enabling TCP keepalives to ensure that it notices when the upstream server disappears unexpectedly. To disable them add `keepalives = 0` to `spock.extra_connection_options`. + The name of the replication set to remove from the subscription. -### EXAMPLE +### EXAMPLE -`spock.sub_remove_repset ('sub_n2n1', 'demo_repset')` - -### POSITIONAL ARGUMENTS - subscription_name - The name of the existing subscription. - replication_set - The name of replication set to remove. + SELECT spock.sub_remove_repset('sub_n2_n1', 'custom_repset'); diff --git a/docs/spock_functions/functions/spock_sub_show_status.md b/docs/spock_functions/functions/spock_sub_show_status.md index 79207c69..de6adfc0 100644 --- a/docs/spock_functions/functions/spock_sub_show_status.md +++ b/docs/spock_functions/functions/spock_sub_show_status.md @@ -4,16 +4,53 @@ ### SYNOPSIS -`spock.sub_show_status (subscription_name name)` - -### DESCRIPTION +spock.sub_show_status (subscription_name name) -Show the status and basic information of a subscription. +### RETURNS -### EXAMPLE +A set of rows describing the status of one or more Spock subscriptions. -`spock.sub_show_status ('sub_n2n1')` - -### ARGUMENTS - subscription_name - The optional name of the existing subscription. If no name is provided, the function will show status for all subscriptions on the local node. +Each row contains: + + - The name of the subscription. + + — Current replication status. + + — The name of the provider node. + + — The connection string used to reach the provider. + + — The logical replication slot used by the subscription. + + — The replication sets associated with the subscription. + + — The origins that are forwarded by this subscription. + +DESCRIPTION + +Displays detailed runtime information about Spock subscriptions. + +If a specific subscription name is provided, only that subscription is shown. +If NULL (the default), the status of all subscriptions on the node is +returned. + +This function is useful for troubleshooting replication issues, validating +configuration, and verifying which replication sets and origins are in use. + +The information returned is derived from Spock catalog metadata and the +current state of logical replication slots. + +This function does not modify any configuration. + +ARGUMENTS + +subscription_name + + Optional. The name of a specific Spock subscription. The default is NULL; + If NULL, all subscriptions are shown. + +EXAMPLE + + SELECT * FROM spock.sub_show_status(); + + SELECT * FROM spock.sub_show_status('sub_n1_to_n2'); \ No newline at end of file diff --git a/docs/spock_functions/functions/spock_sub_show_table.md b/docs/spock_functions/functions/spock_sub_show_table.md index d8ede539..0f3a3d6f 100644 --- a/docs/spock_functions/functions/spock_sub_show_table.md +++ b/docs/spock_functions/functions/spock_sub_show_table.md @@ -1,21 +1,58 @@ ## NAME -`spock.sub_show_table()` +spock.sub_show_table() ### SYNOPSIS - -`spock.sub_show_table (subscription_name name, relation regclass)` - + +spock.sub_show_table (subscription_name name, relation regclass, +OUT nspname text, OUT relname text, OUT status text) + +### RETURNS + +A record containing synchronization status information for the table: + + - nspname is the schema name of the table. + - relname is the table name. + - status is the current synchronization status of the table. + ### DESCRIPTION - -Shows synchronization status of a table. + +Returns the synchronization status of a specific table within a +subscription. + +This function queries the current state of table synchronization for a +given subscription. The status field indicates where the table is in the +synchronization process, with possible values: + + - 'i' - initial sync requested + - 's' - structure synchronization in progress + - 'd' - data synchronization in progress + - 'c' - constraint synchronization in progress + - 'w' - sync waiting for approval from main thread + - 'u' - catching up with changes + - 'y' - synchronization finished at LSN + - 'r' - ready and actively replicating + +This is useful for monitoring synchronization progress, diagnosing +replication issues, or verifying that a table has completed its initial +sync before performing dependent operations. + +The function must be run on the subscriber node where the subscription +exists. + +This is a read-only query function that does not modify any data. + +### ARGUMENTS + +subscription_name + + The name of the subscription to query. + +relation + + The name of the table to check synchronization status for, + optionally schema-qualified. ### EXAMPLE -`spock.sub_show_table ('sub_n2n1', 'mytable')` - -### POSITIONAL ARGUMENTS - subscription_name - The name of the existing subscription. - relation - The name of existing table, optionally qualified. + SELECT * FROM spock.sub_show_table('sub_n2_n1', 'public.mytable'); diff --git a/docs/spock_functions/functions/spock_sub_sync.md b/docs/spock_functions/functions/spock_sub_sync.md deleted file mode 100644 index f77610e7..00000000 --- a/docs/spock_functions/functions/spock_sub_sync.md +++ /dev/null @@ -1,17 +0,0 @@ -## NAME - -`spock.sub_sync()` - -### SYNOPSIS - -`spock.sub_sync (subscription_name name, truncate bool)` - -### DESCRIPTION - -Call this function to synchronize all unsynchronized tables in all sets in a single operation. Tables are copied and synchronized one by one. The command does not wait for completion before returning to the caller. Use `spock.wait_for_sub_sync` to wait for completion. - -### POSITIONAL ARGUMENTS - subscription_name - The name of the existing subscription. - truncate - If true, tables will be truncated before copy; the default is false. diff --git a/docs/spock_functions/functions/spock_sub_wait_for_sync.md b/docs/spock_functions/functions/spock_sub_wait_for_sync.md index ee6a3597..92075e34 100644 --- a/docs/spock_functions/functions/spock_sub_wait_for_sync.md +++ b/docs/spock_functions/functions/spock_sub_wait_for_sync.md @@ -1,21 +1,43 @@ ## NAME -`spock.sub_wait_for_sync(subscription_name name)` +spock.sub_wait_for_sync() ### SYNOPSIS -`spock.sub_wait_for_sync()` - +spock.sub_wait_for_sync (subscription_name name) + +### RETURNS + +void (returns nothing upon successful completion). + ### DESCRIPTION -This function waits until the subscription's initial schema/data sync, if any, are done, and until any tables pending individual resynchronisation have also finished synchronising. +Waits until the specified subscription has completed its initial schema and +data synchronization, and any pending table resynchronizations have +finished. -For best results, run `SELECT spock.wait_slot_confirm_lsn(NULL, NULL)` on the provider after any replication set changes that requested resyncs, and only then call `spock.sub_wait_for_sync` on the subscriber. +This function blocks the current session until the subscription is fully +synchronized. It monitors both the initial sync phase (if the subscription +is newly created) and any tables that are undergoing individual +resynchronization due to replication set changes or manual resync +operations. -### EXAMPLE +For optimal results, execute spock.wait_slot_confirm_lsn() on the provider +node after making replication set changes that trigger resyncs. This +ensures the provider has flushed all necessary changes before checking sync +status on the subscriber. Only after the provider confirms the LSN should +you call spock.sub_wait_for_sync() on the subscriber. + +The function will continue waiting indefinitely until synchronization +completes, so callers should ensure the subscription is actively processing +and that there are no blocking issues preventing sync completion. -`spock.sub_wait_for_sync ('sub_n2n1')` - ### ARGUMENTS - subscription_name - The name of the subscription. Example: sub_n2n1 + +subscription_name + + The name of the subscription to monitor for sync completion. + +### EXAMPLE + + SELECT spock.sub_wait_for_sync('sub_n2_n1'); diff --git a/docs/spock_functions/functions/spock_sync_event.md b/docs/spock_functions/functions/spock_sync_event.md new file mode 100644 index 00000000..1c6e1a5e --- /dev/null +++ b/docs/spock_functions/functions/spock_sync_event.md @@ -0,0 +1,42 @@ +## NAME + +spock.sync_event() + +### SYNOPSIS + +spock.sync_event () + +### RETURNS + +The Log Sequence Number (LSN) of the created synchronization event as +pg_lsn. + +### DESCRIPTION + +Creates a synchronization event in the replication stream and returns its +LSN. + +This function inserts a special synchronization marker into the logical +replication stream at the current write position. The returned LSN +identifies this marker's location in the write-ahead log and can be used +with functions like spock.wait_slot_confirm_lsn() to verify that +subscribers have processed replication up to this point. + +Synchronization events are useful for coordinating activities across +multiple nodes in a replication topology. By creating a sync event on a +provider and then waiting for subscribers to reach that LSN, you can ensure +all nodes have processed a consistent set of changes before proceeding with +operations that depend on synchronized state. + +The sync event itself does not contain data or modify any tables - it +serves purely as a coordination marker in the replication stream. + +This function modifies the replication stream by inserting a sync marker. + +### ARGUMENTS + +This function takes no arguments. + +### EXAMPLE + + SELECT spock.sync_event(); diff --git a/docs/spock_functions/functions/spock_table_wait_for_sync.md b/docs/spock_functions/functions/spock_table_wait_for_sync.md index a7d611c2..7ea77b67 100644 --- a/docs/spock_functions/functions/spock_table_wait_for_sync.md +++ b/docs/spock_functions/functions/spock_table_wait_for_sync.md @@ -1,24 +1,54 @@ ## NAME -`spock.table_wait_for_sync()` +spock.table_wait_for_sync() ### SYNOPSIS -`spock.table_wait_for_sync (subscription_name name, relation regclass)` - +spock.table_wait_for_sync (subscription_name name, relation regclass) + +### RETURNS + +void (returns nothing upon successful completion). + ### DESCRIPTION - -Pause until a table finishes synchronizing. -For best results, run `SELECT spock.wait_slot_confirm_lsn(NULL, NULL)` on the provider after any replication set changes that requested resyncs, and only then call `spock.sub_wait_for_sync` on the subscriber. +Waits until the specified table has completed synchronization for the given +subscription. + +This function blocks the current session until the specified table finishes +its synchronization process. This is useful when you need to ensure a +specific table is fully synchronized before proceeding with operations that +depend on its data. + +Unlike spock.sub_wait_for_sync() which waits for all tables in a +subscription to sync, this function targets a single table. This allows for +more granular control when monitoring synchronization progress or when only +specific tables are critical to downstream operations. + +For optimal results, execute spock.wait_slot_confirm_lsn() on the provider +node after making replication set changes that trigger table resyncs. This +ensures the provider has flushed all necessary changes before checking sync +status on the subscriber. Only after the provider confirms the LSN should +you call spock.table_wait_for_sync() on the subscriber. + +The function will continue waiting indefinitely until the table +synchronization completes, so callers should ensure the subscription is +actively processing and that there are no blocking issues preventing sync +completion. + +Returns NULL if any argument is NULL. + +### ARGUMENTS + +subscription_name + + The name of the subscription to monitor. + +relation + + The name of the table to wait for, specified as a regclass (can + be schema-qualified). ### EXAMPLE -`spock.table_wait_for_sync ('sub_n2n1', 'mytable')` - -### POSITIONAL ARGUMENTS - subscription_name - The name of the subscription. Example: sub_n2n1 - relation - The name of a table. Example: mytable - + SELECT spock.table_wait_for_sync('sub_n2_n1', 'public.mytable'); diff --git a/docs/spock_functions/functions/spock_version.md b/docs/spock_functions/functions/spock_version.md new file mode 100644 index 00000000..c2e447ab --- /dev/null +++ b/docs/spock_functions/functions/spock_version.md @@ -0,0 +1,35 @@ +## NAME + +spock.spock_version() + +### SYNOPSIS + +spock.spock_version () + +### RETURNS + +The version string of the installed Spock extension. + +### DESCRIPTION + +Returns the version number of the Spock extension. + +This function queries the Spock extension and returns its version as a text +string. The version string typically follows semantic versioning format +(e.g., "4.0.1" or "5.0.4"). + +This is a read-only query function that does not modify any data. + +### ARGUMENTS + +This function takes no arguments. + +### EXAMPLE + +The following command returns the Spock version: + + postgres=# SELECT spock.spock_version(); + spock_version + --------------- + 5.0.4 + (1 row) \ No newline at end of file diff --git a/docs/spock_functions/functions/spock_version_num.md b/docs/spock_functions/functions/spock_version_num.md new file mode 100644 index 00000000..8acdc564 --- /dev/null +++ b/docs/spock_functions/functions/spock_version_num.md @@ -0,0 +1,33 @@ +## NAME + +spock.spock_version_num() + +### SYNOPSIS + +spock.spock_version_num () + +### RETURNS + +The version number of the installed Spock extension as an integer. + +### DESCRIPTION + +Returns the version number of the Spock extension as an integer. + +This function queries the Spock extension and returns its version encoded as +an integer value. The integer format allows for easy version comparison in +SQL queries without string parsing. + +The integer typically represents the version in a format like XXYYZZ where XX +is the major version, YY is the minor version, and ZZ is the patch version. +For example, version 3.1.0 might be represented as 30100. + +This is a read-only query function that does not modify any data. + +### ARGUMENTS + +This function takes no arguments. + +### EXAMPLE + + SELECT spock.spock_version_num(); diff --git a/docs/spock_functions/functions/spock_wait_slot_confirm_lsn.md b/docs/spock_functions/functions/spock_wait_slot_confirm_lsn.md new file mode 100644 index 00000000..68b24bbc --- /dev/null +++ b/docs/spock_functions/functions/spock_wait_slot_confirm_lsn.md @@ -0,0 +1,54 @@ +## NAME + +spock.wait_slot_confirm_lsn() + +### SYNOPSIS + +spock.wait_slot_confirm_lsn (slotname name, target pg_lsn) + +### RETURNS + +void (returns nothing upon successful completion). + +### DESCRIPTION + +Waits until the specified replication slot has confirmed receipt of changes +up to the target Log Sequence Number (LSN). + +This function blocks the current session until the replication slot's +confirmed flush LSN reaches or exceeds the specified target LSN. This is +useful for ensuring that a subscriber has received and acknowledged +replication changes before proceeding with subsequent operations. + +The confirmed flush LSN represents the position up to which the subscriber +has successfully received, applied, and durably stored the replicated +changes. This function provides a synchronization mechanism to coordinate +activities between the provider and subscriber nodes. + +If the replication slot does not exist, the function will raise an error. +The function will continue waiting indefinitely until the target LSN is +reached, so callers should ensure the target LSN is reachable and that the +subscription is actively processing changes. + +This function does not modify any data but may block for an extended period +depending on replication lag and network conditions. + +### ARGUMENTS + +slotname + + The name of the replication slot to monitor. + +target + + The target Log Sequence Number to wait for. The function returns + when the slot's confirmed flush LSN reaches or exceeds this + value. + +### EXAMPLE + +The following command waits for a slot named spock_local_sync_346_79a to +confirm that it has received changes to the target LSN (0/3000000): + + SELECT spock.wait_slot_confirm_lsn('spock_local_sync_346_79a', + '0/3000000'); diff --git a/docs/spock_functions/functions/spock_xact_timestamp_origin.md b/docs/spock_functions/functions/spock_xact_timestamp_origin.md new file mode 100644 index 00000000..f87cdece --- /dev/null +++ b/docs/spock_functions/functions/spock_xact_timestamp_origin.md @@ -0,0 +1,48 @@ +## NAME + +spock.xact_commit_timestamp_origin() + +### SYNOPSIS + +spock.xact_commit_timestamp_origin (xid xid, OUT timestamp timestamptz, +OUT roident oid) + +### RETURNS + +A record containing transaction commit information: + + - timestamp is the commit timestamp of the transaction. + - roident is the replication origin identifier of the transaction. + +### DESCRIPTION + +Returns the commit timestamp and replication origin identifier for a given +transaction ID. + +This function queries PostgreSQL's commit timestamp tracking system to +retrieve when a specific transaction was committed and which replication +origin it came from. The replication origin identifies the source node in a +multi-node replication topology, allowing you to trace where changes +originated. + +The commit timestamp tracking feature (track_commit_timestamp) must be +enabled in PostgreSQL for this function to return meaningful results. If +commit timestamp tracking is disabled, the function may return NULL or +raise an error. + +The replication origin identifier (roident) will be 0 for locally-committed +transactions that did not come through replication. For replicated +transactions, it identifies the origin node from which the changes were +received. + +This is a read-only query function that does not modify any data. + +### ARGUMENTS + +xid + + The transaction ID to query. + +### EXAMPLE + + SELECT * FROM spock.xact_commit_timestamp_origin('12345'::xid); diff --git a/docs/two_node_cluster.md b/docs/two_node_cluster.md index 5abab8e2..3f64c490 100644 --- a/docs/two_node_cluster.md +++ b/docs/two_node_cluster.md @@ -1,83 +1,137 @@ # Using Spock to Create a Two-Node Cluster -After installing and initializing Postgres and creating the Spock Extension, you can use the following steps to configure a two-node cluster. In the examples that follow, we'll be creating a cluster that contains two nodes, named `n1` and `n2` that listen for Postgres server connections on port `5432`. - -1. Connect to each node, and use the following commands to initialize a cluster: - - `sudo /usr/pgsql-17/bin/postgresql-17-setup initdb` - - `sudo systemctl enable postgresql-17` - -2. Use your choice of editor to open the `postgresql.conf` file (located in `/var/lib/pgsql/17/data/postgresql.conf`) and add the following parameters to the bottom of the file: - - `wal_level = 'logical'` - - `max_worker_processes = 10` - - `max_replication_slots = 10` - - `max_wal_senders = 10` - - `shared_preload_libraries = 'spock'` - - `track_commit_timestamp = on` - -3. Edit the [`pg_hba.conf` file](https://www.postgresql.org/docs/current/auth-pg-hba-conf.html) (located in `/var/lib/pgsql/current/data/postgresql.conf`) and allow connections between `n1` and `n2`; the following commands are provided as an example only, and are not recommended for production systems as they will open your system for connection from any client: - - `host all all 0.0.0.0/0 trust` - - `local replication all trust` - - `host replication all 0.0.0.0/0 trust` - -4. Use the [`spock.node_create`](spock_functions/functions/spock_node_create.md) command to create the provider and subscriber nodes; on `n1`, use the command: - - `SELECT spock.node_create (node_name := 'n1', dsn := 'host= port= dbname=');` +After installing Postgres, you can use the following steps to create the +Spock extension and configure a two-node cluster. In the +examples that follow, we'll be creating a cluster that contains two nodes, +named `n1` and `n2` that listen for Postgres server connections on port +`5432`. + + + ```sql + sudo /usr/pgsql-17/bin/postgresql-17-setup initdb + sudo systemctl enable postgresql-17 + ``` + +!!! hint + + SELECT version(); SHOW data_directory; SHOW config_file; + +1. After installing Postgres and initializing a cluster, on each node, use + your choice of editor to open the `postgresql.conf` file and add the + following parameters to the bottom of the file: + + ```sql + wal_level = 'logical' + max_worker_processes = 10 + max_replication_slots = 10 + max_wal_senders = 10 + shared_preload_libraries = 'spock' + track_commit_timestamp = on + ``` + +2. On each node that will host spock, edit the + [`pg_hba.conf` file](https://www.postgresql.org/docs/current/auth-pg-hba-conf.html) + (located in `/var/lib/pgsql/17/data/pg_hba.conf`) and allow + connections between `n1` and `n2`. The following commands are provided as + an example only, and are not recommended for production systems as they + will open your system for connection from any client: + + ```sql + host all all /32 trust + host all all /32 trust + + host replication all /32 trust + host replication all /32 trust + ``` + +3. Use your platform-specific command to restart the server on each node. + +4. Then, on each node, connect to the Postgres server with psql and create the + spock extension: + + ```sql + CREATE EXTENSION spock; + ``` + +5. Then, use the + [`spock.node_create`](spock_functions/functions/spock_node_create.md) + command to create the provider and subscriber nodes; on node `n1`: + + ```sql + SELECT spock.node_create + (node_name := 'n1', + dsn :='host= port= dbname='); + ``` On `n2`: - `SELECT spock.node_create (node_name := 'n2', dsn := 'host= port= =');` - -5. On `n1`, use the [`spock.repset_add_all_tables`](spock_functions/functions/spock_repset_add_all_tables.md) command to add the tables in the `public` schema to the `default` replication set. If you are working in another schema, customize this command as needed: - - `SELECT spock.repset_add_all_tables('default', ARRAY['public']);` - -6. On n2, use the [`spock.sub_create`](spock_functions/functions/spock_sub_create.md) command to create the subscription between n2 and n1; the name of the subscription is `sub_n2_n1`: - - `SELECT spock.sub_create (subscription_name := 'sub_n2_n1', provider_dsn := 'host= port= dbname=');` - - `SELECT spock.sub_wait_for_sync('sub_n2_n1');` - -7. On `n1`, create a corresponding subscription to `n2` named `sub_n1_n2`: - - `SELECT spock.sub_create (subscription_name := 'sub_n1_n2', subscriber_dsn := 'host= port= dbname=');` - -8. To ensure that modifications to your [DDL statements are automatically replicated](managing/spock_autoddl.md), connect to each node with a Postgres client and invoke the following SQL commands: - - `ALTER SYSTEM SET spock.enable_ddl_replication=on;` - - `ALTER SYSTEM SET spock.include_ddl_repset=on;` - - `ALTER SYSTEM SET spock.allow_ddl_from_functions=on;` - - `SELECT pg_reload_conf();` - -9. Then, check the status of each node: - -```sql - SELECT * FROM spock.node; - node_id | node_name | location | country | info - ---------+-------------+----------+---------+------ - 22201 | subscriber1 | | | - 53107 | provider1 | | | - (2 rows) - - SELECT * FROM spock.sub_show_status(); - subscription_name | status | provider_node | provider_dsn | slot_name | replication_sets | forward_origins - -------------------+-------------+---------------+------------------------------------------+--------------------------------------+---------------------------------------+----------------- - subscription1 | replicating | provider1 | host=localhost port=5432 dbname=postgres | spk_postgres_provider1_subscription1 | {default,default_insert_only,ddl_sql} | - (1 row) -``` - -A simple test to check that your system is replicating is to connect to the Postgres server on `n1` and add an object (like a table), and then confirm that it is available on `n2`. Similarly, you can create an object on `n2`, and confirm that it has been created on `n1`. + ```sql + SELECT spock.node_create (node_name := 'n2', + dsn :='host= port= dbname=');` + ``` + +6. On `n1`, use the + [`spock.repset_add_all_tables`](spock_functions/functions/spock_repset_add_all_tables.md) + command to add the tables in the `public` schema to the `default` + replication set. If you are working in another schema, customize this + command as needed: + + ```sql + SELECT spock.repset_add_all_tables('default', ARRAY['public']); + ``` + +7. On n2, use the following commands + to create the subscription between n2 and n1 and wait for the subscription + to sync; the name of the subscription is `sub_n2_n1`: + + ```sql + SELECT spock.sub_create (subscription_name := 'sub_n2_n1', + provider_dsn := 'host= port= dbname='); + + SELECT spock.sub_wait_for_sync('sub_n2_n1'); + ``` + +8. On `n1`, create a corresponding subscription to `n2` named `sub_n1_n2`: + + ```sql + SELECT spock.sub_create (subscription_name := 'sub_n1_n2', + subscriber_dsn := 'host= port= dbname='); + ``` + +9. To ensure that modifications to your [DDL statements are automatically + replicated](managing/spock_autoddl.md), connect to each node with a + Postgres client and invoke the following SQL commands: + + ```sql + ALTER SYSTEM SET spock.enable_ddl_replication=on; + ALTER SYSTEM SET spock.include_ddl_repset=on; + ALTER SYSTEM SET spock.allow_ddl_from_functions=on; + SELECT pg_reload_conf(); + ``` + +10. Then, check the status of each node: + + ```sql + SELECT * FROM spock.node; + node_id | node_name | location | country | info + ---------+-------------+----------+---------+------ + 22201 | subscriber1 | | | + 53107 | provider1 | | | + (2 rows) + + SELECT * FROM spock.sub_show_status(); + subscription_name | status | provider_node | provider_dsn | + slot_name | replication_sets | forward_origins + -------------------+-------------+---------------+--------------+ + -------------+----------------------+----------------- + subscription1 | replicating | provider1 | host=localhost + port=5432 dbname=postgres | spk_postgres_provider1_subscription1 | + {default,default_insert_only,ddl_sql} | + (1 row) + ``` + +A simple test to check that your system is replicating is to connect to the +Postgres server on `n1` and add an object (like a table), and then confirm +that it is available on `n2`. Similarly, you can create an object on `n2`, +and confirm that it has been created on `n1`. diff --git a/mkdocs.yml b/mkdocs.yml index d1c6fae8..f0285ec9 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -52,6 +52,7 @@ nav: - Introduction: index.md - Installing and Configuring Spock: install_spock.md - Creating a Two-Node Cluster: two_node_cluster.md + - Using Advanced Configuration Options: configuring.md - Upgrading a Spock Installation: upgrading_spock.md - Spock's Conflict Avoidance Options: conflicts.md - Spock's Management Features: @@ -83,10 +84,15 @@ nav: - Subscription Mgmt Functions: spock_functions/sub_mgmt.md - Cluster Mgmt Functions: spock_functions/gen_mgmt.md - Function Reference: + - spock.gen_slot_name(): spock_functions/functions/spock_gen_slot_name.md + - spock.get_country(): spock_functions/functions/spock_get_country.md + - spock.max_proto_version(): spock_functions/functions/spock_max_proto_version.md + - spock.min_proto_version(): spock_functions/functions/spock_min_proto_version.md - spock.node_add_interface(): spock_functions/functions/spock_node_add_interface.md - spock.node_create(): spock_functions/functions/spock_node_create.md - spock.node_drop(): spock_functions/functions/spock_node_drop.md - spock.node_drop_interface(): spock_functions/functions/spock_node_drop_interface.md + - spock.node_info(): spock_functions/functions/spock_node_info.md - spock.replicate_ddl(): spock_functions/functions/spock_replicate_ddl.md - spock.repset_add_seq(): spock_functions/functions/spock_repset_add_seq.md - spock.repset_add_table(): spock_functions/functions/spock_repset_add_table.md @@ -96,14 +102,15 @@ nav: - spock.repset_alter(): spock_functions/functions/spock_repset_alter.md - spock.repset_create(): spock_functions/functions/spock_repset_create.md - spock.repset_drop(): spock_functions/functions/spock_repset_drop.md - - spock.repset_list_tables(): spock_functions/functions/spock_repset_list_tables.md - spock.repset_remove_partition(): spock_functions/functions/spock_repset_remove_partition.md - spock.repset_remove_seq(): spock_functions/functions/spock_repset_remove_seq.md - spock.repset_remove_table(): spock_functions/functions/spock_repset_remove_table.md + - spock.repset_show_table(): spock_functions/functions/spock_repset_show_table.md - spock.table_wait_for_sync(): spock_functions/functions/spock_table_wait_for_sync.md - spock.seq_sync(): spock_functions/functions/spock_seq_sync.md - spock.sub_add_repset(): spock_functions/functions/spock_sub_add_repset.md - spock.sub_alter_interface(): spock_functions/functions/spock_sub_alter_interface.md + - spock.sub_alter_skiplsn(): spock_functions/functions/spock_sub_alter_skiplsn.md - spock.sub_create(): spock_functions/functions/spock_sub_create.md - spock.sub_disable(): spock_functions/functions/spock_sub_disable.md - spock.sub_drop(): spock_functions/functions/spock_sub_drop.md @@ -111,9 +118,12 @@ nav: - spock.sub_remove_repset(): spock_functions/functions/spock_sub_remove_repset.md - spock.sub_resync_table(): spock_functions/functions/spock_sub_resync_table.md - spock.sub_show_status(): spock_functions/functions/spock_sub_show_status.md - - spock.sub_show_table: spock_functions/functions/spock_sub_show_table.md - - spock.sub_sync(): spock_functions/functions/spock_sub_sync.md + - spock.sub_show_table(): spock_functions/functions/spock_sub_show_table.md + - spock.sync_event(): spock_functions/functions/spock_sync_event.md - spock.sub_wait_for_sync(): spock_functions/functions/spock_sub_wait_for_sync.md + - spock.version(): spock_functions/functions/spock_version.md + - spock.version_num(): spock_functions/functions/spock_version_num.md + - spock.wait_slot_confirm_lsn(): spock_functions/functions/spock_wait_slot_confirm_lsn.md - Limitations: limitations.md - FAQ: FAQ.md - Release Notes: spock_release_notes.md