Add IN CLUSTER ... REPLICA ... syntax for materialized views#35235
Draft
antiguru wants to merge 6 commits intoMaterializeInc:mainfrom
Draft
Add IN CLUSTER ... REPLICA ... syntax for materialized views#35235antiguru wants to merge 6 commits intoMaterializeInc:mainfrom
antiguru wants to merge 6 commits intoMaterializeInc:mainfrom
Conversation
Add a `target_replica` field on `DataflowDescription` that pins a dataflow to a specific replica. The controller ensures only the targeted replica receives the dataflow and its related commands (`Schedule`, `AllowWrites`, `AllowCompaction`). The field defaults to `None` in `DataflowDescription::new()`, so all existing callers retain broadcast behavior unchanged. Co-Authored-By: Claude Opus 4.6 <[email protected]>
|
Thanks for opening this PR! Here are a few tips to help make the review process smooth for everyone. PR title guidelines
Pre-merge checklist
|
Thread a `target_replica` through the SQL layer so materialized views can be pinned to a specific cluster replica. The parser accepts `IN CLUSTER <cluster> REPLICA <replica>` on CREATE MATERIALIZED VIEW statements. The planner resolves the replica name to a ReplicaId, the catalog stores it, and the sequencer sets `DataflowDescription.target_replica` before shipping the dataflow. On bootstrap, the field is restored from the catalog entry. The syntax is internal-only. All existing callers continue to pass `None`, so behavior is unchanged. Co-Authored-By: Claude Opus 4.6 <[email protected]>
antiguru
force-pushed
the
replica_targeted_objects
branch
from
37af36a to
b86bc61
Compare
The pretty-printer for `SHOW CREATE MATERIALIZED VIEW` was not emitting the `REPLICA <name>` clause after `IN CLUSTER <name>`. Also fix the error message expectation in the SLT test. Co-Authored-By: Claude Opus 4.6 <[email protected]>
…dropped When a replica is dropped, any materialized views that target it via `IN CLUSTER ... REPLICA ...` are also dropped. Without this, the persist shard's upper frontier stops advancing (no replica advances it), causing reads to hang indefinitely. The cascade is implemented in `ObjectsToDrop::add_item` for the `ClusterReplica` case, which covers both `DROP CLUSTER REPLICA` and `ALTER CLUSTER SET (REPLICATION FACTOR ...)` code paths. Co-Authored-By: Claude Opus 4.6 <[email protected]>
antiguru
commented
Feb 27, 2026
Member
Author
antiguru
left a comment
antiguru
left a comment
There was a problem hiding this comment.
We need to ensure that replica-targeted objects can only be created by privileged users, i.e., mz_system for the moment.
* Restrict replica-targeted MV creation to internal users (mz_system). Non-internal users get "permission denied" error. * Extract `ComputeCommand::target_replica()` method and use it in the rehydration filtering logic. * Add comment explaining the target_replica existence check in `create_dataflow`. Co-Authored-By: Claude Opus 4.6 <[email protected]>
antiguru
commented
Feb 27, 2026
Comment on lines
1149
to
1156
| match &command { | ||
| ComputeCommand::Schedule(cid) if skipped_ids.contains(cid) => continue, | ||
| ComputeCommand::AllowCompaction { id: cid, .. } if skipped_ids.contains(cid) => { | ||
| continue; | ||
| } | ||
| ComputeCommand::AllowWrites(cid) if skipped_ids.contains(cid) => continue, | ||
| _ => {} | ||
| } |
Member
Author
There was a problem hiding this comment.
Please extract a collection_id function here.
antiguru
force-pushed
the
replica_targeted_objects
branch
3 times, most recently
from
da3de8c to
969657d
Compare
The privilege check requires an internal user for creating replica-targeted materialized views. Update both SLT and testdrive tests to use mz_system connections for these statements. Co-Authored-By: Claude Opus 4.6 <[email protected]>
antiguru
force-pushed
the
replica_targeted_objects
branch
from
969657d to
76f337d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Per-replica introspection needs materialized views that render only on a specific replica.
This adds
IN CLUSTER <cluster> REPLICA <replica>syntax toCREATE MATERIALIZED VIEW, threading atarget_replicathrough the compute controller, SQL parser, planner, catalog, and sequencer.The first commit adds the
target_replicafield toDataflowDescriptionand the compute controller, including thesend_to_collection_replicashelper that routes commands only to the replica hosting a targeted collection.The second commit threads the field through the SQL layer: the parser accepts the new syntax, the planner resolves the replica name, the catalog stores it, and the sequencer sets it on the dataflow before shipping.
On bootstrap,
target_replicais restored from the catalog.The syntax is internal-only.
All existing callers pass
None, so behavior is unchanged.Tests added:
test/sqllogictest/materialized_view_replica_targeted.slt— parsing, error cases, round-trip, data correctnesstest/testdrive/materialized-view-replica-targeted.td— introspection (hydration status, per-replica frontiers), replica drop/recreate, cluster restart🤖 Generated with Claude Code