[Source Versioning] SS-60 update docs for SQL server source versioning by patrickwwbutler · Pull Request #34918 · MaterializeInc/materialize

patrickwwbutler · 2026-02-05T18:59:21Z

Updates docs describing process to add/remove columns from SQL Server upstream sources

Motivation

This PR adds a known-desirable feature.
https://linear.app/materializeinc/issue/SS-60/write-docs-for-sql-server-source-versioning

Checklist

This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

martykulma · 2026-02-06T12:38:09Z

Should also update the SQL pages for CREATE SOURCE and CREATE TABLE to include SQL Server.

patrickwwbutler · 2026-02-06T16:00:13Z

Should also update the SQL pages for CREATE SOURCE and CREATE TABLE to include SQL Server.

CREATE SOURCE contains a "pointer" to the sql-server-considerations.md file, so it should receive this change notifying users of the source versioning workflow as well.

As for CREATE TABLE, it looks like we provide 2 different syntaxes, "Read/write table" and "PostgreSQL source table" https://materialize.com/docs/sql/create-table/

From what I can tell, isn't the syntax for creating a source table the same for each source? And it's only certain functionality that is actually dependent on the source type? Perhaps that section should just be renamed to "read-only source table"?

cc @aljoscha for some confirmation on my understanding of the mzsql syntax here.

martykulma · 2026-02-06T18:35:40Z

Should also update the SQL pages for CREATE SOURCE and CREATE TABLE to include SQL Server.

CREATE SOURCE contains a "pointer" to the sql-server-considerations.md file, so it should receive this change notifying users of the source versioning workflow as well.

As for CREATE TABLE, it looks like we provide 2 different syntaxes, "Read/write table" and "PostgreSQL source table" https://materialize.com/docs/sql/create-table/

From what I can tell, isn't the syntax for creating a source table the same for each source? And it's only certain functionality that is actually dependent on the source type? Perhaps that section should just be renamed to "read-only source table"?

cc @aljoscha for some confirmation on my understanding of the mzsql syntax here.

The syntax of CREATE TABLE .. FROM SOURCE is the roughly the same (I think it's the same for OLTP sources, Kafka is a little different). Renaming "postgres source table" sounds like the best approach. Just need to make sure to specify PG & SQL Server in the sections.

kay-kim · 2026-02-06T19:01:33Z

Could we actually create a separate tab for SQL Server? Reasons:

SQL Server doesn't support TEXT COLUMNS, yes?
Once we get more source versioning turned on, we'll actually separate out the tabs into separate pages ... so that people don't have to wade into the page for content relevant to their source.

martykulma · 2026-02-06T19:07:12Z

SQL Server doesn't support TEXT COLUMNS, yes?

SQL Server does support text columns!

kay-kim · 2026-02-06T19:13:07Z

Oh ... then, FYI -- the current create source for sql server only lists EXCLUDE COLUMNS (...)

CREATE SOURCE [IF NOT EXISTS] <src_name>
[IN CLUSTER <cluster_name>]
FROM SQL SERVER CONNECTION <connection_name>
  [ ( EXCLUDE COLUMNS (<col1> [, ...]) ) ]
<FOR ALL TABLES | FOR TABLES ( <table1> [AS <subsrc_name>] [, ...] )>
[WITH (RETAIN HISTORY FOR <retention_period>)]

and we also state that we don't support types text and ntext ...

martykulma · 2026-02-06T19:21:59Z

Oh ... then, FYI -- the current create source for sql server only lists EXCLUDE COLUMNS (...)
CREATE SOURCE [IF NOT EXISTS] <src_name>
[IN CLUSTER <cluster_name>]
FROM SQL SERVER CONNECTION <connection_name>
  [ ( EXCLUDE COLUMNS (<col1> [, ...]) ) ]
<FOR ALL TABLES | FOR TABLES ( <table1> [AS <subsrc_name>] [, ...] )>
[WITH (RETAIN HISTORY FOR <retention_period>)]
and we also state that we don't support types text and ntext ...

whoops - should add TEXT COLUMNS there!

The unsupported types are correct. SQL Server CDC doesn't play well with text and ntext, and they are deprecated types. Customers can use varchar(max) / nvarchar(max)

patrickwwbutler · 2026-02-09T17:01:54Z

Oh ... then, FYI -- the current create source for sql server only lists EXCLUDE COLUMNS (...)
CREATE SOURCE [IF NOT EXISTS] <src_name>
[IN CLUSTER <cluster_name>]
FROM SQL SERVER CONNECTION <connection_name>
  [ ( EXCLUDE COLUMNS (<col1> [, ...]) ) ]
<FOR ALL TABLES | FOR TABLES ( <table1> [AS <subsrc_name>] [, ...] )>
[WITH (RETAIN HISTORY FOR <retention_period>)]
and we also state that we don't support types text and ntext ...

Do the TEXT COLUMNS and EXCLUDE COLUMNS clauses actually go in the CREATE SOURCE statement? I would have assumed that they go in CREATE TABLE FROM SOURCE statement. Even the syntax we have in the docs here doesn't really make sense, because there is no table specified for which to exclude/text columns from?

kay-kim · 2026-02-09T18:47:05Z

Do the TEXT COLUMNS and EXCLUDE COLUMNS clauses actually go in the CREATE SOURCE statement? I would have assumed that they go in CREATE TABLE FROM SOURCE
So. ...

for people using the old syntax, it'd be in the CREATE SOURCE statement.
for people using the new syntax, it'd be in the CREATE TABLE statement.

doc/user/data/examples/create_source_sql_server.yml

doc/user/content/ingest-data/sql-server/source-versioning.md

kay-kim · 2026-02-19T18:14:18Z

doc/user/content/sql/create-table.md

 ### Handling table schema changes

 {{% include-from-yaml data="postgres_source_details"
 name="postgres-compatible-schema-changes" %}}


So ... this now only link people to the postgres guide. Will need to tweak.

doc/user/content/ingest-data/sql-server/source-versioning.md

doc/user/content/sql/create-table.md

…xt and ntext columns

doc/user/content/headless/sql-server-considerations.md

doc/user/data/examples/create_source_sql_server_legacy.yml

doc/user/content/sql/create-source/sql-server-v2.md

doc/user/content/sql/create-table.md

doc/user/content/ingest-data/sql-server/source-versioning.md

kay-kim

almost... left a question.

kay-kim · 2026-02-26T20:30:19Z

doc/user/content/headless/sql-server-considerations.md

+{{< note >}}
+This section refers to the legacy [`CREATE SOURCE ... FOR ...` syntax](/sql/create-source/sql-server/) that creates subsources as part of the `CREATE SOURCE` operation. see the [SQL Server source versioning guide](/ingest-data/sql-server/source-versioning/) for information on using the new syntax to handle upstream schema changes.
+{{< /note >}}
+{{% schema-changes %}}


Q: the Incompatible schema changes ... wouldn't that be for both New and Legacy syntax?

https://preview.materialize.com/materialize/34918/ingest-data/sql-server/#incompatible-schema-changes

kay-kim · 2026-02-26T20:30:36Z

doc/user/content/headless/sql-server-considerations.md

-{{% include-headless "/headless/schema-changes-in-progress" %}}
+#### New Syntax
+
+Certain additional schema changes (such as dropping/altering existing columns) to SQL Server tables are supported when using the new [`CREATE SOURCE` syntax](/sql/create-source/sql-server-v2/) as detailed by the [source versioning guide](/ingest-data/sql-server/source-versioning/)


Given the note above, we can get rid of this, yes?

patrickwwbutler added 2 commits February 5, 2026 13:12

update docs for SQL server source versioning

83a4fb1

fix whitespace

39c7063

patrickwwbutler requested review from a team, DAlperin, kay-kim, martykulma and petrosagg February 5, 2026 21:01

patrickwwbutler marked this pull request as ready for review February 5, 2026 21:02

added bolding to make source versioning more visible

d830c40

patrickwwbutler added 2 commits February 17, 2026 14:52

updated create table and create source docs for sql server

1a82b31

add tab to data types section

5145e44

kay-kim reviewed Feb 19, 2026

View reviewed changes

fix errors, add new create source page for sql server, add info on te…

954fb00

…xt and ntext columns

patrickwwbutler requested a review from kay-kim February 25, 2026 21:02

kay-kim reviewed Feb 25, 2026

View reviewed changes

patrickwwbutler added 2 commits February 26, 2026 12:19

address comments

b89ea1f

fix lint error and missing hash

405a360

patrickwwbutler requested a review from kay-kim February 26, 2026 18:38

kay-kim reviewed Feb 26, 2026

View reviewed changes

Conversation

patrickwwbutler commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Checklist

Uh oh!

martykulma commented Feb 6, 2026

Uh oh!

patrickwwbutler commented Feb 6, 2026

Uh oh!

martykulma commented Feb 6, 2026

Uh oh!

kay-kim commented Feb 6, 2026

Uh oh!

martykulma commented Feb 6, 2026

Uh oh!

kay-kim commented Feb 6, 2026

Uh oh!

martykulma commented Feb 6, 2026

Uh oh!

patrickwwbutler commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kay-kim commented Feb 9, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kay-kim Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kay-kim left a comment

Choose a reason for hiding this comment

Uh oh!

kay-kim Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

kay-kim Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

patrickwwbutler commented Feb 5, 2026 •

edited

Loading

patrickwwbutler commented Feb 9, 2026 •

edited

Loading