docs: various documentation updates in preparation for next release #3254
+117
−27
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add documentation for expressions that were implemented but not documented in the supported expressions list: - Left (string function) - DateDiff, DateFormat, LastDay, UnixDate, UnixTimestamp (date/time) - Sha1 (hashing) - JsonToStructs (struct) Also fixes TruncTimestamp SQL from `trunc_date` to `date_trunc`. Co-Authored-By: Claude Opus 4.5 <[email protected]>
andygrove
changed the title
…to native_comet Updates contributor guide documentation following the change in c9af2c6: - parquet_scans.md: clarify that auto mode falls back to Spark's native scan, not native_comet - roadmap.md: reflect that the switch to native_iceberg_compat for auto mode is complete Co-Authored-By: Claude Opus 4.5 <[email protected]>
Adds documentation showing how to configure Comet's native Iceberg scan with a REST catalog, including example Spark configuration and sample queries demonstrating namespace creation. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Adds comprehensive documentation of native Iceberg reader capabilities: - Table spec versions (v1, v2 supported; v3 falls back) - Schema and data types including complex types and schema evolution - Time travel and branch reads - MOR table delete handling (positional and equality deletes) - Filter pushdown operations - Partitioning with various transform types - Storage backends (local, HDFS, S3) Also improves the limitations section with clearer explanations. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Co-Authored-By: Claude Opus 4.5 <[email protected]>
The datasources.md previously stated that Comet does not provide native CSV scan, but experimental native CSV support was added in commit f538424. Updated to reflect the new spark.comet.scan.csv.v2.enabled option. Co-Authored-By: Claude Opus 4.5 <[email protected]>
- parquet_scans.md: complete the truncated sentence in S3 Support section - roadmap.md: update native_comet removal section to reflect that auto mode now uses native_iceberg_compat (milestone achieved) - roadmap.md: fix typo "originally" -> "original" Co-Authored-By: Claude Opus 4.5 <[email protected]>
mbutrovich
approved these changes
Jan 23, 2026
Contributor
mbutrovich
left a comment
mbutrovich
left a comment
There was a problem hiding this comment.
LGTM, thanks @andygrove!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Documentation Updates
User Guide
Iceberg Guide (
docs/source/user-guide/latest/iceberg.md):VERSION AS OF, branch reads)Data Sources (
docs/source/user-guide/latest/datasources.md):spark.comet.scan.csv.v2.enabled)Expressions (
docs/source/user-guide/latest/expressions.md):Contributor Guide
Parquet Scans (
docs/source/contributor-guide/parquet_scans.md):native_comet)Roadmap (
docs/source/contributor-guide/roadmap.md):native_iceberg_compatnative_cometremoval section to indicate the milestone has been achieved