Validate WEB_FEATURES.yml #4701
Conversation
jugglinmike
force-pushed
the
lint-web-features
branch
2 times, most recently
from
9f5274b to
aa74a3b
Compare
For every pull request, verify that the `WEB_FEATURES.yml` file is internally consistent and that the pull request does not reduce the number of tests associated with a given classifier. Failing for any reduction in the number of matched tests is intended to alert contributors during file renaming operations. This failure will need to be ignored when tests are simply deleted, and this could potentially become a distraction for maintainers. However, test deletion has historically been so rare that spurious failures are not expected to be common enough to substantively interfere with routine maintenance.
jugglinmike
force-pushed
the
lint-web-features
branch
from
3f00976 to
dd54213
Compare
I think the main place where this would come up is when moving tests out of staging, into the main tree. They might get merged into existing tests. I don't actually know if the classification applies to the staging folder in the first place, so maybe this isn't a problem. |
That's a good call-out! We haven't classified tests in staging, so fortunately those kinds of moves won't be an issue. |
gibson042
left a comment
gibson042
left a comment
There was a problem hiding this comment.
I'd like the directory used by testing to be ignored by git (either explicitly or by redefinition under $TMPDIR) to avoid filesystem pollution even in the event of a process crash, but otherwise it's just a couple of typo fixes.
tools/web-features/README.md
Outdated
| `--manifest`, this utility will write a "manifest" of all web-features and | ||
| their associated test file names. | ||
|
|
||
| ### `check-coverage.py' |
There was a problem hiding this comment.
| ### `check-coverage.py' | |
| ### `check-coverage.py` |
| import json | ||
|
|
||
| testDir = os.path.dirname(os.path.abspath(__file__)) | ||
| OUT_DIR = os.path.join(testDir, 'out') |
There was a problem hiding this comment.
Is this directory ignored by git?
.github/workflows/checks.yml
Outdated
| - name: debug | ||
| run: ls -lah | ||
|
|
||
| # Checking out the original branch ensure that if the patch under review |
There was a problem hiding this comment.
| # Checking out the original branch ensure that if the patch under review | |
| # Checking out the original branch ensures that if the patch under review |
|
Thanks for the review, @gibson042! In addition to adding the file to the project's |
3 participants
For every pull request, verify that the
WEB_FEATURES.ymlfile is internally consistent and that the pull request does not reduce the number of tests associated with a given classifier.Failing for any reduction in the number of matched tests is intended to alert contributors during file renaming operations. This failure will need to be ignored when tests are simply deleted, and this could potentially become a distraction for maintainers. However, test deletion has historically been so rare that spurious failures are not expected to be common enough to substantively interfere with routine maintenance.