Add pants-plugins/pack_metadata to inform pants about our packs

[cognifloyd]

Background

This is another part of introducing pants, as discussed in various TSC meetings.

Related PRs can be found in:

• pantsbuild milestone (which includes PRs since Initialize pantsbuild #5713)
• pantsbuild label (which also includes preparatory PRs that came before adding pantsbuild)

Overview of this PR

This PR makes it possible to inform pants about all of our pack metadata files by adding a new pack_metadata target. The pack_metadata target encompasses the following files:

st2/pants-plugins/pack_metadata/target_types.py

Lines 29 to 40 in 185e871

	default = (
	# metadata does not include any python, shell, or other sources.
	"pack.yaml",
	"config.schema.yaml",
	"*.yaml.example",
	"**/*.yaml",
	"**/*.yml",
	"icon.png", # used in st2web ui
	# "requirements*.txt", # including this causes target conflicts
	# "README.md",
	# "HISTORY.md",
	)

Python, shell, and requirements files all need to be handled separately so that pants can lint/fmt them appropriately. Pants does not have a markdown plugin yet, so I put them in the list as a reminder that they will need to be dealt with, but I commented them out.

One of our packs is in a git submodule (st2tests/st2tests/fixtures/packs/test_content_version), so it also adds a pack_metadata_in_git_submodule target to cleanly handle a case where someone has not checked out the git submodules. This improves the Developer Experience by raising an error as early as possible to let people know that they need to checkout the git submodule and how to do that. Otherwise, they won't get an error until they run tests where we have some code to give the same instructions for the tests that need it.

Note that I have not enabled the plugin yet:

st2/pants.toml

Line 28 in 185e871

#"pack_metadata",

That keeps this PR focused on the plugin itself including adding tests for it. The next PR will add the pack_metadata targets (at least in part using ./pants tailor ::) in our BUILD files. If you want to explore some of that, you can look at: pants-plugins-pack_metadata...pants-pack_metadata

Developing pants plugins

Pants has extensive documentation on the plugin API, including how to create targets, how to write rules, and there's even a mini tutorial about how to add a formatter (adding formatters is a very common thing for plugins to do).

• Plugins overview
• The Target API
  • Concepts eg: defines targets and fields.
  • Creating new fields
  • Creating new targets
• The Rules API
• Testing Plugins

[amanda11]

Couple of typos

[cognifloyd]

Good catch! Fixed.

[amanda11]

LGTM