CLI: ensure new collections are added on deploy #1262
Description
I don't think we do any maintenance of the collections array
When we pull a project, we take the collections key from state and save it to the yaml [citation needed]. This is fine.
But if the user wants to add a new collection, right now they'll need to go into the app and it through the UI.
What we really need to do is add the new collection to the yaml on upload.
The problem is that right now, there's nowhere for the collection to be declared. I can add collections.get('wibble') to my job code, but I need to do something more to get it added on deploy.
I see two options:
- Add a collections array to openfn.yaml, which pulls the collections out of the project yaml file. This invites the user to modify them and add new collections as needed
- Have the user modify the local project yaml directly. Just go in and add to the array. But I have a feeling that the merge will prefer the source collections to the remote one. So not sure if it'll work (and it's crude)
So I think this is a small feature to expose collections in in openfn.yaml.project, and ensure that we use that value in the merge