RFE: Allow every field to reference a remote resource

[eriksjolund]

I noticed at
https://github.com/coreos/fcct/blob/master/docs/configuration-v1_1.md
that systemd -> contents does not have variant for including the contents from a local file. For me it would have been convenient to have that feature.

Why not generalize the data input, so that strings and bytearrays always can be included from URLs, local files or inline?
Something like two new "datatypes":
general_text_data and general_binary_data

Just a sketch:

• general_text_data (object): the config that will replace the current.
  • source (string): the URL of the config. Supported schemes are http, https, s3, tftp, and [data][rfc2397]. Note: When using http, it is advisable to use the verification option to ensure the contents haven't been modified. Mutually exclusive with inline and local.
  • inline (string): the contents of the config. Mutually exclusive with source and local.
  • local (string): a local path to the contents of the config, relative to the directory specified by the --files-dir command-line argument. Mutually exclusive with source and inline.
  • http_headers (list of objects): a list of HTTP headers to be added to the request. Available for http and https source schemes only.
    • name (string): the header name.
    • value (string): the header contents.
  • verification (object): options related to the verification of the config.
    • hash (string): the hash of the config, in the form <type>-<value> where type is either sha512 or sha256.

If it would be possible to reference the string contents of one of

passwd -> users -> ssh_authorized_keys

with a general_text_data, you could start referencing your public ssh key from Github, as they are downloadable like this

wget https://github.com/eriksjolund.keys

[LorbusChris]

I think this is reasonable. Something similar was already done for the Ignition types: #953

[bgilbert]

I suspect that allowing arbitrary fields to come from URLs or local files would introduce more complexity to the spec than the feature is worth.

For your original use case, see #986.

[eriksjolund]

At first I thought the URL:s were downloaded by fcct (at "transpile-time" so to say) but now I see the URL:s
are given to the Ignition file.
So this Github issue probably belongs here
https://github.com/coreos/ignition
instead.
(Should I close this one and open a Github issue there instead?)

Regarding introducing more complexity:
The change would be like changing a datatype,
i.e. from string to general_text_data. In the source code,
shouldn't that just be a small change?

I understand that this RFE would introduce more challenges
in how to easily present the file format standard in the documentation
for the users.

If many of the string would be replaced in the file
https://github.com/coreos/fcct/blob/master/docs/configuration-v1_1.md
with a general_text_data sub tree, that file would become very big.

One way to solve it would be to define general_text_data seperately
outside the normal tree structure.
Another way to solve it could be to generate out some HTML documentation
where the file format specification could be browsed by a web browser.
Then it would be easier to represent a "sub tree" data type.

Another idea:
The fcct could be given a new command-line argument:
--download-content-and-embed-into-ignition-file
so that such general_text_data -> source (with an URL) could be converted
to a general_text_data -> inline (where the string content is embedded into the Ignition file)

That could be useful when a Fedora CoreOS installation is performed without internet access.

[bgilbert]

Okay, I've moved the issue over to Ignition.

We already have a reusable data type for this, Resource, which you can see in action e.g. here. Expanding every field to use Resource would be a major breaking change and would make configs much more verbose. If you have a set of use cases where this would be really helpful, though, please do share them here.

[bgilbert]

You can also use the files section to write services and SSH keys:

variant: fcos
version: 1.1.0
storage:
  directories:
    - path: /home/core/.ssh
      user:
        name: core
      group:
        name: core
      mode: 0700
    - path: /home/core/.ssh/authorized_keys.d
      user:
        name: core
      group:
        name: core
      mode: 0700
  files:
    - path: /home/core/.ssh/authorized_keys.d/github
      user:
        name: core
      group:
        name: core
      mode: 0600
      contents:
        source: https://github.com/bgilbert.keys
    - path: /etc/systemd/system/example.service
      contents:
        source: https://example.com/example.service
systemd:
  units:
    - name: example.service
      enabled: true

[eriksjolund]

List of use cases

Use case 1. Including a systemd unit file from the file system

It seems to be addressed in
#986
that @bgilbert mentioned in
#1097 (comment)

This use case can also be worked around by using
storage -> files -> contents -> local

I did that in an experiment of mine
https://github.com/Clinical-Genomics/scout/blob/systemd_services_podman/systemd/scout.fcc

storage:
  files:
    - path: /home/core/.config/systemd/user/scout-pod.service
      user:
        name: core
      group:
        name: core
      mode: 0644
      contents:
        local: scout-pod.service

(although that example uses systemd user units)

Use case 2. Being able to specify an SSH public key as an URL

Being able to specify an URL, it would be easier to identify the person being granted access just by looking
at an Ignition/FCC file.
For instance URL:s could be used in the form of https://github.com/<githubusername>.keys

@bgilbert Yes, your suggestion in #1097 (comment) to use
storage -> files -> contents -> source
would solve that.

Some more thoughts

My first thought by creating this RFE was that over time there will always pop up RFE:s requesting conversion of a string datatype into the Resource datatype in the spec. For every such change the spec would break. To avoid that situation,
why not right away change all occurrences of the string datatype into the general Resource datatype?

I've been thinking a bit more about this RFE. As there are workarounds, I don't have such a big need for it. If you want you could close this issue, or just keep it open to gather user feedback.