Copy mechanism

[mottosso]

Do not merge

Related to #43 (comment)

[tokejepsen]

I guess the question is whether we want to copy from the docker Samba where there is an existing zip file, or download from a release.

If we copy from the docker Samba, we would ensure all machines have the same version, but might tricky to show what has changed since last time.

[mottosso]

Downloading from a release might be the sensible thing. Samba was only really useful for interactively hosting and mounting the files.

Either way, we need a version number included in the release, something we can check against. Like a version.py or VERSION txt file.

[tokejepsen]

If we are downloading from a release, we probably won't need Samba at all in the end?

Also this PR looks like it's focussing on a local zip file rather than downloading from a release?

[mottosso]

If we are downloading from a release, we probably won't need Samba at all in the end?

Yep, could skip it altogether.

Also this PR looks like it's focussing on a local zip file rather than downloading from a release?

Yes, I think we should be able to replace the local zip path with the zip from a GitHub release. It'd need to be a release as opposed to downloading from master as I don't think submodules are included in a master zip.

[tokejepsen]

I'm struggling to think through the workflow of downloading a release. Here is the release workflow I can see:

1. The administrator/developer clones the repository, and starts the docker containers, which would mainly be the database.
2. The user executes a bat/shell script, from where?
3. The bat/shell script downloads the latest release to .avalon directory, and executes files from there.

I can see a benefit with this, in that anywhere in the world users would get the latest releases. But they would still need to connect to a database which would be within a company.

If we were to copy from a local Samba share, here is the workflow as I can see it;

1. The administrator/developer clones the repository, and starts the docker containers, which would be the database and Samba share.
2. The user executes a bat/shell script on the Samba share.
3. The bat/shell script copies a premade zip file from the Samba and extracts it to .avalon directory. Files are executed from the .avalon directory.

I see this as the safer workflow for companies. They can test out a new release before committing the production to it. Also the administrators/developers would know which version is on each machine.
Lastly a company could be testing out new changes they have developed in production before merging into the official build.

[mottosso]

The administrator/developer clones the repository

One of the advantages of distributing with Docker is that we can git out of the loop.

Instead, they could launch a Docker container containing the tools necessary for downloading a release.

$ docker run --rm -v /install/dir:/volume avalon/install

Where /install/dir could be e.g. c:\users\marcus\.avalon, and the container would download and extract the zip into that install dir.

[tokejepsen]

That is very true!

That could essentially be our install part, where the user could install it anywhere they want.
For network usage they could even just copy those files to a network location, or have some batch script for installing locally on each machine.

[tokejepsen]

Done some experimenting. The fastest way of copying from a container seems to be with docker cp. Think this might be same mechanism that Docker uses when sending the context to build.

Dockerfile-install

FROM alpine
COPY ./volume /avalon

Install

docker run -t -d --rm --name avalon-install avalon/install
docker cp avalon-install:avalon /Users/admin/.avalon
docker kill avalon-install

[tokejepsen]

We could also go the whole way and make an installer executable with python scripts and PyInstaller or py2exe.

But I haven't got any experience with installers, so don't know how difficult it is to make cross-platform installers.

[mottosso]

Dockerfile-install

What I had in mind is a little different, let me know what you think of this.

1. Dockerfile-install contains just a Python interpreter, and script capable of downloading a Zip from a GitHub release; that way it'd only be a few mb
2. The release is updated manually and versioned
3. The dockerfile could offer a --version to install a particular version, and otherwise just download the latest release (may need some logic to determine which on GitHub is the latest, but that's ok)
4. The image is available on Docker Hub, such that docker run avalon/install works without any setup
5. The installation path being provided via the --volume argument means it can be installed anywhere
6. Running the container without arguments could print a "Help" message of how to use it
7. The container could potentially offer an interactive CLI, with options to install other containers like Mongo and CGWire. That way, Dockerfile-install becomes the entrypoint to Avalon and if it also handles updates, effectively becomes the Avalon management CLI
8. On performance of docker cp, I wouldn't worry about it. Installation could take a minute or two without being troublesome, as it only happens once and is expected to take some time.
9. On cross-platform installers, where I can see that being helpful is in distributing Python+PyQt (did you have something else in mind?), but we would need a distribution for every major Linux flavour (as they aren't compatible, like the many Windows versions) so I don't think it's a goal worth chasing. I would focus on Windows for now, and let Linux users install Python+PyQt themselves. Then we could have a look at distributing one for CentOS7 where VFX seems to be heading, and then Ubuntu for the rest. Then OSX is a different beast altogether.

[tokejepsen]

• 1 Yeah, that sounds good. Easy manage and update, even test.
• 2 Guessing this will be automated with Travis in the future?
• 3 👍
• 4 Should probably build from Alpine to minimize the size.
• 6 Or maybe with --help since no arguments would install the latest version.
• 7 Sounds interesting. Could you elaborate on how you see this happening?
• 8 The performance of copying from a container to a volume, was very poor when I tried. Took around 11 min to copy the files in /volume. Hope it better for downloading.

[mottosso]

The performance of copying from a container to a volume, was very poor when I tried. Took around 11 min to copy the files in /volume. Hope it better for downloading.

Likely Samba related. When we mount e.g. c:\users\toke into the container, and download into an internal directory of the container, e.g. /home/root/Downloads, and finally unpack it into the mounted directory it should be almost as fast as unpacking it without Docker.

7 Sounds interesting. Could you elaborate on how you see this happening?

Sure, you can mount the Docker engine into a Docker container, giving a container access to the host's Docker command. It's a little hacky, but I've used a few containers now that are doing that and it's been working fine. Look at some of these mad docker commands: 1, 2

6 Or maybe with --help since no arguments would install the latest version.

Actually now that I think about it, I think running it without argument should launch a little installation guide. Just something to confirm the install, and the destination path.

$ docker run -ti --rm ...
...

Where would you like to install Avalon?
[c:\users\toke\.avalon]

Allright, everything is ready to go, proceed?
[yes]

Installing...

Finished successfully! Type `avalon` to start.

Where every choice is pre-filled with a suggestion, and can be overridden by just typing something. Similar to the VMWare install on Linux

This could then also check for the --volume argument, whether it's mounted and what to do if it's not.

Could be a simple Python script. What do you think?

[tokejepsen]

Yeah, sounds great to me :)