CAF::Download class: API proposal

[stdweird]

There is no functional code here, only an API.
Based on #120
Part of #62

[stdweird]

@ned21 @jrha this is a PR to define the interface to use and have it reviewed/discussed before adding more work.

[stdweird]

retest this please

[ned21]

https as well?

[stdweird]

well, yes and no. i'm confused myself about the naming with all the available combinations and tools.

there are a number of "scenarios" i want to support, but not sure splitting them up in protocol/method/auth makes sense

• file: read from file (could require authentication like kinit+file, e.g. reading from afs (although i'm not sure if that isn't authorisation instead of authentication))
• http: non-encrypted read from server using HTTP GET etc etc (could require authentication, like 'Basic auth', but is pretty non-safe, so there's no authentication called passwd or something).
• plain http like MS uses this, i.e. by encrypting the payload, it becomes secure (authentication via decryption and encryption), and it requires a decrypting postprocessing otherwise the downloaded data is garbage. so calling this protocol http is maybe also not ok. i was thinking to put this under kinit+http (or gssapi+http) and detect the MS scenario via the same trick that CCM uses (the krbencrypt encoding https://github.com/quattor/CCM/blob/master/src/main/perl/Fetch/Download.pm#L96; i guess krbencrypt is something MS made up or is it a standardised name?)
• https implies a valid client certificate, so x509+http is basically https (https without a valid client certificate is just not smart)
• and there's the stuff i want to add for freeipa: http + authentication via spnego and kerberos, which is another flavour of kinit+http or gssapi+http, so a bit confusing with MS scenario, so i might call this kinit+httpn (HTTP Negotiate is a term used by microsoft for this, but they also support ntlm as authentication).
• in the future, i would also add kinit+ipavault or other custom tools used to manage sensitive data

the first 3 are currently supported by ccm-fetch and ncm-download (afai understand the code). the 4th one is to address quattor/CCM#54, the 5th one is there so we don't have to try to setup our own secure datastore and be able to use existing tools.

the difference between kinit and gssapi mainly lies in the way the token is generated/managed: kinit/kdestroy binary or perl GSSAPI, but they both mean 'use kerberos`.

[jrha]

@stdweird where did this get to?

[jrha]

Still a good idea, several components still implement their own download mechanisms.