feat: Add API to download likelihood patchset archives

[matthewfeickert]

Description

Resolves #1045

$ pyhf contrib download --help
Usage: pyhf contrib download [OPTIONS] [ARCHIVE_URL] [OUTPUT_DIRECTORY]

  Download the patchset archive from the remote URL and extract it in a
  directory at the path given.

  Example:

  .. code-block:: shell

      $ pyhf contrib download --verbose
      https://www.hepdata.net/record/resource/1408476?view=true 1Lbb-
      likelihoods

          1Lbb-likelihoods/patchset.json
          1Lbb-likelihoods/README.md
          1Lbb-likelihoods/BkgOnly.json

  Raises:     :class:`~pyhf.exceptions.InvalidArchiveHost`: if the provided
  archive host name is not known to be valid

Options:
  -v, --verbose   Enables verbose mode
  -f, --force     Force download from non-approved host
  -c, --compress  Keep the archive in a compressed tar.gz form
  -h, --help      Show this message and exit.

CLI API Example:

$ pyhf contrib download https://www.hepdata.net/record/resource/1408476?view=true 1Lbb-likelihoods
$ ls 1Lbb-likelihoods/
BkgOnly.json  patchset.json  README.md
$ rm -rf 1Lbb-likelihoods && pyhf contrib download -v https://www.hepdata.net/record/resource/1408476?view=true 1Lbb-likelihoods
1Lbb-likelihoods/patchset.json
1Lbb-likelihoods/README.md
1Lbb-likelihoods/BkgOnly.json
$ pyhf contrib download --compress https://www.hepdata.net/record/resource/1408476?view=true 1Lbb-likelihoods.tar.gz
$ file 1Lbb-likelihoods.tar.gz 
1Lbb-likelihoods.tar.gz: gzip compressed data, from Unix, original size modulo 2^32 6177792

ReadTheDocs build:

• https://pyhf.readthedocs.io/en/feat-add-patchset-download-cli-api/_generated/pyhf.contrib.utils.download.html
• https://pyhf.readthedocs.io/en/feat-add-patchset-download-cli-api/cli.html#pyhf-contrib

Checklist Before Requesting Reviewer

• Tests are passing
• "WIP" removed from the title of the pull request
• Selected an Assignee for the PR to be responsible for the log summary

Before Merging

For the PR Assignees:

• Summarize commit messages into a comprehensive review of the PR

* Add download command to Python contrib API
* Add download command to pyhf CLI API and add contrib group
* Add pyhf custom exception if an untrusted archive host is given
* Add tests of Python and CLI API versions of contrib download
   - Thanks to Anthony Sottile (@asottile) for advice: https://youtu.be/PXu3KCMT3l4
* Add requests as dependency of contrib extra
* Add Python contrib API to docs

[codecov]

Codecov Report

Merging #1046 into master will increase coverage by 0.04%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1046      +/-   ##
==========================================
+ Coverage   96.83%   96.87%   +0.04%     
==========================================
  Files          60       62       +2     
  Lines        3507     3556      +49     
  Branches      504      510       +6     
==========================================
+ Hits         3396     3445      +49     
  Misses         68       68              
  Partials       43       43              

Flag	Coverage Δ
#unittests	96.87% <100.00%> (+0.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/pyhf/cli/__init__.py	100.00% <100.00%> (ø)
src/pyhf/cli/cli.py	100.00% <100.00%> (ø)
src/pyhf/cli/contrib.py	100.00% <100.00%> (ø)
src/pyhf/contrib/utils.py	100.00% <100.00%> (ø)
src/pyhf/exceptions/__init__.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 75f2a87...b72e399. Read the comment docs.

[matthewfeickert]

@kratsg Given your comment on Issue #1045 RE: implementing Issue #947 what are your thoughts on this PR?

It seems to me that having something like this PRs functionality of downloading a tarball and then extracting it into a directory is still in scope of pyhf utilities, but susyudio is another separate planned library(?) given that your example starts with import susyudio. Is that correct?

[matthewfeickert]

I guess the only other thing to think about is if this should be made into a decorator.

[kratsg]

what if pyhf contrib download was hidden away if it didn't import correctly?

additionally, there should be a non-script/CLI test to check the python API itself.

[matthewfeickert]

additionally, there should be a non-script/CLI test to check the python API itself.

Hm. Maybe I misunderstand what you mean, but we don't have a public Python API for this, we only have the CLI API that's exposed.

[kratsg]

Hm. Maybe I misunderstand what you mean, but we don't have a public Python API for this, we only have the CLI API that's exposed.

It would generally make sense that we don't provide a CLI for anything that does not have a python API (for consistency) no? Maybe we need the python API first.

[matthewfeickert]

It would generally make sense that we don't provide a CLI for anything that does not have a python API (for consistency) no? Maybe we need the python API first.

I think I generally agree with that statement but if you're at the Python API do you really care about extracting files into directories rather than just reading them out from the in memory representation of the tarfile? I guess what you're thinking though is that if you wanted to script the download and unpacking of multiple likelihoods through the Python API you should be able to do that as well (correct?).

[kratsg]

I think I generally agree with that statement but if you're at the Python API do you really care about extracting files into directories rather than just reading them out from the in memory representation of the tarfile? I guess what you're thinking though is that if you wanted to script the download and unpacking of multiple likelihoods through the Python API you should be able to do that as well (correct?).

Yes, or likely -- someone will want to parallelize and do a signal grid... But I'm realizing that this doesn't make sense given we only really ever have two files per grid anyway (bkgonly + patchset)...