How To: Update dependencies in requirements.txt

Updating requirements.txt

TL;DR

Updating a dependency

1. Update the version of the dependency in fixed-requirements.txt:

    pyignore==1.0.0
   -pyinotify==0.9.2
   +pyinotify==0.9.6
    pypi==2.0.19

2. Run make requirements and commit the changed files to git:

   make requirements
   git add -u
   git commit -m "Update <dependency> to <version>"

Adding a dependency

1. Add the dependency, including the specific version, to fixed-requirements.txt:

    pyignore==0.0.8
   +pyinotify==0.9.6
    pypi==3.8.4

   Note that you can also use environment markers in fixed-requirements.txt:

    pyignore==0.0.8
   +pyinotify==0.9.6 ; platform_system == "Linux"
    pypi==3.8.4

   And you can also specify Git URLs:

    lock==0.0.5
   +git+https://github.com/StackStorm/logshipper.git@stackstorm_patched#egg=logshipper
    loquacious==0.1.0

2. For the specific component that uses the dependency, add it to the appropriate in-requirements.txt file without specifying a version:

    pyignore
   +pyinotify
    pypi

   You can also add specify Git URLs as well, if you need to:

    lock
   +git+https://github.com/StackStorm/logshipper.git@stackstorm_patched#egg=logshipper
    loquacious

3. Run make requirements and commit the changed files to git:

   make requirements
   git add -u
   git commit -m "Use <dependency> in <component/s>"

Overview

As a developer you may want to bump a version in requirements.txt. This is a bit more involved in StackStorm due to the various subpackages included in the project. There are a few files that control this for each of the ST2 components (also referred to as subpackages):

• fixed-requirements.txt - controls the versions and environment markers for all dependencies of StackStorm components, including st2api, st2client, and all runners
• st2*/in-requirements.txt and contrib/runners/*/in-requirements.txt - these files specify (without versions or environment markers) the dependencies for each StackStorm component
• requirements.txt - generated by fixate-requirements.py (run automatically when you run make requirements), the union of all dependencies of all StackStorm components and runners
• st2*/requirements.txt and contrib/runners/*/requirements.txt - generated by fixate-requirements.py, the requirements for a specific StackStorm component

The fixate-requirements.py script reads in the fixed-requirements.txt file, and reads in the in-requirements.txt for each component, and generates the requirements.txt containing only the dependencies that are used/imported by that component:

make requirements (via fixate-requirements.py)
 |
 +-- st2actions --- in-requirements.txt => requirements.txt
 |
 +-- st2api ------- in-requirements.txt => requirements.txt
 |
 .
 |
 +-- st2stream ---- in-requirements.txt => requirements.txt
 |
 +-- st2tests ----- in-requirements.txt => requirements.txt
 |
 +-- contrib
 |    |
 |    +-- runners
 |         |
 |         +-- action_chain_runner -- in-requirements.txt => requirements.txt
 |         |
 |         +-- announcement_runner -- in-requirements.txt => requirements.txt
 |         |
 .         .
 |         |
 |         +-- remote_runner -------- in-requirements.txt => requirements.txt
 |         |
 |         +-- winrm_runner --------- in-requirements.txt => requirements.txt
 |
 +-- => requirements.txt

Below are the steps to make this happen in the proper way:

1. Update the version in fixed-requirements.txt
2. Run make requirements a. Every component/sub-package that is already marked as using the package in their in-requirements.txt will get updated with the new version.
3. Add and commit all changed files to git and create a PR

Using environment markers

Quick Example

pyinotify==0.9.6 ; platform_system == "Linux"

Available Environment Markers

Marker	Python equivalent	Sample values
os_name	os.name	posix, java
sys_platform	sys.platform	linux, linux2, darwin, java1.8.0_51 (note that "linux" is from Python3 and "linux2" from Python2)
platform_machine	platform.machine()	x86_64
platform_python_implementation	platform.python_implementation()	CPython, Jython
platform_release	platform.release()`	3.14.1-x86_64-linode39, 14.5.0, 1.8.0_51
platform_system	platform.system()`	Linux, Windows, Java
platform_version	platform.version()	#1 SMP Fri Apr 25 13:07:35 EDT 2014, Java HotSpot(TM) 64-Bit Server VM, 25.51-b03, Oracle Corporation, Darwin Kernel Version 14.5.0: Wed Jul 29 02:18:53 PDT 2015; root:xnu-2782.40.9~2/RELEASE_X86_64
python_version	'.'.join(platform.python_version_tuple()[:2]	3.4, 2.7
python_full_version	platform.python_version()	3.4.0, 3.5.0b1
implementation_name	sys.implementation.name	cpython
implementation_version	see definition below	3.4.0, 3.5.0b1
extra	An error except when defined by the context interpreting the specification.	test