1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
|
# Dependency Management
The `requirements.txt` file is generated from `requirements.in` and `requirements_git.txt`, using `pip-tools` and `pip-compile`.
## How To Use
Commands should be run in the awx container from inside the `./requirements` directory of the awx repository.
### Upgrading or Adding Select Libraries
If you need to add or upgrade one targeted library, then modify `requirements.in`,
then run the script:
`./updater.sh run`
#### Upgrading Unpinned Dependency
If you require a new version of a dependency that does not have a pinned version
for a fix or feature, pin a minimum version in `requirements.in` and run `./updater.sh run`. For example,
replace the line `asgi-amqp` with `asgi-amqp>=1.1.4`, and consider leaving a
note.
Then next time that a general upgrade is performed, the minimum version specifiers
can be removed, because `*.txt` files are upgraded to latest.
### Upgrading Dependencies
You can upgrade (`pip-compile --upgrade`) the dependencies by running
`./updater.sh upgrade`.
## Licenses and Source Files
If any library has a change to its license with the upgrade, then the license for that library
inside of `licenses` needs to be updated.
For libraries that have source distribution requirements (LGPL as an example),
a tarball of the library is kept along with the license.
To download the PyPI tarball, you can run this command:
```
pip download <pypi library name> -d licenses/ --no-binary :all: --no-deps
```
Make sure to delete the old tarball if it is an upgrade.
## UPGRADE BLOCKERs
Anything pinned in `*.in` files involves additional manual work in
order to upgrade. Some information related to that work is outlined here.
### Django
For any upgrade of Django, it must be confirmed that
we don't regress on FIPS support before merging.
See internal integration test knowledge base article `how_to_test_FIPS`
for instructions.
If operating in a FIPS environment, `hashlib.md5()` will raise a `ValueError`,
but will support the `usedforsecurity` keyword on RHEL and Centos systems.
Keep an eye on https://code.djangoproject.com/ticket/28401
The override of `names_digest` could easily be broken in a future version.
Check that the import remains the same in the desired version.
https://github.com/django/django/blob/af5ec222ccd24e81f9fec6c34836a4e503e7ccf7/django/db/backends/base/schema.py#L7
### social-auth-app-django
django-social keeps a list of backends in memory that it gathers
based on the value of `settings.AUTHENTICATION_BACKENDS` *at import time*:
https://github.com/python-social-auth/social-app-django/blob/c1e2795b00b753d58a81fa6a0261d8dae1d9c73d/social_django/utils.py#L13
Our `settings.AUTHENTICATION_BACKENDS` can *change*
dynamically as settings are changed (i.e., if somebody
configures Github OAuth2 integration), so we need to
_overwrite_ this in-memory value at the top of every request so
that we have the latest version
### django-oauth-toolkit
Versions later than 1.4.1 throw an error about id_token_id, due to the
OpenID Connect work that was done in
https://github.com/jazzband/django-oauth-toolkit/pull/915. This may
be fixable by creating a migration on our end?
### azure-keyvault
Upgrading to 4.0.0 causes error because imports changed.
```
File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/credential_plugins/azure_kv.py", line 4, in <module>
from azure.keyvault import KeyVaultClient, KeyVaultAuthentication
ImportError: cannot import name 'KeyVaultClient'
```
### pip, setuptools and setuptools_scm
The offline installer needs to have functionality confirmed before upgrading these.
Versions need to match the versions used in the pip bootstrapping step
in the top-level Makefile.
### cryptography
The offline installer needs to have functionality confirmed before upgrading these.
## Library Notes
### pexpect
Version 4.8 makes us a little bit nervous with changes to `searchwindowsize` https://github.com/pexpect/pexpect/pull/579/files
Pin to `pexpect==4.7.x` until we have more time to move to `4.8` and test.
|
