1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
|
# Dependency Management
The `requirements.txt` file is generated from `requirements.in`, using `pip-tools` `pip-compile`.
## How To Use
Commands should be run from inside the `./requirements` directory of the awx repository.
### Upgrading or Adding Select Libraries
If you need to add or upgrade one targeted library, then modify `requirements.in`,
then run the script:
`./updater.sh`
NOTE: `./updater.sh` uses /usr/bin/python3.6, to match the current python version
(3.6) used to build releases.
##### Note - watch out for the updater script, using paths local to your machine instead of generalized paths; ie
```bash
# via -r /awx_devel/requirements/requirements.in <-RIGHT
# via -r /home/foo/bar/awx/requirements/requirements.in <-WRONG
```
#### Upgrading Unpinned Dependency
If you require a new version of a dependency that does not have a pinned version
for a fix or feature, pin a minimum version and run `./updater.sh`. For example,
replace the line `asgi-amqp` with `asgi-amqp>=1.1.4`, and consider leaving a
note.
Then next time that a general upgrade is performed, the minimum version specifiers
can be removed, because `*.txt` files are upgraded to latest.
### Upgrading Dependencies
You can upgrade (`pip-compile --upgrade`) the dependencies by running
`./updater.sh upgrade`.
## Licenses and Source Files
If any library has a change to its license with the upgrade, then the license for that library
inside of `docs/licenses` needs to be updated.
For libraries that have source distribution requirements (LGPL as an example),
a tarball of the library is kept along with the license.
To download the PyPI tarball, you can run this command:
```
pip download <pypi library name> -d docs/licenses/ --no-binary :all: --no-deps
```
Make sure to delete the old tarball if it is an upgrade.
## UPGRADE BLOCKERs
Anything pinned in `*.in` files involves additional manual work in
order to upgrade. Some information related to that work is outlined here.
### django
For any upgrade of Django, it must be confirmed that
we don't regress on FIPS support before merging.
See internal integration test knowledge base article `how_to_test_FIPS`
for instructions.
If operating in a FIPS environment, `hashlib.md5()` will raise a `ValueError`,
but will support the `usedforsecurity` keyword on RHEL and Centos systems.
Keep an eye on https://code.djangoproject.com/ticket/28401
The override of `names_digest` could easily be broken in a future version.
Check that the import remains the same in the desired version.
https://github.com/django/django/blob/af5ec222ccd24e81f9fec6c34836a4e503e7ccf7/django/db/backends/base/schema.py#L7
### social-auth-app-django
django-social keeps a list of backends in memory that it gathers
based on the value of `settings.AUTHENTICATION_BACKENDS` *at import time*:
https://github.com/python-social-auth/social-app-django/blob/c1e2795b00b753d58a81fa6a0261d8dae1d9c73d/social_django/utils.py#L13
Our `settings.AUTHENTICATION_BACKENDS` can *change*
dynamically as settings are changed (i.e., if somebody
configures Github OAuth2 integration), so we need to
_overwrite_ this in-memory value at the top of every request so
that we have the latest version
### django-oauth-toolkit
Version 1.2.0 of this project has a bug that error when revoking tokens.
This is fixed in the master branch but is not yet released.
When upgrading past 1.2.0 in the future, the `0025` migration needs to be
edited, just like the old migration was edited in the project:
https://github.com/jazzband/django-oauth-toolkit/commit/96538876d0d7ea0319ba5286f9bde842a906e1c5
The field can simply have the validator method `validate_uris` removed.
### azure-keyvault
Upgrading to 4.0.0 causes error because imports changed.
```
File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/credential_plugins/azure_kv.py", line 4, in <module>
from azure.keyvault import KeyVaultClient, KeyVaultAuthentication
ImportError: cannot import name 'KeyVaultClient'
```
### slackclient
Imports as used in `awx/main/notifications/slack_backend.py` changed
in version 2.0. This plugin code will need to change and be re-tested
as the upgrade takes place.
### django-jsonfield
Instead of calling a `loads()` operation, the returned value is casted into
a string in some cases, introduced in the change:
https://github.com/adamchainz/django-jsonfield/pull/14
This breaks a very large amount of AWX code that assumes these fields
are returned as dicts. Upgrading this library will require a refactor
to accomidate this change.
### pip and setuptools
The offline installer needs to have functionality confirmed before upgrading these.
Versions need to match the versions used in the pip bootstrapping step
in the top-level Makefile.
### cryptography
The offline installer needs to have functionality confirmed before upgrading these.
## Library Notes
### pexpect
Version 4.8 makes us a little bit nervous with changes to `searchwindowsize` https://github.com/pexpect/pexpect/pull/579/files
Pin to `pexpect==4.7.x` until we have more time to move to `4.8` and test.
|
