Rename schema job to be more clear about its purpose

The make target fails when it detects schema changes, not when schema is invalid.

Also update CONTRIBUTING.md to include information about zuul jobs.

3 files changed, 21 insertions, 4 deletions

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 29d4e4d7c4..bae19f30d9 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -331,6 +331,23 @@ Sometimes it might take us a while to fully review your PR. We try to keep the `
 
 All submitted PRs will have the linter and unit tests run against them via Zuul, and the status reported in the PR.
 
+## PR Checks ran by Zuul
+Zuul jobs for awx are defined in the [zuul-jobs](https://github.com/ansible/zuul-jobs) repo.
+
+Zuul runs the following checks that must pass:
+1) `tox-awx-api-lint`
+2) `tox-awx-ui-lint`
+3) `tox-awx-api`
+4) `tox-awx-ui`
+5) `tox-awx-swagger`
+
+Zuul runs the following checks that are non-voting (can not pass but serve to inform PR reviewers):
+1) `tox-awx-detect-schema-change`
+    This check generates the schema and diffs it against a reference copy of the `devel` version of the schema.
+    Reviewers should inspect the `job-output.txt.gz` related to the check if their is a failure (grep for `diff -u -b` to find beginning of diff).
+    If the schema change is expected and makes sense in relation to the changes made by the PR, then you are good to go!
+    If not, the schema changes should be fixed, but this decision must be enforced by reviewers.
+
 ## Reporting Issues
 
 We welcome your feedback, and encourage you to file an issue when you run into a problem. But before opening a new issues, we ask that you please view our [Issues guide](./ISSUES.md).
diff --git a/Makefile b/Makefile
index d979141ec8..7e8a919425 100644
--- a/Makefile
+++ b/Makefile
@@ -575,7 +575,7 @@ docker-compose-genschema:
 	cd tools && CURRENT_UID=$(shell id -u) TAG=$(COMPOSE_TAG) DEV_DOCKER_TAG_BASE=$(DEV_DOCKER_TAG_BASE) docker-compose run --rm --service-ports awx /start_tests.sh genschema
 	mv swagger.json schema.json
 
-docker-compose-validate-schema:
+docker-compose-detect-schema-change:
 	$(MAKE) docker-compose-genschema
 	curl https://s3.amazonaws.com/awx-public-ci-files/schema.json -o reference-schema.json
 	# Ignore differences in whitespace with -b
diff --git a/tox.ini b/tox.ini
index adbf79d225..55e1427f82 100644
--- a/tox.ini
+++ b/tox.ini
@@ -6,7 +6,7 @@ envlist =
     api,
     ui,
     swagger,
-    validate-schema,
+    detect-schema-change,
 
 [testenv]
 ;basepython = python2.7
@@ -73,9 +73,9 @@ commands =
     make docker-compose-build
     make docker-compose-build-swagger
 
-[testenv:validate-schema]
+[testenv:detect-schema-change]
 deps =
     nodeenv
 commands =
     make docker-compose-build
-    make docker-compose-validate-schema
+    make docker-compose-detect-schema-change