Improve PR check (#778)

* Simplify this script

Before it was using the jenkins-jobs internal API to get the list of
jobs. It also supported multiple ways of fetching the list of jobs to
compare it with.

All this complexity has been removed. Now the list of jobs is read from
STDIN and it always uses the CentOS CI Jenkins as the list of jobs to
compare it with.

* Improve the PR check

- USE_LOCAL to run the PR job in the same folder and NOT remove the temp
  files.
- Support overriding of the Jenkins Jobs path.
- Improved way to handle removing the temp files (using a trap).
- Make sure the admin list is in alphabetical order.
- Use the newly simplified jenkins-jobs-diff.py

* sort the admin list

* Remove alphabetical check since we will move to org auth

* Revert "sort the admin list"

This reverts commit 584a7c0.

* do not hardcode path

* typo

* cannot use quotes in this case

cico_pr_test.sh

@@ -1,24 +1,33 @@
 #!/usr/bin/bash
 
-set -x
+# - Set `USE_LOCAL=yes` to use current folder to generate old and new jobs. The
+#   old and new folder will not be removed if this is set.
+# - Set `JENKINS_JOBS=/path/to/jenkins-jobs` to specify the path to jenkins-jobs
 
-NEW_JOBS=$(mktemp -d)
-MASTER_JOBS=$(mktemp -d)
+set -exo pipefail
+
+if [ -z "$USE_LOCAL" ]; then
+    NEW_JOBS=$(mktemp -d)
+    MASTER_JOBS=$(mktemp -d)
+else
+    NEW_JOBS="new"
+    MASTER_JOBS="old"
+
+    rm -rf "$NEW_JOBS" "$MASTER_JOBS"
+fi
 
 JJB_INDEX="devtools-ci-index.yaml"
-JENKINS_JOBS=~/venv/env/bin/jenkins-jobs
+JENKINS_JOBS=${JENKINS_JOBS-~/venv/env/bin/jenkins-jobs}
 
 delete_tmp() {
-    rm -rf $NEW_JOBS $MASTER_JOBS
+    if [ -z "$USE_LOCAL" ]; then
+        set -x
+        rm -rf $NEW_JOBS $MASTER_JOBS
+    fi
 }
+trap delete_tmp EXIT
 
 $JENKINS_JOBS test --config-xml $JJB_INDEX -o $NEW_JOBS
-ret=$?
-
-if [ "$ret" != "0" ]; then
-    delete_tmp
-    exit $ret
-fi
 
 git show origin/master:$JJB_INDEX | $JENKINS_JOBS test --config-xml -o $MASTER_JOBS
 
@@ -29,12 +38,12 @@ echo "-------------------------------------------------------------------------"
 echo diff -uNr '$MASTER_JOBS' '$NEW_JOBS'
 echo diff -uNr $MASTER_JOBS $NEW_JOBS
 echo
-diff -uNr $MASTER_JOBS $NEW_JOBS
+diff -uNr $MASTER_JOBS $NEW_JOBS || :
 echo "-------------------------------------------------------------------------"
 echo diff -qr '$MASTER_JOBS' '$NEW_JOBS'
 echo diff -qr $MASTER_JOBS $NEW_JOBS
 echo
-diff -qr $MASTER_JOBS $NEW_JOBS
+diff -qr $MASTER_JOBS $NEW_JOBS || :
 echo "-------------------------------------------------------------------------"
 echo "Credential changes:"
 echo
@@ -43,14 +52,9 @@ grep -rh credentialsId $MASTER_JOBS|sed "$SED_EXPR"|sort -u > $MASTER_JOBS/old_c
 grep -rh credentialsId $NEW_JOBS|sed "$SED_EXPR"|sort -u > $NEW_JOBS/new_creds.txt
 diff -U0 $MASTER_JOBS/old_creds.txt $NEW_JOBS/new_creds.txt
 echo "-------------------------------------------------------------------------"
-
-delete_tmp
-
-# Check if job list is empty
-JOB_DIFF=$(scripts/jenkins-jobs-diff.py $JJB_INDEX 2>/dev/null)
-
-echo
 echo "JOB_DIFF:"
+echo
+JOB_DIFF=$(find "$NEW_JOBS" -type d -printf '%P\n' | scripts/jenkins-jobs-diff.py 2>/dev/null)
 echo "$JOB_DIFF"
 
 if echo "$JOB_DIFF" | grep -q 'removed_jobs'; then

scripts/jenkins-jobs-diff.py

@@ -1,24 +1,12 @@
 #!/usr/bin/env python
 
 """
-This script prints the difference between the jobs defined in devtools cico yaml
-(JJB_URL) and those defined in Jenkins (JENKINS_JOBS_URL).
+This script prints the difference between the jobs received by STDIN Jenkins
+(JENKINS_JOBS_URL).
 
 Usage:
 
-    ./jenkins-jobs-diff.py [new_index] [old_index]
-
-If new_index is not supplied, it will connect use the current version of the index:
-https://github.com/openshiftio/openshiftio-cico-jobs/blob/master/devtools-ci-index.yaml
-
-Note that the index be a URL (make sure it's RAW in case of GH) or a path.
-
-If old_index is not supplied, it will query jenkins for the list of defined
-jobs. If it is supplied, it can be a URL or a path.
-
-Requirements:
-
-    pip install jenkins-job-builder==1.6.1
+    echo job1 job2 ... | ./jenkins-jobs-diff.py
 """
 
 import ast
@@ -28,32 +16,10 @@
 import sys
 import os
 
-from jenkins_jobs.builder import Builder
 
 JENKINS_JOBS_URL = 'https://ci.centos.org/view/Devtools/api/python'
-JJB_URL = 'https://raw.githubusercontent.com/openshiftio/openshiftio-cico-jobs/master/devtools-ci-index.yaml'
 UNTRACKED_JOBS = set(['devtools-jjb-service'])
 
-def get_jjb_jobs(index_raw):
-    """
-    Returns an array with job names. +index_raw+ can be a URL or a path.
-    This method uses the actual jenkins_jobs builder package to process the jobs.
-    """
-
-    if index_raw.startswith('http'):
-        index_fp = urllib2.urlopen(index_raw)
-    else:
-        index_fp = open(index_raw, 'r')
-
-    builder = Builder("None", None, None, None, plugins_list={})
-
-    builder.load_files(index_fp)
-    builder.parser.expandYaml()
-    builder.parser.generateXML()
-
-    index_fp.close()
-
-    return [job.name for job in builder.parser.xml_jobs]
 
 def get_jenkins_jobs(url):
     """
@@ -63,23 +29,10 @@ def get_jenkins_jobs(url):
 
     return [job['name'] for job in jenkins_jobs['jobs']]
 
-def main():
-    old_index_yaml = None
-
-    if len(sys.argv) == 2:
-        new_index_yaml = sys.argv[1]
-    elif len(sys.argv) == 3:
-        new_index_yaml = sys.argv[1]
-        old_index_yaml = sys.argv[2]
-    else:
-        new_index_yaml = JJB_URL
-
-    new_jobs_list = get_jjb_jobs(new_index_yaml)
 
-    if old_index_yaml:
-        old_jobs_list = get_jjb_jobs(old_index_yaml)
-    else:
-        old_jobs_list = get_jenkins_jobs(JENKINS_JOBS_URL)
+def main():
+    new_jobs_list = sys.stdin.read().split()
+    old_jobs_list = get_jenkins_jobs(JENKINS_JOBS_URL)
 
     new_jobs = set(new_jobs_list) - set(old_jobs_list) - UNTRACKED_JOBS
     removed_jobs = set(old_jobs_list) - set(new_jobs_list) - UNTRACKED_JOBS
@@ -95,5 +48,6 @@ def main():
     else:
         print yaml.dump(diff, default_flow_style=False)
 
+
 if __name__ == '__main__':
     main()