Test Reporter Improvements (#1)

* Recursively Get Suites and Tests * Processing Tests not in a Suite * Include Suite Name, failed, skipped, and time * Following Linting Rules * Fix error type issue * Set fail-on-error to false * Add sample tests
phoenix-actions · Feb 21, 2022 · a1708a9 · a1708a9
1 parent a3560da
commit a1708a9
Show file tree

Hide file tree

Showing 14 changed files with 1,045 additions and 79 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -42,3 +42,4 @@ jobs:
           name: Mochawesome Tests
           path: __tests__/fixtures/mochawesome-json.json
           reporter: mochawesome-json
+          fail-on-error: false
diff --git a/.github/workflows/dirty-laundry.yml b/.github/workflows/dirty-laundry.yml
@@ -0,0 +1,27 @@
+name: 'Dirty Laundry Test'
+on:
+  pull_request:
+    paths-ignore: [ '**.md' ]
+  push:
+    paths-ignore: [ '**.md' ]
+  workflow_dispatch:
+
+jobs:
+  build-test:
+    name: Build & Test
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - run: npm ci
+      - run: npm run build
+      - run: npm run format-check
+      - run: npm run lint
+
+      - name: Create mochawesome report
+        uses: ./
+        if: success() || failure()
+        with:
+          name: Mochawesome Tests
+          path: __tests__/fixtures/mochawesome1-json.json
+          reporter: mochawesome-json
+          fail-on-error: false
diff --git a/README.md b/README.md
@@ -1,4 +1,4 @@
-# Test Reporter
+# Test Reporting
 
 This [Github Action](https://github.com/features/actions) displays test results from popular testing frameworks directly in GitHub.
 

diff --git a/__tests__/__outputs__/mochawesome-json.md b/__tests__/__outputs__/mochawesome-json.md
@@ -1,11 +1,28 @@
-![Tests passed successfully](https://img.shields.io/badge/tests-1%20passed-success)
-## ✔️ <a id="user-content-r0" href="#r0">fixtures/mochawesome-json.json</a>
-**1** tests were completed in **14ms** with **1** passed, **0** failed and **0** skipped.
+![Tests failed](https://img.shields.io/badge/tests-1%20passed%2C%204%20failed-critical)
+## ❌ <a id="user-content-r0" href="#r0">fixtures/mochawesome-json.json</a>
+**5** tests were completed in **14ms** with **1** passed, **4** failed and **0** skipped.
 |Test suite|Passed|Failed|Skipped|Time|
 |:---|---:|---:|---:|---:|
-|[](#r0s0)|1✔️|||0ms|
-### ✔️ <a id="user-content-r0s0" href="#r0s0"></a>
+|[nofile](#r0s0)||1❌||3ms|
+|[test/main.test.js](#r0s1)|1✔️|3❌||2ms|
+### ❌ <a id="user-content-r0s0" href="#r0s0">nofile</a>
+```
+❌ Timeout test
+	Error: Timeout of 1ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/Users/work/Source/Repos/thirdparty/phoenix-test-reporter/reports/mochawesome/test/second.test.js)
+```
+### ❌ <a id="user-content-r0s1" href="#r0s1">test/main.test.js</a>
 ```
 Test 1
   ✔️ Passing test
+Test 1 Test 1.1
+  ❌ Exception in target unit
+	Error: Some error
+  ❌ Failing test
+	AssertionError: Expected values to be strictly equal:
+	
+	false !== true
+	
+Test 2
+  ❌ Exception in test
+	Error: Some error
 ```
diff --git a/__tests__/__snapshots__/mochawesome-json.test.ts.snap b/__tests__/__snapshots__/mochawesome-json.test.ts.snap
@@ -4,6 +4,30 @@ exports[`mochawesome-json tests report from ./reports/mochawesome-json test resu
 TestRunResult {
   "path": "fixtures/mochawesome-json.json",
   "suites": Array [
+    TestSuiteResult {
+      "groups": Array [
+        TestGroupResult {
+          "name": null,
+          "tests": Array [
+            TestCaseResult {
+              "error": Object {
+                "details": "Error: Timeout of 1ms exceeded. For async tests and hooks, ensure \\"done()\\" is called; if returning a Promise, ensure it resolves. (/Users/work/Source/Repos/thirdparty/phoenix-test-reporter/reports/mochawesome/test/second.test.js)
+    at listOnTimeout (node:internal/timers:557:17)
+    at processTimers (node:internal/timers:500:7)",
+                "line": undefined,
+                "message": "Error: Timeout of 1ms exceeded. For async tests and hooks, ensure \\"done()\\" is called; if returning a Promise, ensure it resolves. (/Users/work/Source/Repos/thirdparty/phoenix-test-reporter/reports/mochawesome/test/second.test.js)",
+                "path": undefined,
+              },
+              "name": "Timeout test",
+              "result": "failed",
+              "time": 3,
+            },
+          ],
+        },
+      ],
+      "name": "nofile",
+      "totalTime": undefined,
+    },
     TestSuiteResult {
       "groups": Array [
         TestGroupResult {
@@ -17,8 +41,64 @@ TestRunResult {
             },
           ],
         },
+        TestGroupResult {
+          "name": "Test 1 Test 1.1",
+          "tests": Array [
+            TestCaseResult {
+              "error": Object {
+                "details": "Error: Some error
+    at Object.throwError (lib/main.js:2:9)
+    at Context.<anonymous> (test/main.test.js:15:11)
+    at processImmediate (node:internal/timers:464:21)",
+                "line": 2,
+                "message": "Error: Some error",
+                "path": "lib/main.js",
+              },
+              "name": "Exception in target unit",
+              "result": "failed",
+              "time": 0,
+            },
+            TestCaseResult {
+              "error": Object {
+                "details": "AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+
+false !== true
+
+    at Context.<anonymous> (test/main.test.js:11:14)
+    at processImmediate (node:internal/timers:464:21)",
+                "line": 11,
+                "message": "AssertionError: Expected values to be strictly equal:
+
+false !== true
+",
+                "path": "test/main.test.js",
+              },
+              "name": "Failing test",
+              "result": "failed",
+              "time": 2,
+            },
+          ],
+        },
+        TestGroupResult {
+          "name": "Test 2",
+          "tests": Array [
+            TestCaseResult {
+              "error": Object {
+                "details": "Error: Some error
+    at Context.<anonymous> (test/main.test.js:22:11)
+    at processImmediate (node:internal/timers:464:21)",
+                "line": 22,
+                "message": "Error: Some error",
+                "path": "test/main.test.js",
+              },
+              "name": "Exception in test",
+              "result": "failed",
+              "time": 0,
+            },
+          ],
+        },
       ],
-      "name": "",
+      "name": "test/main.test.js",
       "totalTime": undefined,
     },
   ],

diff --git a/__tests__/fixtures/mocha-json.json b/__tests__/fixtures/mocha-json.json
@@ -155,4 +155,4 @@
       "err": {}
     }
   ]
-}
+}
diff --git a/__tests__/fixtures/mochawesome-json.json b/__tests__/fixtures/mochawesome-json.json
@@ -19,9 +19,9 @@
   "results": [
     {
       "uuid": "e74145e8-4f10-432c-8fe1-79fab8a0fe12",
-      "title": "",
-      "fullFile": "",
-      "file": "",
+      "title": "In-line test with no file",
+      "fullFile": "nofile",
+      "file": "nofile",
       "beforeHooks": [],
       "afterHooks": [],
       "tests": [
@@ -150,10 +150,7 @@
               ],
               "suites": [],
               "passes": [],
-              "failures": [
-                "e154e320-a501-49bd-b0dc-fcdf8fb6460a",
-                "bee0d977-da09-43a9-9683-6e2676952f60"
-              ],
+              "failures": ["e154e320-a501-49bd-b0dc-fcdf8fb6460a", "bee0d977-da09-43a9-9683-6e2676952f60"],
               "pending": [],
               "skipped": [],
               "duration": 2,
@@ -162,9 +159,7 @@
               "_timeout": 2000
             }
           ],
-          "passes": [
-            "3ebb7382-adde-438c-9d31-694714656ac3"
-          ],
+          "passes": ["3ebb7382-adde-438c-9d31-694714656ac3"],
           "failures": [],
           "pending": [],
           "skipped": [],
@@ -206,9 +201,7 @@
           ],
           "suites": [],
           "passes": [],
-          "failures": [
-            "add99db1-a040-4ab8-86cf-91051665fb47"
-          ],
+          "failures": ["add99db1-a040-4ab8-86cf-91051665fb47"],
           "pending": [],
           "skipped": [],
           "duration": 0,
@@ -218,12 +211,8 @@
         }
       ],
       "passes": [],
-      "failures": [
-        "3ad24933-108e-426f-9cc3-0d71ab9f35c2"
-      ],
-      "pending": [
-        "f1b2a440-4080-44da-9346-bb5df85581e0"
-      ],
+      "failures": ["3ad24933-108e-426f-9cc3-0d71ab9f35c2"],
+      "pending": ["f1b2a440-4080-44da-9346-bb5df85581e0"],
       "skipped": [],
       "duration": 3,
       "root": true,
@@ -255,4 +244,4 @@
       "version": "6.0.1"
     }
   }
-}
+}
-Original file line number
+Diff line change
@@ Expand Up / @@ -155,4 +155,4 @@ @@
           "err": {}
         }
       ]
-    }
+    }