Skip to content

Refactor the custom reporters for testing#20828

Merged
timvandermeij merged 3 commits intomozilla:masterfrom
timvandermeij:refactor-reporter
Mar 10, 2026
Merged

Refactor the custom reporters for testing#20828
timvandermeij merged 3 commits intomozilla:masterfrom
timvandermeij:refactor-reporter

Conversation

@timvandermeij
Copy link
Contributor

The commit messages contain more details about the individual changes.

Related to #11851.

@codecov-commenter
Copy link

codecov-commenter commented Mar 8, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.62%. Comparing base (44a6354) to head (5fb9f12).
⚠️ Report is 6 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #20828      +/-   ##
==========================================
- Coverage   62.65%   62.62%   -0.03%     
==========================================
  Files         169      169              
  Lines      120729   120729              
==========================================
- Hits        75644    75612      -32     
- Misses      45085    45117      +32     
Flag Coverage Δ
fonttest ?
unittestcli 62.62% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@timvandermeij
Copy link
Contributor Author

/botio test

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)


Received

Command cmd_test from @timvandermeij received. Current queue size: 0

Live output at: http://54.241.84.105:8877/f733b4a99f88ef8/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)


Received

Command cmd_test from @timvandermeij received. Current queue size: 1

Live output at: http://54.193.163.58:8877/5f06a06fc51a971/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)


Failed

Full output at http://54.241.84.105:8877/f733b4a99f88ef8/output.txt

Total script time: 43.62 mins

  • Unit tests: Passed
  • Integration Tests: FAILED
  • Regression tests: FAILED
  different ref/snapshot: 1

Image differences available at: http://54.241.84.105:8877/f733b4a99f88ef8/reftest-analyzer.html#web=eq.log

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)


Failed

Full output at http://54.193.163.58:8877/5f06a06fc51a971/output.txt

Total script time: 72.28 mins

  • Unit tests: Passed
  • Integration Tests: FAILED
  • Regression tests: FAILED
  different ref/snapshot: 1

Image differences available at: http://54.193.163.58:8877/5f06a06fc51a971/reftest-analyzer.html#web=eq.log

This makes the order of checks consistent with the one in
`test/reporter.js` and improves safety because now any status other
than passed will be treated as a failure (also if Jasmine adds more
statuses later on).
…rters

This improves readability of the code and makes it consistent with the
recently added check for excluded tests.
Similar to excluded tests pending tests should not count towards runs or
result in console logging because they are effectively not (fully) run.
This reduces visual noise and helps to get the tests running on GitHub
Actions where non-passed tests will count towards a non-zero exit code.
Copy link
Contributor

@calixteman calixteman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you.

@timvandermeij timvandermeij merged commit 6541903 into mozilla:master Mar 10, 2026
10 checks passed
@timvandermeij timvandermeij deleted the refactor-reporter branch March 10, 2026 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants