Skip to content

Project 3: Mutation Testing

This assignment is a focused, hands-on mini‑project that extends your practice from Exercise 3 and contributes to your project grade component.

You and your team will apply mutation testing on a real codebase to evaluate and improve the effectiveness of your test suite.

Codebase Requirements

Choose a codebase that:

  • Has a runnable unit test suite
  • Has a mutation testing tool compatible with the language and build tool used.
  • Contains non‑trivial classes with business logic (conditionals/loops/edge cases).
  • Is hosted on GitLab CSUI or mirrored there, with CI able to run tests.

You may reuse your team’s codebase from Project 1 or Project 2, provided it satisfies the above criteria.

Tasks

Complete the following tasks as a team. Where indicated, some parts are per‑member.

  1. Repository setup
  2. Fork or mirror the chosen repository into your team’s namespace on GitLab CSUI.
  3. Ensure all team members can build and run tests locally.
  4. Optional but recommended: set up CI to run unit tests on MRs and main.
  5. Configure mutation testing tool (once per repo)
  6. Add mutation testing tool into the codebase and configure it.
  7. Verify unit tests pass before running the mutation tests.
  8. Run mutation tests locally and generate a report from the tool (usually in HTML format or capture the plaintext via standard output stream)
  9. Per‑member analysis of killed mutants (individual)
  10. Each team member selects at least one killed mutant from the report.
  11. Identify and record the mutation operator used by mutation testing tool.
  12. Explain why the mutant is killed by the existing tests.
  13. Per‑member analysis and fix for a surviving mutant (individual)
  14. Each team member selects at least one surviving mutant.
  15. Explain why the current test suite did not kill the mutant (e.g., missing boundary case, insufficient assertions, path not exercised).
  16. Add or update a focused unit test to kill the mutant.
  17. Re‑run the mutation testing tool to confirm the target mutant is now killed. Capture before/after status.
  18. Team‑level consolidation
  19. Make sure the CI pipeline still pass after all changes.
  20. Create a group report that aggregates each member’s work (see Deliverables).
  21. Ensure all new/updated tests are committed and the project builds cleanly.

Deliverables

At the end of this project, you are required to prepare the following artefacts in your team's repository:

  • Configured a mutation testing tool in the codebase.
  • Updated unit tests that kill the selected surviving mutants.
  • A Markdown report named PROJECT_3_REPORT.md at the repository root containing, for each team member:
  • Member name and NPM.
  • Killed mutant: operator, location (class#method:line), why it was killed, and the relevant test(s) involved.
  • Surviving mutant (before fix): operator, location, why it survived (testing gap), and your hypothesis.
  • Test improvement: description of the new/updated test (path/inputs/assertions) and the result after re‑running the mutation testing tool (status changed to KILLED).
  • A short team summary (1–2 paragraphs) describing overall mutation score before/after (if applicable), notable patterns in surviving mutants, and lessons learned.

The due date of this project: Friday, 10 October 2025, 23:55 UTC+7. Submit the URL of your fork repository to the designated submission slot on SCELE. Only a representative from the group is required to submit the URL. Please ensure any updates to the fork repository related to this project were made and pushed before the due date.

Report Structure Template

Use the following structure in PROJECT_3_REPORT.md:

  • Team info: members and repository URL
  • How to run: commands and environment to reproduce mutation testing tool runs
  • Per‑member sections (repeat for each member):
  • Killed Mutant
    • Operator:
    • Location: <Class#method:line>
    • Why killed:
    • Reference test(s): <TestClass#testMethod>
  • Surviving Mutant (Before Fix)
    • Operator:
    • Location: <Class#method:line>
    • Why survived:
  • Test Improvement (After Fix)
    • What changed:
    • Result:

Generative AI Usage Policy

You may use generative AI tools for this mini‑project, but you must understand that you are trying to learn and practice the subject, not the AI itself. Generative AI may produce incorrect results; review outputs critically.

If you do use generative AI, you must agree on the following constraints:

Allowed Uses

  • Code generation: suggesting unit test scaffolds or refactorings.
  • Explanations: clarifying mutation testing tool configuration and mutation operators.
  • Debugging help: interpreting failing tests.

For every AI‑assisted change, document it in the written report under a section called AI Assistance Log.

Prohibited Uses

  • Direct submission of AI‑generated content without understanding or validating it.
  • Bypassing learning by outsourcing the entire analysis or report to AI.

Generative AI Use Disclosure

GitHub Copilot with GPT‑5 is used to draft and proofread this document.

Last update: 2025-10-02 02:03:52
Created: 2025-10-02 02:03:52