Skip to content

Exercise 1: AI-Assisted Software Quality Assurance Activities

In this exercise, you will practice setting up automated static analysis, security scanning in CI/CD, and modern AI-assisted Software Quality Assurance (SQA) workflows using the Model Context Protocol (MCP). The tasks include configuring a new SonarQube project on SonarQube CSUI, integrating built-in GitLab CI security analysers, configuring an official SonarQube MCP server in your local development environment, and comparing a refactoring experiment guided by automated test verification.

The open-source project used in this exercise is a fork of Spring Petclinic REST. The project's codebase provides both production code (i.e., the application source code) and test code (i.e., unit and integration tests). Currently, the test suite covers approximately 85% of the production code when run locally using Maven.

Getting Started

Complete the following steps before starting the exercise:

  1. Ensure you are enrolled in the SQA (PMPL) course and have an active account on GitLab CSUI. If you do not have an account on GitLab CSUI or had been disabled, please notify the instructors immediately.
  2. Log in to SonarQube CSUI using your GitLab CSUI account.
  3. Ensure the required tools are installed on your local development machine:
    • Git
    • Java JDK version 17
    • A container runtime, either Docker or Podman (required to run the SonarQube MCP container image locally)
    • An AI agent or text editor/IDE that supports MCP clients (e.g., Visual Studio Code with GitHub Copilot, Antigravity, Codex)
  4. Open the project page on GitLab CSUI.
  5. Fork it to your own namespace on GitLab CSUI.
  6. Clone the codebase onto your local development machine:
1
2
git clone git@gitlab.cs.ui.ac.id:<YOUR_USERNAME>/<YOUR_FORK>.git <path-to-local-workspace>
cd <path-to-local-workspace>
  1. Confirm that the default branch is csui:
1
2
3
# git status
On branch csui
# ... (omitted for brevity)
  1. Run the test suite locally and generate the JaCoCo coverage report:
1
./mvnw test jacoco:report

If everything goes well, all tests should pass and the HTML coverage report will be located at target/site/jacoco/index.html. Open it in a web browser and confirm the initial baseline coverage (~85%).

Task 1: Configure SonarQube

Even though the original, upstream repository has a SonarQube configuration, your cloned repository needs its own dedicated project on SonarQube CSUI to track analyses independently.

Follow these steps to set up your SonarQube project on SonarQube CSUI:

  1. Open the Projects page on SonarQube CSUI. Click Create Project and choose Local project.

  2. Fill in the required fields:

    • Project display name: spring-petclinic-<your-name>
    • Project key: spring-petclinic-<your-name>
    • Main branch name: csui

    See the provided screenshot to see an example using burhan as the placeholder name. alt 3. On the next screen, select Use the global setting and click Create project. 4. In the setup wizard under Analysis Method, select With GitLab CI and generate a new SONAR_TOKEN. Save this token securely. You will need it for both GitLab CI and your local MCP configuration later in the corresponding task. 5. In your forked repository on GitLab CSUI, navigate to Settings > CI/CD > Variables. Add a new masked variable named SONAR_TOKEN with the generated token value. 6. In your local repository, create a new branch:

1
git switch -c task/1-setup-sonarqube
  1. Open pom.xml, locate the <properties> block, and replace sonar.projectKey with your own project key (spring-petclinic-<your-name>).
  2. Commit and push the changes:
1
2
3
git add pom.xml
git commit -m "Configure SonarQube project key"
git push -u origin task/1-setup-sonarqube
  1. Create a Merge Request (MR) targeting the csui branch of your own fork repository (not the upstream class repository).
  2. Ensure the pipeline passes, merge the branch into csui, and verify that the sonarqube-check job reports analysis results on SonarQube CSUI.

Task 2: Explore GitLab CI Job Templates

To keep stay within the GitLab platform, GitLab provides several built-in CI job templates that can be included into a project's CI/CD configuration. By doing so, we can minimise the tools need to be maintained and also focus on utilising the built-in features on GitLab.

Note: GitLab does offer a code quality analysis using CodeClimate. However, it is being deprecated. Hence the emphasis on using SonarQube.

As part of the exercise, you need to modify the existing CI/CD pipeline configuration of the forked project to include the following CI job:

1
2
git switch csui
git pull origin csui

  1. Create a new branch : 

    1
    git switch -c task/2-update-gitlab-ci

  2. Open .gitlab-ci.yml. Notice the TODO comments indicating where security scanning templates should be included.

  3. Add the SAST job template using GitLab's include syntax according to the GitLab SAST documentation.
  4. Enable Advanced SAST by adding the environment variable GITLAB_ADVANCED_SAST_ENABLED: "true" either in .gitlab-ci.yml or under Settings > CI/CD > Variables on GitLab.
  5. Add the Dependency Scanning template using the include syntax according to the GitLab Dependency Scanning documentation.
  6. Commit, push, and create a Merge Request to csui:
1
2
3
git add .gitlab-ci.yml
git commit -m "Enable SAST and Dependency Scanning templates in GitLab CI"
git push -u origin task/2-update-gitlab-ci
  1. Verify that the SAST and Dependency Scanning jobs execute in the MR pipeline, then merge the branch into csui.

At initial configuration for the first time, both SAST and Dependency Scanning jobs may not produce interesting findings because they have not established the baseline based on the current state of the main (csui) branch. In the subsequent tasks, both jobs should be able to provide findings by comparing analysis results in the new MR pipelines with the existing baseline.

Task 3: Configure SonarQube MCP Server

Large Language Models (LLMs) used in AI assistants/agents lack direct access to external static analysis platforms unless equipped with standardized tools. The Model Context Protocol (MCP) provides an open standard for AI clients to communicate with developer toolchains, including SonarQube.

SonarSource provides an official MCP server container image (docker.io/sonarsource/sonarqube-mcp) that exposes SonarQube APIs, such as issue searches, rule descriptions, code metrics, and quality gate evaluations. These exposed APIs are made available as function signatures and inserted to the context window of the LLM when being operated by an AI assistant. Hence, the AI assistant will have the knowledge of functions that can be invoked to complete the user's prompt.

Follow these steps to configure the SonarQube MCP server in your local development environment:

  1. Ensure your container engine (Docker or Podman) is running.
  2. Open the official SonarQube MCP Configuration Generator.
  3. Use the following parameters in the generator:
    • Deployment / Instance Type: Self-managed / SonarQube Server
    • SonarQube Server URL: https://sonarqube.cs.ui.ac.id
    • Authentication Token: Your SONAR_TOKEN generated in Task 1
    • Client: Choose your AI assistant (e.g., GitHub Copilot in VS Code, Antigravity, Codex, Claude Code)
  4. Copy the generated configuration snippet into your client's MCP configuration file (e.g., mcp_config.json, claude_desktop_config.json, or your IDE's MCP settings).
  5. Verify the MCP connection: Prompt your AI assistant to verify MCP server connectivity to SonarQube:
1
2
Use the SonarQube MCP server to inspect the project named 'spring-petclinic-<your-name>'.
What is the current Quality Gate status and how many open issues are reported?

Confirm that your AI assistant/agent successfully invokes the MCP functions (or sometimes called as tools), such as get_project_quality_gate_status or search_sonar_issues_in_projects, and returns real analysis data from SonarQube CSUI.

Task 4: AI-Assisted SQA & Comparative Refactoring

One of main responsibilities of a QA Engineer is automating repetitive works pertaining to testing and quality improvement, along with maintaining correctness of the software. While Generative AI (GenAI) can propose and apply code refactoring quickly, it is also non-deterministic and prone to regressions or hallucinations. In this task, you will run a controlled 4-step experiment demonstrating how automated tests serve as a guardrail when orchestrating AI-driven refactoring.

Create a new working branch from csui:

1
2
3
git switch csui
git pull origin csui
git switch -c task/3-ai-assisted-refactoring

Step 4.1: Manual Identification & Triage

  1. Review the JaCoCo coverage report (target/site/jacoco/index.html) generated during the Getting Started phase.
  2. Review the list of open issues in your project on SonarQube CSUI or through your AI assistant via MCP.
  3. Identify a Java class that meets two criteria:
    • It has existing automated test coverage (preferably >75% line or branch coverage).
    • It contains multiple reported SonarQube issues or code smells (e.g., classes under org.springframework.samples.petclinic.service, org.springframework.samples.petclinic.model, or org.springframework.samples.petclinic.rest.advice).

Step 4.2: Manual Refactoring Baseline

  1. Choose one specific code smell in the selected class (e.g., unoptimized regex syntax, use of legacy synchronized classes like StringBuffer, redundant parentheses, or raw types).
  2. Manually refactor the code to resolve the smell according to SonarQube's recommendation.
  3. Run the test suite locally to verify the refactoring did not break existing behavior:
1
./mvnw test
  1. Once tests pass, commit your change:
1
2
git add <modified-file>
git commit -m "refactor: Manual fix for <rule-key> in <ClassName>"

Note: <rule-key> is the unique identifier of the violated SonarQube rule.

Step 4.3: Tool-Assisted AI Refactoring (Prompt Iteration 1)

Now, automate the remediation of a second code smell in the same class (or another covered class) using your AI assistant connected via the SonarQube MCP server.

  1. Write a prompt to your AI assistant instructing it to:
    • Look up open issues for your target class using the SonarQube MCP server.
    • Select one unresolved issue and refactor the code to fix it, mimicking the reasoning you applied manually in Step 4.2.
  2. Constraint for this step: Write a natural prompt describing the desired fix, but do not explicitly instruct the assistant to run the test suite or verify the build. Write the prompt as if you are talking naturally with your colleague.
  3. Record the following in your report:
  4. The exact prompt used
  5. The MCP tool calls made by the assistant
  6. The resulting code diff
  7. The LLM model used by the assistant, e.g., google/gemini-3.8-flash, gpt-5.6-sol
  8. Manually run ./mvnw test locally to evaluate whether the assistant's proposed fix passed the test suite on its first try. If tests fail, document the failure before rectifying it.

Step 4.4: Agentic Verification Loop (Prompt Iteration 2)

Refactor a third code smell, but this time apply prompt engineering to enforce an agentic verification loop.

  1. Reiterate on your prompt from Step 4.3 by adding strict operational constraints:
  2. The assistant must use SonarQube MCP to locate the issue and apply the refactoring.
  3. Mandatory verification constraint: The assistant must execute the project's test suite (e.g., ./mvnw test -Dtest=<TargetTest> or ./mvnw test) within its execution environment.
  4. Self-correction constraint: The assistant must inspect the test exit code and output logs. If any test fails or compilation errors occur, it must analyze the failure, adjust its code changes, and re-run the tests until the suite passes cleanly before concluding.
  5. Observe how the assistant behaves under this constrained prompt.
  6. Record the following in your report:
  7. The full prompt
  8. Tool invocation sequence (file edits, test commands, log inspection)
  9. Your evaluation

Task 5: Quality Gate & Full Verification

Ensure overall project quality meets the SonarQube's Quality Gate and then make it better:

  1. Maintain or increase the overall project code coverage to at least 90%.
  2. Ensure at least 5 code quality or security issues (including those resolved in Task 4) are remediated across the project.
  3. Run a complete local check:
1
./mvnw clean test jacoco:report
  1. Push task/3-ai-assisted-refactoring to GitLab CSUI and create a Merge Request into csui.
  2. Verify that all pipeline jobs, including sonarqube-check, sast, and dependency_scanning, succeed and that SonarQube reports a passing Quality Gate.

After you have finished this exercise, you can proceed to work on the first project to apply what you have learned in your own software project work.

GenAI Usage Policy

This exercise allows GenAI and LLM tools as learning materials and tools. Even so, it is still imperative for you as course participant of SQA and user of GenAI to factually check the responses generated by GenAI.

The use of GenAI does not mean you can entirely delegate your learning process to GenAI. You enrolled in a course to learn and obtain knowledge and skills. Do not make GenAI to be the one that does the learning, making you as a "meat proxy" for them.

Allowed Uses

  • Querying static analysis data and metrics via MCP servers.
  • Requesting refactoring proposals for identified code smells and rule violations.
  • Directing assistants to execute test suites and self-correct based on test feedback.
  • Asking for explanations of compiler errors, test assertions, or SonarQube rule rationales.

Mandatory Requirements

  • All AI-generated changes must be guarded and verified by automated tests.
  • You must critically review all diffs produced by an AI assistant before committing.

Deliverables

At the end of this exercise, you are required to prepare the following artefacts:

  • A forked repository of Spring Petclinic REST on GitLab CSUI with all task branches merged into csui.
  • Working CI/CD pipeline configuration (.gitlab-ci.yml) with SonarQube, SAST, and Dependency Scanning enabled.
  • An updated codebase demonstrating improved code quality, passing Quality Gates, and maintained/improved test coverage.
  • A report file named EXERCISE_1_REPORT.md committed at the root of your repository using the provided template.

The due date of this exercise is 8 November 2026, 23:55 UTC+7. Submit the URL of your repository to the designated submission slot on SCELE. Please make sure any updates to the repository related to this exercise were made and pushed before the due date.

Report Template

You can use the following Markdown template as a starting point for your report that goes into EXERCISE_1_REPORT.md:

Note: you can write your report in English or Bahasa Indonesia.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
# Exercise 1 Report: AI-Assisted SQA & Code Quality

- **Name:** <YOUR_NAME>
- **NPM:** <YOUR_NPM>
- **GitLab Repository URL:** <URL>
- **SonarQube Project Key:** <PROJECT_KEY>

## 1. Refactoring Log

> In tabular format. The entries in the following table are only example!

| Method | Target Issue / Rule | Refactoring Strategy | Test Verification Result |
| :--- | :--- | :--- | :--- |
| Manual (Step 4.2) | `<e.g., java:S1234>` | Replaced character class `[a-zA-Z]` with `\d`. | Passed `./mvnw test` on first run. |
| AI Prompt v1 (Step 4.3) | `<e.g., java:S5678>` | Assistant replaced `StringBuffer` with `String`. | `<Passed / Failed initial run>` |
| AI Agentic Loop (Step 4.4) | `<e.g., java:S9999>` | Agent removed redundant parameter parentheses and ran tests autonomously. | Passed after `<N>` test iteration(s). |

## 2. MCP Configuration & Tool Inspection

- **MCP Client Used:** `<e.g., VS Code / Antigravity / Codex / Claude>`
- **Observed MCP Tool Calls:**
  - `<e.g., search_sonar_issues_in_projects(projectKeys=[...])>`
  - `<e.g., get_component_measures(...)>`

## 3. Prompt Engineering & Agentic Evolution

### Prompt Iteration 1 (Without Explicit Test Verification)

- **Prompt Text:**
  > `<Paste your exact prompt from Step 4.3>`
- **Assistant Behavior & MCP Usage:** `<Describe what tools the assistant called and whether it checked the code.>`
- **Evaluation:** `<Did the code compile? Did existing tests pass without intervention?>`

### Prompt Iteration 2 (With Agentic Test Verification Loop)

- **Prompt Text:**
  > `<Paste your exact prompt from Step 4.4>`
- **Autonomous Actions Observed:** `<Describe the assistant's actions, e.g.,: file edit -> test command execution -> log inspection.>`
- **Evaluation & Self-Correction:** `<Did the assistant encounter test failures? Did it fix them autonomously?>`

## 4. Interesting/Relevant Findings (optional)

> Write down your interesting findings here, if any, that not necessarily related to the exercise but still relevant to the SQA as a whole.
Last update: 2026-10-29 11:37:49
Created: 2026-10-29 08:11:33