Background
Recently, I redesigned my team's pipeline running on a multi-module Maven monorepo using GitLab CI. It wasn't that the previous setup was broken, but my team faced a few persistent issues that I hoped to resolve to bring the pipeline into a more robust state. I was the one who did the initial setup as well, but that was done when I was still relatively new to GitLab CI. As I worked on this redesign, I learned quite a few lessons along the way that I hope to share with the "future me."
Objective
I set out to resolve several key pain points:
Double Pipelines: Preventing both branch and MR pipelines from triggering for the same commit.
Artifact Management: Resolving issues where build artifacts weren't reliably uploaded or shared.
Visibility: Fixing cases where test coverage (JaCoCo) wasn't being correctly captured.
Beyond just fixing issues, I also wanted to implement several strategic improvements:
Scalability: Creating a more robust setup using !reference tags and reusable job templates.
Security: Integrating automated SAST scans.
Performance: Reducing overall pipeline runtime through optimized cache strategies and surgical artifact sharing.
Read on to see the architectural patterns and specific pipeline designs that resolved these pain points and enabled us to scale effectively.
Pipeline design
Project Structure
This is a sample setup, but quite similar to one that I do have. In the actual setup, there's around 10 - 12 modules.
. (Root Aggregator)
├── .gitlab-ci.yml # Root Orchestrator
├── .gitlab-ci-base.yml # Central CI Blueprint
├── pom.xml # Root Aggregator POM
├── parent-pom/ # Shared configuration & dependency management
│ └── pom.xml
└── project/ # Functional module aggregator
├── pom.xml
├── mmm-security/ # Foundation module
│ ├── .gitlab-ci.yml # Child Pipeline
│ └── pom.xml
├── mmm-core/ # Core application module
│ ├── .gitlab-ci.yml # Child Pipeline
│ └── pom.xml
├── mmm-search/ # Search module
│ ├── .gitlab-ci.yml # Child Pipeline
│ └── pom.xml
└── mmm-report/ # Coverage reporter
└── pom.xml
Pipeline setup
Configure a parent-child pipeline with the following setup
Root Orchestrator (.gitlab-ci.yml)
Central Blueprint (.gitlab-ci-base.yml)
Independent Child Pipelines (.gitlab-ci.yml housed within each module)
Overview
Hands-On: The Interactive CI/CD Playbook
If you'd like to explore this architecture in a more visual and hands-on way, I've built an Interactive CI/CD Playbook. It allows you to dive into the project structure, explore how we manage artifact sharing across parent-child pipelines, and see the code snippets in action.
Explore the Playbook →
If you prefer to dig into the implementation details and the "why" behind these design decisions, read on—the following best practices break down the architecture component by component.
Best Practices
Root Orchestrator
This is the main controller that defines the workflow rules, the stages available, and when to trigger the downstream child pipelines.
workflow:
rules:
- if: $CI_FULL_PIPELINE == "true" # Manual override
- if: $CI_PIPELINE_SOURCE == "parent_pipeline"
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: \(CI_COMMIT_BRANCH && \)CI_OPEN_MERGE_REQUESTS # Double-pipeline suppression
when: never
- if: $CI_COMMIT_BRANCH # All branch pushes (including main)
CI_FULL_PIPELINE allows me to manually trigger full pipeline run
CI_COMMIT_BRANCH && CI_OPEN_MERGE_REQUESTS prevents Double Pipelines: a common issue where GitLab triggers both a branch pipeline and an MR pipeline for the same commit.
stages:
- configuration
- foundation
- application
- test # setup to run Gitlab built-in SAST scans
- report
pre:
stage: .pre
script:
- env
.pre is a built-in default stage that always run first. This is useful to see what are all the available environment variables available to the job for ease of troubleshooting.
While env is great for debugging, it should be used with caution (or removed before production) to avoid accidentally logging environment metadata, even though GitLab masks secrets by default.
.trigger-rules:
rules:
- if: "$CI_FULL_PIPELINE == 'true'"
when: always
# Merge Request: Accuracy check using compare_to
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
changes:
compare_to: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME
paths:
- $MODULE_PATH/**/*
- parent-pom/**/*
# Main Branch: Standard change detection
- if: \(CI_COMMIT_BRANCH == \)CI_DEFAULT_BRANCH && $CI_COMMIT_BEFORE_SHA != "0000000000000000000000000000000000000000"
changes:
- $MODULE_PATH/**/*
- parent-pom/**/*
trigger-core:
stage: application
extends: .trigger-rules
needs:
- job: trigger-security
optional: true
- job: deploy-parent-pom
optional: true
variables:
MODULE_PATH: "project/mmm-core"
PARENT_SOURCE: $CI_PIPELINE_SOURCE
PARENT_BRANCH: $CI_COMMIT_BRANCH
trigger:
include: project/mmm-core/.gitlab-ci.yml
strategy: mirror
.trigger-rules: Define reusable rules to determine when the job should be triggered. I used compare_to: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME to ensure change detection is calculated accurately against the target branch—eliminating the "rebase noise" that often plagues monorepo pipelines.
extends: .trigger-rules: By extending this template, it inherits the centralized change-detection logic. This ensures that only file changes under a specific module (or the parent-pom) trigger a pipeline run—preventing unnecessary resource waste.
needs: These are upstream jobs that this job depends on, and if there are changes to those modules, then this job has to wait until those are done before it can run. Having the optional: true is the secret sauce for monorepos: it prevents the pipeline from failing if an upstream module wasn't triggered due to lack of changes.
variables: This is especially important in parent-child pipelines as this allows us to pass context to the child pipeline in order to evaluate the rules correctly. When the child pipeline runs, the value for $CI_PIPELINE_SOURCE could be different from when it is run in the parent pipeline.
trigger:mirror: Ensures the parent pipeline reflects the downstream (child pipeline) status accurately. See docs for more detailed explanation.
This parent-child isolation means a failure in the "Search" module doesn't block the "Core" module's deployment—significantly reducing the "blast radius" of failures in a large monorepo.
Central Blueprint
This is where all the global variables, job-templates, stages, cache strategy, and reusable snippets are declared.
I treated our CI templates like "functions" with a defined interface using spec:inputs. This allows the Root Orchestrator to pass specific configurations (like forcing a full pipeline) without relying on fragile global variables.
# In .gitlab-ci-base.yml
spec:
inputs:
full_pipeline:
default: "false"
# Usage in .gitlab-ci.yml
include:
- local: '.gitlab-ci-base.yml'
inputs:
full_pipeline: $CI_FULL_PIPELINE
Cache Strategy
cache:
key: "maven-$CI_COMMIT_REF_SLUG"
paths:
- .m2/repository/
policy: pull-push
Using $CI_COMMIT_REF_SLUG is a standard strategy to define a shared branch-level cache, allowing modules to share internal dependencies. This ensures that once the first job has pulled the necessary dependencies, they are cached locally on the runner (or remotely, like in Minio in my setup). Subsequent jobs or pipelines in the same branch will reuse this cache to prevent redundant downloads, saving bandwidth, preventing race conditions, and shaving time off every run.
.deploy-snapshot-template:
extends: .base-maven-job
stage: release
interruptible: false # Ensure deployment finishes once started
cache:
key: "maven-$CI_COMMIT_REF_SLUG"
paths:
- .m2/repository/
policy: pull
rules:
- if: \(CI_COMMIT_BRANCH == \)CI_DEFAULT_BRANCH
script:
- |
echo "Deploying SNAPSHOT for $MODULE_PATH to GitLab Maven Registry..."
mvn \(MAVEN_CLI_OPTS deploy -pl \)MODULE_PATH -am -DskipTests
It is important to know when to override the cache policy so that it will not unnecessarily push the updated .m2/repository back to the remote cache if I am sure this job only uses the dependencies to do its job.
!reference tag
.coverage-parser:
script:
- |
echo "Extracting coverage percentage for GitLab UI..."
REPORT_PATH=\({JACOCO_XML_PATH:-"\)MODULE_PATH/target/site/jacoco/jacoco.xml"}
echo "DEBUG: Parsing JaCoCo report at: $REPORT_PATH"
if [ -f "$REPORT_PATH" ]; then
# Use grep -o to extract ONLY the matching tag, then tail -1 to get the aggregate
# This works correctly even if the entire XML is on a single line.
LINE_COUNTER=\((grep -o '<counter type="LINE"[^>]*/>' "\)REPORT_PATH" | tail -1 || true)
if [ -n "$LINE_COUNTER" ]; then
MISSED=\((echo "\)LINE_COUNTER" | sed -n 's/.*missed="\([0-9]*\)".*/\1/p')
COVERED=\((echo "\)LINE_COUNTER" | sed -n 's/.*covered="\([0-9]*\)".*/\1/p')
TOTAL=$((MISSED + COVERED))
echo "DEBUG: Found LINE counter: missed=\(MISSED, covered=\)COVERED, total=$TOTAL"
if [ "$TOTAL" -gt 0 ]; then
PERCENT=\((awk -v c="\)COVERED" -v t="$TOTAL" 'BEGIN {printf "%.2f", (c / t) * 100}')
echo "Coverage: $PERCENT%"
else
echo "Coverage: 0.00%"
fi
else
echo "DEBUG: No LINE counter line found in $REPORT_PATH"
fi
fi
# Template for application modules
.application-template:
extends: .base-maven-job
stage: build
variables:
JACOCO_XML_PATH: "$MODULE_PATH/target/site/jacoco/jacoco.xml"
script:
- mvn \(MAVEN_CLI_OPTS verify -pl \)MODULE_PATH -am
- !reference [.coverage-parser, script]
This is similar to YAML Anchors, which allows you to reuse snippets across jobs. While that is the case, I find !reference more developer-friendly, as it allows you to select specific keys (like script) to reuse.
Note that YAML Anchors are a native YAML feature and work outside of GitLab, while the !reference tag is a GitLab-specific feature.
Variables
variables:
# Performance: Use a project-relative path for caching
MAVEN_REPO_LOCAL: ".m2/repository"
SONAR_USER_HOME: ".sonar" # Defines the location of the analysis task cache
# JVM Tuning for CI (Merged with user preferences)
MAVEN_OPTS: >-
-Dhttps.protocols=TLSv1.2
-Dorg.slf4j.simpleLogger.showDateTime=true
-Djava.awt.headless=true
-Dfile.encoding=UTF-8
-Xmx2048m
-XX:+TieredCompilation
-XX:TieredStopAtLevel=1
# Maven CLI optimization (Merged with user preferences)
MAVEN_CLI_OPTS: >-
--batch-mode
--errors
--fail-at-end
--show-version
--no-transfer-progress
--threads 1C
-DinstallAtEnd=true
-DdeployAtEnd=true
-s .mvn/settings-ci.xml
GIT_DEPTH: "0" # Tells git to fetch all the branches of the project, required by the analysis task
# GitLab FastZip and Compression
FF_USE_FASTZIP: "true"
ARTIFACT_COMPRESSION_LEVEL: "fast"
CACHE_COMPRESSION_LEVEL: "fast"
I want to draw attention to MAVEN_CLI_OPTS where -s .mvn/settings-ci.xml is defined. This is the unsung hero of our pipeline: it maps ${env.CI_JOB_TOKEN} to our GitLab Maven repository, allowing seamless, credential-free publishing and dependency resolution within the CI environment.
I also enabled FF_USE_FASTZIP and fast compression. In a monorepo with 10+ modules, the time saved zipping and unzipping artifacts and cache across dozens of jobs adds up to several minutes per pipeline.
Artifacts
.application-template:
extends: .base-maven-job
stage: build
script:
- mvn \(MAVEN_CLI_OPTS verify -pl \)MODULE_PATH -am
artifacts:
when: always
paths:
- "$MODULE_PATH/target/"
exclude:
- "$MODULE_PATH/target/*.jar"
reports:
junit:
- "$MODULE_PATH/target/surefire-reports/TEST-*.xml"
- "$MODULE_PATH/target/failsafe-reports/TEST-*.xml"
coverage_report:
coverage_format: jacoco
path: "$MODULE_PATH/target/site/jacoco/jacoco.xml"
expire_in: 1 hour
coverage: '/Coverage: (\d+(?:\.\d+)?)%/'
when: Set to always so that artifacts are uploaded even on failure, allowing debugging via HTML reports.
paths, exclude: Target specific files for upload. Since fat-jar files can be massive, excluding them prevents 413 Request Entity Too Large errors.
reports: Ensures all relevant reports are submitted to provide coverage data in the GitLab Pipeline UI and the MR widget.
Sharing artifacts across jobs ensures that build products (like /classes) can be reused in subsequent steps, such as jib:build. This significantly reduces container build times by skipping redundant source code re-compilation.
Sharing artifacts across parent-child Pipeline
One of the biggest hurdles in Parent-Child pipelines is that child pipelines run in isolated workspaces. Standard needs: artifacts: true cannot pull files from a triggered child pipeline back into the root orchestrator.
To solve this for our aggregated reports, I implemented an ID-based API Collection pattern:
Bridge API: Query the parent pipeline's bridges to find the downstream_pipeline.id.
Jobs API: Query that child pipeline to find the specific build job ID.
Artifact Download: Use a Project Access Token to download the artifact zip directly via the API.
Why a Project Access Token (PAT)? Because GitLab's standard $CI_JOB_TOKEN is restricted for security and often cannot cross the pipeline boundary. A PAT with read_api scope ensures our aggregator has the necessary authority.
# Simplified aggregation logic in root orchestrator
for module in mmm-core mmm-search; do
# 1. Resolve Child Pipeline ID
CHILD_PIPELINE_ID=\((curl --silent --header "PRIVATE-TOKEN: \){PAT_TOKEN}" \
"\({CI_API_V4_URL}/projects/\){CI_PROJECT_ID}/pipelines/${CI_PIPELINE_ID}/bridges" \
| jq -r ".[] | select(.name==\"trigger-${module#mmm-}\") | .downstream_pipeline.id")
# 2. Resolve specific Job ID
CHILD_JOB_ID=\((curl --silent --header "PRIVATE-TOKEN: \){PAT_TOKEN}" \
"\({CI_API_V4_URL}/projects/\){CHILD_PIPELINE_ID}/jobs" \
| jq -r ".[] | select(.name==\"build-${module#mmm-}\") | .id")
# 3. Securely Download
curl --location --header "PRIVATE-TOKEN: ${PAT_TOKEN}" \
"\({CI_API_V4_URL}/projects/\){CI_PROJECT_ID}/jobs/${CHILD_JOB_ID}/artifacts" \
--output "${module}.zip"
done
Independent Child Pipelines
With .gitlab-ci-base.yml providing the blueprint for all child modules, the child pipelines are simple to set up and configure.
# project/mmm-core/.gitlab-ci.yml
include:
- local: '.gitlab-ci-base.yml'
variables:
MODULE_PATH: "project/mmm-core"
build-core:
extends: .application-template
deploy-core-snapshot:
extends: .deploy-snapshot-template
needs: ["build-core"]
deploy-core-image:
extends: .deploy-image-template
needs: ["build-core"]
deploy-manifests-prod:
extends: .deploy-cluster-template
needs: ["deploy-core-image"]
resource_group: core-deploy-prod
environment:
name: prod
variables:
OVERLAY_PATH: "project/mmm-core/k8s/overlays/prod"
DEPLOYMENT_NAME: "mmm-core"
Deployment Safety (resource_group)
In a monorepo, multiple modules often deploy to the same namespace. If two pipelines trigger simultaneously, they might both try to run kubectl apply for the same module, leading to race conditions.
To prevent this, I implemented module-specific Resource Groups as a CI-level mutex:
deploy-manifests-prod:
extends: .deploy-cluster-template
resource_group: core-deploy-prod
environment:
name: prod
By using the naming convention [module]-deploy-[env]:
No matter how robust your pipeline is, things can still go wrong. To provide a safety net, I paired every deployment job with a manual Rollback job.
rollback-prod:
extends: .rollback-template
needs: ["deploy-manifests-prod"]
resource_group: core-deploy-prod
variables:
DEPLOYMENT_NAME: "mmm-core"
K8S_NAMESPACE: "prod"
Using kubectl rollout undo, this job allows any team member to immediately revert a failed deployment to its previous stable revision with a single click in the GitLab UI. This "Panic Button" is essential for maintaining high availability in a fast-moving monorepo.
Shift Left Security
GitLab provides comprehensive built-in security scanning templates that you can adopt quickly and easily.
Shift left security means building security testing, compliance checks, and secure coding practices into the earliest phases of the software development life cycle (SDLC).
Source: https://about.gitlab.com/topics/devsecops/shift-left-security/
# .gitlab-ci.yml
include:
- local: '.gitlab-ci-base.yml'
- template: Jobs/SAST.gitlab-ci.yml
- template: Jobs/Secret-Detection.gitlab-ci.yml
- template: Security/Dependency-Scanning.gitlab-ci.yml
stages:
- ...
- test # setup to run GitLab built-in scans
- report
That's it! You could override the individual job if you want to, but it's optional and some could be overwritten using global variable.
<properties>
<sonar.version>5.5.0.6356</sonar.version>
<sonar.projectKey>bwgjoseph:${project.artifactId}</sonar.projectKey>
<sonar.projectName>bwgjoseph:${project.artifactId}</sonar.projectName>
<!-- Not necessary for self-hosted -->
<sonar.organization>bwgjoseph</sonar.organization>
<sonar.coverage.jacoco.xmlReportPaths>${project.build.directory}/site/jacoco/jacoco.xml</sonar.coverage.jacoco.xmlReportPaths>
<!-- SonarQube Scanner Properties -->
<sonar.scanner.skipSystemTruststore>true</sonar.scanner.skipSystemTruststore>
<sonar.scanner.skipJreProvisioning>true</sonar.scanner.skipJreProvisioning>
<!-- Java 25 support: Allow Sonar plugins (like IaC) to access restricted native methods -->
<sonar.scanner.javaOpts>--enable-native-access=ALL-UNNAMED</sonar.scanner.javaOpts>
<!-- Use CI project dir for scanner home if available, fallback to local .sonar -->
<sonar.userHome>${env.CI_PROJECT_DIR}/.sonar</sonar.userHome>
</properties>
To ensure independent Quality Gates in SonarQube, I enforced unique sonar.projectKey values for each module (e.g., bwgjoseph:${project.artifactId}). Without this, every module analysis would overwrite the previous one in the Sonar dashboard!
To take a pipeline from "Functional" to "Enterprise-Ready," I added these high-impact features:
The Engineering Portal: I used GitLab Pages to host a unified site for aggregated JaCoCo coverage at /coverage and Maven Documentation at /site. I added a "Gateway Index" in CI that dynamically builds a landing page to navigate between them.
Human-Friendly Triggers: I leveraged GitLab's variables:options to create a dropdown menu in the UI. Now, anyone on the team can manually trigger a full build without needing to remember CLI flags.
The Ghost Aggregator: I introduced a "code-free" module (mmm-report) purely for aggregation. It provides a clean target for our ID-based artifact collection and prevents our functional modules from being cluttered with aggregation logic.
Inner-Loop Development: Validate your complex parent-child YAMLs locally using gitlab-ci-local. This shaves hours off the debugging cycle by allowing you to run jobs directly on your machine.
Conclusion
At the end of the day, a CI/CD pipeline is only as good as the developer experience it provides. In this redesign, our goal was to make the monorepo feel "small" again—ensuring that a developer working on a single module isn't burdened by the weight of the entire project.
By automating the complex bits—like ID-based artifact collection and cross-pipeline reporting—and providing human-friendly tools like UI-driven triggers and local validation, I’ve built a system that stays out of the way while providing a safety net of "Shift Left" security.
Scaling a monorepo parent-child pipeline isn't just about build commands; it's about orchestration, visibility, and developer experience. Hopefully, these patterns help you (and the future me!) build better pipelines.
Source Code
As usual, the full source code is available on GitHub
