ResponseOne DevSecOps Pipeline

The problem

Most teams ship code with known vulnerabilities because security checks happen too late, post-deployment, during a quarterly pen test, or never. ResponseOne's brief was to instrument a real application with SAST, DAST, dependency scanning, and secrets detection at the CI/CD level, then build a SOAR layer that ingests every finding, enriches it, and triggers the right response automatically, without a human copying data between tools.

OWASP Juice Shop deliberately vulnerable web application running as the pipeline's target — OWASP Juice Shop, the deliberately vulnerable target application the pipeline scans against.

My role

Formal roles on the final report: Team Lead, Solution Architect, and Technical Lead. What that covered in practice:

Originally assigned, and delivered

Deploying and configuring the Shuffle SOAR platform on RES-SVR-L001
Building and debugging the "Critical CVE Alert" workflow, including diagnosing and fixing the Router/startnode structural bug
Designing and enforcing severity gates across all four scanner jobs (Secrets, SAST, SCA, DAST)
Building the automated playbooks (Secrets Alert, CVE Alert, OWASP Alert branches)

Absorbed beyond original scope, covering gaps in teammates' areas

Scanner integration debugging: found and fixed the jq/logic bugs in the scanner jobs, including the riskdesc substring-matching defect in the DAST gate and a related SCA bug where the Shuffle-reporting step read the unfiltered finding array instead of the severity-filtered one
Environment and infrastructure troubleshooting: Docker networking faults, a port conflict with Wazuh's indexer, an OpenSearch health failure traced to container resource contention, ngrok tunnel stability, GitHub Codespace git conflicts, and workflow YAML syntax errors
Wrote every vulnerability finding entry (description, severity, CVSS, remediation) for Secrets, SAST, SCA, and DAST, plus the project's executive summary
Designed and executed the plant-detect-gate-alert verification method used consistently across all four scan types, including planting synthetic vulnerabilities where real ones weren't otherwise available, and narrowing scope to surface a genuine one where planting wasn't needed
Repeatedly diagnosed and resolved git branch divergence between local Codespace commits and remote edits made via the GitHub web editor

SOAR platform and playbook build

Shuffle runs on RES-SVR-L001 as the automation core. Scanner findings from GitHub Actions POST to a webhook, and three conditional branches route each finding to the right playbook: a Secrets Alert straight to Slack, and a CVE or OWASP finding through NVD enrichment first for full CVE/CVSS context before alerting.

Shuffle SOAR platform running with its containers healthy on RES-SVR-L001 — Shuffle deployed and healthy on RES-SVR-L001, the SOAR core for the whole pipeline.

Two structural bugs surfaced while wiring the workflow, both fixed rather than worked around. The first: an early version of the canvas had an unintended connection between branches that would have caused cross-triggering. It was found during testing and corrected.

Shuffle canvas showing the branch connection corrected between conditional paths — Correcting an unintended branch connection in the Shuffle canvas before it could misroute a finding.

The second was more fundamental: Shuffle derives a workflow's startnode from the actions[0]/graph structure rather than accepting a manually set start/isStartNode value. With three parallel conditional branches wired directly from the trigger, only one branch was ever structurally reachable, the CVE and OWASP branches were silently skipped regardless of condition or payload, no error, just no execution. The fix was to insert a single "Router" pass-through node between the trigger and the three branches, funnelling every conditional path through it. Shuffle then correctly assigned the Router as the startnode, resolving routing for all three branches.

Shuffle canvas showing the Router pass-through node fixing the startnode bug that was skipping two of three branches — The Router fix: all three branches now sit under a shared pass-through node, resolving the startnode bug.

With NVD enrichment wired in, a full test using CVE-2021-44228 (Log4Shell) confirmed the complete chain, enrichment lookup, CVSS/description detail, and a formatted Slack alert with real context rather than raw JSON.

Slack alert showing full NVD-enriched CVE detail for CVE-2021-44228 — NVD-enriched CVE alert delivered to Slack with full CVSS and description detail.

CI/CD pipeline and severity gates

Four scanner jobs, Secrets (Gitleaks), SAST (Semgrep), SCA (Trivy), and DAST (OWASP ZAP), run in GitHub Actions on every push, each gated to block the build on a High or Critical finding.

GitHub Actions security-scanners.yml workflow defining the four scanner jobs — The security-scanners.yml workflow defining all four scanner jobs.

GitHub Actions run showing all four scanner jobs completing successfully — All four scanner jobs completing end to end, confirming the pipeline is operational.

Severity gate logic added to the SAST and SCA jobs in the workflow YAML — Severity gate logic added to the SAST and SCA jobs, blocking the build on High/Critical findings.

Severity gate logic added to the Secrets and DAST jobs in the workflow YAML — Severity gate logic added to the Secrets and DAST jobs, completing the gate set across all four scanners.

Verifying the gates surfaced a real logic bug rather than just confirming they ran. The DAST gate's jq query used test("High"), a substring match against ZAP's full riskdesc field (format: "<Risk> (<Confidence>)"). That incorrectly matched entries like "Medium (High)", where "High" described ZAP's confidence, not its risk rating, a false-positive block on a Medium-risk finding. The fix anchored the match with startswith("High") instead, verified against both the live report and a synthetic test confirming the corrected query isolates only genuine High-risk entries.

GitHub Actions log showing the severity gate correctly blocking on a High-risk finding after the riskdesc bug fix — The corrected gate blocking cleanly on a genuine High-risk finding, the first run where it reached real evaluation logic rather than failing on a script error.

Verifying all four scanners with real findings

Each scan type was verified the same way: plant or surface a genuine finding, confirm the gate blocks the build, and confirm the alert reaches Shuffle and the notification channel with accurate data, not just that the job ran without error.

Secrets (Gitleaks)

A synthetic AWS access key was planted in a test file to trigger detection. Gitleaks flagged it, the gate blocked the job (secrets are always treated as Critical), and Slack received a real-time alert with accurate repo, file, and line data.

Creating and committing a test file containing a synthetic AWS access key to trigger the secrets scan — Planting a synthetic AWS key to trigger the secrets scan, avoiding AWS's own allowlisted documentation placeholder.

Gitleaks secrets scan completing and the severity gate blocking the build on the detected secret — Gitleaks detecting the planted secret and the severity gate blocking the build.

Slack alerts confirming the secret detection with accurate repo, file, and line data — Real-time Slack alerts confirming the secret detection with accurate file and line data.

SAST (Semgrep)

A deliberate SQL injection pattern, an unsanitized string-concatenated query, was planted in a test file. Semgrep flagged it as the sole error-level finding, the gate blocked the job, and the alert reached Shuffle with the exact rule and file that triggered it.

Shuffle execution details showing the exact Semgrep rule, file, and repo that triggered the SAST alert — Shuffle execution detail confirming the exact rule (sqlalchemy-execute-raw-query) and file that triggered the alert.

Slack alert confirming the SAST finding with rule and file detail — Slack alert confirming the SAST finding, matching the rule and file Shuffle received.

SCA (Trivy)

Juice Shop ships as a prebuilt container with no manifest for Trivy to scan, so a synthetic package.json declaring lodash 4.17.4, a version with a known Critical prototype pollution CVE, was planted to give Trivy a real dependency to flag.

Planting the outdated lodash 4.17.4 dependency in package.json to trigger the SCA scan — Planting a known-vulnerable lodash version, confirmed by npm's own audit output before Trivy ever ran.

Trivy SCA scan completing and correctly flagging the planted vulnerable dependency — Trivy correctly flagging CVE-2019-10744 (CVSS 9.1) and the gate blocking on the Critical finding.

Slack alert confirming the SCA finding for the planted lodash CVE.

DAST (OWASP ZAP)

The baseline ZAP scan came back clean, leaving no genuine High-severity finding to verify against. Rather than fabricate one, the scan was switched from passive baseline mode to an active full scan against Juice Shop's known-vulnerable endpoints, surfacing two real High-severity findings: SQL Injection and Source Code Disclosure via File Inclusion.

Switching the ZAP scan step from action-baseline to action-full-scan on the GitHub Codespace terminal — Switching to an active ZAP full scan after the passive baseline scan returned no genuine High-risk finding.

ZAP full scan completing with the two genuine High-severity findings and the gate blocking the build — The full ZAP scan completing, both High-severity findings caught, and the gate blocking the build.

Slack alert confirming the DAST findings with risk and confidence detail — Slack alert confirming the DAST findings, correctly isolated to genuine High-risk entries after the riskdesc fix.

Shuffle execution details showing the DAST finding routed and enriched correctly — Shuffle execution detail confirming the DAST finding routed and enriched correctly.

Email notification showing the full pipeline run status across all four scanner jobs — Email notification confirming the full pipeline run, all four jobs visible with their pass/fail status in one place.

Challenges and resolutions

Beyond the two Shuffle structural bugs and the riskdesc gate defect above, verification surfaced a steady stream of environment and logic issues, each diagnosed from logs and host-level diagnostics rather than assumption:

Docker wasn't pre-installed on RES-SVR-L001 despite the initial assumption it would be, installed from Docker's official repository before deployment could begin
OpenSearch failed to bind to port 9200 because Wazuh's indexer already held it, resolved by fully purging Wazuh, which wasn't needed for this project
A 502 on the Shuffle webhook traced back to OpenSearch failing to reach a healthy state under CPU contention from a backlog of Orborus-spawned execution containers, resolved with a full docker compose down/up cycle
RES-SVR-L001 sits behind NAT with no public IP, making the Shuffle webhook unreachable from GitHub's cloud runners, resolved with an ngrok tunnel that stayed stable across a full host restart
A missing continue-on-error on the ZAP step and missing closing fi statements in nested bash conditionals caused job failures on three separate occasions across the DAST job alone
A prior file edit accidentally removed the permissions block required for SARIF upload, breaking two scanner jobs with access errors after an otherwise clean merge
The "Send to Shuffle" step originally triggered on any alert rather than specifically High-risk ones, firing with null alert data whenever only Medium/Low findings were present, fixed by aligning the trigger to the same threshold the gate itself uses

Fixes were tested on isolated branches before merging into main, a practice adopted after an early direct-to-main merge introduced two untested regressions at once.

Dashboard and outcome

Shuffle's built-in dashboard tracked workflow executions directly, no separate Grafana/Prometheus stack needed. At the point captured below: 9 workflows, 100% success rate, 0 failed runs, and 257 total workflow executions.

Shuffle dashboard showing 9 workflows, 100 percent success rate, and 257 total executions — Shuffle's live dashboard: 9 workflows, 100% success rate, 257 executions.

Brief vs. delivered

The brief called for four scanners gating every commit, SOAR ingestion with no manual import, CVE/CVSS enrichment, three automated playbooks, a live dashboard, and a before/after MTTT comparison. What shipped matches the core of that closely: all four scanners gate real commits with live-tested logic bugs found and fixed rather than assumed away, ingestion runs webhook to webhook with zero manual steps, enrichment pulls real NVD data, and all three playbooks (Secrets, CVE, OWASP) are wired and verified with evidence across GitHub Actions, Shuffle, Slack, and email.

Due to tight submission deadline, a few brief items were not realized and hence are not evidenced in the final report: an explicit before/after MTTT figure with methodology, demonstrated deduplication logic across reruns, and a written threat model of the pipeline itself (what happens if a scanner is tampered with or bypassed).

The full project report, including every finding write-up, the complete challenges log, and additional screenshots, is documented on Notion.

View full report on Notion →

ResponseOne: DevSecOps Pipeline & SOAR Automation Platform