Stop Treating Pentests Like Final Exams: Why Security Can’t Be Your Last Step

A close-up photo of a part closed laptop. The photo is taken from the profile of the laptop, and there is a light reflection of the screen (which is blurred) showing on the keys, giving the keys a slight ethereal glow.

“The pentest report just came back. We have 47 critical vulnerabilities. Release is tomorrow.”

I’ve heard variations of this panic-inducing statement from dozens of clients. They treated their penetration test (aka “pentest”) like a final exam they could cram for, only to discover they’d been building on quicksand for months.

Here’s the uncomfortable truth: if your pentest reveals critical vulnerabilities, it’s not doing its job. It’s exposing the failure of your entire development process.

The Pentest-as-Final-Exam Antipattern

Let me paint a familiar picture:

  1. Team builds features for 3-6 months
  2. Two weeks before release: “Oh, we need that security checkmark”
  3. External pentest scheduled
  4. Report arrives: disaster
  5. Panic mode: delay release or ship vulnerabilities
  6. Promise to “do better next time”
  7. Repeat

This approach treats security like spell-check: a final step before publishing. But unlike typos, security vulnerabilities can destroy your business.

The Real Purpose of Penetration Testing

A properly positioned pentest isn’t a discovery tool, it’s a verification tool. It should confirm that your security practices work, not reveal that they don’t exist.

Think of it like this:

  • Bad: Pentest as metal detector finding landmines
  • Good: Pentest as final inspection confirming the area is clear

The difference? In the good scenario, you’ve been clearing mines all along.

Building Security Into Your SDLC

As Tanya Janca, application security expert, author, and OWASP Lifetime Distinguished Member, explained in a recent conversation with me about secure coding:

If you add a security step to each phase, you’ll build way better software. And that is a whole bunch of activities, right? It’s not really one thing, but if your organization adds it to the way you build software, every single time you’ll release better software.

Here’s what that looks like in practice:

Design Phase: Threat Modeling

Before writing code, ask: “How could someone abuse this feature?”

  • Document trust boundaries
  • Identify potential attack vectors
  • Design with least privilege
  • Plan security controls upfront

Development Phase: Secure Coding

While writing code:

Build Phase: Automated Security Scanning

Every build should include:

  • Static Application Security Testing (SAST)
  • Dependency vulnerability scanning
  • Container image scanning
  • Secret detection

Testing Phase: Continuous Security Testing

Not just before release:

  • Dynamic Application Security Testing (DAST)
  • API security testing
  • Infrastructure scanning
  • Security unit tests

Deployment Phase: Runtime Protection

Deploying to Production isn’t the end:

  • Web Application Firewalls (WAF)
  • Runtime Application Self-Protection (RASP)
  • Continuous monitoring
  • Incident response planning

ℹ️ Note

The above lists are non-exhaustive. You should be doing all of these things as a bare minimum

Your Internal Pentest Arsenal

“But we don’t have a security team!”

You don’t need one to start. The same tools external pentesters use are available to you:

ZAP (Zed Attack Proxy)

Free, open-source, and automation-friendly:

# Add to your CI/CD pipeline
- name: ZAP Scan
  uses: zaproxy/action-full-scan@v0.4.0
  with:
    target: 'https://staging.yourapp.com'
    rules_file_name: '.zap/rules.tsv'
    cmd_options: '-a'

More information about ZAP can be found at zaproxy.org.

Burp Suite

Professional-grade scanning with CI/CD integration:

# Automated Burp scan
java -jar burp-ci-driver.jar \
  --burp.jar=burpsuite_pro.jar \
  --project-file=myproject.burp \
  --target-url=https://staging.yourapp.com

More information about Burp Suite can be found at PortSwigger.

SemGrep

Static analysis that catches vulnerabilities before runtime:

# GitHub Actions example
- name: SemGrep Security Scan
  uses: returntocorp/semgrep-action@v1
  with:
    config: >-
      p/security-audit
      p/owasp-top-ten

More information about SemGrep can be found at semgrep.dev

Integration Strategy

These aren’t one-time tools—they’re continuous guardians:

  1. Every Pull Request: SemGrep for code analysis
  2. Every Merge to Main: ZAP basic scan
  3. Every Release Candidate: Full security suite
  4. Weekly: Comprehensive infrastructure scan
  5. Before External Pentest: All of the above

The Pentest Preparation Playbook

When you do schedule that external pentest (and you should), here’s how to maximize its value:

8 Weeks Before: Internal Security Sprint

  • Run all automated tools
  • Fix critical and high findings
  • Update dependencies
  • Review security configurations

6 Weeks Before: Mirror Production

Create an exact replica:

  • Same infrastructure
  • Same OS versions
  • Same dependencies
  • Same configurations
  • Same data patterns (sanitized)

4 Weeks Before: Internal Pentest

Your team attempts to break the system:

  • Use OWASP Top Ten as a checklist
  • Test authentication and authorization
  • Verify input validation
  • Check for information disclosure

2 Weeks Before: Fix and Verify

  • Address internal findings
  • Re-test fixed issues
  • Document security measures
  • Prepare security documentation

Pentest Day: Confirmation, Not Discovery

The external pentest should find:

  • Advanced attack vectors
  • Business logic flaws
  • Complex chaining attacks
  • Things you genuinely missed

Not:

  • SQL injection in login forms
  • Unpatched dependencies
  • Default credentials
  • Missing HTTPS

The CI/CD Security Pipeline

Here’s a complete security pipeline that runs automatically:

name: Security Pipeline

on: [push, pull_request]

jobs:
  security-scan:
    runs-on: ubuntu-latest
    steps:
      # Code Analysis
      - name: SemGrep Scan
        uses: returntocorp/semgrep-action@v1
        
      # Dependency Check
      - name: OWASP Dependency Check
        uses: dependency-check/Dependency-Check_Action@v2
        
      # Container Scanning (if applicable)
      - name: Trivy Container Scan
        uses: aquasecurity/trivy-action@master
        
      # SAST
      - name: SonarQube Analysis
        uses: SonarSource/sonarqube-scan-action@master
        
      # DAST (on staging)
      - name: ZAP Scan
        if: github.ref == 'refs/heads/main'
        uses: zaproxy/action-full-scan@v0.4.0
        
      # Results
      - name: Upload Security Reports
        uses: actions/upload-artifact@v3
        with:
          name: security-reports
          path: |
            *-report.html
            *-report.json

ℹ️ Note

The above is provided as an example, please edit it to fit your purposes.

ℹ️ Warning

The above example was correct at the time of writing. However, we take no responsibility for any harm, misconfiguration, or other negative effects by your use of it.

As with all code you find on the internet, it is your responsibility to vet it’s contents before using it.

Beyond Tools: The Security Mindset

Tools catch vulnerabilities, but secure design prevents them. Consider:

Input Validation

// Bad: Trusting user input
public IActionResult Search(string query)
{
    var sql = $"SELECT * FROM products WHERE name LIKE '%{query}%'";
    // SQL injection vulnerability
}

// Good: Parameterized queries
public IActionResult Search(string query)
{
    var results = _context.Products
        .Where(p => p.Name.Contains(query))
        .ToList();
}

Authentication and Authorization

  • Implement proper session management
  • Use strong password policies
  • Enable MFA where possible
  • Apply principle of least privilege
  • Log security events

Data Protection

  • Encrypt sensitive data at rest
  • Use TLS for data in transit
  • Implement proper key management
  • Sanitize logs and error messages

The Timeline That Works

Here’s a realistic timeline for integrating security:

Month 1: Foundation

  • Set up basic security scanning in CI/CD
  • Train team on OWASP Top 10
  • Implement security code reviews
  • Start threat modeling for new features

Month 2: Automation

  • Add SAST to pull requests
  • Implement dependency scanning
  • Set up DAST for staging
  • Create security test cases

Month 3: Maturation

  • Internal penetration testing
  • Security champions in each team
  • Regular security training
  • Incident response planning

Month 4+: Continuous Improvement

  • Regular internal assessments
  • Quarterly external pentests
  • Security metrics tracking
  • Continuous learning

The Connection to SBOMs and Attestations

As I’ve discussed in previous posts about SBOMs and Attestations, security is a chain. SBOMs tell you what’s in your software, attestations prove it hasn’t been tampered with, but neither protects against application-level vulnerabilities.

You need all three:

  1. SBOMs: Know your components
  2. Attestations: Verify your supply chain
  3. Pentesting: Validate your implementation

The Business Case for Continuous Security

Let’s talk money:

The "Last-Minute Pentest" Cost:

  • External pentest: £15,000
  • Emergency fixes: £50,000 (overtime, delays)
  • Delayed release: £200,000 (lost revenue)
  • Reputation damage: Incalculable
  • Total: £265,000+

The "Continuous Security" Cost:

  • Tool licenses: £5,000/year
  • Training: £10,000
  • Additional development time: 10%
  • Regular pentests: £30,000/year
  • Total: £45,000 + 10% time

The maths is clear: prevention is cheaper than panic.

Your Security Evolution Roadmap

This Week:

  1. Install ZAP locally
  2. Run it against your staging environment
  3. Share findings with your team
  4. Pick one tool to add to CI/CD

This Month:

  1. Implement basic security scanning
  2. Schedule team security training
  3. Start threat modeling new features
  4. Plan your first internal pentest

This Quarter:

  1. Full security pipeline operational
  2. First internal pentest completed
  3. External pentest scheduled
  4. Security metrics dashboard live

This Year:

  1. Security integrated into every phase
  2. Pentests finding minor issues only
  3. Team security-conscious by default
  4. Compliance achieved with confidence

The Bottom Line

Pentests aren’t final exams, they’re progress reports. If you’re discovering critical vulnerabilities during a pentest, you’ve already failed. Security isn’t a phase; it’s a practice.

Start small. Run ZAP this week. Add SemGrep to your PR process. Build security habits. When that external pentest arrives, it should confirm your security, not reveal its absence.

Your customers trust you with their data. Honour that trust by building security in, not bolting it on.

The best time to start was at project kickoff. The second best time is now.

Ready to transform your security posture from reactive to proactive? I provide DevSecOps consultation that helps enterprises build verifiable, secure software supply chains.

Reach out using the contact form below, and I’ll be in touch to schedule a free discovery call.

We'll never share your name with anyone else.
We'll never share your email with anyone else.
Providing a subject can help us deal with your request sooner.
Please be as specific as possible; it will help us to provide a more specific response.
Un-checking this box will ensure that your data is deleted (in line with our privacy policy after we have dealt with your request. Leaving this box checked will auto enrol you into our email communications, which are used for marketing purposes only.