Penetration Testing Requirements: Written Into the Standard, Expected in Practice, or Only Proposed

Penetration testing is a gray area. If you ask five teams which security frameworks require pen tests, you’ll often get five different answers. That’s partly because what’s “required” isn’t always crystal clear, and sometimes, even if a framework doesn't strictly need a pen test, auditors and customers will still expect one.

Frameworks like FedRAMP and PCI DSS have pen testing written into their standards. SOC 2, ISO 27001, and HIPAA, on the other hand, leave it up to you (though many customers and auditors will still expect one). AI-focused frameworks are also split: AIUC-1 specifically requires a pen test, while NIST AI RMF and ISO 42001 are more open-ended.

Penetration Testing Requirements at a Glance

Here’s a breakdown of a number of security frameworks and standards alongside their penetration testing requirements:

Which Frameworks Require a Penetration Test?

PCI DSS and FedRAMP both name a penetration test in their requirements, as do a range of recent laws and acts like DORA and NYDFS.

What Does PCI DSS Require for Penetration Testing?

Under Requirement 11.4 in version 4.0.1, PCI DSS (Payment Card Industry Data Security Standard) states that penetration testing is required at least once every 12 months and after any significant changes. It also requires tests to follow a defined, documented methodology, and the tester to be qualified and independent. If segmentation has been used to reduce scope, it requires the segmentation controls to be tested every 12 months (every six months for service providers).

Does FedRAMP Rev5 Require a Pen Test?

FedRAMP Rev 5 requires penetration testing for all authorized organizations under control CA-08. Your initial pen test must be completed no more than six months before the assessment and must be run by a 3PAO (third-party assessment organization). Post-authorization, penetration tests must be completed at least every 12 months.

The 2026 Consolidated Rules now describe penetration testing as part of vulnerability detection, subject to the vulnerability detection and response rules. The vulnerability detection rules become mandatory on December 7, 2026, with a grace period that runs to March 7, 2027.

Does FedRAMP 20x Require a Pen Test?

FedRAMP 20x follows the same vulnerability detection rules as Rev5, with organizations required to systematically, persistently, and promptly discover and identify vulnerabilities. Penetration testing is one way organizations can find those vulnerabilities, alongside options like scanning, bug bounties, and automated control testing.

In practice, with FedRAMP 20x Class B and C, penetration testing becomes one way to show that vulnerability detection works, rather than a scheduled test on its own.

Which Other Regulations and Standards Require a Pen Test?

• CMMC (Cybersecurity Maturity Model Certification) Level 3 requires a pen test at least annually and after significant security changes. Levels 1 and 2 don't strictly require a test, though many defense contractors will still complete one to satisfy risk assessment needs.
• NYDFS (New York Department of Financial Services) Part 500 requires annual testing from inside and outside the system boundary, by a qualified internal or external party.
• DORA (the EU's Digital Operational Resilience Act) requires threat-led penetration testing at least every three years, but only for financial entities their regulator designates.
• NIST SP 800-53, the federal control catalog from the National Institute of Standards and Technology (NIST), puts control CA-8 in its High baseline.

Which Frameworks Leave the Pen Test to You?

SOC 2, ISO 27001, and HIPAA don't require a penetration test today. Teams run one anyway, because customers and auditors expect it. HIPAA is the one that may change.

Does SOC 2 Require a Penetration Test?

Neither SOC 2 Type 1 nor Type 2 requires a pen test for attestation. Pen testing is mentioned once in the Trust Services Criteria, under CC4.1, where it's listed as one of the ways a company might meet CC4.1's requirements, alongside other options like internal audits, compliance and third-party assessments, and vulnerability scans.

Does ISO 27001 Require a Penetration Test?

ISO 27001 doesn't require a penetration test. The management of technical vulnerabilities (8.8) and security testing in development and acceptance (8.29) fall under its Annex A, but a pen test is never listed as a set requirement. Like SOC 2, though, every auditor will ask for one, even though it's not named in the framework's documentation.

Does HIPAA Require a Penetration Test?

Currently, HIPAA doesn't require penetration testing; its Security Rule has no explicit penetration testing requirement. However, in January 2025, the Department of Health and Human Services (HHS) proposed an update that would require a penetration test by a qualified person every 12 months, or more frequently if a risk analysis calls for it. This update is still a proposal, though, so as things stand HIPAA doesn't require pen testing. HHS is targeting July 2027 for a final rule, and compliance would be required 180 days after it takes effect.

Why Do Teams Run a Pen Test When It Isn't Required?

When we talk about SOC 2 or ISO 27001 not requiring a penetration test, it's always with a caveat, because customers who ask to see those reports will almost always expect to see a pen test alongside them. I think of pen testing as a sibling to SOC 2: it's separate, but almost always alongside it.

That's what we tell companies: you don't have to do a pen test for SOC 2, but you might as well. The standards don't ask for pen tests specifically, but for most teams it makes sense to do one.

Want to dig deeper? Check out our guide to penetration testing for startups.

Is a Vulnerability Scan Enough?

More often than not, a vulnerability scan isn't enough to satisfy a buyer that wants a pen test. You also need to be careful with offerings sold as automated pen tests, as these are often just rebranded vulnerability scans.

If a buyer wants a pen test, that's all they'll accept. A true pen test will scan your systems to see what exploits are available, with human or AI agents also trying to get access and exploit gaps. It offers a level of testing that SOC 2 or ISO 27001 don't, which is why buyers like to see them alongside your reports.

What Do AI Frameworks Require for Pen Testing and Red Teaming?

For AI, the question changes slightly. AI brings with it a whole host of challenges, like prompt injection, jailbreaks, and data leakage, that may not be covered by a traditional pen test unless specifically requested. When it comes to AI, tests are usually called adversarial testing or red teaming: a term that's become the standard for testing AI models and the apps built around them with a specific goal, like making an AI system leak data or ignore its instructions.

Here's what's required in the EU AI Act, NIST AI RMF, AIUC-1, and ISO 42001.

Does the EU AI Act Require Pen Testing?

The EU AI Act requires adversarial testing for providers of general-purpose AI models with systemic risk under Article 55. In practice, that means labs like OpenAI, Anthropic, and Google that build and train foundation AI models.

For high-risk AI systems, Article 9 requires testing against prior defined metrics before the system is placed on the market. Article 15 requires resilience against attempts to exploit vulnerabilities, including data poisoning, model poisoning, adversarial examples, and confidentiality attacks. Neither names a penetration test or a frequency, and the duties sit mainly with providers rather than the companies deploying a system.

Does ISO 42001 Require Red Teaming?

ISO 42001 doesn’t specifically name red teaming. It asks you to define and document measures for each AI system, along with the criteria for using them (Annex A, control A.6.2.4). How far the testing goes depends on your organization’s risk assessment and Statement of Applicability.

Does the NIST AI RMF Require Red Teaming?

MEASURE 2.7 of the NIST AI RMF (AI Risk Management Framework) expects AI system security and resilience to be evaluated and documented. It suggests red-team exercises under adversarial or stress conditions, with the results documented. However, the framework itself is voluntary.

What Does AIUC-1 Require for Adversarial Testing?

AIUC-1 is the world's first AI agent standard. Its control B001 requires adversarial robustness testing at least every three months, run by a third party. That includes red teaming, prompt injection assessments, jailbreak attempts, and simulated malicious tool calls, built on a taxonomy of adversarial risks.

A second control, B004, is about preventing scraping of your AI endpoints and runs every 12 months. Its expected evidence includes a third-party pen test of those endpoints, covering scraping, brute force, reconnaissance, and rate limiting.

The OWASP Top 10 for LLMs

The OWASP Top 10 for LLM Applications is made by the Open Worldwide Application Security Project. It's a widely used list of risks for large language model (LLM) apps with no certification or audit behind it. Its 10 items include prompt injection, sensitive information disclosure, and improper output handling. It's a practical starting point for scoping what an AI test should cover, and AIUC-1 maps B001 to four of its items.

Putting This Into Action: How to Know When You Need a Pen Test, Adversarial Testing, or Red Teaming?

When a customer or auditor asks for a pen test, it's best to find out exactly what they want before you begin the process. The answer will generally help you decide which test to run, who needs to perform it, and what the report needs to show.

For most startups getting their first pen test, gray box testing tends to give you the most useful information for the money. With gray box testing, the tester is spending time exploring vulnerabilities and looking for weaknesses rather than on reconnaissance and gaining initial access. (Our guide to penetration testing for startups digs deeper into this.)

For teams focused on AI testing, the OWASP list is a good starting point to understand the threats that come with LLMs. From there, it's all about the framework you're pursuing and what your customers want to see. For example, if you're building agentic AI solutions and pursuing AIUC-1, you'll need to run adversarial testing at least every three months.

At Workstreet, we assist teams of all sizes and requirements with pen testing, adversarial testing, and red teaming. We size your testing to your company stage and needs. If a customer has asked you for a pen test, or you're looking for one as you work toward a new framework, and you want to work out what's actually required, talk to our team.