What is a Penetration Test?
Free-form security testing, or penetration testing (often called pen testing), is a minimally structured examination of the cybersecurity for a target device or system. For those familiar with requirements-based verification and validation (V&V) testing, penetration testing is different altogether. The intent of penetration testing is to find unknown security vulnerabilities or architectural concerns rather than confirm that implementation matches design. The latter is the focus of security V&V. Penetration testing can also be described as attacker-like testing because the same tactics, techniques, and procedures (TTPs) an adversary might use are exercised by the penetration tester. From a medical device perspective, penetration testing and its complete findings are now expected by the US FDA as part of a manufacturer’s premarket regulatory submission.
What do they test?
By definition, penetration test coverage is variable and incomplete. Mirroring the real world, where a potential threat actor has a specific set of skills, so do penetration testers. While they have a broad set of capabilities, every tester will have their own specialties. This is important for two reasons: you want to match the capabilities of the tester to your focus area as well as mix up scope and testing capabilities over time to minimize blind spots.
Testing of network-connected systems is common practice and a familiar space for many testers. However, medical devices come in all shapes and sizes, making it critical to select a tester who best matches the capabilities of the device. For networked embedded systems, a tester who is familiar with testing networking protocols as well as embedded systems is a great fit. Testing of mobile apps, physical access, and standard or custom wireless protocols all require different skill sets. For example, an embedded system where the test includes physical access may require the tester to be familiar with embedded CPU architectures, flash memory, various serial and parallel interfaces, and Joint Test Action Group or JTAG. Highly skilled testing may be necessary in cases where the target is a custom wireless link, non-standard protocols are used, or significant hardware engineering knowledge is required.
Defining testing scope and parameters is a critical planning step that will help select the appropriate tester as well as further refine potential target areas for the test.
Great! Now that the tester is selected, we are good to go, right?
Once a tester has been identified, you will need to work with them to confirm the scope and rules of engagement (RoE) for the test, essentially what is “in” and “out” of bounds for them to test. This is especially important for network connected devices which may connect to a back-end infrastructure for software updates or other tasks. Well defined RoE’s will focus efforts on your desired target and prevent a tester from probing into systems that should not be included in scope.
Finally, penetration test results are not pass/fail. Instead, the results will be compiled into a report which itemizes the potential issues found and may even suggest ways to improve. Interpretation of the results are then required. This should apply well-defined risk assessment criteria to determine whether the identified vulnerabilities represent an acceptable risk or if patches, mitigations, or other actions are required.
Putting it all together
Penetration testing is a critical part of a healthy product security program and is increasingly a key requirement for regulatory compliance. While it provides reasonable assurance that any gaps are identified, it is important to keep in mind that its coverage is incomplete and should not be relied upon as the only security check. Good security requirements, threat modeling, V&V, and ongoing risk management are just as important to ensure the long term security of medical devices.
About MedSec
MedSec is uniquely prepared to meet the specific challenges of medical device and healthcare cybersecurity. MedSec partners with medical device manufacturers and healthcare delivery organizations to address cybersecurity in medical devices throughout all stages of the device lifecycle. MedSec leverages its cybersecurity expertise, coupled with its intimate knowledge of the healthcare regulatory and operating environments, to offer support in design, architecture, verification, penetration testing, risk assessments, regulatory filings and SBOM development.