Updated: Mar 11, 2020
The broad definition of Offensive Security (OffSec) includes any activities, automated or manual, that take an offensive (but ethical) hacking and exploitation approach to software and systems. There is very wide range in OffSec offerings and approaches spanning the related capabilities, resources, and tools. This post will explore OffSec with a focus on the Penetration (Pen) Testing offering.
What is Penetration Testing? It can depend on who you ask and, unfortunately, it is usually easier to explain what it is not. Pen Testing is a gray area under OffSec and getting grayer as the complexity of solutions increase. Technologies including client-server, mobile device, IoT, services (web, cloud, etc.), distributed data stores, and public cloud are evolving past perimeters into active monitoring and automated counter-measures. A solution and each of the technologies in that stack has its own requirements and challenges, so defining Pen Testing (or any OffSec area) is dictated by the target solution and business/market.
There is no "cookie cutter" or "one size fits all" definition or approach. Vendors that focus on enterprises and use these types of terms often provide an end-to-end, long duration, and expensive proposal that is not what you need and without the ROI you want. You need a tailored approach that is developed with an understanding of your business. This allows for a properly scoped engagement with a reasonable signal-to-noise ratio, which is the real, actionable findings measured against false positives or other noise. It is important in OffSec to have the right signal/noise ratio baseline and the ability to tune the tooling and automation to involve humans only when needed. This provides a segue to a Pen Testing definition.
A Pen Test is aptly named — highly trained OffSec technicians attempt to penetrate a solution to find vulnerabilities and do exploits. In terms of signal/noise ratio, there should be no false positives in Pen Testing given that its more manual than automated, more "inside-out" than "outside-in," and all findings are validated. (Inside-out is when a foothold is established or given inside the solution, and the tester moves sideways and elevates leveraging vulnerabilities to do an exploit. A Pen Test will do some outside-in employing vulnerability scanning and hacking techniques, but that itself is not Pen Testing.)
Pen Testing approaches are highly manual with automation and tooling run by technicians performing the right amount of manual validation to filter out the false positives and doing validations to provide verfied, actionable findings. These technicians must also be able to develop reports for any audience (internal or external to the business) and provide guidance on remediation. Ethical hacking techniques are employed with the testing going beyond the technology stack into the physical and user security aspects (e.g., facilities, servers/devices, social engineering, phishing, etc.). The definitions, approaches, targets, and rules of engagement are all part of scoping and planning of the Pen Test. Setting clear goals with well-defined roles/responsibilities and rules of engagement is important so the technicians know how to approach each target and how far they should take the exploits. Definitions, scoping, planning, and especially the rules of engagement are more important and take on finer detail when partner/client systems or infrastructure a business doesn't own or directly manage are involved (e.g., public clouds like AWS and Azure).
You should always take care not to compromise the systems, configurations, or data, but that isn't always possible in OffSec, so Cylidify recommends:
Not targeting actual production systems or exercising great care when any production assets are involved (systems, services, applications, components, or data)
Make sure that the target system or solution is dedicated to the OffSec testing effort and not shared with other testing (e.g., regression or performance) noting that this may be for an extended period (7+ days)
Having fallback systems and databases as well as involvement and awareness of operational and support teams to keep the solution available and unblock testing as needed
Pen Testing should target production equivalent solutions extending out to the defenses and configurations. The defenses and countermeasures should be also be tested, but configured and managed so they do not hinder the testing or prevent certain types of testing, including chained attacks and inside-out approaches (e.g., adversarial simulations or red vs. blue or others focused on insider threats).
Cylidify believes there is very high value to be recognized from a properly planned, scoped, and implemented and we are available to assist. In the pre-engagement, we can discuss OffSec and Pen Testing in even greater detail defining our approach, detailing the value propositions, covering roles and responsibilities, and delving into some of the more complicated aspects such as rules of engagement and disclosure. (See the disclosure discussions in our Ransomware blogs.) You can use our website contact form, email, or call to schedule a free initial consultation. Many of our OffSec engagements start (and end) with an assessment and plan covering scoping and implementation of an engagement that is implemented by another vendor including less traditional approaches such as crowd-sourced testing or bounty programs. However we can also provide a full range of OffSec consulting and services as listed in our offerings.