If you asked the average person what information security or cybersecurity companies do, they’re likely to give you an explanation that closely resembles penetration tests. The technical and hacking side of information security is the most well-known. It’s also portrayed as the most glamorous.
How many times have you seen footage of a person in a dark room, a hoodie covering their face, relentlessly scrolling through pages and pages of code?
How many times have you seen footage of a person popping onto a computer for roughly 33 seconds and immediately infiltrating a sophisticated network and infrastructure?
Don’t get me wrong, those people exist, and some of them are capable of doing that. The reality, though, is that most penetration tests are significantly less flashy. They take planning, research, a breadth of technical knowledge, and (frankly) a lot of trial and error.
When performed properly, a penetration test can do incredible things for your organization’s security. You can quickly learn how easy (or challenging) it is for attackers to break through your infrastructure, where your security gaps are, what could be done if an attacker were to get in, and what the risk is with having those security gaps. However, not all penetration tests are created equal, and not all security initiatives require penetration tests. Sometimes the smoke and mirrors, the glitz and glam, and the excessive hype are very real.
Here are the 7 ways a penetration test results can be overrated:
1. Not All Pen Testers Are Created Equally
This is pretty obvious. It’s true with most things. You have your greats at the top, your not-so-greats at the bottom, and most fall somewhere in the middle. Who you get to conduct your test has a dramatic impact on penetration test results. Skill, experience, expertise, tools used, methodology, deliverables. These factors and more contribute to the success of your engagement.
Ranging from a single person with a computer in their garage to multi-million-dollar organizations employing people who only pen test, not every tester is going to tackle the engagement the same way.
Ultimately, it’s important that you select a tester with the right amount of experience and one that can conduct an objective, risk-based engagement that helps your organization improve on the security measures it has set for the business.
If you don’t, you’re likely going to end up with penetration test results that falls short of your expectations—or one that costs way too much.
2. Penetration Tests Can Be Overkill
We often use home security systems as an example of penetration tests. How do you test if your home alarm system is working? You stage a break-in. That’s essentially what a penetration test does. It stages a break-in on
If you haven’t completed the security system install yet, the staged break-in would be too easy (and wouldn’t even actually test the system). It’s important that you do the preliminary work before testing. The same is true with a penetration test.
If you’ve never had a proper security assessment done on your organization and security practices (or if you don’t have security practices in place at all), finding out how vulnerable they are is both premature and a waste of your time and money.
3. Penetration Test Results Don’t Always Gauge Security
I just mentioned how without the right initial measures in place, a penetration test can be like walking through a wide-open door.
Imagine the engagement stopping there. Well, often, it does. Depending on the scope of your pen test engagement, the tester’s job may be to literally just see if they can get in. Not how many different ways, not how much they can do when they’re inside, not how fast. Were they able to get in or not?
How is that for penetration test results?
What are you learning about your organization’s security measures with an engagement like that? You’re learning someone can get in. Big whoop. We already know that it’s impossible to prevent all breaches, so we already knew someone can get in.
If your penetration test stops there, it’s not making your security program better.
4. Results Are Sometimes Open for Interpretation
A pen test is not standard across all engagements. Penetration test results can vary based on the person who’s conducting your test. For example, it’s certainly possible for a penetration tester to have infiltrated your system within 18 minutes of starting the engagement.
18 minutes sounds terrifyingly quick at initial glance, but that number requires some analysis and interpretation.
Was your penetration test conducted by a professional who has been doing them for nearly a decade? Was it conducted by someone who just finished training and this is their first engagement?
If it’s closer to the first, your network may not actually be as vulnerable as 18 minutes would indicate. That tester is a seasoned professional who has seen nearly everything. For that tester to get in that quickly, it might actually be a testament to their skills more so than your security program.
On the flip side of that, if your penetration tester is doing an engagement for the first time or just doesn’t have nearly the same experience as the tester in the first example, your organization should probably be extremely worried about that 18-minute conclusion.
Penetration test results may be overrated if you’re not properly taking into consideration the entire scope of the engagement. It’s important to work with an organization that will pair you with a tester who matches your current security objectives and has your best interests in mind.
5. Penetration Tests Often Have Limited Parameters
One of the most challenging components of penetration testing is the fact that they’re not always true tests. The point of a penetration test is to simulate a real-world attack and determine how an attacker could potentially infiltrate networks and steal information.
Simulated attacks are just that though—simulated. Often, the organization requesting the test will put limitations on the test, only allowing the penetration tester to attempt certain things or attack specific concerns.
In the wild, an attacker will use whatever vector they see fit to attempt to maintain your information. And, they’ll likely try to escalate the attack once they’ve made it through, too.
If you’re putting limitations (especially just to meet a compliance requirement) on what the tester can do or where they can test, your penetration test results are not giving you the whole picture. The less you allow, the less similar it becomes
6. Penetration Tests Can Be Largely Automated
In today’s bustling security landscape, it’s not uncommon to see the promise of “easy button” solutions to complex security issues. Penetration testing has fallen under the same trap. Pen testing engagements are often automated through the use of tools and solutions available on the market.
While this has its benefits (consistency, time, and cost), automating penetration testing is not always the right way to go about it.
Ultimately, you need an engagement that is suited best for your organization’s needs, goals, objectives, vulnerabilities, important areas or concerns, regulations, and more. At a minimum, penetration test results should be interpreted for you based on your own risk threshold—and should include an executive summary..
Penetration test results are overrated if they use automation in a way that doesn’t align with your business and security objectives or in a way that doesn’t provide you with an explanation of the results beyond “pass/fail.”
7. They Can Provide a False Sense of Security
I’ll never say that doing well on a penetration test is a bad thing. You’re likely doing some things very well if you’re able to stave off a penetration tester’s advances for a significant period of time or limit what they get access to.
However, having good penetration test results is just a small piece of a much bigger puzzle.
As mentioned before, the parameters of a penetration test can be pretty narrow. It’s that way most often because the organization commissioning the test has a specific area they have in mind for testing. They’ve likely done a lot of work to bolster that portion of their security measures and want to ensure it’s as protected as they think it is.
In cases like these, they’re missing the bigger picture.
Similarly, penetration tests are only looking at the technical side of security. This only covers one portion of a more encompassing look at handling your security measures.
A good security initiative takes into account external and internal technical controls (including web applications), administrative controls (policies, procedures, onboarding, offboarding, etc.), and physical security (locks, cameras, badges, etc.).
If you’re only taking a look at the technical controls (which network pen testing does), you’re not getting a comprehensive look at your entire security posture.
Penetration tests can be powerful tools in understanding where some of the vulnerabilities in your security measures exist.
However, if you’re not selecting an organization or penetration tester that will work within your goals, security objectives, and business objectives, you’re going to end up dissatisfied with penetration test results, engagement time, cost, or all of the above. Additionally, if you’re not using a penetration test as a single initiative in a much grander security plan, you’re likely to miss critical vulnerabilities in areas that are not covered by technical testing.
Moral of the story?
If you’re not going about your penetration testing in the proper way, you’re not setting your organization up for success. Find a security provider who is willing to work with your organization to test your vulnerabilities in a way that makes sense to you, that provides you with expert recommendations and remediation based on the results, and is available to provide you with more security services that will fill the gaps where a penetration test won’t.
If you do those things, your penetration test results will be far from overrated.
For more information on penetration testing, and to find out how to schedule one for your organization, visit frsecure.com.
