A short while back, a discussion broke out on a mailing list about the nature of being a pen-tester. The discussion quickly gravitated towards the number of “security” companies where numbers of projects far out-weigh the interestingness of projects, leading rapidly to a cookie-cutter mentality to pen-test engagements..

Of course if you have spent any time in the industry, you already know this to be true.. the obvious danger with this is that you have a lot of unhappy pen-testers giving shoddy output to (eventually) very unhappy customers. Sadly this soon follows the well published “market for lemons” problem where eventually due to information asymmetry, bad products will soon push out good ones.. i.e. because its hard for customers to tell the difference between good pen-tests and lame pen-tests, eventually the market price drops towards low grade pen-tests (since the customer is paying for what they expect) and at the low prices, good pen-test teams will close shop and move on to other lines of work..

The list discussion was dominated by guys who had been in the pen-testing game for years who contended that “you sign up for a cool dynamic job pushing the envelope, and you end up running scannerX for the next Y years of your life..”

I replied with a quasi essay, mainly because we have had these sorts of discussions at the office for years.. i think the bottom line is that if you in the right company, just about any line of work will be ok, and if you in the wrong company just about any project can be made to suck.. Paul Graham once covered this when describing great hackers.. a piece that sometimes went over the edge, but mostly warms the heart..

We handle the problem of cookie-cutter projects by following 2 simple rules:

a) we always try hard to push the envelope,

b) we always double and triple check to make sure we are adding value.

(a) is the easy one (relatively) and we are really fortunate because over the years we have ended up with a culture that reinforces this sort of behavior. Although the office has great teamwork (which you can easily judge by the number of people helping other people on project till the wee hours of the morning) it also has a healthy amount of competition. Everyone wants to pull the next great piece of leetness, and everyone works hard for it.. In time it reaches the state where you almost feel dirty for not getting it..
(b) is also relatively simple.. We have had a number of projects over the years where we have literally turned down business, because even though the customer thought he needed us, at that point we didnt think he did.. we felt his money was better spent at that point in time doing something else, and we pointed him towards it.. its a win for the customer who doesnt just walk away with a cool report he is unable to effectively work on and its a win for the analyst who doesnt end up with his work being in vein.

Now both come with obvious downsides.. (b) wouldnt please short sighted investors / business folks and (a) is cool for some but breaks some people to pieces..

My answer for both is the same (but is clearly my opinion.. blah blah std disclaimer.h): Let them go elsewhere..

If your business folks / investors dont realise that eventually customers like this would be happier, and that the resultant goodwill will hold your company in better stead than the short term gains they get from just that piece of work, then you guys were probably going to kill each other before long anyway..

If your analysts dont wake up itching to go, and wanting to dent the world then they probably not right for your company either.. there are lots of jobs in the world where people can kinda half-ass their way through the day but thats not the company you want to work for.. so you and ultimately they (the half-assers) will probably be happier with them finding one of those companies instead..

Doing good work constantly, and constantly pushing the limits are def. their own reward, and most people i know who live like that, probably dont know how to live any other way, but make no mistake it _is_ hard.. Being good at something could happen with luck (good genes, good background, etc) but being great at something and being consistently great at it _does_ take work.. anyone who says otherwise is probably still just sailing along on his luck component that never lasts forever..

(i first blogged some posts to this effect waaaaaaaaaay back.. the original posts reference Richard Hammings – you and your research paper which always bears repeating)

OK.. now we bump into another pen-testers dilemma.. a good friend of mine and i used to have this discussion often a few years ago where we pondered how come if we were breaking into banks / huge companies all the time, that this was not happening more often? a logical conclusion has to be that such attacks are not as likely as we imagined and that our work while fun was largely academic. Around 2003 this argument was at its peak since our attacks were increasingly more in-depth, with the sorts of attacks required to achieve our goal needing a level of technical skill far greater than the level of attacks actually being discovered in the wild.

It was a low time for the argument on my side (i always held that what we do actually does add value despite the attacks being arcane..) and of course we started to question the value we were adding.. if the number of people likely to pull off such attacks were increasingly small (due to the complexity level of the attack) then surely the customers wouldn’t really need to pay good money to find this out.. they could just pay for the cheap assessment and mitigate the threats at the level of the attackers they are likely to face..

Of course.. as time went on this started to change.. Attacker sophistication is always on the rise, and attacks that were once publicly decried as a black-art, are point and click weaponised a few years later.. Customers that have been protecting their code / networks / etc from these attacks (because you have been using it against them since 2002/3) are much better off, and much further down the path because you took them there years ago.. the argument swings again in my favor.. :>

Ultimately.. for me its pretty simple.. do it cause you love it.. do it as hard as you can.. make sure your company and you are on the same page.. and make sure you adding value.. the rest just falls into place..