I’m angry. The full back story isn’t worth getting into, but there was a company where I applied for a job in the spring of 2013: to build a company’s machine learning infrastructure from scratch. It was a position of technical leadership (Director equivalent, but writing code with no reports) and I would have been able to use Clojure. As it were, I didn’t get it. They were looking for someone more experienced, who’d built those kinds of systems before, and wouldn’t take 6 months to train up to the job. That, itself, is not worth getting angry about. Being turned down happens, especially at high levels.
I found out, just now, that the position was not filled. Not then. Not 6 months later. Not to this day, more than a year later. It has taken them longer to fill the role than it would have taken for me to grow into it.
When they turned me down, it didn’t faze me. I thought they’d found a better candidate. That happens; only thing I can do is make myself better. I found myself, however, a bit irked when I found out that they hadn’t filled the position for longer than it would have taken me to gain the necessary experience. I lost, and so did they.
That’s not what makes me angry. Rationally, I realize that most companies aren’t going to call back a pretty-good candidate they rejected because they had just opened the position and they thought they could do better (if you’re the first 37% of candidates for a job, it makes sense for them not to choose you and, empirically, first and second applicants for a high-level position rarely get it). That’s the sort of potentially beneficial but extremely awkward social process that just won’t happen. What makes me angry is the realization of how common a certain sort of decision is in the technology world. We make a lot of lose-lose decisions that hurt all of us. Extremely specific hiring requirements (that, in bulk, cost the company more in waiting time than training a 90% match up to the role) are just the tip of the iceberg.
You know those people who complain about the lack of decent <gender of sexual interest> but (a) reject people for the shallowest, stupidest reasons, (b) aren’t much of a prize and don’t work to better themselves, and (c) generally refuse to acknowledge that the problem is rooted in their own inflated perception of their market value? That’s how I feel every time I hear some corporate asswipe complain about a “talent shortage” in technology. No, there isn’t one. You’re either too stingy or too picky or completely inept at recruiting, because there’s a ton of underemployed talent out there.
Few of us, as programmers, call the initial shots. We’ve done a poor job of making The Business listen to us. However, when we do have power, we tend to fuck it up. One of the problems is that we over-comply with what The Business tells us it whats. For example, when a nontechnical CEO says, “I only want you to hire absolute rock stars”, what he actually means is, “Don’t hire an idiot just to have a warm body or plug a hole”. However, because they tend to be literal, over-compliant, and suboptimal, programmers will interpret that to mean, “Reject any candidate who isn’t 3 standard deviations above the mean.” The leads to positions not being filled, because The Business is rarely willing to pay what one standard deviation above the mean costs, let alone three.
Both sides now
I’ve been on both sides of the interviewing and hiring process. I’ve seen programmers’ code samples described with the most vicious language over the most trivial mistakes, or even stylistic differences. I’ve seen job candidates rejected for the most god-awful stupid reasons. In one case, the interviewer clearly screwed up (he misstated the problem in a way that made it impossible) but, refusing to risk face by admitting the problem was on his end, he claimed the candidate failed the question. Another was dinged on a back-channel reference (don’t get me started on that sleazy practice, which ought to be illegal) claiming, without any evidence, that “he didn’t do much” on a notable project four years ago. I once saw an intern denied a full-time offer because he lived in an unstylish neighborhood. (The justification was that one had to be “hungry”, mandating Manhattan.) Many of us programmers are so butthurt about not being allowed to sit at the cool kids’ table that, when given the petty power associated with interviewing other programmers, the bitch-claws come out in a major way.
Having been involved in interviewing and recruiting, I’ll concur that there are a significant number of untalented applicants. If it’s 99.5 percent, you’re doing a lot of things wrong, but most resumes do come from people way out of their depth. Moreover, as with dating, there’s an adverse weighting in play. Most people aren’t broken, but broken people go on orders of magnitude more dates than everyone else, which is why most peoples’ dating histories have a disproportionate representation of horror stories, losers, and weirdos. It’s the same with hiring, but phone screening should filter against that. If you’re at all good at it, about half of the people brought in-office will be solid candidates.
Of course, each requirement cuts down the pool. Plenty of companies (in finance, some officially) have a “no job hopper” or “no unemployeds” rule. Many mandate high levels of experience in new technologies (even though learning new technologies is what we’re good at). Then, there are those who are hung up on reference checking in weird and creepy ways. I know of one person who proudly admits that his reference checking protocol is to cold-call a random person (again, back-channel) is the candidate’s network and ask the question, without context, “Who is the best person you’ve ever worked with?” If anyone other than the candidate is named, the candidate is rejected. That’s not being selective. That’s being an invasive, narcissistic idiot. Since each requirement reduces the size of qualified people, it doesn’t take long before the prejudices winnow an applicant pool down to zero.
Programmers? Let’s be real here, we kinda suck…
As programmers, we’re not very well-respected, and when we’re finally paid moderately well, we let useless business executives (who work 10-to-3 and think HashMap is a pot-finding app) claim that “programmer salaries are ridiculous”. (Not so.) Sometimes (to my horror) you’ll hear a programmer even agree that our salaries are “ridiculous”. Fuck that bullshit; it’s factually untrue. The Business is, in general, pretty horrible to us. We suffer under closed allocation, deal with arbitrary deadlines, and if we don’t answer to an idiot, we usually answer to someone else who does. Where does the low status of programmers come from? Why are we treated as cost centers instead of partners in the business? Honestly… much of the problem is us. We’ve failed to manage The Business, and the result is that it takes ownership of us.
Most of the time, when a group of people is disproportionately successful, the cause isn’t any superiority of the average individual, but a trait of the group: they help each other out. People tend to call these formations “<X> Mafia” where X might be an ethnicity, a school, or a company. Y Combinator is an explicit, pre-planned attempt to create a similar network; time will tell if it succeeds. True professions have it. Doctors look out for the profession. With programmers, we don’t see this. There isn’t a collective spirit: just long email flamewars about tabs versus spaces. We don’t look out for each other. We beat each other down. We sell each other out to non-technical management (outsiders) for a shockingly low bounty, or for no reason at all.
In many investment banks, there’s an established status hierarchy in which traders and soft-skills operators (“true bankers”) are at the top, quants are in the middle, and programmers (non-quant programmers are called “IT”) are even lower. I asked a high-ranking quant why it was this way, and he explained it in terms of the “360 degree” performance reviews. Bankers and traders all gave each other top ratings, and wrote glowing feedback for minor favors. They were savvy enough to figure out that it was best for them to give great reviews up, down, and sideways, regardless of their actual opinions. Quants tended to give above-average ratings and occasionally wrote positive feedback. IT gave average ratings for average work and plenty of negative feedback. The programmers were being the most honest, but hurting each other in the process. The bankers and traders were being political, and that’s a good thing. They were savvy enough to know that it didn’t benefit them to sell each other out to HR and upper management. Instead, they arranged it so they all got good ratings and the business had to, at baseline, appreciate and reward all of them. While it might seem that this hurt top performers, it had the opposite effect. If everyone got a 50 percent bonus and 20% raise, management had to give the top people (and, in trading, it’s pretty obvious who those are) even more.
Management loves to turn high performers against the weak, because this enables management to be stingy on both sides. The low performers are fired (they’re never mentored or reassigned) and the high performers can be paid a pittance and still have a huge bonus in relative terms (not being fired vs. being fired). What the bankers were smart enough to realize (and programmers, in general, are not) is that performance is highly context-driven. Put eight people of exactly equal ability on a team to do a task and there will be one leader, two or three contributors, and the rest will be marginal or stragglers. It’s just more efficient to have the key knowledge in a small number of heads. Open source projects work this way. What this means is that, even if you have excellent people and no bad hires, you’ll probably have some who end up with not much to show for their time (which is why open allocation is superior; they can reassign themselves until they end up in a high-impact role). If management can see who is in what role, it can fire the stragglers and under-reward the key players (who, because they’re already high performers, are probably motivated by things other than money… at least, for now). The bankers and traders (and, to a lesser extent, the quants) had the social savvy and sense to realize that it was best that upper management not know exactly who was doing what. They protected each other, and it worked for them. The programmers, on the other hand, did not, and this hurt top performers as well as those on the bottom.
Let’s say that an investment bank tried to impose tech-company stack ranking on its employees, associate level and higher. (Analyst programs are another matter, not to be discussed here.) Realizing the mutual benefit in protecting each other, the bankers would find a way to sabotage the process by giving everyone top ratings, ranking the worst employees highly, or simply refusing to do the paperwork. And good for them! Far from being unethical, this is what they should do: collectively work The Business to get what they’re actually worth. Only a programmer would be clueless enough to go along with that nonsense.
In my more pessimistic moods, I tend to think that we, as programmers, deserve our low status and subordinacy. As much as we love to hate those “business douchebags” there’s one thing I will say for them. They tend to help each other out a lot more than we do. Why is this? Because they’re more political and, again, that might not be a bad thing. Ask a programmer to rate the performance of a completely average colleague and you’ll get an honest answer: he was mediocre, we could have done without him. These are factual statements about average workers, but devastating when put into words. Ask a product manager or an executive about an average colleague and you’ll hear nothing but praise: he was indispensable, a world-class player, best hire in ten years. They realize that it’s politically better for them, individually and as a group, to keep their real opinions to themselves and never say anything that could remotely endanger another’s career. Even if that person’s performance was only average, why make an enemy when one can make a friend?
“Bad code”
Let’s get to another thing that we do, as programmers, that really keeps us down. We bash the shit out of each other’s code and technical decision-making, often over minutiae.
I hate bad code. I really do. I’ve seen plenty of it. (I’ve written some, but I won’t talk about that.) I understand why programmers complain about each other’s code. Everyone seems to have an independent (and poorly documented) in-head culture that informs how he or she writes code, and reading another person’s induces a certain “culture shock”. Even good code can be difficult to read, especially under time pressure. And yes, most large codebases have a lot of code in them that’s truly shitty, sometimes to the point of being nearly impossible to reason about. Businesses have failed because of code quality problems, although (to tell the whole story) it’s rare that one bad programmer can do that much damage. The worst software out there isn’t the result of one inept author, but the result of code having too many authors, often over years. It doesn’t help that most companies assign maintenance work to either to junior programmers, or demoted (and disengaged) senior ones, neither category having the power to do it right.
I’d be the last one to come out and defend bad code. That said, I think we spend too much time complaining about each other’s code– and, worse yet, we tend toward the unforgivable sin of complaining to the wrong people. A technical manager has, at least, the experience and perspective to know that, at some level, every programmer hates other peoples’ code. But if that programmer snitches to a non-technical manager and executive, well… you’ve just invited a 5-year-old with a gun to the party. Someone might get fired because “tabs versus spaces” went telephone-game into “Tom does shoddy work and is going to destroy the business”. Because executives are politically savvy enough to protect the group, and only sell each other out in extreme circumstances, what started out as a stylistic disagreement sounds (to the executive ear) like Tom (who used his girlfriend’s computer to fix a production problem at 11:45 on a Friday night, the tabs/spaces issue being for want of an .emacs.d) is deliberately destroying the codebase and putting the whole company at risk.
As programmers, we sell each other out all the time. If we want to advance beyond reasonable but merely upper-working class salaries, and be more respected by The Business, we have to be more careful about this kind of shit. I’ve heard a great number of software engineers say things like, “Half of all programmers should just be fired.” Now, I’ll readily agree that there are a lot of badly-trained programmers out there whose lack of skill causes a lot of pain. But I’m old enough to know that people come to a specific point from a multitude of paths and that it’s not useful to personalize this sort of thing. Also, regardless of what we may think as individuals, almost no doctor or banker would ever say, to someone outside his profession, “half of us should be fired”. They’re savvy enough to realize the value of protecting the group, and handling competence and disciplinary matters internally. Whether to fire, censure, mentor or praise is too important a decision to let it happen outside of our walls.
There are two observations about low-quality code, one minor and one major. The minor one is that code has a “all of us is worse than any of us” dynamic. As more hands pass over code, it tends to get worse. People hack the code needing specific features, never tending to the slow growth of complexity, and the program evolves over time into something that nobody understands because too many people were involved in it. Most software systems fall to pieces not because of incompetent individuals, but because of unmanaged growth of complexity. The major point on code-quality is: it’s almost always management’s fault.
Bad code comes from a multitude of causes, only one of which is low skill in programmers. Others include unreasonable deadlines, unwillingness to attack technical debt (a poor metaphor, because the interest rate on technical “debt” is both usurious and unpredictable), bad architecture and tooling choices, and poor matching of programmers to projects. Being stingy, management wants to hire the cheapest people it can find and give them the least time possible in which to do the work. That produces a lot of awful code, even if the individual programmers are capable. Most of the things that would improve code quality (and, in the long term, the health and performance of the business) are things that management won’t let the programmers have: more competitive salaries, more autonomy, longer timeframes, time for refactoring. The only thing that management and the engineers can agree on is firing (or demoting, because their work is often still in use and The Business needs someone who understands it) those who wrote bad code in the past.
One thing I’ve noticed is that technology companies do a horrible job of internal promotion. Why is that? Because launching anything will typically involve compromises with the business on timeframe and headcount, resulting in bad code. Any internal candidate for a promotion has left too many angles for attack. Somewhere out there, someone dislikes a line of code he wrote (or, if he’s a manager, something about a project he oversaw). Unsullied external candidates win, because no one can say anything bad about them. Hence, programming has the culture of mandatory (but, still, somewhat stigmatized) job hopping we know and love.
What’s really at the heart of angry programmers and their raging against all that low-quality code? Dishonest attribution. The programmer can’t do shit about the dickhead executive who set the unreasonable deadlines, or the penny-pinching asswipe managers who wouldn’t allow enough salary to hire anyone good. Nor can he do much about the product managers or “architects” who sit above and make his life hell on a daily basis. But he can attack Tom, his same-rank colleague, over that commit that really should have been split into two. Because they’re socially unskilled and will generally gleefully swallow whatever ration of shit is fed to them by management, most programmers can very easily be made to blame each other for “bad code” before blaming the management that required them to use the bad code in the first place.
Losers
As a group, software engineers are losers. In this usage, I’m not using the MacLeod definition (which is more nuanced) and my usage is halfway pejorative. I generally dislike calling someone a loser, because the pejorative, colloquial meaning of that word conflates unfortunate circumstance (one who loses) with deserved failure. Here, however, it applies. Why do we lose? Because we play against each other, instead of working together to beat the outside world. As a group, we create our own source of loss.
Often, we engage in zero- or negative-sum plays just to beat the other guy. It’s stupid. It’s why we can’t have nice things. We slug each other in the office and wonder why external hires get placed over us. We get into flamewars about minutiae of programming languages, spread FUD, and eventually some snot-nosed dipshit gets the “brilliant” idea to invite nontechnical management to weigh in. The end result is that The Business comes in, mushroom stamps all participants, and says, “Everything has to be Java“.
Part of the problem is that we’re too honest, and we impute honesty in others when it isn’t there. We actually believe in the corporate meritocracy. When executives claim that “low performers” are more of a threat to the company than their astronomical, undeserved salaries and their doomed-from-the-start pet projects, programmers are the only people stupid enough to believe them, and will often gleefully implement those “performance-based” witch hunts that bankers would be smart enough to evade (by looking for better jobs, and arranging for axes to fall on people planning exits anyway). Programmers attempt to be apolitical, but that ends up being very political, because the stance of not getting political means that one accepts the status quo. That’s radically conservative, whether one admits it or not.
Of course, the bankers and traders realize the necessity of appearing to speak from a stance of professional apolitical-ness. Every corporation claims itself to be an apolitical meritocracy, and it’s not socially acceptable to admit otherwise. Only a software engineer would believe in that nonsense. Programmers hear “Tom’s not delivering” or “Andrea’s not a team player” and conceive of it as an objective fact, failing to recognize that, 99% of the time, it means absolutely nothing more or less than “I don’t like that person”.
Because we’re so easily swayed, misled, and divided, The Business can very easily take advantage of us. So, of course, it does. It knows that we’ll sell each other out for even a chance at a seat at the table. I know a software engineer who committed felony perjury against his colleagues just to get a middle-management position and the right to sit in on a couple of investor meetings. Given that this is how little we respect each other, ourselves, and our work, is it any wonder that software engineers have such low status?
Our gender issues
I’m going to talk, just briefly, about our issues with women. Whatever the ultimate cause of our lack of gender diversity– possibly sexism, possibly that the career ain’t so great– it’s a major indictment of us. My best guess? I think sexism is a part of it, but I think that most of it is general hostility. Women often enter programming and find their colleagues hostile, arrogant, and condescending. They attribute that to their gender, and I’m sure that it’s a small factor, but men experience all of that nonsense as well. To call it “professional hazing” would be too kind. There’s often nothing professional about it. I’ve dealt with rotten personalities, fanaticism about technical preference or style, and condescension and, honestly, don’t think there’s a programmer out there who hasn’t. When you get into private-sector technology, one of the first things you learn is that it’s full of assholes, especially at higher levels.
Women who are brave enough to get into this unfriendly industry take a look and, I would argue, most decide that it’s not worth it to put up with the bullshit. Law and medicine offer higher pay and status, more job security, fewer obnoxious colleagues, and enough professional structure in place that the guy who cracks rape jokes at work isn’t retained just because he’s a “rockstar ninja”.
“I thought we were the good guys?”
I’ve often written from a perspective that makes me seem pro-tech. Originally, I approached the satirical MacLeod pyramid with the belief that “Technocrat” should be used to distinguish positive high-performers (apart from Sociopaths). I’ve talked about how we are a colonized people, as technologists. It might seem that I’m making businesspeople out to be “the bad guys” and treating programmers as “the good guys”. Often, I’m biased in that very direction. But I also have to be objective. There are good business people out there, obviously. (They’re just rare in Silicon Valley, and I’ll get to that.) Likewise, software engineers aren’t all great people, either. I don’t think either “tribe” has a monopoly on moral superiority. As in Lost, “we’re the good guys” doesn’t mean much.
We do get the worst (in terms of ethics and competence) of the management/business tribe in the startup world. That’s been discussed at length, in the essay linked above. The people who run Silicon Valley aren’t technologists or “nerds” but machiavellian businessmen who’ve swooped in to the Valley to take advantage of said nerds. The appeal of the Valley, for the venture capitalists and non-technical bro executives who run it, isn’t technology or the creation of value, but the unparalleled opportunity to take advantage of too-smart, earnest hard workers (often foreign) who are so competent technically that they often unintentionally generate value, but don’t know the first thing about how to fight for their own interests.
It’s easy to think ourselves morally superior, just because the specific subset of business people who end up in our game tends to be the worst of that crowd. It’s also a trap. We have a lot to learn form the traders and bankers of the world about how to defend ourselves politically, how to stand a chance of capturing some of the value we create, and how to prevent ourselves from being robbed blind by people who may have lower IQs, but have been hacking humans for longer than we could have possibly been using computers. Besides, we’re not all good. Many of us aren’t much better than our non-technical overlords. Plenty of software engineers would gladly join the bad guys if invited to their table. The Valley is full of turncoat software engineers who don’t give a shit about the greater mission of technology (using knowledge to make peoples’ lives better) and who’d gladly sell their colleagues out to cost-cutting assholes in management.
Then there are the losers. Losers aren’t “the bad guys”. They don’t have the focus or originality that would enable them to pull off anything complicated. Their preferred sin is typically sloth. They’ll fail you when you need them the most, and that ‘s what makes them infuriating. They just want to put their heads down and work, and the problem is that they can’t be trusted to “get political” when that’s exactly what’s needed. The danger of losers is in numbers. The problem is that so many software engineers are clueless, willing losers who’ll gladly let political operators take everything from them.
When you’re young and don’t know any better, one of the appeals of software engineering is that it appears, superficially, to tolerate people of low social ability. To people used to artificial competition against their peers, this seems like an attractive trait of the industry; it’s not full of those “smooth assholes” and “alpha jocks”. After several years observing various industries, I’ve come to the conclusion that this attitude is not merely misguided, but counterproductive. You want socially skilled colleagues. Being the biggest fish in a small pond just means that there are no big fish to protect you when the sharks come in. Most of those “alpha jocks” aren’t assholes or idiots (talk to them, nerds; you’ll be surprised) and, when The Business comes in and is looking for a fight, it’s always best to have strong colleagues who’ve got your back.
Here’s an alternate, and quite possible hypothesis: maybe The Business isn’t actually full of bad guys. One thing that I’ve realized is that people tend to push blame upward. For example, the reputation of venture capitalists has been harmed by founders blaming “the VCs” for their own greed and mismanagement. It gives the grunt workers an external enemy, and the clueless can be tricked into working harder than they should (“they don’t really like us and haven’t given us much, but if we kill it on this project and prove them wrong, maybe they’ll change their minds!”). It actually often seems that most of the awfulness of the software industry doesn’t come directly from The Business, but from turncoat engineers (and ex-engineers) trying to impress The Business. In the same way that young gang members are more prone to violence than elder dons, the most creative forms of evil seem to come from ex-programmers who’ve changed their colors.
The common enemy
So long as software engineers can easily be divided against each other on trivial matters like tabs versus spaces and scrotum versus kanban, we’ll never get the respect (and, more importantly, the compensation) that we’re due. These issues distract us from what we really need to do, which is figure out how to work The Business. Clawing at each other, each trying to become the favored harem queen of the capitalist, is suboptimal compared to the higher goal of getting out of the harem.
I’ve spoken of “The Business” as if it were a faceless, malevolent entity. It might sound like I’m anti-business, and I’m not. Business is just a kind of process. Good people, and bad people, start businesses and some add great value to the world. The enemy isn’t private enterprise itself, but the short-term thinking and harem-queen politics of the established corporation. Business organizations get to a point where they cease having a real reason to exist, and all that’s left is the degenerate social contest for high-ranking positions. We, as programmers, seem to lack the skill to prevent that style of closed-allocation degeneracy from happening. In fact, we seem to unintentionally encourage it.
The evil isn’t that software is a business, but that technical excellence has long since been subordinated entirely to the effectively random emotional ups and downs of non-technical executives who lack the ability to evaluate our work. It’s that our weird ideology of “never get political” is actually intensely political and renders us easy to abuse. Business naturally seems to be at risk of anti-intellectual tendencies and, rather than fight back against this process, we’ve amplified it just to enjoy the illusion of being on the inside, among the “cool kids”, part of The Business. Not only does our lack of will to fight for our own interests leave us at the mercy of more skilled business operators, but it attracts an especially bad kind of them. Most business people, actually, aren’t the sorts of corporate assholes we’re used to seeing run companies. It’s just that our lack of social skill appeals to the worst of that set: people who come in to technology to take advantage of all the clueless, loser nerds who won’t fight for themselves. If we forced ourselves to be more discerning judges of character, and started focusing on ethics and creativity instead of fucking tabs-versus-spaces, we might attract a better sort of business person, and have an industry where stack ranking and bastardized-“Agile” micromanagement aren’t even considered.
If we want to improve our situation, we have to do the “unthinkable” (which is, as I’ve argued, actually quite thinkable). We have to get political.
