Programmer salaries in the age of LLMs
What happens to the distribution of programmer salaries in the age of LLMs? I argue they will separate bimodally, much like what happened to lawyers’ salaries in the 1990s due to the rise of the Internet.
Dan Luu has previously written about lawyers’ salaries. I will paste some graphs from his article. First, lawyers in 1991:
Next, lawyers in 2000:
What happened? Well, if you’re an elite lawyer in the modern age, you probably command a large team of lower-tier lawyers, cheap paralegals, and so on, who are very good at using the Internet to look up case law and draft your opinions and so on. In contrast, if you’re an elite lawyer in the year 1990, it’s much more valuable for you specifically to have an encyclopedic knowledge of all the relevant case law, and it’s much harder for you to delegate modular pieces of work off to less competent people.
Indeed Cowen makes a similar point about lawyers in Who gains and loses from the new AI? where he writes:
The returns to factual knowledge are falling, continuing a trend that started with databases, search engines and Wikipedia. It is no longer so profitable to be a lawyer who knows a large amount of accumulated case law. Instead, the skills of synthesis and persuasion are more critical for success.
I claim that the trend which AI/ML continues for lawyers is one that it starts for programmers. Just like how a partner at Cravath likely sketches an outline of how they want to approach a particular case and swarms of largely replaceable lawyers fill in the details, we are perhaps converging to a future where a FAANG L7 can just sketch out architectural details and the programmer equivalent of paralegals will simply query the latest LLM and clean up the output. Note that querying LLMs and making the outputted code conform to specifications is probably a lot easier than writing the code yourself ー and other LLMs can also help you fix up the code and integrate the different modules together!
More generally, the farther such technologies advance, the more existing technical professions will undergo a sort of “paralegalization” and bifurcation of the talent distribution.
Somewhat-above-average programmers (including myself) had a decent run of it, but it may be over for us before too long.
P.S. If your objection is that you played with ChatGPT and wasn’t impressed, you should try projecting trends beyond the t=1 month timescale.
January 16th, 2023 at 7:58 pm
[…] Programmer salaries in an age of LLMs. And cybercriminals starting to use […]
January 17th, 2023 at 4:27 am
This model for programmers isn’t new, it was already in place 20 years ago at least as early as the dot com bubble. Even then there was an small group of “architectss” and countless nameless “coders”, often in least/best cost countries. This trend is likely to continue.
Another example of this trend is the medical professions with physician assistants, nurse practitioners, and a slew of other para-medical personnel coming to the fore.
January 17th, 2023 at 7:44 am
[…] Programmer salaries in the age of LLMs […]
January 17th, 2023 at 10:11 am
This was the thinking behind software such as Rational Rose in the 1990s: the clever programmer will come up with the class structure, and the others will fill in the blanks.
This has fallen out of favour due to its incompatibility with reality – even the purportedly simplest chunk of code can have a massive blast radius if you write it badly enough and will need reviewing.
The knowledge to review it is the same as it is to write it, so I don’t see that reducing the demand for decent programmers.
The law on the other hand was, as you say, hinging on having a prodigious memory. Programming has never been like that; it’s only been valuable for people to be able to understand a domain, translate it into something digital, and build that digital thing.
January 17th, 2023 at 1:10 pm
> P.S. If your objection is that you played with ChatGPT and wasn’t impressed, you should try projecting trends beyond the t=1 month timescale.
An obnoxious way of expressing a dismissive stance.
You are acting as if an unknowable outcome is fact.
January 17th, 2023 at 1:22 pm
It would have been nice if your article actually explained what an LLM is.
January 17th, 2023 at 4:38 pm
I think of LaLMs as being a positive supply shock to the supply of SWEs. But to know how this will impact SWEs’ wages we need to know the demand curves: depending in the elasticity of demand for SWEs their earnings, at a particular skill level, may go up or down when faced with a 2x increase of productivity. It may be (probably will be) that relative earnings between SWEs at different skill levels will change but without knowing demand curves, or how LaLMs will differentially impact different skill levels, no one can predict in which direction.
You say that that 60% percentile SWEs may be negatively impacted. Or maybe: LaLMs will increase your productivity enough that you + LaLM ~= L7 + LaLM so, actually, you earn more and an L7 earns less. Or maybe there’s so much software to write that everyone earns more. Or maybe productivity goes up a lot, the amount of software written goes up a lot, and SWE wages go down because SWEs’ productivity outpaces the elasticity of demand.
So: I have no idea if SWEs will earn more or less but I’m quite certain that more software will be produced. And the job of writing that software will fundamentally change.
January 17th, 2023 at 7:44 pm
I don’t really agree with this. It’s very hard to know how salary distributions will change in time but it’s not really that meaningful anyway. Why? Because what’s a software developer? Or what’s a data scientist or whatever? They are just job titles. You don’t really want to use job titles as the filter for selecting a group. Data scientists for example, aren’t what they used to be. There was a huge demand for them so we lowered the bar for what type of person could be called a data scientist. And that’s all it took for stabilization (perhaps even decline) in data scientist salary. But that doesn’t mean that the people who made $150K 5 years ago don’t make $300K today. Their career opportunities are better than ever. It doesn’t matter who they share job titles with. Software developer jobs have also gone through plenty of change and specialization. So it’s not going to be meaningful to compare salary distributions over long periods of time. It’s not like law where a law degree is a well defined stable criteria and law itself is a somewhat static industry.
Surely there will be more specialization and gradation in software jobs and new job titles but I don’t think it’s worth worrying about. What matters for anyone’s job prospects and salary is supply of people with skills like theirs and demand for those skills. These technologies just make developers of any kind more productive. And software unlike law or food production of whatever do not have fixed demand. The cheaper it gets to create software, the more people are going to want software. The more we can do with software, the more innovative products there will be requiring it.
It is never a bad thing to be in a profession that is on the cutting of edge of technology. So just make sure you stay at that cutting edge. Make sure that leverage any new technology that comes along to make you more productive.
And really, it’s not usually worth trying to forecast where salaries are heading in general. No one knows. The price of things in a market for anything are themselves the signals that tell people what to do and where to specialize. People adjust to these things in real time. If someone will pay them more to do something else, they go and do something else.
January 17th, 2023 at 8:17 pm
There are still a lot of unknowns with it comes to whether or not the output of an LLM is copyright infringement. A recent episode of All In discussed this. Once we are required to pay for an LLM’s output, it could trigger a bunch of lawsuits and might kill a large number of LLMs trained on publicly available / possibly protected material.
How useful is an LLM that is trained on data the company had to pay for?
January 17th, 2023 at 9:24 pm
In my opinion since Google dominated the search engine market, did anyone using it ever reflect if the results are useful and why they are ranked the way they were? no probably not, so Google used Bots and provided a list with output imagine apple will combine googles search engine with graphic design and help of ChatGPT that will be a new era since machine learning is not 100% reproducible there will be a shift in the industry as for mechanics who now use tablets for predictive maintenance AI will assert coders to improve quality and other aspects, critical thinking will still remain in huma hand.
January 17th, 2023 at 11:31 pm
quality and quantity will decide
yes, LLMs will create a quantity of code, but, its ability to meet functional requirements will be determined by the quality of the design.
i suspect there will be many variants of any LLM produced code, and, keeping track of which is the most useful and why wil be a whole new discipline.
LLMs will not be free, and, if the function of the code being created relies upon igneous thought in design that ingenuity will not be easily created by the current style of LLMs / GPT systems.
after all, the AI fanatics still insist that there will more jobs created by AI than dispense with.
i personally feel there will be fewer jobs and only super humans will actually get paid.
further growth in inequality in society will be the real output, the coming impact of AI on lawyers is not discussed here as it hasn’t happened yet
January 18th, 2023 at 5:45 am
Very interesting post! From a statistical perspective, I find the graphs interesting in that the spread increased as well. So yes, it’s bimodal, but I’d say 30-40% are actually in the area between the two modes. That could be evidence against the claim of a discrete boundary between “higher-level” and “lower-level” salaries in programming.
From a story perspective, the story does make sense. An analogue would be in manufacturing, where skilled labor went towards design of objects or design of processes rather than the actual work of making objects or running processes.
However, I think the end of that story is also worth questioning, and this goes back to that sizeable middle in the second graph. Above-average engineers are going to have more impact with LLMs. So why wouldn’t they build more quickly, execute more milestones, release more features, and thus move up?
In other words, the charts above called out the median. What about the average?
January 19th, 2023 at 11:46 am
“Somewhat-above-average programmers (including myself) had a decent run of it” – Talk about the arrogance in it!
I strange trend I noticed is that, all the crypto fans or bros somehow magically shifted over to the AI hype now that cryptos lost its ploy. AI advances have been long going on under the hood in academia and industries, its only now the hype because of something released publicly. That does not change anything, good developers or Somewhat-above-average programmers are always adapting, and its nothing that will significantly affect salaries.