Adam Caudill

Security Leader, Researcher, Developer, Writer, & Photographer

Millions of Jobs

or: On AI, Job Creation & Destruction, and The Race to Oblivion

Image: Photo by Jenessaa Lu on Unsplash

It has been 20 years since I first used machine learning to solve a complex business problem. The underlying problem was simple: the company was selling a new service and wanted to know who was most likely to buy it. We had millions of records, and each record had hundreds of fields. A vast amount of data, but no idea how to extract insight from it. Countless hours from various data analysts had been invested into finding a pattern, but none was forthcoming. Months turned into years, still no idea of who was most likely to actually buy. Enter machine learning. Over the course of a couple hours one evening, I was testing new techniques with machine learning, and there was the answer. The company met it’s entire monthly sales goal the next day. It found what every human had missed.

This was long before machine learning was rebranded as Artificial Intelligence (AI), but I couldn’t deny the power and potential that it could offer. It was also obvious that once it evolved to the point that it could be broadly applied, it would be a revolution.

We are racing towards that revolution, though to understand what lies ahead, we must first spend a little time understanding the past. This revolution isn’t without precedent, and that precedent is vital to understanding what comes next.

The Industrial Revolution #

Going back as far as 1760 may seem irrelevant when talking about the latest technologies that are still in their nascent stages, but the situation was actually quite similar. In a relatively short period of time, new technologies completely changed the way most people worked, how they were paid, how businesses functioned, and both created and destroyed vast numbers of jobs. Without understanding this period of time, without understanding our history, it’s impossible to understand our future.

Looking at textile workers during this time is particularly illustrative. Most of those that spun cotton and wove it into fabrics worked from their homes, working by hand, using simple tools like spinning wheels, slowly and steadily turning raw cotton into a useful product. Over the course of a surprisingly small number of years, some of these workers were in large factories instead, operating dangerous machines, and doing the work of 500 people. Yes, you read that right. A new technology that replaced 500 people that had honed their skills over years, replaced by a machine and a single person. Of those that weren’t hired to keep these machines fed were forced to look for other work, or effectively left the workforce entirely.

This 500 to 1 ratio for cotton spinning is likely the most drastic, and it would be disingenuous to say that these losses were entirely permanent. As these new technologies paid for themselves, businesses invested in more machines to expand their production capacity, which meant hiring more people to keep them fed, plus the occasional work to maintain and repair them. The jobs being created were very different than those they eliminated, with different skill levels, and different pay. The fact that it enabled Britain to effectively compete with India in the production of textiles, despite the fact that wages in India were 1/6th of those in Britain at the time, should tell you everything you need to know about the pay of these new jobs.

Turning this cotton thread into woven fabrics similarly saw massive changes, with a 40 to 1 ratio. A skill carefully developed, replaced by a machine that would do the work of 40 people. This is repeated countless times across various industries, with the new technologies enabling substantially greater efficiency, while costing countless jobs, and creating some number of new, but different, jobs.

This went on, waxing and waning, all the way up to 1913, when manufacturing found the optimal form in Henry Ford’s assembly lines. Turning cars from hand-built & carefully crafted machines, into the ubiquitous mass-produced vehicles we know today.

The trend here is clear and simple: revolutionary technologies and techniques are highly effective at replacing large numbers of workers with specific knowledge and skills, with a smaller number of less skilled workers. Plus creating a few highly skilled workers to support these technologies. Nothing about this is really surprising, and should be well understood, but it’s a preview of what the future holds.

Quiet Part, Out Loud #

A world in which human wages crash from AI – logically, necessarily – is a world in which productivity growth goes through the roof, and prices for goods and services crash to near zero. Consumer cornucopia. Everything you need and want for pennies. - Marc Andreessen

The above quote is from a highly active investor in AI companies, and one of the most influential voices in the venture capital community. Before going into the quote, and why this is saying the quiet part out loud, we need to cover some background.

There are two general pricing models for goods and services, the first is the cost to provide the product or service, plus a (small) margin for profit. The second is whatever the market will pay, regardless of cost. In reality, the cost + margin model provides only a floor for what can be charged, while most pricing is based on what the market will pay, optimised for leaving as little money on the table as possible, unless there is so much competitive pressure that the price must be reduced to that floor.

While your milk and eggs have enough competitive pressure to reduce pricing down to cost + margin, while the medications you may take, the electronics you buy, and most other products that you buy are based on what the market will allow, what people are willing to pay. There’s a reason that some medications constantly go up in price and are priced at hundreds of times the manufacturer’s cost - because they can get it. This is the same reason that an Apple macBook costs two to three times that of a PC with equivalent performance - they can get it.

Why is this at all important?

Marc’s point that prices will drop substantially after wages collapse is based on a flawed and dangerous assumption, that goods will be priced on the cost + margin model alone. That simply isn’t how the world works. There’s also the unmentioned issue that to people with no income, it doesn’t matter how cheap products become.

He is likely correct that wages will collapse, that is largely the point. AI, as it’s being developed today, has one purpose: improve efficiency to the point that jobs can be eliminated. This provides allows provides to charge more and more for their products, and results in huge returns for investors.

How wages are likely to collapse is important to consider, as it will almost certainly be a very unevenly distributed collapse. The average income for any given region is likely to drop precariously, primarily as a result of job eliminations. However, we’ve seen that for some roles, such as those building these AI systems are climbing, and there’s no reason to think that will change. Those that can’t be replaced will see great demand and high pay, those that can be replaced, will be left to scramble to find new work.

I believe that there’s good reason to believe this type of stratification will be seen across the board, in all areas impacted by these AI initiatives. Those with certain skills seeing strong demand and high pay, especially at the most senior roles, with greater difficultly for those seeking to enter impacted areas.

Impacts & Future Challenges #

Based on the impacts we’re seeing today, and the already signalled advances that we can safely assume will come, there are some things that we can began to make assumptions about, with varying levels of uncertainty and timelines to see these in full effect.

I’d like to talk about those that have been on my mind as I researcher more into where these advances are going. This is far from exhaustive, and may turn out to be incorrect (though I know what outcomes I’d bet on). There’s also a timeline issue that I’d like to acknowledge: some of these could occur next year, others may be a decade away. This is not a short-term analysis, but a long-term look at how these technologies are likely to evolve and impact people.

Software Development #

Generally speaking, there are two kinds of developers:

  • Those that carefully craft their code, attend or speak at conferences, write blog posts pontificating on the correct way to build software, and would often rather leave their job than be forced to write bad code1.
  • The other 95%.

The fact is that the vast majority of developers are buried deep in companies, facing tight deadlines, primarily work on line-of-business software, and don’t have time to worry about the best way to do something, because they are too busy just keeping the business running. This silent majority are both the most likely to leverage AI to make their jobs easier, and the most likely to face downsizing as those efficiency improvements allow smaller teams to do the same work.

We are seeing the first steps of what AI will do to software development. The tooling will improve, accuracy will improve, higher-level tools will be built on top of the code generation used today. Many common line-of-business application will eventually be able to be (mostly) generated, needing more oversight and review than active development. It will take a lot longer before the tools become good enough that they’ll start displacing those building commercial software, but commercial software is only a small portion of the software being built today.

Within the next decade, it’s likely that more and more development teams will be replaced by a single business analyst using generative AI to create the internal software powering companies. This builds on trends that have been going on for a decade or more already, though will almost certainly be greatly accelerated in terms of impact.

Creative Arts & Writing #

Not long ago, I had an idea for a short story, but it was too ridiculous to invest the time in writing it2, so I did the next best thing: I asked ChatGPT to do it. I provided it with the abstract and a couple sentences to explain the plot and message. I waited and expected to get a laugh. What I got was a shock. It was clear, expanded nicely on the idea, and would have passed for thoughtful and creative. I spent months wondering if there was actually a point in writing after this. It did in seconds what I would have spent weeks doing.

This experience if far from unique, and business leaders are seeing that in many cases they can get ‘good enough’ results using current AI tools, compared to paying someone to do the same work. We are seeing this across the web, from journalists being supplanted, AI generated marketing copy everywhere, video games racing to cut costs by using AI generated assets and even replacing voice actors.

Good enough here is critical; in many cases business are fine with a loss of quality when it means that the price to produce a work drops to nearly zero. It’s a powerful motivation, and even a challenge for management to justify paying someone to do work that could be done for free. These roles where a portion of their work can be replaced with ‘good enough’ will see (and are already seeing) some of the earliest impact of AI3. Job opening in these spaces are already seeing increased competition as positions are eliminated.

Truck & Taxi Drivers #

While Tesla’s most recent timeline for their long awaited self-driving semi may or may not happen (though I won’t be holding my breath), the fact is that true self-driving vehicles are coming, and coming in mass. There are more than 250,000 taxi drivers in the U.S. today (not counting Uber and other ride-sharing drivers), and a somewhat astounding 3,600,000 professional truck drivers in the country. Most of those jobs are simply doomed. Self-driving vehicles will be able to do the job both with greater efficiency and at a lower cost. With no need to sleep, no need for breaks, these vehicles will quickly wipe out many of these jobs once the technology has sufficiently matured.

Of all industries, professional drivers will likely be hit the hardest by percentage of jobs lost. The industry will be largely obliterated.

Management #

Management roles provide both opportunity and challenge for AI replacement. I’ve spent much of my career in management roles, and to be honest, much of the time is simply lost due to things like inefficient communication. So many meetings that could have been an email, so many emails that could have been a couple bullet points, all without losing anything useful. Middle management roles are likely the most vulnerable to being eliminated by AI, as much of the role comes down to overcoming inefficiency. When eliminated, I would expect to see their responsibilities moving up the chain of command, as flatter organisational structures become more popular for their lower costs.

At least one CEO has noted that even he could be replaced by AI.

Information Security #

I have been predicting for years that automation will steadily erode more and more work within the security industry. This includes better tooling that reduces work for penetration testers by automating more of the work, and detecting and fixing issues earlier in development cycles. As well as systems to automate a growing percentage of routine work, allowing smaller teams to do more. If you look at the state of the security product market today, this is obvious.

As tooling evolves, and likely evolves at an accelerating pace, the impact on security jobs will likely also accelerate. That said, while these systems will be able to make many of those in security roles substantially more effective, the lack of creativity and ability to perform deep reasoning and analysis will ensure that jobs remain, and are in high demand.

How many jobs will be impacted is difficult to guess, at least based on where things stand today. This is likely the most difficult field to judge due to the various complexities and perceptions4 involved.

Plumbing, Construction, Mechanics #

Many of the jobs that were once considered the best of white-collar jobs now face greater uncertainty and the worst job stability since these roles came into existence, while some blue-collar jobs are likely more secure than ever.

Plumbers are becoming a new millionaire class, seeing small businesses becoming increasingly successful. This is just one of many jobs that will become more attractive as technology evolves due to the reliable income, constant demand, and job stability.

After the Industrial Revolution, white-collar jobs were often the best paying, most stable, and most desirable. After the AI Revolution, it’s possible that it’s the blue-collar jobs that will start to see competition for jobs, as white-collar jobs evaporate.

Inevitability #

This post may have you wondering if I’m actually a Luddite arguing against the evolution of technology. Far from it to be honest, I have spent most of my life pushing technology to the limit. The earliest of early adopters. The cutting edge often old news to me. I’ve always pushed for technical improvement.

I’m also a realist, and having studied history and carefully watching where technology is going, talking about the likely future is critical to be prepaid for it. As much as it’s possible to prepare for revolutionary changes.

While I believe we are still many years from the most important AI breakthrough, Artificial General Intelligence (AGI), though this has largely become a marketing term instead of a technical definition, the fact remains that we are headed to substantial changes. Furthermore, we are headed into this intentionally, with no plan as a society to deal with the repercussions. Leaders on countries are interested in ‘winning’ the race to build the most profitable AI systems, but the impact of deployment hasn’t received any meaningful attention.

Am I predicting an AI disaster? #

Many self-proclaimed technologists have spilled endless amounts of ink photons in rising the alarm about how AI could harm humanity, or even destroy us. This is not only of those arguments. We are sufficiently far from building anything truly intelligent, much less intelligent enough to see us as a threat, I don’t see a reason to even entertain these concerns.

My concerns are more immediate, and more focused on how it will impact individuals. If we manage to build a system that is capable of destroying us, and sees us as such a threat that it “feels” the need to take action against us, we’ll have earned the resulting outcome.

The Writing on the Wall #

As noted above, I could be wrong, or the process could unfold over decades, instead of the decade or less I’m predicting here. The impact could be less than I expect, or could be drastically worse.

Some things are obvious when viewed objectively, and I hope I’ve provided a useful analysis of these things, and the likely consequences for the future.


  1. To be clear, while it’s possible to read this description is a rather negative way, when I worked as a developer, I counted myself in this group. Don’t take this as a slight, but an acknowledgement that the software development industry is not just us. ↩︎

  2. I do enjoy writing, and based on the feedback I’ve received, it seems to be well received. That said, I don’t write quickly; I spend far too much time rewriting, changing my mind and taking a different approach, and just too much time thinking about what I’d like to actually say. It’s not uncommon for a blog post to evolve over months or even years before I get around to actually finishing it. For short stories and creative writing, it’s far worse. ↩︎

  3. When I talk about the impact of AI on creative works, I’m reminded of George Orwell’s 1984, and the Ministry of Truth’s novel-writing machines that created cheap entertainment for the masses, devoid of any actual creativity. We already see Amazon flooded with generated books, and as these tools become more effective at emulating creativity, it’s not unlikely that some of these will eventually start to perform well. ↩︎

  4. When looking at how roles are impacted, it’s important to look at not only the real-world impact of these technologies, but also the likely perception of business leaders. It’s the perception of business leaders, as well as pressure from investors, that will have the greatest impact on how many positions survive or are created. In some areas, this is easy to see based on public information, for others, it’s more complicated to anticipate the broad perception. ↩︎

Adam Caudill


Related Posts

  • On Apple, Privacy, and Device Control

    If you’ve bothered to look at Twitter or any technology news source, you’ve seen that Apple made a major announcement: Expanded Protections for Children. This has been written about by countless outlets, so I’ll assume you’re familiar with the basics. The announcement covered a few new features being added to the next version of Apple’s operating systems, namely: Scanning of inbound and outbound messages for sexually explicit images. Scanning images being uploaded to iCloud for CSAM.

  • Assange, Ecuador, and the Cyberarmy

    ALERT: Julian Assange has requested political asylum and is under the protection of the Ecuadorian embassy in London http://t.co/bz4O9bjF — WikiLeaks (@wikileaks) June 19, 2012 The news that Julian Assange has asked Ecuador for political asylum is flooding twitter as his supporters do their best to attract attention to his cause, and rally everyone they can to contact Ecuador’s embassy in London to urge them to grant Assange’s request. While I’ve watched the Assange case since before the first allegations came out of Sweden – that’s not my interest here, my interest is in what Ecuador could do if they wanted to.

  • Gpg4win & IDEA

    Huge PGP files, an ancient version of PGP, and errors every time they tried to decrypt a file – that was my completely unexpected challenge on Friday. Dealing with file processing issues really isn’t part of my job description, but I’m the closest thing my company has to an expert when it comes to encryption, so the task fell to me. After looking at the options and issues to get the server upgraded to a non-stone-age version of the PGP software, the easiest answer looked like decrypting the files with GPG – it wasn’t as easy as expected, but I did get some useful information that may help others.

  • The (Questionable) Future of YAWAST

    The last release of YAWAST was on January 1, 2020; while the release history was sometimes unpredictable, the goal was a new release each month with new features and bug fixes. I intentionally took January off from the project. In February, I left the company I was at; the team of penetration testers there had helped to inspire new features while looking for ways to make them more productive. But something else happened in February, an issue was opened – something that appeared to be simple, but in fact, made me realize that the entire project was in doubt.

  • Developers, Developers, Developers

    Note: This was written in 2012, but not published at the time. The point is still valid, perhaps moreso than ever and deserves to be made publicly. The content has been updated as appropriate, though the core of this article remains intact from the 2012 draft. I would like to note that this doesn’t apply to every environment, there are some where developers are very knowledgeable about security, and write code with minimal issues – my current employer happens to be one of those rare & exciting places.