Blog on Adam Caudill

Five Years with Hugo

Sat, 10 Jan 2026 01:29:22 +0000

During a break over Christmas 2020, I rebuilt this site, moving from WordPress to Hugo. After more than 5 years of publishing with Hugo, I’d like to share what I’ve learned, what’s worked, what hasn’t, and why for once, I’m happy with the platform I’m using.

This review builds on two recent articles, Five Hundred, a retrospective of 500 posts to this site, and ‌Lessons Learned from 20 Years & Why You Should Blog, a look back at 20 years of publishing here, and the value of writing & blogging more generally. In this post, I will be diving into publishing with Hugo specifically, what’s good, what’s not, and what you should think about if you are considering it.

Writing Friction

In the Lessons Learned post I spoke about the need to carefully consider the friction of the tools you use, as any bit of friction can reduce the motivation and time dedicated to writing. As such, I should address this first and foremost.

In 2012, I moved to Octopress, and away from WordPress, there were a variety of reasons for this, though security and performance were major factors. By the end of 2015, I moved back to WordPress. The reason? Friction. With static state generation, the process of creating a new post was slower, took more time & effort, and as a result, I did so less frequently.

By 2020, I was ready to embrace that friction. When I switched from WordPress to Hugo, I knew that each post would take more time, that I wouldn’t post as often. Yet, this isn’t a flaw for me, it’s a feature.

During the 2010s, I posted here 146 times, with an average length of 817 words. So far, in the 2020s, I’ve posted 52 articles, with an average length of 1,629 words. I post less often, though each sees greater effort and more depth, each is a greater investment to ensure that it’s worth the time to read.

While I will loudly caution people to watch for friction, and warn that it can lead to less motivation to spend limited free time writing, it’s not always a flaw. If you are already going to spend several hours (or more) on an article, the additional friction of using a static generator really does become insignificant.

Hugo works great if you don’t mind the friction; at this point, the friction isn’t a problem for how I write today.

Markdown All The Things

Hugo supports various formats for content, though the best support is for Markdown, and is what this site uses exclusively. When I switched this site from WordPress to Octopress in 2012, I had manually converted a decade’s worth of content to Markdown from the HTML that WordPress generates. Thankfully, after that all content has been in Markdown.

Even when I switched from Octopress back to WordPress, I took advantage of the fact that WordPress supports Markdown as well. This made later moves much, much easier.

I could explain why you do want to write in Markdown, for a variety of reasons, though Anil Dash recently documented the success of Markdown quite well, as such, you should read his article if you want a deeper dive.

Themes & Design

Hugo has a rich collection of themes available, many of these are remarkably well designed and impressive contributions from their talented designers. I selected a clean and simple theme, Kiera, and spent a week extending it, expanding the features, redesigning the home page, adding the sidebar, and countless other changes.

Hugo builds on the Go Template system, which allows the themes to be quite robust and flexible, supporting extensive embedded logic. While this doesn’t offer the degree of flexibility that I would truly like - such as having full access to Go¹ - it does work well and can do almost anything one would need.

The template files are HTML², and easy to edit and customise for your particular desires, it’s one of the easier and more pleasant templating systems that I’ve worked with.

Frequent Changes

The team behind Hugo are constantly working to improve the framework, fix bugs, add features, improve flexibility, and address design constraints. As I write this, on the 10th day of January, there have already been 4 releases this year.

This constant change does come at a price though, as there is a steady stream of new deprecation warnings that are added, signalling the future change or removal of some feature. There are also infrequent but very real breaking changes that slip in, without any clear warning. Over the last 5 years, there’s been at least two of these breaking changes each year, leaving the site broken in some way.

Thankfully, most of these breaking changes have been in less important areas, or places where I’m leveraging functionality that’s very much non-standard (as is my wont to do).

To help detect these issues, I’ve found it to be quite useful to use two different GitHub Actions jobs in the repository that houses this site, one performs a build against the version of Hugo that I’m currently using, and the other builds against the latest version. While this won’t catch all issues, it does surface many issues before they start breaking deployments.

Hosting

To keep the costs low, avoid maintaining another server, and ensure good performance, I opted to use Cloudflare Pages to host the site. This integrates cleanly with GitHub, it supports preview URLs, including for branches, which is an ideal way to get feedback before publishing, and allows me to write on any device, even without a local build environment, as it’s built on Cloudflare’s servers.

While there are quite a few providers that now offer similar services for static or otherwise edge-deployed sites, at the time Cloudflare offered the best features and the lowest price.

One beauty of Hugo and other static generators is that the site can be hosted anywhere. I intentionally don’t perform any edge compute, so this site could be hosted on a Raspberry Pi if I wanted. This adds great flexibility to move the site should the need arise, without needing to change or rebuild anything.

Easy Extensibility

One thing I love about Hugo and other static generators, is that I can easily extend the build process to add new functionality that isn’t part of the framework itself. I’ve done this to add automated related content, generating content stats, and others. This allows you to achieve things that would normally require a dynamic website, or even perform more compute-intensive operations that simply couldn’t be performed in real-time in a server.

This allows you to think outside of the box in a way that would be difficult, and build rich and innovative features. It also allows you to extend these custom features without being bound to the framework. An example of this is the fairly simple content stats script mentioned above, which counts simple things like the number of words. Today, that has expanded, and even includes details such as Flesch reading ease score or the hapax legomenon count. It’s evolved, and without needing to touch the framework itself.

What else would I use?

If I was to rethink my use of Hugo, if I was to switch again, what would I use? After having used many platforms & frameworks, there are a few (and only a few) candidates I’d consider:

Ghost - This would be my first choice, and the most likely thing I would move to. It’s very clean, easy to use, minimal friction, and an overall great writing experience.
Jekyll - This is a very robust static generator, well designed, and extremely flexible. It has most of the benefits of Hugo, though does have some sharp edges in my experience.
WordPress - This is the classic and ubiquitous option, well established, massive collection of themes available, huge plugin ecosystem. While I was once a big fan of WordPress, it has evolved in ways that seem to be more suited being a CRM for more complex sites, and less focused on being a great writing platform³.

There are many, many other options, though most of them I would avoid for one of two reasons:

Platform Lock-in - I don’t want my writing to be restricted to a single platform that’s eagerly waiting for the next payment. Of the options here, all can be self-hosted.
Writing not a Priority - Any framework or platform used needs to be focused on writing and making it easy and pleasant to actually write. There are dozens of other static generators, but many are too focused on other use cases to be good for writers.

Will I switch again?

This is a great question, and I’ve no idea. My writing habits may change, my needs may shift again, but for now, Hugo is ideal for my needs.

I’ve become quite a fan of Hugo, built at least a dozen sites using it, and I don’t plan on switching to anything else for the foreseeable future. If you are re-evaluating your writing platform, I would wholeheartedly recommend Hugo.

With the Hugo Modules system, you can bring in the full power of Go, though this does create a more complex build environment. For my uses, I’ve avoided that level of complexity. ↩︎
While the files are HTML, the embedded Go Template code is obviously not standard HTML, and thus unlikely to receive proper syntax highlighting. As the code can be rather complex, this can make reading it somewhat difficult. There is a plugin for VSCode that addresses this, though it’s sadly broken and abandoned; I’ve forked the project to correct the worst issues. ↩︎
For those familiar with the WordPress history, Project Gutenberg represented a substantial shift in focus and approach, and one that I’ve never cared for. It added undue complexity to the writing experience, which, with the complexity that has grown over the years as it’s evolved into a more general purpose CRM, makes it less and less fit for a dedicated writing platform. ↩︎

AI & IAM: Focus on Fundamentals

Fri, 09 Jan 2026 06:29:22 +0000

A recent article, The Future of Cybersecurity Includes Non-Human Employees¹, discussed the growing need to manage access granted to the rapidly expanding number of AI agents being deployed in companies. This is a deeply important topic, and particularly timely, as many are facing this challenge today. While I do want to address that topic, I also want to address how this is being framed.

Aside from leaning into an inflammatory tone with the use of “Non-Human Employees” as part of the title of the piece, there’s a deeper issue I see with how this is being framed, and it’s also related to AI. Importantly, this is far from unique to this article, but a trend across discussions of AI (and other new & emerging technologies). For the pragmatic security practitioner, a clear understanding of this framing device and the danger of accepting it unchallenged is of particular import.

Let’s take a closer look.

Reframing for AI

“Everything old is new again,” a phrase first uttered by Jonathan Swift, or Winston Churchill, or maybe it was Mark Twain² - whoever it was, it’s a simple phrase that’s apt as ever. With each substantial new technology, every existing challenge, every solved problem, every known risk gets reframed and repackaged and relabelled. For marketing teams, this is a fantastic opportunity to stake a claim on new territory, to assert leadership in solving some critical issue, to announce their solutions for these “new” problems.

AI is here, and everything is new again.

Non-human employees are becoming the future of cybersecurity, and enterprises need to prepare accordingly. As organizations scale Artificial Intelligence (AI) and cloud automation, there is exponential growth in Non-Human Identities (NHIs), including bots, AI agents, service accounts and automation scripts. In fact, 51% of respondents in ConductorOne’s 2025 Future of Identity Security Report said the security of NHIs is now just as important as that of human accounts. - The Hacker News

Here, in these first two sentences, my main objections to the framing are already clear.

Many of the challenges and risks aren’t actually new at all; they are the same as those that we in the security industry have been addressing for years. It’s new names for an old problem. They call it “Non-Human Identities” - for the last few decades this was called a service account. A new name for an old and well understood thing.

The danger of this framing is that it seeks to leverage new names and the association with a new technology to separate it from the well-established and well-understood approaches that have been refined over years of diligent work and careful analysis. This is great when you can sell something that addresses the problem. It’s the only thing this reframing is good for.

In reality, service accounts for AI are effectively the same as service accounts for any other form of automation³. As such, the mitigations for service accounts apply to agentic AI, and should be applied. This includes all of the controls that should always be applied, such as:

Least Privilege: As always, accounts should have the tightest possible set of privileges to complete the task, no more.
Separation of Concerns: Credentials and other secrets should be used for a single well-defined purpose, and nothing else. They should not be re-used for other processes or systems.
Credential Lifetime: Credentials should have the shortest possible lifetime, and be easily rotated.
Actionable Logging & Alerts: All automation should have clear logs, and alerts for unusual activity.
IP & Location Restrictions: Credentials used for automation should only be permitted within a known environment, any use outside of this controlled environment, should trigger accounts being disabled and immediate alerts.
Leak Detection: Logs and other automation artefacts should be automatically checked for secrets, authentication tokens, and other sensitive material.
Clear Ownership: The automated processes, data, and credential should have clear documented owners, with clear responsibilities - especially should issues arise.
Human Validation: When taking substantial actions, actions that can’t be reverted, or otherwise have significant impact, a human should be in the loop to validate that the action is intended and the results are desired.

These apply to agentic AI just as much as they apply to any other service accounts. In other words: the key is to focus on the fundamentals. Regardless of the underlying technology, the key to achieving meaningful security is to focus on the fundamentals first, then moving on to looking at the unique challenges.

Non-Deterministic Automation

Agentic AI and other forms of LLM-based automation are fundamentally similar to other forms of automation, though there is a substantial point that is critical to understand: it’s non-deterministic. If I automate a process using Python, the process is fixed, and the behaviour is deterministic - which is to say, it will always do the same thing⁴. With AI, it will by its very nature, do something different each time it’s used⁵.

It is this factor that presents the first meaningful difference between traditional automation and LLM-based automation. As the automation can’t be assumed to produce a consistent result, and may misbehave in truly unpredictable ways, this introduces some new risks that need to be addressed.

AI may attempt to perform actions beyond the intended scope, making the limitation of permissions particularly important. Implementing the principle of least privilege and maintaining strict separation of concerns are critical to avoid unintended activities.
AI may send secrets to search engines or other third-parties, making it important to restrict traffic and build deeper monitoring & leak detection⁶ into the process. This results in short credential lifetimes and IP & location restrictions being more important than normal⁷. It also shows the value of integrating proxies for web traffic, and wrappers & frameworks for code, to provide additional controls directly in the automation.
AI is subject to prompt injection, intentional or otherwise, which can cause radical departures from the intended behaviour, and thus any credentials or access the automation has can be abused in ways that would entirely defy expectations. This results in all input being potentially dangerous, and thus putting secrets at risk. This furthers the need to take the most restrictive and cautious approach possible.
AI agents can take unexpected paths to achieve results, as such, it’s critical that the permissions granted don’t include the ability to mint new tokens, assign permissions, or assume other roles. It would not be surprising to see an AI agent attempt this to circumvent an access control that resulted in an error message.
et cetera. This list could grow, though there is a clear pattern, and it’s that pattern that matters.

There are a number of unique risks that come with giving agentic AI credentials and access to systems, though all of them are addressed by existing high-level controls. Yes, the details of the controls, and the best approach to effectuate those controls may be different, the underlying controls are the same.

It’s this change that matters, where a secret is used for a defined intent, bound to a deterministic process, is now unbound to intent, and can be used in response to unexpected error messages, poisoned prompts, unanticipated data, or simply hallucinated steps in the process. Of course, with this change to unbind a secret to intent, it also can shift the impact to entirely different parts of the system or different data. This is where the practitioner needs to change how they reason about these risks and how they apply the controls to mitigate the risks.

Fundamentals Always Come First

If you are facing the question of how to secure access for agentic AI⁸, focus on fundamentals first. Follow the best practices for any service account, ensure that those controls are in place, solid, and validated. Once that’s done, then start looking at the unique risks.

If you focus excessively on what’s different about AI-based automation, you risk missing the basic controls that will address most of the unique issues - and the issues that impact your other service accounts.

As is always the case in security, follow the fundamentals.

The Hacker News, January 7, 2026. The article does share insight, though ends as a sales pitch for a Keeper product. ↩︎
When in doubt, it’s common to attribute most any quote to Mark Twain, as he was an incredible source of fantastic quotes, though the collection of apocryphal quotes are far larger and far more impressive. In this case, finding the true source of this statement is far from clear, though Jonathan Swift is likely the best fit for oldest variant of it. Or, you can just go with Stephen King’s “sooner or later, everything old is new again” line from The Colorado Kid. ↩︎
The article uses the term “Non-Human Employees”, though I reject this framing entirely. While I have written exploring the further-future issue of artificial life and intelligence, though we aren’t remotely close to this. As noted in the linked article, from what I believe to be a reasonably educated position on the topic, I don’t anticipate seeing this in my lifetime. As such, these are not, by any accurate definition, employees - this is automation. ↩︎
This is a slight oversimplification, though is fundamentally accurate, in that the instructions executed will be the same, though depending on how the code is written, it may still result in somewhat different outcomes. For the sake of this comparison, executing a fixed and pre-defined set of instructions is sufficient to call this type of automation deterministic. ↩︎
As above, this is a slight oversimplification, though to a lesser extent. LLMs are, by definition, statistical models and produce inconsistent results, even when given identical prompts. While the level of entropy involved can be tuned, reducing the variance in results, the fact remains the the models are statistical, and it’s not possible to ensure consistent results. ↩︎
Leak detection for credentials and API keys is particularly important for all cases, though there is one that is very much unique to LLMs: secrets have been known to find their way into LLM training materials, resulting in others gaining access to API keys that may still be valid. This is just one of the various issues that can occur if prompts are used for training. ↩︎
There is also the matter of PII and other sensitive data that can be leaked, but for the sake of this article, I will maintain a focus on IAM and associated secrets management. Broader questions around AI safety are best addressed separately. ↩︎
The question of if you should use agentic AI is left as an exercise for the reader. ↩︎

On Privacy Nihilism

Tue, 06 Jan 2026 20:29:22 +0000

Amongst the steady stream of marketing emails for gift cards and other last minute gifts in the days before Christmas, buried in the noise sent when people are least likely to see it, was a notice. It was an all-too-familiar “we take your privacy seriously, but” email. Perfectly timed to make it clear that privacy wasn’t that important.

This wasn’t just my email address being leaked, this was everything. Name, address, income, employer, social security number. Each record stolen was essentially an identity theft kit; everything needed in one place. From a privacy and data security perspective, few things are worse.

Yet the only thing remarkable about my reaction to that notice was that it was entirely unremarkable, more blasé than nonplussed. This was far from the first such notice, not even in the first dozen such notices. This has in fact become so routine that I’ve simply lost count of the number of occurrences thus far, it’s in the dozens.

The state of privacy and data security has become so deficient & dysfunctional that my oldest child had his first data breach notification, revealing his social security number to attackers, at less than 6 months old. He never had a chance to protect his information.

While I’ve focused on - and fought for - user privacy for much of my career, this unending series of failures has changed how I see my own privacy.

Failure Made Personal

As an industry, we have solved some of the hardest technical problems, we’ve devised solutions to many of the thorniest issues, and we’ve created tools and techniques to make finding and fixing issues easier than ever. Annual spending on security is now estimated to be an incredible $200,000,000,000¹. That’s a remarkable number of zeros.

Yet, hacks have far from stopped. But for some, myself included, the caring has.

While I still fight for the privacy of users, I’m generally a privacy nihilist when it comes to my own information. I’ve given up. Call it emotional exhaustion or cynical realism, I’ve accepted that my data has been leaked and stolen so many times that there’s no point in making any extraordinary effort to protect it.

That said, that doesn’t mean we should abandon protecting others, forgetting what we owe each other in a moral sense, or abdicating the responsibility that comes with skills and abilities we’ve developed.

Incentives

We can, and should, do better. We can, and should avoid such absurd situations as babies receiving data breach notifications, yet this is the world that we live in. And I have some thoughts as to why.

In 2018, I drafted an article about a breach - that article wasn’t published at the time - I’d like to share a portion of it here. It seems to have aged fairly well.

There is much said about the cost of a breach, on stock prices to fines, fees, and lawsuits. However there is little agreement on these numbers and there is a common opinion that many of these estimates overstate the cost. One analysis from ~~this year~~ 2018 puts the cost at $141 per record stolen, however looking at the impact to specific well known breaches, this number appears to be vastly higher than the actual cost.

When a business makes budget decisions, they look at many things, from regulatory and contractual requirements to estimates of breach costs (legal, public relations, sales, &c). All of these factors feed into the decision on what can be spent on security while maximising profit. This makes the cost of a breach extremely important: the lower the cost, the easier it is to justify spending less to prevent one. This is an unfortunate reality of business; in every decision there is a certain amount of risk accepted, and reducing that risk by too much means a loss of profits that investors expect to see.

There have been efforts in the United States and around the world to fine those companies that fail to properly protect sensitive data, however at least in the United States, the regulations that have made it into law so far have fairly limited impact. In the case of credit card data being stolen, the issuing banks absorb much of the cost. In some cases the banks are able to recover some of the losses, however the amount recovered isn’t substantial.

In the 2013 Target breach, approximately 40 million cards were exposed, banks sued Target claiming hundreds of millions in losses. Target settled for $106.4M, or roughly $2.66 per card lost. According to the 2016 Target annual report (the most recent available in 2018), the cost of the breach was approximately $300M, though with insurance and tax deductions, this was reduced substantially. This is for a company with around $3B in annual profits. While having 40M cards stolen seems like a devastating breach, the long term cost and impact has been effectively inconsequential.

This leads us to a disturbing realisation if you care about your data: many companies are investing just enough to achieve legal & regulatory compliance, but not enough to keep your data from being stolen.

In the 8 years since I wrote those words, they have weighed on me, and my view of how most companies treat the data they’re entrusted with. Some companies work hard to protect data, limit what they collect, and invest heavily in avoiding the need to send out one of those emails. Others, not so much.

Show me the incentive and I’ll show you the outcome. - Charlie Munger

Incentives are often aligned towards simple solutions, shifting responsibility, compliance often as theatre, and security too often seen as a means to avoid liability instead of achieving actual results.

One of the simplest ways to look at business leadership is that it’s a continuous balancing act of risks. Often, investments are focused on bringing these risks down to the point that the probabilities shift to the favour of the business. Once that’s done, it can be difficult to justify spending more. This is especially true for costs that are largely externalised to other parties, and especially individuals who have little opportunities to resist the imposition of these costs.

The challenge for those in the industry has long been finding ways to protect users, with the resources available.

Fighting the Good Fight

For much of my career I’ve focused on protecting the privacy and security of users, from my work in applied cryptography, to writing and speaking on privacy & data security, and of course, advocating for end-user privacy protections everywhere I go. Not because I’m zealously protecting my own information - that ship has sailed - but because we can and should do a better job of protecting people.

One thing I’ve always loved about the security community is that we will always do everything we can to help others, to protect people, to put in the time, effort, and energy needed to go beyond what should be possible with the resources available. A group of people that still solve problems just because they are hard.

Personally, I’ve little hope for my information, though I will never give up on protecting everyone else. Some who saw the title of this essay likely assumed that this was an abdication of privacy and the need to protect data. I hope you see the point is actually quite different: even if the effort is futile for some of us, the collective effort is more important than ever to achieve results.

We can and should do better.

Gartner forecast - $212B in 2025. ↩︎

Dynamic Social Media Images for Hugo

Tue, 06 Jan 2026 03:29:22 +0000

I’m a big fan of Hugo as a publishing platform, it’s the framework behind this site, and is incredibly flexible - if you are willing to invest the time and effort to make it truly yours. It’s fast, versatile, and has robust theming support. However, it’s also a static site generator, so doing anything dynamic means doing some extra work (as you have to do it at build time).

My life philosophy can be summed up to “work hard to be lazy” - in this case, that means I want a solution for social media sharing (OpenGraph) images that I will work without me needing to think about them again. This way, when a link is shared to Bluesky, Mastodon, LinkedIn, or other platforms, a reasonable image will be shown - even if I didn’t include an image for the article.

More importantly, I was a solution that’s smart about what images it uses.

The solution I use here will select the image to be used, with a series of fallback options, so that there is always an image. In the case of the code below, it uses the following options, in order:

images from the frontmatter.
photo from the frontmatter.
Image resources for the page with “feature” in the name.
Image resources for the page with “cover” or “thumbnail” in the name.
The first image on the page.
A generated image with the page title.

This ensures that there will always be an image, that it will make the most reasonable choice about what that image is, and, importantly, that I don’t need to lift a finger. There’s nothing to forget, no risk of there being no image.

This is an ideal solution for people like me that sometimes will add images, but often won’t This ensures that when shared on social media, it always looks good.

Generating Images

The final fallback option, the generated image with the page title is the most recent addition to this design, as in the past the code simply used a single static image. These images are generated using Hugo’s native images.Text filter, making it quite easy to apply text to an image (it does just what it says on the box). This is a surprisingly friendly and feature.

This does require a little setup, there are two files that will need to be added to your assets/ directory.

Font: You will need to add the font you want to use to the assists/fonts directory. A broad selection of these can be found on the Google Fonts Github repo. Personally, I used IM Fell English, as I love the history behind it, though there are countless options available to suit your site and taste.
Background Image: You will also need the background image, 1200 x 630 pixels, in the assets/og-image directory. This is the image that the text will be applied to.

In the code below, you will need to update the file names as appropriate.

In the code below, you’ll see the heavy lifting for the image generation:

{{ $fontPath := "fonts/IMFeENrm28P.ttf" }}{{ $imagePath := "og-image/bg.jpeg" }}{{ $filter := "" }}{{ $r := "" }}{{ $text := .Title }}{{ $font := resources.Get $fontPath }}{{ with $r = resources.Get $imagePath }} {{ $opts := dict "alignx" "left" "aligny" "bottom" "color" "#ffffff" "font" $font "linespacing" 8 "size" 80 "x" 10 "y" (sub .Height 25) }} {{ $filter = images.Text $text $opts }}{{ end }}

This is a thankfully straightforward implementation of this feature, loading the font and image as resources, collecting the page title as the text to apply, then defining the set of options we’ll use to apply the text. In my case, to match the site’s header, the text is white, the text is placed at the bottom left, and the offset are selected to keep the text visible without running off the image.

This image processing is quite fast, though for large sites, it can slow the generation process down. In my case, the slowdown is approximately 4 seconds.

The Code

The theme for this site started as the publicly available Kiera theme, though has been almost entirely rebuilt, and this incudes often replacing the use of Hugo’s built-in theme components, and replacing or extending them. The code for my OpenGraph image handling is no different, starting as a 11 lines of code built into Hugo, it grew substantially.

{{- if .IsPage }} {{ $imgs := findRE "<img.*src=(?:\"|')(.*?)(?:\"|').*?>" .Content 1 }} {{ if $imgs }} {{ $img := index ($imgs) 0 }} {{ $first_img := replaceRE "<img.*src=(?:\"|')(.*?)(?:\"|').*?>" "$1" $img }} {{ if $first_img }} {{ $first_img := $first_img | absURL}} {{ $.Scratch.Set "first_img" $first_img }} {{ end }} {{ end }}{{ end -}}{{ $fontPath := "fonts/IMFeENrm28P.ttf" }}{{ $imagePath := "og-image/bg.jpeg" }}{{ $filter := "" }}{{ $r := "" }}{{ $text := .Title }}{{ $font := resources.Get $fontPath }}{{ with $r = resources.Get $imagePath }} {{ $opts := dict "alignx" "left" "aligny" "bottom" "color" "#ffffff" "font" $font "linespacing" 8 "size" 80 "x" 10 "y" (sub .Height 25) }} {{ $filter = images.Text $text $opts }}{{ end }}{{- with $.Params.images -}} {{- range first 6 . }} <meta property="og:image" content="{{ . | absURL }}" /> {{ end -}}{{- else -}} {{- with $.Params.photo -}} <meta property="og:image" content="{{ . | absURL }}" /> {{- else -}} {{- $images := $.Resources.ByType "image" -}} {{- $featured := $images.GetMatch "*feature*" -}} {{- if not $featured }} {{ $featured = $images.GetMatch "{*cover*,*thumbnail*}" }} {{ end -}} {{- with $featured -}} <meta property="og:image" content="{{ $featured.Permalink }}"/> {{- else -}} {{- with $.Scratch.Get "first_img" }} <meta property="og:image" content="{{ . }}"/> {{- else -}} {{ with $r }} {{ with . | images.Filter $filter }} <meta property="og:image" content="{{ .Permalink }}"/> {{ end }} {{ end }} {{- end -}} {{- end -}} {{- end -}}{{- end -}}

For my site, I’ve placed this in a dedicated file, opengraph_image.html, which is included as a partial in my header file. Whenever possible, I keep self-contained blocks such as this in separate files to improve readability and keep any other files from uncontrollable growth.

This provides an approach that is incredibly simple for day-to-day use, while being extremely flexible and ensuring a positive sharing experience for readers. This can be implemented in only a few minutes but will save you time and leave you with one less thing to worry about in the future.

If you are building or improving a Hugo site, I hope this was useful to you.

Lessons Learned from 20 Years & Why You Should Blog

Sun, 04 Jan 2026 08:29:22 +0000

Twenty years ago, I started publishing articles and essays here, and I recently published the 500th post to this site. After writing 267,897 words here and investing 2,100 hours into this site, I’ve learned a few things, made some mistakes, and I’d like to share some of these insights with you. Whether you are a veteran of the blogosphere or questioning if you should dip your toes in the waters (you should), I think you will find some useful information here.

This isn’t a how-to, this isn’t a collection of best practices, and this certainly isn’t a “how to get rich as a blogger” guide. This is what I’ve learned from making mistakes over the last 20+ years, and how you can avoid them, or maybe this will help you see that you’ve been making the same mistake.

I hope that you will find this to be helpful. Also, there’s no narrative arc, so feel free to jump around and focus on the sections that are most interesting to you.

Tools Matter

The tools you use to collect ideas, draft posts, and systems used to publish your writing are far more important than they may seem at first. This isn’t just a matter of fighting bad tools, but because each tool or system used has a tendency to add a bit of friction to the process, and the greater the friction, the less you will write over time.

This friction builds up over time, compounding, and eventually thwarts the desire to actually write.

Minimise Friction

Whether you are using a (self-)hosted platform or a static generator, pick a framework that doesn’t get in your way. Pick something that allows you to write with the smallest number of annoyances and extra steps.

A great example of this is Ghost, it’s clean and simple. Or write.as, which has a beautiful focus on simplicity & elegance. Or any of a dozen other options.

While it’s important to look at the features, to ensure it’ll do what you need, it’s more important to choose an option that stays out of your way. Don’t compromise on ease of use for any other feature, as nothing else matters if you eventually lose interest in writing.

Dynamic vs. Static

There are two types of publishing frameworks, those that dynamically generate the pages on the server, such as Ghost or WordPress, and static generators that build the files needed, and can be served without any code running on the server.

These static generators shift all the work to your computer over the server, allowing the site you publish to be more secure, incredibly fast, and opens the door to far more (and cheaper) hosting options. The tradeoff is that adding content is more time consuming and complex.

This site uses Hugo, as with Hugo, the added complexity is low enough that the tradeoffs are worth it. There are a huge number of options for static generators, each with a different set of tradeoffs.

Tools are Temporary. Data is Forever

It’s critical to carefully select the tools you use, and re-evaluate these from time to time, ensuring that they are actually serving your workflow well. If your tools aren’t working, throw them out. While moving data and overhauling workflows may be a temporary inconvenience, that pain won’t last as long as using the wrong tools.

Thankfully, nearly all blogging platforms offer export & import systems, making it a straightforward process to move between platforms. While this isn’t always as easy as one would like, the important thing is that the option is there. It’s important that you aren’t forced into a single platform.

One important thing to remember if you look at changing platforms, make sure you pay attention to anything that could break links. From the URL structure for your posts, to the location of your RSS feed, make sure that you either match these after the move, or add redirects to ensure that nothing breaks.

Own Your Name

No matter what you use, make sure that you are using a domain name that you own. This isn’t just for branding, it’s to ensure that you always have the freedom to switch platforms, tools, underlying technologies, you name it. This way you always have options.

While this does add some cost, it also ensures that you aren’t beholden to any single company. If they make a change you don’t like, just move your site.

My Tool Stack

As noted above, this site is built using Hugo, though it didn’t start there. Originally, back in 2006, this site used WordPress, then moved to Octopress, then to Jekyll, then back to WordPress, and then, in 2021, to Hugo.

For writing, most posts start in Drafts, a great app for iOS and macOS, that has fantastic markdown support and using iCloud to sync between devices. I will typically use this to capture the first notes of an idea, and then work on the prose on and off, sometimes over years. This is one of my favourite parts of my workflow, it’s fantastic.

Once a piece is ready for serious focus, I then move to iA Writer, one of the best dedicated writings apps I’ve used. This is where the real work happens, and it has a feature set that’s ideal for actually getting that work done.

If I need to share an early copy of a piece for review or editing, I use Ellipsus, due to its great collaboration features.

Writing & Content Choices

What you write, how you approach it, and even how you think about where you’re publishing this work is extremely important. And, unsurprisingly for anyone who’s been a reader for a while, I have thoughts.

Build a Website, Not a Blog

While I will say that I’ve been blogging for decades, this is not a blog. This site includes a blog, but it’s not just a blog. I publish articles about current events, essays, research, fiction, photography, it’s also where I keep my resume, and a whole variety of other things. This goes well beyond a blog, and I encourage others to do the same.

Why? Let me explain.

Your site is an archive of your work, your growth, your career, and your life. Wins and losses, lessons learned, and insight into who you are - not just one facet of you - all of these things matter, and all deserve to be given space.

Use your space on the internet for everything you can. It’s not just a place to share thoughts, but a place to share what matters to you, what people should know, and gives you a place to bring everything together. It’s important to understand and accept that not every reader will be interested in everything you post. You can’t please all the people all the time, so the best approach is to instead tailor each piece to a specific audience, and if others appreciate it, all the better.

Grow beyond blogging, own the space and make it yours.

Ignore the Analytics

This site used to be hosted on a platform that would count views for each post, and then I used Google Analytics to gather more data about the traffic this site got, what topics were popular, and which ones didn’t perform well. Using analytics like that is basically blogging 101.

Years ago I moved to a platform that can’t count views, then I removed Google Analytics. This is one of the most freeing decisions I’ve made in running this site. It allowed me to focus not on what would get the most reliable traffic, but instead on topics that mattered to me, or were learned through painful experience, or were being poorly addressed in media.

When you focus on the numbers, you soon find yourself serving the numbers, instead of those that could learn the most from your knowledge and experience. Ignore the numbers, write what you care about, and say what you need to say.

Think In Decades, Not Quarters

When you are planning your posts, it’s helpful to think about the intended use lifespan of the post. Will it still be useful in a month? In a year? Five years, or even 10 years? While it’s useful to write content about what’s happening today, there’s substantial value in writing for the next decade.

Writing content that will still be valid and, more importantly, useful in the next decade means that you’re building up a corpus that goes so far beyond news cycles. Your content is likely going beyond specific technologies. This creates a rich set of references that you can link to, and content that will always be useful, and will help you to grow an audience that values your insight.

In the special 500th post retrospective, I’ve cited quite a few posts, many of which are several years old, yet are still useful and actionable today.

Be Authentic

Readers want to hear your voice, and even if you don’t know what it is yet, every writer has a voice. One major difference between sites that get traffic and those that get lost in the noise, is providing an authentic voice that people can sympathise with.

Be you, not a fake persona, not the person that you think people want, and especially, don’t use AI as a cheat code as a way to create content faster.

Providing a real, relatable, and authentic view of yourself is the most important thing you can do to build an audience that will come back, will give you feedback, and will share your work with others that can benefit from it.

Quality Over Quantity

The early posts here weren’t great, to say the least. The posts were short and frequent, several posts each month, yet the quality was far from ideal. Over the years, I’ve shifted more and more to exclusively long-form content, with the average length of posts in the 2020s being over 1,600 words, with several exceeding 5,000 words.

While the typical guide for bloggers is to push for a regular & frequent posting cadence, I’ve found this to be a mistake. Taking time, as much as needed, to create a better and deeper post is far more useful to readers than a short aside that provokes little thought.

The posts here are sometimes built over months (or more), and may have dozens of hours of work invested in them. This level of effort is clear to readers, and provides far more long-term value.

Link, Don’t Repeat

Once you’ve invested the time to cover a topic well, don’t repeat yourself in later posts, link back to the original. This both keeps your older content alive, it allows you to focus on being clear as possible for the topic at hand, instead of trying to cover multiple topics at once.

As you build up your corpus of long-lived content, you will find plenty of opportunities build on your existing content, and write topics that build on what you’ve written before. By creating these “base posts”, posts that exist primarily to make an underlying point and be linked to, you can create an evolving argument.

While I still struggle with remembering to look for opportunities to link to prior posts, it’s worth the effort.

Own Your Space & Write

While I’m concerned about the state of reading, I’m a firm believer that it’s more important than ever to share knowledge and experience. Now isn’t the time to retreat, much the opposite, it’s time to lift everyone else up.

I started to draft a longer argument here about why you should blog if you aren’t already, though JA Westenberg recently made this argument beautifully in The Case for Blogging in the Ruins. You should go read that.

The Next Decade

The next decade of this website has begun, and I’ve no intention of stopping anytime soon. This is a way to refine my skills, to share my experiences, to help others avoid my mistakes, and to pay a debt to all those that helped me when I was just getting started.

Writing like this, while time consuming, is a gift to yourself and the world, and one that I believe is worthwhile. Here’s to the next decade.

Five Hundred

Fri, 02 Jan 2026 08:29:22 +0000

The year was 2006 when I registered adamcaudill.com and set up WordPress to host this site. I had recently moved, started a new job as a software developer, and I wanted a new place to share thoughts, code, and the insight I was gathering along the way. I made the very first post.

It will be 20 years, next month, since that first post, a short note from someone still finding his legs in the industry and far from finding his legs as a writer. Through the 2000s, the average length of the posts was only 240 words. Far from the long-windedness common in my more recent work.

This post marks the 500th post here.

During 2006, I posted frequently, several times each month. Most of these posts were short, filled with typos, and were generally short asides about news in technology. These early, and often embarrassing posts, are easily browsed on the archive page. Over time, activity died down, sometime only posting once in a year. Though the changes to the content were even larger.

My philosophy about how and what I wrote evolved, and eventually articulated, to long-form articles and essays that are less about breaking news, and more often about topics that will be relevant for years. Discussions meant to be thought provoking. Insight earned from hard experience. Original research, and technology tips meant to address challenges not documented elsewhere. These extensive tomes even explored philosophy and religion.

From the articles and essays, I started publishing short stories and other fiction, expanding my writing efforts and refining those skills. Investing in a skill I view as critical.

Based on the latest calculations, I’ve invested 1,391 hours writing the 267,897 words you see here. A huge investment in sharing the lessons I’ve learned, though an investment I’ve always viewed as worthwhile. If I can help people to learn something new, refine a skill, grow as a leader, or see something from a new perspective, then it has been a worthwhile investment. We stand on the shoulders of those that came before; sharing what I can feels like what I owe to the community.

Blogging in an Age of Decreased Reading

While the number of blogs is growing, reading seems to be dying, the value of maintaining a blog and sharing lessons is timeless and even if fewer people are reading, some will, and hopefully they will find something useful.

From studies on Crew Resource Management that took weeks of work, to arguing for changes to how the industry handles vulnerability disclosure that is still cited to this day (a decade later), all of these efforts have resulted in great feedback and, more importantly, led people to think about how they work, these efforts matter.

Not everything here has been happy, the loss of friends, and coworkers and family. There are others that deserve to be memorialised here, though I didn’t have the heart to write the words. I still don’t.

This site has been witness to many things, including much that should have been written but wasn’t. There are many stories, both personal and professional, that, looking back, I do wish I had written. Stories that should have been told & recorded. Alas, such is hindsight.

All in all, I am proud of what you see here, despite the flaws & omissions.

Writing is More Important than Ever

The security industry is changing, unemployment is high and jobs are at risk, sharing knowledge is critical to the next generation of professionals, who are among the hardest hit. From how to work productively, avoiding pitfalls in communication, how to lead people more experienced than them, to learning how to evaluate risk, there is much we can teach, if we take the time. And when the time comes, we can make sure they get their turn in the spotlight, and set them up for success.

We need to provide solid guidance on avoiding security issues, we need to refute obvious mistakes, help people avoid disinformation, and provide the insight to end common threats.

With the rise of AI, and the incredible implications it brings to all knowledge workers, it’s by sharing knowledge, sharing insight, and lifting all up, that they have the best chance to not just survive, but thrive in this uncertain future.

While generative AI can sound, for a time, like an expert, it’s knowledge, insight, understanding, and logic that is critical to the future of the security industry, and the brilliant community that powers it.

We owe it to the next generation to share what we’ve learned.

Looking Ahead

Despite the challenges facing the economy, the job market, the rise of generative AI, and political turmoil, the security community is a truly amazing group, and a community that I’ve always been honoured to be a member of.

For me, I’ve been sharing my experience throughout my career, and the content of this site stretches back to 2003. I intend to continue to write and share for as long as I am able.

Whose Monkeys Are These?

Thu, 01 Jan 2026 08:29:22 +0000

Over the course of my career, I’ve found that there are some principles that are key for people and teams to be effective. One of these is that everything should have an owner. Everything should have someone that is responsible. Everything should have a designated person whose job it is to care about it. This might a be bug or vulnerability reports in software, it could be routine processes, or who responds to certain emails.

When there isn’t clear ownership, things slip through the cracks, things are forgotten about, and minor issues can quickly become anything but minor.

This works well enough for many things where ownership is clear and agreed upon, but things get ugly when that thing is a problem. Here, human nature starts getting in the way, and leads to unfortunate outcomes. In this essay, I’d like to explore how people deal with the problems they see.

Not My Circus, Not My Monkeys

The Polish idiom “nie mój cyrk, nie moje małpy” is, perhaps, my favourite phrasing of the statement “that’s not my problem.” Some variant of this statement is made countless times everyday in Slack threads and email exchanges, sometimes rightfully staying out of other people’s area of responsibility, but far more often, to avoid a problem that they don’t see as theirs to address.

An SEP is something we can’t see, or don’t see, or our brain doesn’t let us see, because we think that it’s somebody else’s problem. That’s what SEP means. Somebody Else’s Problem. The brain just edits it out, it’s like a blind spot. - Douglas Adams, Life, the Universe and Everything

In reality, when problems are found, people react in very different ways. From my experience, there are 3 kinds of people, when they see a problem, they will do one of these things:

Unless it’s clearly theirs, they assume it’s somebody else’s problem and stop thinking about it.
Unless they’re told it’s theirs, they assume it’s somebody else’s problem and stop thinking about it.
Unless they see somebody else handling it, they assume it’s their problem.

When a person says “that’s not my problem,” they are probably in the first group. It’s not clearly theirs, so they drop it, move on, and focus on other things. As they don’t perceive it as their responsibility to address (or ensure that someone else is addressing it), it drops from their mind.

This form of automatic anti-ownership is both understandable, and, to me at least, disappointing. Instead of claiming ownership, at least until a better fitting owner is identified, they avoid making a concerted effort to not have ownership. This strategy greatly increases the odds of the problem slipping through the cracks.

Where the first group exhibits an active aversion to owning issues they see, the second group exhibits an indifference to it. Unless given the order to take ownership, they adopt a more passive approach and pay little heed to this issue unless told to.

This is not to say that anyone in either of the first two groups are avoiding work, or unwilling to take ownership, but that due to the nature of different personality types, incentives, and organisational dynamics, people can be disinclined to adopt a problem that isn’t clearly theirs.

We’ve all seen this, right?

None of these traits though should be particularly surprising to anyone with a non-trivial amount of leadership experience. These are common traits, and lead to common issues, and are simply part of the routine within teams. Enough ~~ink~~ electrons have been spilled covering these archetypes, so I won’t belabour the point on them. Instead, I will focus on the third personality type.

The Cost of Ownership

For those people that assume a problem is theirs unless (or until) they see that someone else is handling it, those that are proactive about accepting ownership, they do so at both visible and invisible costs. It’s these costs and the motivation that pushes them to proactively accept ownership that we will be discussing here.

In general, people that are proactive are seen positively, are more likely to be promoted, are often given greater responsibility. After all, they are already accepting greater responsibility without being asked to.

Ownership: Proactive Acceptance vs. Seeking

Before we move on, I would like to look at a seemingly subtle difference, which is, in reality, an important distinction in both behaviour and motivation.

The archetype - the people - I’m describing here are those that proactively accept ownership of problems they see, though they do not seek ownership. The difference is critical: one accepts what they see, the other seeks to expand what they own. The latter may be seeking greater control or influence, while the former only seeks to ensure that problems are addressed.

This oft misunderstood distinction can lead to conflict or discord, though the motivations are entirely different, as those that seek to expand control are less likely to be motivated to actually address problems, but to leverage problems to achieve other aims. Where those that proactively accept are instead motivated to achieve a resolution to the problem, or at least a resolution to their perceived responsibility for it.

As this essay is a study of how problems are perceived, and not how they can be leveraged, I will not be further exploring those that seek to use problems (and other forces and events) for their advantage.

The Responsibility of Knowledge

For those that proactively accept ownership, they feel a deep sense of responsibility for issues they see, even if they are outside the direct scope of their role. Some portion of this may be rooted in loyalty and concern for their company, some may be linked to a broader sense of what they owe to coworkers, either way, that sense of responsibility is real. It’s also difficult to ignore, creating an unavoidable and inescapable weight once they become aware of a problem.

Once the knowledge of the problem comes into existence, the increasingly oppressive weight of the problem grows. And it will continue to grow until that responsibility is, one way or another, discharged.

This practice tends to result in fewer problems being left until they develop into a large issue, but it weighs on the people that feel obliged to pursue these problems.

The Squeaky Wheel

While the people that are the most dedicated to ensuring success, to avoiding the deferred consequences of ignored problems, also are those that are seen as the squeaky wheel. They are the proverbial messenger that are too often the first in the line of fire. By opting to say something, to acknowledge the problem, to seek a solution or an appropriate owner, they are faced with continuous guilt by association, they are blamed for bringing negativity into discussions.

This creates a singularly unfortunate combination of pressures: the perceived weight of responsibility and the dread of blame and resistance for acknowledging that the problem exists.

When you combine this personality type with certain careers, such as anything in security, a field whose members are often seen as a particular inconvenience when added to a conversation, the results are worse (at least for those that feel compelled to open their mouths).

Yet, for all the benefits of this archetype in terms of results, the self-imposed pressures speed burnout, magnify discontent due to the recurring instances of shooting the messenger, and creates an impenetrable cloud of stress that shades everything. Useful for a team, bad for the person.

Owning a Circus

If you look in the mirror and see the person that has no choice but to adopt each and every unowned problem they encounter: welcome to the annoyingly stressful club. I very much identify with this archetype as well.

Thanks to this deep sense of responsibility, one ends up caring for a substantial number of proverbial monkeys. Though it’s important to understand how much of this pressure is self-inflicted, and not organisationally imposed. More importantly though, it’s important to remember that the vast majority of these problems aren’t actually yours: you don’t own them, you’re a caretaker until the rightful owner is identified.

When viewed through this critical lens, that there is a key distinction between ownership and custody, the pressure is lowered and the path forward is simplified.

If you are in a leadership role, you need to understand which members of your team exercise this type of proactive acceptance, and actively aid them in finding the proper owners of these problems. This work to redirect these problems will substantially lower stress and slow burnout, and will demonstrate that they don’t need to own these issues, only care for them temporarily.

Just as importantly, you need to take due care when discussing these issues to ensure that you don’t create a feeling that they are being seen as a problem themselves. The repeated negative receptions to speaking up eventually creates a constant fear, a latent pressure in every conversation.

For those of us that have shared this lived experience, we need to identify this in others, so that we can leverage our positions & experience to relieve the pressure from others.

Is Long-form Writing Dead?

Mon, 29 Dec 2025 02:29:22 +0000

In a world where attention spans have been reduced to seconds, college students aren’t expected to read full books, AI is used to summarise anything more than a few sentences, and blogs have been largely replaced with microblogging platforms, is there still a place for long-form writing? In this essay I would like to explore that question; from how we got here to what hope we have for the future.

This is a complex topic, and to properly explore this, we need to go back several decades. What we see today are the symptoms of decades of changes and issues, layered one upon the other, building up to a far larger issue than what any of these could explain in isolation. While this essay goes on quite a journey, I believe that the journey is worthwhile and enlightening.

Before we truly begin, I must confess an important fact: despite the length of this document, the countless hours that have been spent in research, writing, and building up to this essay, the question it presents won’t be answered. While I intend to explore the question in detail, I won’t provide the answer. In reality, I don’t know. Long-form writing, as we know it, may be a dying art.

Three-cueing vs. Phonics

We start at the start, which is to say, how we learn to read. If you are not familiar with the three-cueing method, I suggest you read this, as this is critical to our analysis. If you are familiar, feel free to skip ahead.

Since as early as 1570, children in the English-speaking world have largely learned to read through a system that’s now called phonics. A system that focuses on the sounds associated with the letters & letter groups, allowing new readers to break down words into smaller components and work through those components individually. Phonics has strengths and weaknesses; it allows a reader encountering a new word to work through it to discover what it is with some effort. It’s also slow to show progress and frustrating for those learning to read.

In the 1960s, a new method was introduced that aimed to simplify learning, and allow new readers to achieve literacy faster. Three-cueing doesn’t focus on breaking words down to understand them, but using context clues to allow readers to make guided guesses¹. This system taught readers to look at three elements to identify context.

When a reader encounters a word they aren’t familiar with, they are taught to look for the following cues to identify the word:

Visual: Looking at the first letters, word shape, letter clusters, visual alignment to other known words.
Structure: Looking at the structure of the sentence to identify patterns, grammatical requirements, or other syntactic hints.
Meaning: Looking for semantic hints, such as meaning of the sentence or prior sentences, context of the document so far, illustrations, and other hints that provide broader context.

Using these three-cues, new readers are taught to identify likely words, guess which is the best fitting, and if needed, to mentally replace the word with another that the reader knows.

This system is easier and faster to teach, and performs well in standardised tests where students can be prepared for a limited and consistent set of words that they need to know. However, it has problems. Very serious problems.

The system was based on a flawed understanding of cognitive development, and resulted in teaching students a system that allowed them to gain some understanding of written words quickly. Though the system also resulted in students often not understanding texts, and different students coming away with substantially different interpretations of the same text². Those that learned to read with this method struggle to understand new words, and struggle with comprehension, due to replacing or skipping unknown words.

In 2020, one study showed that 75% of teachers in the US were teaching the three-cues method. However, this is changing as the issues with the method are becoming better known, and at least 12 states have now banned the method.

[…] the student told Dames that, at her public high school, she had never been required to read an entire book. She had been assigned excerpts, poetry, and news articles, but not a single book cover to cover. - The Elite College Students Who Can’t Read Books

As this method of teaching reading has spread, schools have also reduced the quantity of reading expected of students. Students are struggling to finish even simple reading assignments, struggling to understand what they read³, and become overwhelmed when asked to read lengthy or complex texts.

[…] we are in new territory when even highly motivated honors students struggle to grasp the basic argument of a 20-page article - Adam Kotsko

The simple version of all this is, unfortunately, teaching methods have failed to prepare children to read. Choices that were made over the last few decades are coming into full relief, and the picture is disturbing.

Micro-Content & Micro-Dopamine

With the words “just setting up my twttr” Jack Dorsey started a revolution. Twitter was focused on short but high volume content, with a focus on interaction over quality. Micro-blogging was born. Short posts, drops in a firehose, most with a half-life measured in hours or even minutes⁴.

If you didn’t get a re-tweet or reply or like within the first few minutes, it was probably a miss. Might as well delete it and try something else to see if that one lands. The search for those tiny hits of dopamine started as soon as Twitter started.

Due to the small post size supported, thoughts had to be compressed, vocabulary limited to the shortest words available, commentary limited. Like the previous sentence, the posts were limited to 140 characters, a painful limitation that made clarity difficult and precision nearly impossible.

Then there was Instagram. Photos, just photos. A way to share filtered views of everyday life. The next step was Vine, a service that allowed users to view, create, and share 6-second looping videos. Tiny content, tiny dopamine hits.

From here, we all know how things have evolved. TikTok, Threads, Bluesky, Mastodon, and countless others.

The impacts of poor reading skills continue to compound as social media was reshaped to focus on shorter content, with compressed vocabularies, and an increased focus on achieving quick dopamine hits over meaningful communication.

Writing but not Reading

If you include self-published books, it’s estimated that up to 4,000,000 books are published every year. Yet sales are often far worse than disappointing. A self-published author is likely to see 5 copies sold. For professionally published books, that number climbs to roughly 1,000.

[…] based on data on 58,000 books published in a year, ‘90% sold less than 2,000 copies, and 50% sold less than 12 copies.’ - The Elysian

While this isn’t to say that people have given up on books, as popular authors are still selling well, it does show that there is a massive disconnect between those writing books and the market that’s actually available⁵.

Given the sales figures that can be reasonably expected, writing a book is, sadly, often more about either passion or ego than viable income.

Schrödinger’s Blog

Blogging is alive and well and simultaneously a dead relic, it just depends on who you ask. There are now more than 600 million blogs, a number that has been growing rapidly. However, while the number of blogs is climbing, the authenticity is dropping. For those focused on company blogs, 57.4% are using AI to create content, and various studies show an intent to use AI more in the future. Studies show interest in blogs dropping, especially among young people, as early as 2010.

Estimates vary, though approximately 1.6B-2.5B blog posts are published every year. Yet, it’s also becoming harder to convince people to actually look at these posts: roughly 60% of Google searches result in no clicks, searches where the user either relies on an AI summary or what they saw on site previews. But that’s not the worst thing going on, it gets worse.

According to one source, the average reader only spends 52 seconds reading a blog post. The average blog post across the industry is roughly 1,400 words; at a normal reading speed, that means on average, a reader will cover about 150 words before leaving.

AI: Machines Writing for Machines

AI is everywhere, and it’s critical to this discussion as well. As noted above, 57.4% of corporate bloggers are using AI to draft content. Another, a survey of professional writers, show that 63% of writers use AI to generate text that they later edit. While some avoid using AI to generate content, including myself, the percentage is not high, and seems to be trending lower.

In the book market, the number of books that are all or largely AI generated is climbing rapidly. In one study of books available on Amazon, in a sampling of books in one category, 82% were found to likely be AI. A broader industry analysis put the number at 20% of all self-published books. The number of books generated by AI has grown to such an extent that Amazon now limits self-publishers to three new books per day.

Once we get through writing, we then land on the other side: AI summaries. A survey shows that 87% of adults in the US use AI summaries while searching, 60% use it to search for information, 48% use it to summarise long texts.

This can be combined with AI being integrated directly into browsers, or browsers being released by AI companies to offer deeper integration, making it easy to get an AI generated summary without even leaving the tab. Or email platforms integrating AI summaries. Or corporate chat platforms integrating AI summaries. Or meeting platforms integrating AI summaries. It keeps going. Anywhere there is long text, there’s an AI summary button within a few clicks.

Is Long-form Reading Dead?

At the beginning of this essay I asked the question, is long-form writing dead? At this point I think it’s clear that we’ve been asking the wrong question. There is a great deal of long-form writing in the world, and it’s quite possible that it’s growing (though the quality is increasingly questionable). The real question comes down to, is reading long-form content dying? Is it getting to the point that it’s pointless to write because it won’t be read?

That is a question that we will have to leave to the future, though I suspect, with what we have seen here, that there is reason to worry.

This article, for the sake of brevity, keeps the description brief and focused on how it applies to the points made herein. The issues with the three-cueing system are well documented and widely available. If you are unfamiliar with this system, I suggest that you research the issue and its history. From my experience as a person that learned phonics as a child and as a parent with children that were taught with three-cueing at school, and seeing the complications it has caused, I have strong opinions on this topic. ↩︎
A study asked a group of student volunteers to read the first 7 paragraphs of Bleak House by Charles Dickens. Subjects were asked to rewrite the passage in modern English, with free access to to dictionaries, reference material, and their phones. 58% were unable to understand the material to the point that they would be unable to read the novel alone. Only 5% were found to be proficient and properly understood the material. These students were English majors. See They Don’t Read Very Well: A Study of the Reading Comprehension Skills of English Majors at Two Midwestern Universities. See also Teaching with Whole Books Boosts Comprehension and Engagement for further reading; this article extensively documents the challenges with slipping reading comprehension. ↩︎
There are a variety of causes of reduced reading comprehension, and the use of flawed methods to teach new readers is just one. While I will explore this more, I should be clear that this can’t be entirely blamed on three-cueing. ↩︎
I talk more about the view of seeing online content in terms of its half-life in Write Like You Are Running Out of Time. I find this to be a particularly useful way to view the effective lifespan of what we write. ↩︎
Exact figures for the book industry are notoriously hard to find. Much of the data is treated as confidential, some publishers & platforms don’t share useful data, including Amazon, making it difficult to find consistent and well-sourced statistics. You will likely see somewhat different figures from different sources, so these should be seen as illustrative instead of exact. ↩︎

Why I Will Never Write With AI

Sat, 27 Dec 2025 08:29:22 +0000

Every person has rules that they apply to themself, red lines that they won’t cross. For me, signing my name to anything generated by AI is one of those. I’d like to take a few minutes to talk about why.

I spend a lot of time writing. Some is on this site, though much more of my work isn’t public, is published elsewhere, not yet published, or ghost written for others. On this site, according to the stats, I’ve spent at least 864 hours writing the content you see here. That’s over a month of continuous writing. That’s over a month of my life dedicated to sharing information and working to prompt further discussion and contemplation. Yet I never have, nor ever will, sign my name to anything written with AI¹.

For some readers, this will seem to be a given due to objections² to generative AI. After all, generative AI may be the most hated technology since the industrial revolution³. For others, they focus on benefits and the ability to “create” more content in less time, or overcome physical limitations⁴, or write in languages they haven’t mastered, or any of a thousand other reasons they could cite. This essay isn’t about either of those positions.

In fact, this article isn’t truly about AI at all. While it does center on generative AI, it’s not the AI that matters here. Allow me to explain.

Writing as Creation

Whether one writes to document a system, answer a question, share experiences, or tell a story, it’s a form of art. Conveying knowledge and understanding to another through crafted prose and thoughtful constructs, this is both an art and a critical skill. For those of us that work in the technology industry, our ability to clearly and effectively convey information is vital to our success. For those that write to tell stories, true or fiction, they raise this art higher, to convey emotions.

What starts as a simple effort to express a thought or idea becomes a craft with experience, and an art as skill grows. Each word selected to achieve the desired effect. Intent laden and time consuming to do well (or well enough, as in my case⁵), good writing requires more than a catalog of words and a working understanding of a language. It requires an understanding of the reader, their experiences, their knowledge, their perspective. The best writers use their knowledge to write for not a single reader, but many readers, each with different backgrounds. They blend all this carefully, thoughtfully, and with exacting detail.

When you encounter anything from a technical analysis to a fictional tale, if you walked away understanding more than you did before, it’s because of that careful intent. And intent is what this article is actually about.

Why I Started Writing

Starting when I was frequently answering programming questions in forums as a teenager, providing detailed and lengthy answers to complex questions, it was clear that the single most valuable skill I could develop wasn’t memoizing more development trivia or adding more programming languages, it was writing. Clear, careful, thoughtful, and targeted communication is the difference between building a career and being sidelined. Ensuring that others understand exactly what you need them to understand⁶ is the single most important skill anyone can develop. Without that, other skills are far less useful.

I did not start writing for the sake of writing, or for the sake of creating, but as an effort to develop a critical skill. Many millions of words later, this skill has served me well, and is one that I continue to invest in.

In the nearly 1,000 hours that I have spent writing content for this site, it has never been part of my income, it has never paid a single bill. Essays such as this, are instead a gift, given freely to readers, in hopes that it will spark discussion or thought. The only benefit I receive for the hours invested is an opportunity to learn new things through the research required, and some slow polishing of the skills I’ve developed for conveying information and ideas.

AI as a Cheat-code

For many readers, the letters iddqd will be immediately recognisable, the ultimate cheat code for Doom. Full health, no damage. Or maybe dnstuff was your ultimate cheat; granting all keys, weapons, and ammo in Duke Nukem 3D. A few keystrokes and the challenge was gone. You could play through the game quickly, easily, and without a worry.

When the game was over though, nothing was accomplished. There was nothing to be proud of. No skill was improved. No knowledge gained. It was done, but a meaningless and empty event.

In the mid to late 1990s, I spent some time making custom maps for both of these games, this included countless tweaks, hundreds of tests, and hour upon hour of playing the same levels over and over. While working on the maps it made perfect sense to use the cheat codes. There was a time and place where the point wasn’t the journey, but getting to the end to achieve something else.

AI, for many, is a cheat code. It’s a way to move through a task quickly, to get to the end without investing the level of time, energy, effort, or thought otherwise needed.

There are times, in my opinion, that using a cheat code makes sense. You need to get around something that would take time, you wouldn’t get anything useful from the journey, and it’s in the way of getting to something more important.

There are other times that using a cheat code only cheats yourself. You miss out on the journey, the research, the knowledge, the experience. Generative AI has proven itself to be a highly effective cheat code for many, especially when it comes to writing. A person can spend 30 seconds writing a prompt, and they get a 1,500 word essay. An essay devoid of intent. An experience devoid of benefit.

The Journey Matters

If you’ll indulge me, I’ll continue with the cheat code analogy.

Some games can provide deep immersion, great acting, compelling stories, and journeys that matter far more than the destination. A few examples, for me, have been Expedition 33, Baldur’s Gate 3, Cyberpunk 2077, and Alan Wake - all of these can provide a remarkable journey when you invest the time and effort into them. But, there’s a better example: MyHouse.wad⁷. This appears to be a custom map for Doom, one that is superficially simple. However, through exploration, there is vastly more to be found than what appears on the surface. In reality, it’s an expansive experience that goes beyond the carefully crafted in-game journey.

I use MyHouse.wad as an example here, as it’s a perfect analogy. If you take the easy route and complete the map quickly, if you judge the game by what you see on the surface, you miss a journey that will stay with you for weeks. Worse, you’d have no idea what you had missed. Missing out and not even knowing it simply adds insult to injury.

Some time ago, I wrote an article about how aviation safety lessons can be applied to security. This required weeks of research, identifying source materials, reading countless PDFs and investigative reports. This was one of the most time consuming articles I’ve ever written. It was also a great experience, and gave me a number of opportunities to learn more, dive deeper, find new parallels, and improve the work.

Today, with a well structured prompt, I could produce a blog post on the same topic in about 5 minutes. It would read well enough and it would likely make good points⁸. Yet it wouldn’t have intent, it wouldn’t have lived experience, it wouldn’t have insight that built on understanding of separate industries that can learn from each other.

Yet, going with the cheat code route would have cheated me out of valuable opportunities to learn and grow. It also would have cheated readers out of their valuable time. Asking a person to read, to take their time and attention, and spending it on something that wasn’t worth the time to write is simply insulting. Time is too limited and too valuable to ask another to waste it.

Everyone is Doing It

As frequent readers are likely aware, I’ve written quite a few articles about AI - from impact to jobs to debating the questionable creative value of AI content, and even explored far future implications of artificial life (and more). I’ve explored these topics as, regardless of how anyone feels about it, generative AI is one of the most consequential technologies developed in decades. Even if one believes that it’s all hype, that doesn’t change the impact it’s having to the economy, jobs, and the growing impact to how people work on a daily basis.

There are wide gulfs in how people perceive generative AI, and nuance is becoming harder to find in these conversations. Yet the reality is that use is spreading, and in some areas, it’s nearly impossible to avoid. At this point I would argue that nearly all codebases with a non-trivial number of developers includes AI generated code⁹. The only question is if developers and companies are willing to be honest about that fact.

In management, there is fear of being left behind if it’s not used, that competitors that embrace it will be able to pull ahead. Venture capital firms make it clear that heavy AI integration is now table stakes - new startups that don’t make extensive use of AI are considered to not be viable. Universities increasingly make AI usage a standard part of their programs, as it’s assumed that understanding AI will be necessary to work in the future.

Saying that everyone else is doing it is an argument that can be seductive, yet changes nothing. It’s still a cheat code and using cheat codes comes with consequences. One should understand those consequences well before using them, and in which scenarios it’s appropriate for them.

Intellectual Integrity

Some people will use AI for everything, others will use it for nothing. For me, I’m somewhere between these extremes. I would rather spend my limited time to create and spend time with those that matter, and if there’s a cheat code that’ll allow me to waste less time, I will¹⁰. Time is precious, too precious to waste.

When I say this, I’m not just talking about my time, I’m also talking about your time. And I would like to thank you for your time, your attention, and the opportunity to hopefully inspire more detailed and nuanced consideration of a divisive and complex topic.

Like every article on this site, this was written by hand, one keystroke at a time. Like every other article, not a single word was written by generative AI. That won’t change. Out of respect for those that take the time to read these articles, that will never change.

There are roughly three articles here that include AI generated content, used as examples for discussion, and are clearly labeled as such. ↩︎
The moral, ethical, legal, environmental, and economic issues related to generative AI and the companies that develop the underlying models is beyond the scope of this article. Countless articles have been written about these issues, some nuanced and exploring complex issues, others taking a more simplistic view. Addressing these issues in a fair way would require an effort far larger than the scope of this article. As the full complexity is too extensive to fairly explore here, these issues will not be addressed. ↩︎
A recent informal poll on Mastodon showed over 50% of respondents have adopted an approach of avoiding the technology whenever possible. The last major change in technology that resulted in such a substantial popular resistance was likely the Luddite movement of the early 1800s, driven by a group of textile workers that believed that changes in technology would reduce jobs & pay, lower quality, and worsen working conditions. This was exacerbated by a weak economy, with low job security, high underemployment, and raising inflation. This comparison is not made as a slight to those that oppose generative AI or the companies that power it, but to show the closest historical comparison. ↩︎
In a prior essay I shared a personal example of how this, to some degree, applies to me due to a rare hereditary degenerative neurological disorder. This impacts fine motor control and causes a tremor in my hands that has been growing over time, and results in escalating restrictions on my ability to engage in certain activities. The impact on my photography has been growing, and some activities, such as painting, are effectively impossible due to lack of precision. While this is not an argument that I will engage in personally, it is one that I am sympathetic to. ↩︎
Despite the title you see above, I do not truly consider myself a writer, but a person that writes. I’ve had the honour to work with true writers, and their skill and ability far exceeds my own. It is a craft I care about, though a title I don’t feel I’ve earned. ↩︎
I would be remiss to not take this opportunity to remind readers that when writing, or otherwise communicating, what one says matters far less than what others understand. No matter how clear you think you are, if others have a different understanding than that you intended, you have failed. This is one of the most challenging aspects of writing, regardless of venue or type. ↩︎
Instead of providing an overview, such as the one on Wikipedia, if you are unfamiliar with this, I strongly recommend this YouTube video which includes a play though and explanation. This is a long video, at over an hour and a half, though it provides the best way to experience this that I’ve found. This is a remarkably well constructed experience, and one that is well worth the time. ↩︎
While writing this, I asked ChatGPT & Gemini to do just this: create an article covering the same subject matter, to see how they would compare. Both created reasonable drafts with useful though incomplete explanations. Both were at or above the quality one would expect to see on LinkedIn or average corporate blogs. Gemini did provide better narrative and prose, though the insight was lacking and was far from complete. Neither were of sufficient quality to reproduce for the sake of comparison. ↩︎
Based on industry trends, surveys, private conversations with various organisations. For codebases that are under active development, with a non-trivial number of developers, it’s almost certain that either the company is supporting the use of AI generated code, or developers are using generation tools independently. There will be some exceptions, though the number is almost certainly far lower than some would expect. ↩︎
I wrote more about my take on tools that improve productivity in Developer Tools & Productivity. ↩︎

25% Unemployment in Tech?

Tue, 23 Dec 2025 14:05:49 +0000

For more than a year there has been a clear trend: unemployment in the tech industry was climbing. The number of people with an #OpenToWork badge on their picture was climbing. There was something worse though, something that I hadn’t seen before: several months later, they were still looking. Something was wrong. Something had changed.

There are incredibly talented people with years or even decades of experience that are finding themselves spending anywhere from 9 to 24+ months to find new employment. People that are well established in their careers, people that have proven themselves, yet unable to find stable employment. People that just a few years ago would have had several offers within weeks, now unable to find even one after many months.

The strength and resilience of the job market has been collapsing.

A week ago I launched a poll on Mastodon asking a simple question: if you work in tech, are you employed, unemployed, or underemployed? From anecdotal observations, the issue was clear. I wanted to take it to the next step and start collecting data. Today I’d like to discuss the results of this poll, government unemployment data, tax law changes, and other forces that are reshaping employment.

The Numbers

Let’s start by taking a look at the numbers. If you’d like to skip ahead to the analysis, feel free. While analysing poll results and BLS data may be interesting, I understand if it seems a bit boring.

Poll Results

This poll, posted to Mastodon, offered three options. Here are the results:

Unemployed: 25%
Underemployed: 12%
Employed: 63%

This paints a picture that is both clear and disturbing. With 37% of people in the field that are either unemployed or underemployed, the job market is in a disturbing state.

This is an obviously unscientific poll, and there are a wide number of reasons that the numbers won’t reflect reality perfectly. From self-selection bias to lack of regional specific filtering, these numbers aren’t a statistically valid sampling. What they are though, is a reliable pulse-check of the industry. A way to get a feel for how the industry is doing, even if the specific numbers aren’t entirely correct.

While it’s easy to assume that selection bias has resulted in inflating the numbers for being unemployed or underemployed, there are other factors to be considered. One example are those that are either semi-retired or retiring early, because of the difficulty of finding full-time roles. There are also those that have left the tech industry entirely, in search of full-time work.

Based on anecdotal information, the number of people that have left the field or workforce is far from trivial.

The Official Numbers, U.S. Edition

If you look at the headlines of news reports in the U.S., you will see unemployment listed at 4.6%¹, though this is one of many different measurements. The Bureau of Labor Statistics publishes a variety of figures, the specific figure most people are familiar with is the U-3² count.

While U-3 is the standard count, it’s arguable that U-6 (currently at 8.7%) is the more insightful number, as it includes those that are unquestionably underemployed (working part-time), as well as those that have actively looked for a new job in the last 12 months but not in the last 4 weeks.

In our informal poll, the equivalent would be roughly unemployed + underemployed. That would be 37% - far higher than the 8.7% being reported. While we are comparing a tech-focused poll to the broader population, I suspect it would be unwise to assume that the difference is that large³.

Given that our unemployment numbers are so far off, perhaps there are other figures that are more useful.

If we look at the labor force participation rate, we see that it’s currently at 62.5%. This is the percentage of people that are either employed or actively looking for employment. Comparing these charts, it’s clear that in the U.S., the number of people that are part of the labor force has been dropping since 2006 when it was 66.4%.

While the percentage of those unemployed is similar to the 2006 level, the percentage of those in the labor pool has dropped by almost 4%, and has yet to recover to pre-pandemic levels.

Currently, this means that 37.5% of the population is considered to be outside the workforce, and thus not counted in the unemployment numbers that everyone sees.

The Analysis

With these numbers in mind, I would like to take a few minutes to explore what these results mean, how we got here, and look a bit more into what’s going on.

8,000 Applicants

Speaking to a recruiter at a pre-IPO startup⁴, they shared a few interesting details about their open roles. One of these was a fully-remote software engineer role, which had received over 8,000 applications. Another role, a very senior & specialised role had received over 250 applications, a role that a few years ago would have seen less than 20.

I have heard this from various recruiters, and has been building over the last three years or so. Hundreds of applications for roles that are specialised and require uncommon skills, roles that in the past I would have been excited to see two dozen applications. If you look at tech jobs on LinkedIn, you will see “Over 100 applicants” on nearly every one.

According to one hiring platform company, tech jobs are seeing the greatest number of applications per opening across all job roles⁵, with applications up 28% year over year⁶.

There are a number of factors in play with this massive increase:

Unemployment & underemployment are likely at unprecedented levels.
People are applying for roles that they are far over- or under-qualified for.
Substantial increase in fraudulent applications.

This is creating a hiring nightmare for applicants and hiring managers: a flood of applications that drown the viable applicants, hiding them in a sea of noise.

We’re trying to hire a single developer right now and it’s been an absolute nightmare because of the amount of fraudulent applications and scammers. It’s crazy, and I don’t know how anyone gets anybody hired anywhere for a tech job anymore. - Mike Sager

While the supply of those looking seems to far exceed demand, there is indeed demand. There are roles open, though identifying the right candidates is getting harder, and filling roles is taking longer.

Time to fill roles is growing, with one source indicating that the average for a tech role is currently 51 days⁷. However the time for someone to find a new role is, at least anecdotally, substantially longer. Searches of 6 to 12+ months not being uncommon⁸ among those sharing their experiences on social media (and observations of unemployed status on LinkedIn).

Substantially increased competition, longer timelines, roles that will never be filled⁹, potential ageism¹⁰, AI-based screening¹¹, longer and more complex interview processes - all in all, the experience of finding new roles in tech has become far worse in recent years.

Tax Changes

In the U.S., there was a major change to tax law that impacted how research and development expenses are treated in federal income taxes. Critically, this massively changed how the costs of software development are handled. As part of the Tax Cuts and Jobs Act of 2017, signed by Trump in December 2017, and taking effect on January 1, 2022, updated how capitalization & amortization are calculated.

To simplify the critical part of this change, since 1954 R&D expenses (such as building software) could be fully expensed in the year the costs were incurred. After this change, these costs were broken up over 5 years for costs in the U.S., or over 15 years for foreign costs. This resulted in delaying the write-off of tech expenses, and increasing the tax burden for companies.

By increasing the tax burden on companies, it gives them another reason to reduce hiring and limit staff.

As of now, with the enactment of the OBBBA, this change has been partly repealed - at least for costs incurred in the U.S. This includes limited ability for certain firms to implement this change retroactively.

However, the damage is done.

Generative AI & Job Losses

I’ve written before about job losses from AI, analysing the likely impact over the years. Across the U.S., there were nearly 1.2M jobs cut, with 153,536 in the tech industry (17% more than 2024)¹². There are estimates that as many as 55,000 layoffs are directly connected to AI, this is a complicated topic and the data isn’t clear.

One major challenge is that, from observing the market, there are likely multiple drivers for AI-related layoffs:

Automation replacing jobs.
Reduction in staff due to improved efficiency (real or perceived).
Cost-cutting due to AI investments.

There is little doubt that AI is resulting in job losses, and reduced hiring¹³, and it’s the reduced hiring that’s likely having the largest impact. Given the negative media coverage that are associated with layoffs, partial or complete hiring freezes are able to largely stay under the media radar, and avoid unwanted attention, while allowing natural attrition to reduce headcount.

Given survey responses, it’s likely a safe assumption that the number of jobs lost to AI is predominantly in the form of roles that would have been opened otherwise.

Looking Ahead

The more time one spends looking at employment data, the more weakness that is seen. From the increasing number of people that have left the workforce, the reduced number of jobs being created (down 35% from last year¹⁴), the increasing timelines for those seeking employment to find a full-time role - it all points to a weakening economy. Furthermore, AI is leading to layoffs and eliminating jobs before they are opened.

I try to end my analysis on complex topics like this with useful advice, positive notes, or otherwise focusing on the silver lining that exists in most situations. In this case, I struggle to find something in this data that leads to optimism.

Good luck out there folks. There’s nothing else I can say.

While some will argue the accuracy of officially published figures from the U.S. Government, this is the data that’s available and the data that most people will see when looking at media reports. ADP has released data that indicates greater job losses than is being reported by official numbers, some in the U.S. Government believe that the official numbers are overcounting new jobs. I will not be arguing the accuracy of the data here, only that it doesn’t present a realistic image of the state of the tech industry or the experience of those that work in the industry. ↩︎
This is roughly defined as those that 16 or older, not working, and has actively looked within the last 4 weeks. BLS publishes detailed definitions. These are very much worth reviewing to better understand how they view and count employment and unemployment. ↩︎
Based on OEWS data published by BLS, in the U.S., approximately 4% of roles (6.95M of 154M roles) are directly tech-related roles. This is almost certainly undercounting, as it relies on specific job codes, which won’t account for those that are tech-adjacent, being entirely dependent on the health of the tech industry, but may not be coded as such. This data is from May 2024, which is the most recent available. Given the lack of recent data, and the limitations of how often the data is updated, it’s not useful to understand trends over shorter timelines. ↩︎
The company is a FinTech startup that you’ve likely encountered, though its name may not be familiar to those outside of the FinTech space. As such, roles at this company are unlikely to draw the level of attention that would be seen from companies such as Google, Meta, Amazon, etc. ↩︎
The top-level category that includes most tech roles, “computer & mathematical occupations” are seeing 65 applications per opening. Other sources place this figure as high as 83. It should be noted that this does include other job types. iCIMS Insights, December 2025. PDF ↩︎
iCIMS Insights, March 2025, Page 5. PDF ↩︎
iCIMS Insights, March 2025, Page 7. PDF ↩︎
BLS data indicates that across all those currently counted as unemployed, 24.3% have been unemployed for 27 weeks or more. As noted elsewhere, this is based on the 4.6% that are counted as unemployed, not including the 37.5% of the population that they no longer count as part of the labor pool. ↩︎
Greenhouse reported that 18%-22% jobs are ghost jobs that won’t be filled. Another report from LiveCareer lists 45% of HR professionals admit to regularly posting ads for ghost jobs. Some of these ghost jobs are being posted for SEO purposes. ↩︎
Those that are 55+ make up only 5% of applicants, 45-54 are 7%, and 35-44 are 10%. This supports concerned raised herein and elsewhere that an increasing number of people with substantial experience are leaving the field. I will leave further exploration of this issue as an exercise for the reader. iCIMS Insights, March 2025, Page 8. PDF ↩︎
This report from iCIMS is worth reviewing to see how AI is being used, and how companies and applicants perceive it, is worth reading. Determining the wisdom, value, and impact of this trend is left as an exercise for the reader. ↩︎
Challenger, Gray & Christmas, Challenger Report: 71,321 Job Cuts on Restructurings, Closings, Economy, Dec. 2025. ↩︎
Per a survey by the Federal Reserve Bank of New York, 23% of respondents indicated that they planned on reducing hiring due to AI. ↩︎
“Through November, U.S. employers have announced 497,151 planned hires, down 35% from the 761,954 announced at this point in 2024.” Challenger Report: 71,321 Job Cuts on Restructurings, Closings, Economy, Dec. 2025. ↩︎

Good Faith, Moral Duty, and Selfishness

Wed, 10 Dec 2025 03:23:18 -0500

What do we owe to each other? This is a classic question of moral philosophy, and a critical question that defines how society itself functions. It’s also a key question to understand what role each of us plays to make society successful.

These words were started as society was entering a seismic shift, the COVID-19 pandemic had started, people were dying, fear ran deep, and the disease impacted some far more than others. Some reactions to this crisis were based on what was good for society, others came down to the interests and desires of the individual. The question of what we owe to each other had rarely been so imperative.

I am finishing¹ this as society enters another seismic shift. Fear again runs deep, there will be deaths, people are divided to an extent not seen in generations, and, as like before, some see little or no impact while others are disproportionately affected. The question of what we owe each other has not been more pertinent during my lifetime than it is now.

Before we go on though, I’d like to make one thing clear: this essay is intended to spur thought & consideration, and to encourage readers to research this topic more. There is no possible way that this essay can fully cover the topics discussed here. There are places where a single sentence summarises a specialised area of research that a number of people have dedicated their careers to.

To fully cover these topics in depth, it would be to synthesise the work of hundreds of researchers and lifetimes of effort. As such, this is presented not as a definitive statement on these topics, but an introduction to spur further reading.

Why are we like this?

To discuss this, it’s useful to have an understanding of how and why we got here. If social psychology² isn’t your cup of tea, feel free to skip ahead.

Selfishness

Humans are, arguably, born selfish and self-serving, from crying strategies to prevent competition for resources to the “mine” stage toddlers go through claiming ownership over everything within reach. Within the first years of life, humans show themselves to often be uninterested in the needs or feelings of others - it’s part of our DNA. While each child develops differently, and shows care for others at different points & frequency, this selfishness is something that should be only temporary.

We grow, we realise that others have feelings, we learn to share our toys, we develop empathy & understanding. While we may have an innate predisposition to this behaviour (we do come from a very long line of social creatures), it’s something that must be fostered to develop properly.

As time goes on, our sense of self develops more, and we perceive ourselves not just as an isolated individual, but associate with others around us³. We transition from “I” to “we” - we see ourselves as part of something larger, and we learn what our role is, we learn where we fit in.

Why does this matter?

This growth⁴ is necessary for society to function, and the evolutionary steps to achieve this predate the existence of our species by millions of years. In a world without empathy, understanding, or social connections, it would be a world of violence, hate, no cooperation (such as via governments, businesses, or even farms & small communities). There would be no art, no science, no commerce, nothing but a constant and independent fight for survival.

Society exists and functions solely because we have evolved to think of not only ourselves, but others.

Due to some combination of environment & other external influences, not everyone manages this growth. Their worldview progresses little beyond themselves. Once a person learns that there’s a world beyond their own wants and desires, there’s still a bigger challenge to come.

Us versus Them

People perceive themselves as belonging to a group⁵ (and any number of subgroups⁶), and see everyone else as part of an out-group. Thanks to an interesting psychological effect, people see those in the out-group as more different from themselves than they are, just as they those that are part of the in-group as more similar to themselves than they are.

This perception is a source of out-group homogeneity, the belief that “they” are all the same, interchangeable, whoever “they” are. Conversely, it creates in-group heterogeneity, the view individuals who are part of the in-group are more distinct.

All of this compounds into something interesting, and deeply unfortunate. Our ability to understand people is impacted substantially by our primary group that we identify with. Our ability to see commonalities is driven by that group. Our ability to place ourselves in the shoes of another, to feel empathy, is dictated by that group.

The smaller and more homogeneous that primary in-group is, the harder it is to understand those in the out-group. This is a result of there being fewer points of reference to establish commonalities, fewer opportunities to see things from another point of view.

Out-Group Apathy & Antagonism

This in-group centric worldview leads to something far darker, moral exclusion. Members of a group can see their values, history, norms, and culture as superior to those in the out-group. This leads some to see themselves as superior, and treat those in the out-group as lesser - not deserving, not worthy.

This can range from disregarding others and not considering them at all, to outright dehumanising. This is form of belittling and demeaning groups is all too common, from false narratives to marginalising people and their value and contributions. Among those that see themselves as members of a small and limited in-group, they find little common ground, and consider much of the world as being in an out-group that is lesser than their own.

It is this in-group superiority that has contributed to many of the worst atrocities in human history.

By arguing that those in the out-group are of lesser import, they then descend to arguing that others are not deserving of the same rights they enjoy. When you combine these views, it leads people down the darkest of paths and into the depths of evil humans are capable of.

It can start slowly, with hurtful but otherwise small impacts, steadily growing more overt, more direct, more dangerous, and more destructive actions. We see this today in a variety of ways, and in far more overt ways than would be expected even a few years ago⁷. There’s no reason to believe that this trend is slowing down, much the opposite.

Large vs. Small In-Groups

The smaller and more homogeneous the in-group a person identifies with, the less opportunities a person has to build understanding and empathy with others. If a person’s self-selected in-group is based on highly specific filtering factors (e.g. a combination of nationality, ethnic background, religion, socio-economic status, gender), the more likely a person is to view large portions of the population as an out-group that they have negative perception of.

The broader and less filtered the in-group, the more opportunities for understanding, connection, and empathy. The ultimate - though likely rare - in-group is all humans⁸, without a true out-group.

For me, this belief in the importance of a broad in-group that extends beyond common filtering functions is rooted in a document that I read many years ago, which included a vital line for me:

We exist without skin color, without nationality, without religious bias… Loyd Blankenship, The Conscience of a Hacker

This line resonated with me in a way that few things did; it addressed the view that many of the things that separate of are, in the end, artificial constructs that have little truth. There are things that we have in common, there are things we don’t, but in the ways that matter, we’re all alike.

We’re all people with emotions, hopes, fears, dreams, nightmares. Everything else are those things that make us interesting, give us different perspectives, new ideas, and add to the richness of human interaction.

What We Owe Each Other

This is a question that is responsible for numerous books being written, countless hours of lectures in philosophy classes, and millennia of debate. I will not attempt to summarise this here, but rather address this in more direct terms.

it’s about making choices that benefit not just yourself but also the people around you, fostering a society where everyone can thrive

When the COVID-19 vaccine was made available, I booked my appointment the first day I could. I didn’t do this because I didn’t want to get sick, it was in despite of the fact I knew I would. My body doesn’t react well to many vaccinations; the COVID-19 vaccine had me in bed for the next 4 days and it was well over a week before I was back to normal. I knew what I was signing up for and yet I did it, because I didn’t want to risk getting anyone else sick⁹. A inconvenience for me, for something that could be life or death for someone else. It’s what I owed them. It’s what we owed to each other.

When a friend came out to me as transgender, my response was simply to ask what if there was anything I could do to support them. Why? Because we all deserve respect, we all deserve support from those around us. It’s what I owed them.

When a neighbour was having a yard sale selling things that she obviously needed (such as her baby’s car seat), making it painfully clear that she was desperate for money, I picked out something unimportant for $1, then handed her a $100 bill and told her not to worry about the change. Why? She needed it more than I did. I’ve no idea what was going on in her life, but she needed help, and I could help. It’s what I owed her.

Paying for other people’s groceries when they don’t have enough money at the checkout. Because nobody should be hungry. Paying for other people’s medicine because nobody should go without healthcare. I did these things because it’s what I consider to be the least I could do, because this is the behaviour that we owe to each other. These things deserve no thanks, no credit, not even any acknowledgment. These things should be the minimum that any of us should do, if we are able.

We thrive together, or we fall together. That’s what it means to be part of a society. It means we help each other, we support each other, we respect each other. It’s what we owe to each other.

The Golden Rule

If we do not stop to help each other, what will we become? Jeff Atwood

Most are familiar with the “golden rule” - treat others are you wish to be treated. This is a foundational norm in a healthy society, the root of what’s good & right in the world. This presents the most basic check for right and wrong, one can simply ask themselves the question: how would I feel if the roles were reversed?

Those that fail to ask this most simple question will often act instead in the interest of themselves (directly, or for the perceived benefit of their in-group), and fail to comprehend the true impact of their actions. A simple lesson that most learn by kindergarten, yet so many fail to keep this lesson.

Countless words have been written in an attempt to answer the question, what do we owe each other, though in reality, there is a simple and concise answer: treat others are you wish to be treated.

This was drafted in February of 2025, based on a draft that began in 2020. Much has changed since the first words were written; fears have been realised, polarisation has intensified. This article isn’t intended as a political discussion, as I try to avoid overt politicisation here, but a reflection on the challenges that we face. This is a complex topic, and one that I have sat aside multiple times. I don’t believe that this is a political question, but a growing societal issue, and it’s important to consider the various roots of this problem. ↩︎
While I have a deep interest in - and have spent substantial time studying - both psychology and philosophy, the key areas of focus of this article, please read this as a layperson’s exploration of a complex topic instead of an expert analysis. ↩︎
This is a vast oversimplification, based on self-categorization theory, presented in the simplest possible way for brevity. ↩︎
To be clear on this, this is not a reference to neurodivergent people. Nothing herein is intended in any way to criticise, demean, or otherwise belittle those with autism or are otherwise neurodivergent. If anything here is interpreted otherwise, then I apologise for the lack of clarity. ↩︎
At the highest level, the group is humans with the out-group being anything living that isn’t human. In practice, the group is substantially more specific. ↩︎
The model presented here of a group and various subgroups, may be better thought of as one to many groups with highly divergent cardinality and substantial (but not necessarily complete) intersection between the low cardinality groups to the high cardinality groups. The group/subgroup model is used for simplicity and brevity. ↩︎
There are many examples of the type of destructive and exclusionary behaviour that is occurring in today’s environment, though I do not see either a need to repeat these, nor do I see a reason to elevate these harmful statements by recounting them as part of this essay. Hateful statements do not deserve such dignity. ↩︎
An in-group of everyone is, to me at least, the ideal. However, due to the realities of human experience and the environments that we are raised in, this self-selected in-group is likely less common than I would wish. I do believe though, that an aspirational in-group is not substantially different than the effective in-group, as it represents an intentional effort to understand those outside the effective in-group. ↩︎
One of the people I was worried about exposing to COVID-19 was my father, due to his existing health conditions. He would later be exposed COVID-19 while in a medical facility, passing away soon after the symptoms became apparent. Had I not been diligent in taking precautions, I could have been responsible for accelerating his passing. The decisions we make matter, even when we have to pay a price for them. ↩︎

Developer Tools & Productivity

Tue, 09 Dec 2025 01:00:09 +0000

Technology improves and advances ceaselessly, new tools are created and change how people work. Some are small and simple, making people somewhat more productive. Others revolutionise the way people work. These revolutionary tools may come along only once or twice in a generation, and when they do, they tend make people uncomfortable. They can make people question their role, their skills, their future, and their place in the industry.

I would like to take a few minutes to talk about a revolutionary change in how developers work. I kindly ask the reader to reserve judgement on this topic till the end of this article, to fully understand the intent.

Shifting Paradigms

Developer tools have historically been hard to use, complex, and required extensive and specialised knowledge to accomplish anything beyond a simple “Hello World” program. While there have always been efforts to make this better - new tools, new libraries, new frameworks - these made the work more efficient, made developers more productive, but there were still few that could do it well.

This difficulty worked as a moat, a gatekeeper, that kept many people out. Not because they couldn’t learn, but because they had limited time. Not because they weren’t motivated, but because they had a job that took up their time. Not because they were lazy, but because they would prefer to spend their time with their family.

For those that hadn’t had the opportunity to invest the time to learn, they were largely locked out.

There are some that see this as a good thing, as it keeps those that aren’t qualified, those that aren’t dedicated enough, those that haven’t paid their dues, it keeps them out of the field. It ensures that only some can build software.

More importantly, some argue, it keeps them from making messes that others will have to clean up. It keeps them from creating work or introducing bugs that others will have to fix later.

Not all people learn the same way, not all people have the same opportunities to explore and gain skills, not all people have the time to dig into things to the same level. Most importantly, to me at least, some people would just rather spend time with their family. If a tool can help level this field, and help bring more people in - I see it as a good thing.

This is an exclusionary view that I tend to disagree with, though I’ll talk more about it below.

Productivity Matters

When these revolutionary tools are released, it doesn’t only open the door for new developers to join the field, it changes how many current developers work. It allow people to do more in less time. It means it’s easier to complete tasks. It opens the door to going home on time and spending time with loved ones.

For my entire career, I’ve always had a focus on productivity: I care greatly about the quality of my work, and that of the work that my teams do, but I also have a life. I also have a family. This means that the more productive I am when I’m working, the more time I can spend actually having a life. While I do love what I do for a living, it’s not my only priority.

When I’ve managed developers, my goal was to enable them to do their best work, and get it done as efficiently as possible. If you want a happy team, you always make sure that they can actually enjoy their life. That they don’t feel a need to be working constantly.

If there’s a tool, framework, or library that makes me more efficient - without a loss in quality - you can bet I’ll use it, and I’ll make it available to my team (if they want it). I’ve devoted countless hours to building private libraries and frameworks to allow my teams to do more, better, faster. These are some of the accomplishments that I’m most proud of.

Embracing Change

I’ve had many debates over using new tools, and adopting technology that could reshape development. From dealing with fears about jobs being taken over by business analysts that have no background in development, to the worry of new bugs introduced by people that don’t have a traditional computer science background.

These conversations are always complex, always laced with fear & discomfort, and deep worries. Some don’t want things to change. They like the status quo. They like the gatekeeping. They like the feeling of superiority.

By now, you may be thinking that I’m speaking of a particular technology, and I am - but almost certainly not the one you’re thinking of. I’m speaking of Visual Studio¹ and the conversations I had while it was reshaping how developers work.

Visual Studio, with the ability to design a UI with a drag & drop interface, integrated debugging, IntelliSense, and a host of features that made it easy for people to build software. This new breed of IDE allowed new developers to enter the field without needing to memorise complex Windows API calls to create a window or add a button. It allowed people to instantly spot syntax errors and easily fix them. It allowed them to easily find and fix bugs.

The wave of visual IDEs changed development. It opened doors. It welcomed new developers and gave them new & gentler ways to learn and explore. It provided hands-on experience without the level of frustration that had existed. It allowed them to focus on getting things done. It allowed developers to work faster. It allowed them to focus more on what they cared about.

Yet, decades on, I still hear the refrain that “real developers only need vi.”

I’m not talking about those that use the tools they prefer - if you just love vi, good for you - but those that look down on those that use other tools². Some believed, and still do, that if another’s priorities vary from their own, they are doing something wrong. Unless you were using C or C++, you were doing something wrong. If you didn’t opt for the most complex path to achieve something, you didn’t belong.

There are still those, decades on, that see using tools that make people more productive as polluting software development. There are still those that want the gatekeeping.

I am a firm believer that each person should use the tools that are the best for them, and allow them to focus their limited time on this planet on those things that actually matter to them.

I will leave applying this to other technologies, if and as appropriate, as an exercise for the reader³.

This could also be Visual Delphi, Macromedia Dreamweaver, Microsoft FrontPage or any of a thousand tools released in the mid to late 1990s, which started a shift from pure text editors into integrated development environments and similar tools that allowed those without a traditional technology background/education to build things that they wouldn’t have been able to in the past. ↩︎
[insert obligatory joke about people that use emacs on purpose] ↩︎
I apologise if this comes across as a rant, or if it doesn’t properly address your concerns about any particular technology. This post, in reality, isn’t about any technology, but about an exclusionary and self-superior mindset that I’ve seen repeatedly during my years in the technology industry. A mindset that I find deeply problematic. If you would like to argue that this is an oversimplification, I will grant that. All technologies have consequences, and have positive and negative impacts; a full debate on these consequences is beyond the scope of this post. ↩︎

On AI, Art, Writing, and the Distillation of Creativity

Mon, 31 Mar 2025 00:37:41 -0400

Can generative AI create art? Two years ago I took my first swing at answering that, at least from my perspective.¹ As AI systems become more advanced, this question, and the issues surrounding it have become of greater import. With a new release from OpenAI, it’s become a topic of great passion, and one prime to revisit for me.

I would like to explore this topic more deeply than I did previously, both in terms of cultural impact, and historical context. This is something that is easy to take an emotional position on, though as someone that considers themself an artist, it’s worthy of more nuanced examination. I’ll be touching on art broadly, photography specifically, and unlike my last essay on the topic, I’ll spend some time discussing writing.

The Studio Ghibli Connection

Let’s start with the recent events that has brought this to the fore. OpenAI released an update to their image generation feature, using their newer 4o model, to create images and video that substantially improve over the results of prior models.

Shortly after this release, users discovered that they could direct the system to recreate existing images in the style of the beloved-by-many Studio Ghibli. This became a viral trend, and used for everything from creating cute family photos to the cruelest of political machinations. To say this prompted considerable debate would be an understatement.

Art, and the Unbroken Chain

In my prior post on this topic, I spent some time discussing the definition of art, and how it may apply to generative AI. I broke down each element of the definition, providing arguments for and against. In the end, I settled on something simpler than a technical definition: given that art depends so heavily on the eye, opinion, and perspective of the beholder, it may be best to paraphrase Forest Gump and say simply that “art is as art does.”

There is a great unbroken chain in art, starting with the first people that painted on cave walls, and connects over the millennia to the artists of today. Each and every one building on the work of others. No artist exists in isolation.

Every work of art, starting with those crude lines of the first cave paintings, has built on the knowledge, experience, style, and design of others. Some mimic, some contrast, some defy, and some refine, but all artists build on the work of others.

Even those art forms that create ex nihilo, starting with a blank canvas and an idea, draw from the knowledge, ideas, and visions of others. Each artist collects this knowledge, and distills it into their works.

Photography

I am a photographer, and I don’t mean that in terms of it being a hobby I’ve dabbled in. I was a photojournalist - work that has won awards (and death threats), I’ve paid my bills through sports and portrait photography, I’ve been a professional wedding photographer, and now focus on fine art photography.

I’ve invested a non-trivial portion of my life into this art, and learning everything I can about it. The physics of light, the mechanical design of lenses, the chemistry of film, the electrical engineering of digital sensors. I’ve studied countless images to learn what works and what doesn’t. I’ve studied the psychology of how people perceive photos, model posing, facial expressions, and perspective. I’ve refined my style over decades of effort.

Every image I create is a distillation of that knowledge. It’s the result of everything that I’ve learned, and applying that to a specific scene. When a person generates an image using AI, the AI system understands none of that. Yet, critically, it benefits from that knowledge.

Generative AI is trained on massive troves of data, including millions of photographs.² Through these photographs, this knowledge is effectively transferred by proxy; the system may have no understanding of why, but it does gain the understanding of how. Thanks to this training, generative AI still uses this distilled knowledge.

The Experiment, Redux

In the last essay, I shared an image I created using generative AI; today I include a revised version. The prior version was created in Stable Diffusion, after several hours of effort, this version was recreated with OpenAI’s 4o model.

As was the case previously, this image follows my style, my choices of lenses, my affinity for cityscapes, and of course my love of sharp contrasts. It was the result of substantial effort, and more time than I would invest in most of the photographs that I create. Of course, it’s not perfect, though reality is also rarely perfect.

One could argue that this is nothing but “AI slop” - cheapening the work of photographers, models, tattoo artists, and others. Or one could argue that it’s a new way to create photos with greater flexibility and freedom.

Is this art?

Today, as a photographer, I am as much of a purist as possible while using a digital camera. I use film emulation on the camera to achieve the high-contrast black and white style, and otherwise my photos are untouched. No edits, not even cropping.

While not intended to be a slight against those with a different creative process, I have said many times that “my art is photography, not Photoshop.” The idea of altering the moments I capture after the fact is entirely anathema to me.

To me, an AI generated photo is no different than a real photo that has been heavily edited in Photoshop: it’s the product of a tool, and the artistic value is found in the intent and vision of the person controlling the tool.

Is Photography Even Art?

Today, I would say that this question is absurd, though that’s not always been the case.

while the photograph is the mere mechanical reproduction of the physical features or outlines of some object animate or inanimate, and involves no originality of thought or any novelty in the intellectual operation connected with its visible reproduction in shape of a picture

On March 3rd, 1865, President Abraham Lincoln signed an amendment to the existing copyright laws that extended copyright protection to photographs in the US, a move that was later tested before the Supreme Court. There was a time where there was a real argument that photographs were devoid of creativity.

Charles Baudelaire, in a letter titled “Le Public Moderne et la Photographie” said, in a rather famous quote:

As the photographic industry was the refuge of all failed painters, too ill-equipped or too lazy to complete their studies, this universal infatuation bore not only the character of blindness and imbecility, but also the color of vengeance. […] it is obvious that this industry, by invading the territories of art, has become art’s most mortal enemy […]

Baudelaire was far from alone in attacking photography, and denying its value as a form of art. For painters especially, photography was a problem. Not only did this new technology intrude on their domain of capturing moments and the beauty of the world, it severely impacted their incomes.

Portraiture rapidly changed from paintings, where an artist would charge or weeks or months of work, to photography, where the entire process was down to a couple hours. The steady and reliable incomes that artists counted on to survive simply vanished thanks to a new technology. The parallels here should be obvious.

The Ability to Create

In my last essay, I should an AI generated “painting” based off on an idea I had. You see, despite quite a bit of effort, I have no ability to draw or paint. I understand the techniques, I’ve practiced, but I’m terrible at it. I simply don’t have the ability to translate what I can imagine into something that my hands can produce.

One could argue that lacking that ability, I have no right to create, based on the vision in my mind, art that requires those abilities. One could argue that using a tool to execute that vision renders the result hollow and meaningless, because my hands are unable to execute the details.³ One could argue that the only those with certain abilities should be able to execute their vision.

I would not make these arguments personally; as noted earlier, to me, the artistic value is in the intent and vision of the creator, regardless of the tools they use to execute it.

There is, quite obviously, the question of if a person is just generating something, versus executing on a clear vision. I’ll talk more about that later.

Writing

The social media post that pushed me to write this referred to the text generated by AI as “usually a blathering, vacuous, and erroneous essay disrespectful of the reader’s time.” I’m not going to disagree with that, but I feel like there’s both nuance and context that can be added.

Upon reading that line, the first thing that struck me is how often the same can be said for text written by humans. This is especially true of anything labeled as “thought leadership” or an annoying percentage of LinkedIn posts. Human’s are prone to error, too often don’t check facts, and too rarely are focused on grabbing attention over providing useful information or a perspective that’s actually thought provoking.

A common theme in my career has been serving as a ghost writer for others, providing the first draft of an article that will eventually be published under their name. These articles go through editing processes, they go through PR firms, and along the way, much tends to disappear. Too many of those first drafts, touched only by humans, have become the same “blathering, vacuous, and erroneous essay disrespectful of the reader’s time” that AI is criticised for.

Generative AI is highly effective at distilling what humans create, including both the good and the bad. If it has distilled this tendency in how we write, that may say more about us than it.

In addition to my own work as a ghost writer, I’ve worked with very talented writers doing the same. I’ve seen their early drafts. I’ve worked with them to prepare articles. For articles that broadly fit into the “thought leadership” category, there’s not much difference between their work, and what’s generated by the more advanced chain-of-thought LLMs, when used with a well crafted prompt.

My Break from Fiction

I spend some of my free time writing fiction, some is published here. Mostly dystopian, which I’m guessing isn’t a surprise to anyone reading this.

One of my favourite stories about LLM generated text, and one that started the process of changing how I see AI systems more broadly, came from an idea I had for a short story. It was a novel idea, a bit funny, but a little too absurd to justify the time it makes me to write. I quickly decided that I wasn’t going to pursue it.

Before discarding the idea entirely, I wrote up a detailed prompt for ChatGPT to see what it could do with it.

The result was funny, witty, just the right amount of absurdist, and creative in ways I hadn’t anticipated. Quite frankly, I was shocked by the quality. It was good enough that I stopped writing fiction for nearly a year, because it felt like a waste of time. It wasn’t perfect, but it was at least as good as the first draft of the fiction I’ve written.

I had to wonder if there was a point to writing at all, if an LLM could do in 30 seconds, what took me 40 hours.

Distilling Human Creativity

The power (and arguably terror) of generative AI is the fact that it is the distillation of human creativity. It’s the sum total of what humans have created for centuries.

It reflects us, sometimes in ways that we don’t like, because we don’t like what humans have created. It is, in many ways, a mirror. Why is it good at writing vacuous essays? Because humans have written a lot of them.

Just as we learn from each other, and create from what we’ve learned, so to do these systems. They generate from what they’ve seen, from what we have created. For good and ill.

It’s All About the Prompt

One of the oldest acronyms in computing, GIGO, applies today as much as ever. If you use a tool like Photoshop and randomly click buttons, you may end up with an image, but it will certainly be garbage. If you use generative AI the same way, you’ll get the same result. Garbage in, garbage out.

In my experiments, I’ve found that it’s possible to clearly articulate a vision, a structure, a point, enough detail that you produce something that at least resembles what you wanted. When using chain-of-thought (CoT) LLMs⁴, the results can be quite impressive. They are obviously still far from perfect, but in many cases I’ve seen results on-par with what a person would produce.

From my perspective, the prompts are where you go from garbage to something that may become art. It’s the intent and vision expressed in the prompt; it’s how the tool is used and guided, that imbues the result with meaning.

The difference between Jackson Pollock and a random person throwing paint at a canvas is intent and vision. I think there’s an argument that the same applies to generative AI.

Creative Jobs

While this goes beyond the core scope of this article, I would be remiss to end the article without touching on this more clearly. I recently wrote about the likely impact on jobs from AI, which as noted there will likely impact creative and knowledge-based jobs disproportionately. Just as photography put many painters out of work, so too will AI. It seems to be an unavoidable reality at this point. For many companies, AI will be “good enough” for jobs to be eliminated.

Generative AI has the ability to allow those without certain talents and abilities to create in ways they never could before, and at the same time to destroy the livelihoods of the artists that would have been responsible for that creation in the past. Unlike what happened with the introduction of photography, this isn’t a shift towards fewer people being paid less to achieve the same thing, but may represent a broad elimination of jobs.

As AI systems continue to evolve, the impact will only grow, and I believe that we, as a society, need to make broad changes to avoid the worst outcomes.

In Conclusion

Generative AI is likely many other tools, in that the quality and value of the results are dependent on the quality and value in the inputs. To me at least, it is simply another tool, another means by which people who are willing to put effort into it, can express a vision that may not be practical or even possible otherwise.

It’s also a tool that, when given input of little effort or value, will produce output of little value.

For those that have made it this far, may I offer you something amusing, that you may find thought provoking in unexpected ways. While we talk about the errors and potentially worthless nature of what generative AI creates, may I present something of purely human creation: English as She Is Spoke. The section on “Idiotisms and Proverbs” (p. 58) is quite worth the read.

I suggest reading this original post before completing this one, as it lays out my general perspective on the artistic value of generative AI, which this post builds upon. ↩︎
I will not be touching on the issue of copyright here; it is a complex area of law, and the fair use exception is even more complex due to its nature. Determining if the companies that build AI systems have violated copyright laws is a matter for the courts to decide. Personally, I do believe that there is a solid argument for fair use, based on my understanding of the caselaw in play, I believe that there is also a reasonable argument against it. This is a novel topic, and will require considerable litigation to determine where the fair use line sits. Due to the nature of the fair use exception, the answer can only be found through litigation. ↩︎
My family shares a genetic condition, a rare degenerative neurological disorder, that is most easily compared to Parkinson’s, though progresses slower and eventually stops progressing. One result of which is a small tremor in my hands and limitations on fine motor control. In practice, this, so far, has minimal impact on me, beyond the inability to draw and the need to use camera lenses with some form of optical image stabilisation to compensate for the constant movement of the camera. In the case of arts such as painting and drawing, the limitation goes beyond a lack of talent. ↩︎
Chain-of-thought (CoT) models outperform non-CoT models to such a degree, especially in terms of accuracy, that I don’t use the non-CoT models, except for the most basic of tasks. Using CoT models has fundamentally changed how I view LLMs. ↩︎

When AI Becomes I

Mon, 17 Feb 2025 18:39:48 +0000

One of the many joys of being human, is that we constantly face questions about our existence, from the seemingly simple (why is the sky blue), to the labyrinthine (what is the meaning of life, does pineapple go on pizza). Thanks to growing up watching Star Trek, one of these that has fascinated me is the question of artificial life. Thanks to a character named Data, a character that’s both relatable and entirely different, many have found themselves wondering if that’s what the future holds.

This essay began 3 years ago, when some within Google claimed that they had, unintentionally, created an AI system with sentience. A claim that was quickly met with derision. For me though, it was fascinating - not because of claim itself, but the challenge involved in finding the truth. The challenge of finding incontrovertible proof.

This is a topic that has fascinated me for decades. As we building more advanced systems, build LLMs of such complexity that we no longer truly understand their behaviour, observe nascent signs of emergent behaviour, how we interact with these systems is changing in fundamental ways. Marketing parlance has already started to shift to hiring AI systems. The latest AI agents are being designed to act and be treated as just another member of the team instead of a piece of software. Then there’s the soaring popularity of AI companions.

The anthropomorphisation of AI has well and truly begun, both naturally as a result of advancing technical complexity, and intentionally through the marketing and design of the systems.

The question is, if we reach the point where a technology actually becomes some form of life, can we tell the difference between that, and the human tendency to anthropomorphise?

Biological roots

While we generally believe that we know a great deal about life, organic life, in reality we’ve only scratched the surface. Despite a vast number of years and huge amounts of money, no lab on has been able to achieve abiogenesis. Try as we might, we’ve not been able to create life ex nihilo, to take raw ingredients and create something that’s alive, no matter how simple.

A great deal of effort has been invested into finding LUCA, the last universal common ancestor, the thing all life on Earth evolved from. But we don’t know what came before LUCA, we don’t know where LUCA came from, we don’t know how LUCA started.

And that’s just life of Earth. As research finds ever more evidence of the abundance of organic compounds in the universe, it’s all but certain that life, of some form, exists elsewhere.

Given this limited knowledge of life and how it starts, where the line is between not alive and alive at the most primitive level, it makes one question our ability to recognise it in forms of life that are dramatically different from what we know.

Organic vs. Non-Organic Life

Note: This section was written in part by ChatGPT¹. Given the topic, it seemed only fair.

Organic life, to the extent we understand it, is rooted in biochemical processes - cells that metabolise, reproduce, and evolve over generations beyond count.

This definition is built on a foundation of shared genetic material, chemical reactions, and an intrinsic drive for self-preservation. Organic organisms, from single-celled bacteria to complex mammals, operate on principles of energy transformation and homeostasis, ensuring that life continues through mechanisms honed by billions of years of evolution.

Non-organic life, by contrast, challenges our conventional notions of existence. Imagine a system built not of cells and DNA but of silicon and code, yet still capable of self-maintenance, adaptation, and even self-replication. In this view, life becomes a matter of functional attributes: autonomous goal-setting, emergent complexity, and the ability to interact meaningfully with the environment.

Here, the “metabolism” is digital, the reproduction algorithmic, and the evolution driven by iterative learning and self-modification.

The quest to differentiate organic from non-organic life is - at least - as much philosophical as it is scientific. While organic beings carry the tangible legacy of evolutionary biology, non-organic entities might express their ‘aliveness’ through the spontaneity of novel problem-solving, the emergence of self-awareness, and behaviours that surpass their initial programming.

These traits force us to question whether the essence of life is inherently tied to its physical makeup or if it resides in the dynamic processes that support existence, regardless of their substrate.

Ultimately, the distinction between organic and non-organic life may prove to be more of a spectrum than a binary divide. As artificial intelligence systems evolve, they could begin to display properties we once thought were exclusive to living organisms. This convergence invites us to broaden our definitions and reimagine what it means to be “alive,” suggesting that the journey from being a mere tool to becoming a living entity is as much about expanding our conceptual horizons as it is about technological advancement.

There is a further complication to this, one rooted not in biology but psychology: anthropomorphism. There’s a human tendency to attribute familiar, often comforting, human traits to non-human life and inanimate objects. As we interact with increasingly sophisticated AI systems, our minds are predisposed to see intentions, emotions, or consciousness in behaviours that are simply the result of the system’s complexity.

This bias can obscure objective evaluations, leading us to interpret patterned, algorithm-driven responses as evidence of self-awareness or autonomy. Recognising and mitigating this tendency is essential, not only to avoid overestimating the “aliveness” of AI systems but also to ensure that our criteria for life remain grounded in observable, reproducible phenomena rather than our psychological need to relate to the world in human terms.

What is intelligence, anyway?

This is a word that can be defined easily, but the question itself is incredibly profound, and with substantial implications for how we live today, ignoring the possibility or non-organic life.

the ability to learn, understand, and make judgments or have opinions that are based on reason - Cambridge Dictionary

Each time you give a command to a dog, such as to leave something alone, you are seeing all of these things in action. Each time a dog asks to go outside, once again, you are seeing all of these in action. Each time a cat stands on a shelf, looks at you, and knocks something off, you’re seeing all of these things. If you’ve spent more than a few hours with a parrot, you’ll be amazed at how much they have in common with a human child. This list goes on almost endlessly.

We see non-human intelligence around us constantly. We see non-human sentience around us every day. We see non-human emotion around us every day².

One reason it’s easy for us to recognise these in other animals is that thanks to common ancestors and similar evolutionary pressures, we experience these things in fairly similar ways. While human brains are far more complex, they aren’t so profoundly different that we experience the world in an entirely different way.

When thinking about non-organic life, life that is truly entirely different from us, it might be useful to think about an example of organic life that shows sentience, intelligence, and emotions, but has very little in common with us: octopi.

The octopus may be one of the most fascinating and historically misunderstood animals there is. The Octopus vulgaris has about the same number of neurons as a dog. They can solve complex puzzles, learn patterns, enjoy receiving attention, and even recognise individual faces. Today, these things are known and demonstrable fact. Yet 75 years ago, none of that was understood, and likely wouldn’t have been believed.

Octopi evolved in a way that is so entirely different from us, that we simply didn’t understand them, we didn’t understand their physiology, we didn’t understand their neural system.

I do believe that at some point, humanity will create non-organic life that gains sentience, though I would be surprised if that happens in my lifetime³. I also suspect that it will be like the octopus, in that we don’t recognise it initially. Such a life form would be so far outside our experience, so far outside our understanding, that we simply wouldn’t see it staring us in the face.

The initial draft for most of this section is the product of ChatGPT in Reasoning mode, using the o3-mini model. It has been edited for content and style. The full exchange, including prompts, may be found here. ↩︎
As a society, our respect for the sentience, intelligence, and emotions from non-humans has been something between utterly appalling and appalling. Truth be told, even today, these truths are too often ignored for the sake of convenience. Of course, the same could be said of these things from our fellow humans. ↩︎
I must admit, the more I learn about how LLMs learn and execute chain of thought (CoT) techniques, the more I wonder if we are progressing to this point faster than I expected. I have seen examples of CoT that demonstrate a level of logical analysis at or above the level I would expect from most humans. ↩︎

Millions of Jobs

Thu, 30 Jan 2025 08:57:16 +0000

It has been 20 years since I first used machine learning to solve a complex business problem. The underlying problem was simple: the company was selling a new service and wanted to know who was most likely to buy it. We had millions of records, and each record had hundreds of fields. A vast amount of data, but no idea how to extract insight from it. Countless hours from various data analysts had been invested into finding a pattern, but none was forthcoming. Months turned into years, still no idea of who was most likely to actually buy. Enter machine learning. Over the course of a couple hours one evening, I was testing new techniques with machine learning, and there was the answer. The company met it’s entire monthly sales goal the next day. It found what every human had missed.

This was long before machine learning was rebranded as Artificial Intelligence (AI), but I couldn’t deny the power and potential that it could offer. It was also obvious that once it evolved to the point that it could be broadly applied, it would be a revolution.

We are racing towards that revolution, though to understand what lies ahead, we must first spend a little time understanding the past. This revolution isn’t without precedent, and that precedent is vital to understanding what comes next.

The Industrial Revolution

Going back as far as 1760 may seem irrelevant when talking about the latest technologies that are still in their nascent stages, but the situation was actually quite similar. In a relatively short period of time, new technologies completely changed the way most people worked, how they were paid, how businesses functioned, and both created and destroyed vast numbers of jobs. Without understanding this period of time, without understanding our history, it’s impossible to understand our future.

Looking at textile workers during this time is particularly illustrative. Most of those that spun cotton and wove it into fabrics worked from their homes, working by hand, using simple tools like spinning wheels, slowly and steadily turning raw cotton into a useful product. Over the course of a surprisingly small number of years, some of these workers were in large factories instead, operating dangerous machines, and doing the work of 500 people. Yes, you read that right. A new technology that replaced 500 people that had honed their skills over years, replaced by a machine and a single person. Of those that weren’t hired to keep these machines fed were forced to look for other work, or effectively left the workforce entirely.

This 500 to 1 ratio for cotton spinning is likely the most drastic, and it would be disingenuous to say that these losses were entirely permanent. As these new technologies paid for themselves, businesses invested in more machines to expand their production capacity, which meant hiring more people to keep them fed, plus the occasional work to maintain and repair them. The jobs being created were very different than those they eliminated, with different skill levels, and different pay. The fact that it enabled Britain to effectively compete with India in the production of textiles, despite the fact that wages in India were 1/6th of those in Britain at the time, should tell you everything you need to know about the pay of these new jobs.

Turning this cotton thread into woven fabrics similarly saw massive changes, with a 40 to 1 ratio. A skill carefully developed, replaced by a machine that would do the work of 40 people. This is repeated countless times across various industries, with the new technologies enabling substantially greater efficiency, while costing countless jobs, and creating some number of new, but different, jobs.

This went on, waxing and waning, all the way up to 1913, when manufacturing found the optimal form in Henry Ford’s assembly lines. Turning cars from hand-built & carefully crafted machines, into the ubiquitous mass-produced vehicles we know today.

The trend here is clear and simple: revolutionary technologies and techniques are highly effective at replacing large numbers of workers with specific knowledge and skills, with a smaller number of less skilled workers. Plus creating a few highly skilled workers to support these technologies. Nothing about this is really surprising, and should be well understood, but it’s a preview of what the future holds.

Quiet Part, Out Loud

A world in which human wages crash from AI – logically, necessarily – is a world in which productivity growth goes through the roof, and prices for goods and services crash to near zero. Consumer cornucopia. Everything you need and want for pennies. - Marc Andreessen

The above quote is from a highly active investor in AI companies, and one of the most influential voices in the venture capital community. Before going into the quote, and why this is saying the quiet part out loud, we need to cover some background.

There are two general pricing models for goods and services, the first is the cost to provide the product or service, plus a (small) margin for profit. The second is whatever the market will pay, regardless of cost. In reality, the cost + margin model provides only a floor for what can be charged, while most pricing is based on what the market will pay, optimised for leaving as little money on the table as possible, unless there is so much competitive pressure that the price must be reduced to that floor.

While your milk and eggs have enough competitive pressure to reduce pricing down to cost + margin, while the medications you may take, the electronics you buy, and most other products that you buy are based on what the market will allow, what people are willing to pay. There’s a reason that some medications constantly go up in price and are priced at hundreds of times the manufacturer’s cost - because they can get it. This is the same reason that an Apple macBook costs two to three times that of a PC with equivalent performance - they can get it.

Why is this at all important?

Marc’s point that prices will drop substantially after wages collapse is based on a flawed and dangerous assumption, that goods will be priced on the cost + margin model alone. That simply isn’t how the world works. There’s also the unmentioned issue that to people with no income, it doesn’t matter how cheap products become.

He is likely correct that wages will collapse, that is largely the point. AI, as it’s being developed today, has one purpose: improve efficiency to the point that jobs can be eliminated. This provides allows provides to charge more and more for their products, and results in huge returns for investors.

How wages are likely to collapse is important to consider, as it will almost certainly be a very unevenly distributed collapse. The average income for any given region is likely to drop precariously, primarily as a result of job eliminations. However, we’ve seen that for some roles, such as those building these AI systems are climbing, and there’s no reason to think that will change. Those that can’t be replaced will see great demand and high pay, those that can be replaced, will be left to scramble to find new work.

I believe that there’s good reason to believe this type of stratification will be seen across the board, in all areas impacted by these AI initiatives. Those with certain skills seeing strong demand and high pay, especially at the most senior roles, with greater difficultly for those seeking to enter impacted areas.

Impacts & Future Challenges

Based on the impacts we’re seeing today, and the already signalled advances that we can safely assume will come, there are some things that we can began to make assumptions about, with varying levels of uncertainty and timelines to see these in full effect.

I’d like to talk about those that have been on my mind as I researcher more into where these advances are going. This is far from exhaustive, and may turn out to be incorrect (though I know what outcomes I’d bet on). There’s also a timeline issue that I’d like to acknowledge: some of these could occur next year, others may be a decade away. This is not a short-term analysis, but a long-term look at how these technologies are likely to evolve and impact people.

Software Development

Generally speaking, there are two kinds of developers:

Those that carefully craft their code, attend or speak at conferences, write blog posts pontificating on the correct way to build software, and would often rather leave their job than be forced to write bad code¹.
The other 95%.

The fact is that the vast majority of developers are buried deep in companies, facing tight deadlines, primarily work on line-of-business software, and don’t have time to worry about the best way to do something, because they are too busy just keeping the business running. This silent majority are both the most likely to leverage AI to make their jobs easier, and the most likely to face downsizing as those efficiency improvements allow smaller teams to do the same work.

We are seeing the first steps of what AI will do to software development. The tooling will improve, accuracy will improve, higher-level tools will be built on top of the code generation used today. Many common line-of-business application will eventually be able to be (mostly) generated, needing more oversight and review than active development. It will take a lot longer before the tools become good enough that they’ll start displacing those building commercial software, but commercial software is only a small portion of the software being built today.

Within the next decade, it’s likely that more and more development teams will be replaced by a single business analyst using generative AI to create the internal software powering companies. This builds on trends that have been going on for a decade or more already, though will almost certainly be greatly accelerated in terms of impact.

Creative Arts & Writing

Not long ago, I had an idea for a short story, but it was too ridiculous to invest the time in writing it², so I did the next best thing: I asked ChatGPT to do it. I provided it with the abstract and a couple sentences to explain the plot and message. I waited and expected to get a laugh. What I got was a shock. It was clear, expanded nicely on the idea, and would have passed for thoughtful and creative. I spent months wondering if there was actually a point in writing after this. It did in seconds what I would have spent weeks doing.

This experience if far from unique, and business leaders are seeing that in many cases they can get ‘good enough’ results using current AI tools, compared to paying someone to do the same work. We are seeing this across the web, from journalists being supplanted, AI generated marketing copy everywhere, video games racing to cut costs by using AI generated assets and even replacing voice actors.

Good enough here is critical; in many cases business are fine with a loss of quality when it means that the price to produce a work drops to nearly zero. It’s a powerful motivation, and even a challenge for management to justify paying someone to do work that could be done for free. These roles where a portion of their work can be replaced with ‘good enough’ will see (and are already seeing) some of the earliest impact of AI³. Job opening in these spaces are already seeing increased competition as positions are eliminated.

Truck & Taxi Drivers

While Tesla’s most recent timeline for their long awaited self-driving semi may or may not happen (though I won’t be holding my breath), the fact is that true self-driving vehicles are coming, and coming in mass. There are more than 250,000 taxi drivers in the U.S. today (not counting Uber and other ride-sharing drivers), and a somewhat astounding 3,600,000 professional truck drivers in the country. Most of those jobs are simply doomed. Self-driving vehicles will be able to do the job both with greater efficiency and at a lower cost. With no need to sleep, no need for breaks, these vehicles will quickly wipe out many of these jobs once the technology has sufficiently matured.

Of all industries, professional drivers will likely be hit the hardest by percentage of jobs lost. The industry will be largely obliterated.

Management

Management roles provide both opportunity and challenge for AI replacement. I’ve spent much of my career in management roles, and to be honest, much of the time is simply lost due to things like inefficient communication. So many meetings that could have been an email, so many emails that could have been a couple bullet points, all without losing anything useful. Middle management roles are likely the most vulnerable to being eliminated by AI, as much of the role comes down to overcoming inefficiency. When eliminated, I would expect to see their responsibilities moving up the chain of command, as flatter organisational structures become more popular for their lower costs.

At least one CEO has noted that even he could be replaced by AI.

Information Security

I have been predicting for years that automation will steadily erode more and more work within the security industry. This includes better tooling that reduces work for penetration testers by automating more of the work, and detecting and fixing issues earlier in development cycles. As well as systems to automate a growing percentage of routine work, allowing smaller teams to do more. If you look at the state of the security product market today, this is obvious.

As tooling evolves, and likely evolves at an accelerating pace, the impact on security jobs will likely also accelerate. That said, while these systems will be able to make many of those in security roles substantially more effective, the lack of creativity and ability to perform deep reasoning and analysis will ensure that jobs remain, and are in high demand.

How many jobs will be impacted is difficult to guess, at least based on where things stand today. This is likely the most difficult field to judge due to the various complexities and perceptions⁴ involved.

Plumbing, Construction, Mechanics

Many of the jobs that were once considered the best of white-collar jobs now face greater uncertainty and the worst job stability since these roles came into existence, while some blue-collar jobs are likely more secure than ever.

Plumbers are becoming a new millionaire class, seeing small businesses becoming increasingly successful. This is just one of many jobs that will become more attractive as technology evolves due to the reliable income, constant demand, and job stability.

After the Industrial Revolution, white-collar jobs were often the best paying, most stable, and most desirable. After the AI Revolution, it’s possible that it’s the blue-collar jobs that will start to see competition for jobs, as white-collar jobs evaporate.

Inevitability

This post may have you wondering if I’m actually a Luddite arguing against the evolution of technology. Far from it to be honest, I have spent most of my life pushing technology to the limit. The earliest of early adopters. The cutting edge often old news to me. I’ve always pushed for technical improvement.

I’m also a realist, and having studied history and carefully watching where technology is going, talking about the likely future is critical to be prepaid for it. As much as it’s possible to prepare for revolutionary changes.

While I believe we are still many years from the most important AI breakthrough, Artificial General Intelligence (AGI), though this has largely become a marketing term instead of a technical definition, the fact remains that we are headed to substantial changes. Furthermore, we are headed into this intentionally, with no plan as a society to deal with the repercussions. Leaders on countries are interested in ‘winning’ the race to build the most profitable AI systems, but the impact of deployment hasn’t received any meaningful attention.

Am I predicting an AI disaster?

Many self-proclaimed technologists have spilled endless amounts of ~~ink~~ photons in rising the alarm about how AI could harm humanity, or even destroy us. This is not only of those arguments. We are sufficiently far from building anything truly intelligent, much less intelligent enough to see us as a threat, I don’t see a reason to even entertain these concerns.

My concerns are more immediate, and more focused on how it will impact individuals. If we manage to build a system that is capable of destroying us, and sees us as such a threat that it “feels” the need to take action against us, we’ll have earned the resulting outcome.

The Writing on the Wall

As noted above, I could be wrong, or the process could unfold over decades, instead of the decade or less I’m predicting here. The impact could be less than I expect, or could be drastically worse.

Some things are obvious when viewed objectively, and I hope I’ve provided a useful analysis of these things, and the likely consequences for the future.

To be clear, while it’s possible to read this description is a rather negative way, when I worked as a developer, I counted myself in this group. Don’t take this as a slight, but an acknowledgement that the software development industry is not just us. ↩︎
I do enjoy writing, and based on the feedback I’ve received, it seems to be well received. That said, I don’t write quickly; I spend far too much time rewriting, changing my mind and taking a different approach, and just too much time thinking about what I’d like to actually say. It’s not uncommon for a blog post to evolve over months or even years before I get around to actually finishing it. For short stories and creative writing, it’s far worse. ↩︎
When I talk about the impact of AI on creative works, I’m reminded of George Orwell’s 1984, and the Ministry of Truth’s novel-writing machines that created cheap entertainment for the masses, devoid of any actual creativity. We already see Amazon flooded with generated books, and as these tools become more effective at emulating creativity, it’s not unlikely that some of these will eventually start to perform well. ↩︎
When looking at how roles are impacted, it’s important to look at not only the real-world impact of these technologies, but also the likely perception of business leaders. It’s the perception of business leaders, as well as pressure from investors, that will have the greatest impact on how many positions survive or are created. In some areas, this is easy to see based on public information, for others, it’s more complicated to anticipate the broad perception. ↩︎

Security Is a Shell Game

Sat, 05 Oct 2024 14:59:44 +0000

In the world of security, everything comes down to trust; sooner or later you have to trust something. Often, this something is a human. While we are busy building advanced cryptosystems that will survive the heat death of the universe, sooner or later, digging down layer by layer, you get down to a human and their limited memory. While we may build software, hardware, and other systems to protect this chain of trust, it almost always ends with a human.

I was part of a standards body meeting discussing the security controls of a system, and someone complained that without a certain control, the system would be little more than a shell game. Where the ultimate source of trust was covered in layers and moved around, but never truly secured. They were, of course, correct. Though it’s not because of the missing control, but because the chain of trust ended with a human’s memory.

Any security architecture that involves a human can ultimately be reduced to a shell game, with its strength limited to what a human can readily remember.

It’s not that difficult to design systems that are robust, provable, and provide substantial protection. It’s extremely difficult to design such a system if a human is involved at any point. Human memory is limited, fallible, and degrades over time; thus if a system needs to be accessed by a human, the security of the system will be reduced down to what a human can readily remember. Be it a password, a PIN, or any other secret committed to memory, this will always be the weakest link. A password that one can remember will, without doubt, be of lower entropy and more easily guessed than any other secret used in the system.

When designing a new system, it’s common for there to be many layers, each with its own security properties and trust requirements, each layer assuming that the prior layer is just as secure. Yet, these robust defences, strong proofs, large keys, and detailed trust relationships all end up with the same thing: a secret a human needs to remember. For example, you can build an authentication system that provides a strong proof of identity, great phishing resistance, robust cryptography, and in the end, it’s protected by a password like Winter2024!. Everything is great, until a human enters the picture.

Recovering from Failure

Now, you may ask yourself, why not eliminate the human with hardware? This is why we have hardware security keys, right? In theory, this is the perfect solution, as it eliminates a human’s memory from the chain, and allows every secret in a system to be truly random. There’s a problem though, people don’t have perfect memory and tend to lose things, including hardware.

Once you introduce hardware¹ as the final link in the chain of trust, you face a challenge: what happens when that hardware is lost or broken? There are generally two options here:

Say ’too bad’ and accept that whatever was behind this system you’ve designed is lost.
Implement a recovery mechanism, which adds new links to the chain. Links that end with a human².

Option 1 doesn’t sit well with users (and can have devastating effects), and option 2 doesn’t avoid the problem of relying on a human’s poor memory - it just moves the problem to another part of the system. This is often the case with security, the problem just gets moved because there isn’t a good solution.

This is the point where many otherwise incredibly secure systems show their weaknesses. You can use a secure authentication system like passkeys, strong phishing resistance, randomly generated so they are impossible to guess. Yet the security is reduced to the password for the service you use to sync them (such as an Apple account or password manager) and thus the security of the mechanism to reset that password.

Always the Same Problem

Many years ago when I was becoming more involved in the crypto³ community, one of my favourite questions to ask was how key management should be implemented. What’s the right way to protect that final secret? The answer was always the same: “that’s a good question” followed by quickly changing the topic. The reason is that while it’s easy to hide the ultimate root of trust, add layers, move it around, and build complex systems that would protect against even the most advanced attackers, handling that ultimate key is a very hard problem.

A lot of progress has been made to limit the impact of relying on human memory. Passkeys and password managers allow for far stronger secrets to be used, and with the associated improvements against common attacks such as phishing, this represents a vast amount of progress. However, the same underlying issue still exists.

The fact is that this is an effectively unsolved problem; the solutions could address it add far too much complexity for end users to rely on, and no amount of additional layers or moving the key around will fix it.

Key management has always been the greatest challenge for protecting secrets, and it will continue to be for the foreseeable future. As long as the chain ends with a human, the security of the system will be reduced to what that human can remember.

This isn’t to say that we should stop finding ways to better secure users, data, and the secrets that are critical to protecting them. Incredible progress has been made, and is still being made. Data is better protected today than it’s ever been. That said, that protection is often weaker than we would all like it to be.

This applies equally to hardware surrogates, such as password managers that play the same role that a hardware key would, just with a longer chain of trust involved. ↩︎
This is effectively true, though not strictly true. It’s entirely possible to build recovery systems that rely on additional hardware tokens, though due to the complexity to implement and execute recovery, such mechanisms are far from common. For consumers and the vast majority of enterprise systems, systems like this effectively don’t exist. Only systems that need the highest security and can justify the additional costs and complications consider designs like this. As this post is focused on the common systems that most people are exposed to, any design that’s impractical for broad use won’t be considered. ↩︎
Crypto means cryptography. Yes, that’s a hill I’ll die on. ↩︎

Whose CVE Is It Anyway?

Thu, 14 Sep 2023 23:29:22 +0000

The latest vulnerability causing headaches across the world is CVE-2023-4863, issued by Google Chrome and described as “Heap buffer overflow in WebP in Google Chrome prior to 116.0.5845.187 allowed a remote attacker to perform an out of bounds memory write via a crafted HTML page”. This same CVE is cited by a number of other vendors as they are impacted as well. But, is this really a Google Chrome vulnerability?

The fix for this issue is not actually part of Chrome or even Chromium, but rather the libwebp library, which is used by Chromium, and a number of other projects. The library is maintained by the WebM Project, which is a joint effort between Google and a number of other companies. Firefox, for example, uses also uses libwebp, and is impacted by this vulnerability - though they don’t mention Chrome in their advisory, but they do mention Chrome’s CVE.

flowchart LR B[ libwebp] B-->C[ Chromium] C-->D( Chrome) C-->E(Electron) B-->F( Firefox) E-->G(Visual Studio Code) E-->H(Signal) E-->I( Slack) C-->J[ Edge] B-->K[ Android] C-->L[Brave]

Every product mentioned in that chart (and many others) have found themselves impacted to some degree by this vulnerability in libwebp, yet the defining CVE for the vulnerability was issued by the Google Chrome CNA as a Chrome vulnerability, instead of targetting either of the upstream sources. There are countless applications that use Electron, or Chromium, or libwebp, and they are all impacted by this vulnerability, yet the CVE is tied to Chrome. Some vendors have opted to cite the Chrome CVE, some have issued updates without actually explaining what they are patching, and some may end up issuing their own CVEs. This is less than ideal.

The CVE system is a wonder in many ways, both good and bad, while it’s easy to bash the system for its flaws, it has proven useful and sustainable - something that isn’t common in this field. That said, it does certainly have flaws, and the CNAs that issue the CVEs are not always consistent in their approach. In this case, the Chrome CNA has opted to issue a CVE for a vulnerability in a library that is semi-independent of Chrome, and is used by a number of other products.

When talking about this CVE, it could mean Chrome, the layers upstream of Chrome, or the things that are downstream of the upstream components. If asked if you had patched for this vulnerability, would it even be possible to give a confident answer? If you updated Slack, do you know if it was patched? With the mass proliferation of Electron, it’s likely that you have a number of applications that are impacted, and you may not even know it. Would you know from the CVE description that Android was impacted? If you had to explain to a customer that you’ve patched for this, do you reference this CVE, or that you’ve patched for the libwebp vulnerability? If you reference the CVE, you’ll likely confuse them (they’ll undoubtedly what Chrome has to do with it) - if you reference the library, they may not know what you are talking about.

The best advice at the moment, is just update everything.

AI: Art Without Expression?

Sat, 22 Apr 2023 21:19:21 +0000

Generative AI¹ is changing the world, and is doing so faster than most of us could realise. While I don’t share the fear that it’ll destroy humanity (something we’re doing quite well at, without help), I do see that it’s having an impact on how we work, how we interact, and will have a growing impact on what jobs survive into the next generation. Just as switchboard operators, pin setters, and lift operators are all essentially extinct today, advances in technology will steadily eliminate some jobs, while creating new ones.

I have much to say on how it impacts developers, writers, graphic artists, and others, though for the moment it’s photographers I’d like to talk about. Photography is my art form of choice, and I’m something of a purest - I don’t edit my photos (other than minor corrections), and they are published just as they were when they left the camera. It’s something I have strong feelings about.

The Quote

I recently saw a quote that pushed me to think more, and to experiment more. I wanted to understand, is what Generative AI produces art, or something else?

You’re Consuming Art That Has No Expression Behind It - Patrick Stump, Fall Out Boy

To start answering this question, we have to look at both how we define art, and how generative AI models are trained, and I’ll start with the later.

AI Models

For a generative AI system to do anything at all, it needs to be trained on a vast quantity of samples - the more samples, the better the results. For a generative AI that produces photorealistic images, that means being trained on huge numbers of photographs. It then takes this vast dataset and uses elements learned from the countless images to produce what it was asked for. These elements are combined in unique ways, based on the description (known as a prompt) that was provided by the user, along with some randomness to produce different & interesting results. How this prompt is structured, what it contains, what’s left out, and what’s indicated as something unwanted all play a role in what is finally created (this is evolving into a new field of its own, prompt engineering).

While the AI system creates the image, it does so based on what the user tells it. It’s useless without training, and it’s aimless without careful decisions from the user.

What is Art?

the conscious use of skill and creative imagination especially in the production of aesthetic objects - Merriam-Webster

There are some important words here that complicate matters for AI, in fact, almost every one of them:

conscious - AI systems aren’t conscious, so they aren’t able to do anything consciously.
skill - AI has great information to work with, far more than what a human could learn from working with more experienced artists, though it has no innate skill, only the ability to combine elements that it’s learned from other’s works.
creative - AI doesn’t create anything that’s truly new, it can only combine what it’s learned.
imagination - Without consciousness, imagination isn’t possible, seeing what doesn’t exist but could isn’t something that software is suited for, and may never be.

If you were to write a definition or art that was intended to ensure that AI could never meet the definition, this is exactly what you would likely arrive at. The definition is broad in meaning, but exacting in the exclusion of anything lacking clear consciousness.

Though, is it actually excluding AI? Let’s take another look at that list, and see how it could be read if we assumed that AI output is art:

conscious - While there’s no good argument for consciousness, the process is not unlike consciousness. The process is deliberate, goal orientated, and structured to complete the task using what it’s learned.
skill - Humans, like all animals, learn from others; our skill is built on the knowledge and experience of others. We learn what works and what doesn’t from others. We learn the foundations of our skills from others. We learn what others appreciate, what’s atheistically pleasing, what works, and what doesn’t, all from others. AI models do something very similar, though at an incredible scale, a scale no human could match.
creative - Humans create things that are unique constantly. This blog post is a unique work, it has never existed in the history of the universe until I wrote it. AI creates unique things as well, while elements may be built from the things it’s learned (not unlike most art today), the output is still entirely unique in the universe.
imagination - AI systems hallucinate fairly often, creating new things that aren’t based on their training, or anything else identifiable. This isn’t quite imagination as human’s know it - though we just may be better at managing it.

As you can see, there’s an argument to be made that it isn’t as simple as some people make it out to be. The expression of a generative AI system don’t directly represent the expression of a human, but it does indirectly represent the expression of millions.

The United States government has been clear that it doesn’t consider non-human expression to be subject to copyright protection, which doesn’t help the argument for this being art.

Though, given that art depends so heavily on the eye, opinion, and perspective of the beholder, it may be best to paraphrase Forest Gump and say simply that “art is as art does.”

The Experiment

What you see above is an image created with Stable Diffusion, while I don’t often shoot portraits, this does align with my style. Creating this image required rendering a few hundred images and the final prompt consists of approximately 50 terms. Creating this image wasn’t a matter of feeding the system a simple command and getting exactly what I was looking for. It was a matter of trial and error, tweaking parameters, adjusting the prompt, steadily improving the image. After hours of work, I had something I was happy with.

This is something that I created, yet was also created by a machine. It is the product on many hours of effort - and honestly more time than I would put into most of the photographs I make. With most portraits, it is a collaboration between photographer and model, though in this case, the model isn’t the human kind.

Is it art?

This is something I’ve struggled with. Looking at my role in the creation, the role of the generative AI, and the role of the countless people that contributed to the model that was used to produce it - it’s complicated. It’s impossible to say that this is purely my creation, as it isn’t. It’s creative, it’s new & unique, it’s my style, and it’s imaginative - from my own imagination, but is it actually art?

Yes. I would argue that generative AI can be a powerful tool for artists to guide creation of otherwise difficult or impossible works - just as Photoshop has been a powerful tool to enable the creation of otherwise difficult or impossible works. When guided by a human with a vision, generative AI can create amazing things that fulfil that vision. Where it becomes complicated is that it can also create something with no vision - just as a 5 year old can create something that is on the surface comparable to a Jackson Pollock work. It’s the vision, the purpose, the message, that separates a pointless creation into art. The tool used for creation is less important that than the vision behind it.

Greater & Lesser Art Forms

Personally, I see art forms as being on a spectrum, with the extremes being the most difficult to create to the least difficult. The most difficult to create are those that require the most skill, the most creativity, the most imagination, and the most consciousness. These are generally those that are creations from nothing, creating from a blank canvas, starting with nothing but a vision. These are the greater art forms, they require a special level of skill, and a special mind.

The lesser art forms allow the artist to start with something, and build on it. For a photographer, this is the world around them - leveraging what people and nature have created. No true photograph is a work of pure creation, but capturing the world from a unique perspective.

What you see here is a “painting” - created with generative AI, and in about 5 minutes. It’s a unique creation, it’s a unique expression, it’s a unique work of art. It’s also a lesser form of art. Unlike a true painting, the level of skill, vision, and imagination are far lower. It may be art, but it’s a lesser form of it. As someone that has little ability to draw or paint, I would be proud of this if it was a creation of my hand - but it isn’t, it’s a few minutes of typing and GPU time.

Generative AI can serve to greatly reduce the skill, investment, effort, and vision required to create something, and in this, the art itself is less meaningful. While it’s possible to invest heavily in achieving a particular vision, the creation is reduced by the reduced influence of the artist. It’s a lesser creation, and its artistic value is thus limited.

Art is Art

Regardless of the tools used to achieve expression, art is about expression and vision. As technology has changed, from simple beads and cave paintings, to generative AI, art has changed with it. Some forms become less popular, new techniques are created, realism comes and goes, but it’s the vision, the expression that matters. Just as photography changed the world of art, replacing paining as the way the world is captured, AI will push art in new directions. As long as there’s a vision guiding the creation, it’s still art.

While I deeply hate the conflation of Machine Learning and Artificial Intelligence (one we’ve been doing for decades, the other, we aren’t there yet), though I’ve given up that fight. I apologise to those still fighting the good fight. ↩︎

Twitter Becomes a Walled Garden

Sun, 18 Dec 2022 19:27:51 +0000

Today is a red letter day in the history of Twitter, though not in a good way. Twitter has a long reputation of free speech, providing a platform for all that wanted it, easily connecting to the powerful, building communities, and organising against tyranny. This didn’t come without controversy of course; in the effort to keep the platform safe, more and more moderation was implemented - sometimes the got it right, sometimes they got it wrong. While they worked to protect users, there was still a focus on providing as much freedom as possible.

We are the free-speech wing of the free-speech party. - Dick Costolo, CEO of Twitter

In a new policy, coming in on a Saturday and during the final game of the World Cup, Twitter has announced a raft on new restrictions on linking to other social media platforms - a seismic shift in their approach to regulating users’ speech. The list of platforms include:

Facebook
Instagram
Mastodon
Truth Social
Tribel
Post
Nostr
third-party link services such as linktr.ee and lnk.bio

Linking to any of these services, promoting these services, asking users to follow you on another service, or posting something that looks like a link are all banned under this new policy. Want to show off your photos on Instagram? That’s a suspension. Switched to another platform? That’s a suspension. Like many recently, I’ve moved to Mastodon - it’s likely that my Twitter account will be suspended for linking to my Mastodon profile.

Specifically, we will remove accounts created solely for the purpose of promoting other social platforms and content that contains links or usernames for the following platforms: Facebook, Instagram, Mastodon, Truth Social, Tribel, Nostr and Post.
— Twitter Support (@TwitterSupport) December 18, 2022

Twitter is becoming a walled garden, locking users in to avoid the reality that they aren’t the only social media platform around; a fact they seem to be trying not to admit. In fact, the policy starts with a remarkably obnoxious and pretentious line:

Twitter is where the public conversation is happening, and where people from all over the globe come to promote their businesses, art, ideas, and more. - Twitter Policy Announcement

The fear of losing users, in addition to losing advertisers, is clearly making Twitter’s current leadership rethink some long-held positions.

The acid test for any two competing socioeconomic systems is which side needs to build a wall to keep people from escaping? That’s the bad one!
— Elon Musk (@elonmusk) June 6, 2022

Helpfully, Twitter’s (current) owner has made his opinion clear on what it means when companies build walls. If you’re still using Twitter, now may be the time to stop - I’d suggest Mastodon, which is currently a far more reasonable alternative.

On Art, Heritage, Nazis, & 3D Scanners

Tue, 05 Jul 2022 23:39:46 +0000

or: How an 18th century painter saved Warsaw from the Nazis, and how 3D scanners may save Ukraine

While researching applications that use the iPhone’s LiDAR scanner or perform more pure photogrammetry, I came across an effort by Polycam - likely the largest player in this space - to help protect the heritage of Ukraine and its people: Backup Ukraine.

This is an effort to recruit people in Ukraine to leverage Polycam (provided a no cost as part of this program), to create detailed scans of artwork, buildings, and other items of cultural significance. Polycam is then committing to make the full archive of everything captured as part of the program freely available under a Creative Commons license, hosted for at least 5 years after the end of the war, and made available to Unesco and other partners. This presents an option to undo at least some of the damage being inflicted as part of this war.

Destroying a country’s cultural heritage is the fastest way to erase their national identity. In light of Russia’s false denial of Ukrainian sovereign and unique national identity, we take the destruction of the country’s history extremely seriously.

This is a fantastic effort, and I’m happy to see that it’s ongoing, and new artworks and artefacts are being added all the time.

While reading about this effort, it struck me that this isn’t a new idea - while using crowd-sourced 3D scanning to preserve cultural identity may be new, this story goes back further.

The Razing of Warsaw

In 1944, following intense clashes with Polish resistance fighters, the occupying Nazis took their revenge out on the people of Warsaw, and razed much of its historic centre to the ground. More than 150,000 of Warsaw’s citizens were dead, and 85% of the oldest and most culturally significant part of the city was reduced to ruble.

This act of revenge against those that dared to stand against the tyranny and evil of Nazism left a scar on Poland, and threatened the loss of an important part of the country’s identity. A few short years after later thought, a long dead artist would help to heal the county.

Bernardo Bellotto (1722-1780), a Venetian artist, had been appointed by Stanisław August Poniatowski, the King of Poland, as a court painter, and among the works Bellotto completed was a series of cityscapes showing Warsaw in all it’s glory. While these works were stolen by the Nazis 1939, and by Emperor Nicholas I of Russia in 1832, and by Napoleon in 1807 - they seem to always find their way back home. During the rebuilding of Warsaw in the late 1940s and early to mid 1950s, these works played a crucial role in restoring the historic heart of the city to its former glory - or, at least as close to restoration as is possible after such barbaric acts.

It was through Bellotto’s art that Warsaw was given another chance, and Poland was able to recapture an important piece of its identity. As Russia attempts to follow in the path of their once ally-turned-enemy, inflicting barbaric destruction on the people of Ukraine, efforts like this to preserve the cultural identity of the country are vital.

The war will end; no matter the outcome, one day Ukraine will need to heal and find itself again. These efforts are what will ensure that generations from now, the people of Ukraine will know who they are. Today’s fight is for life & land, tomorrow’s fight will be for the soul of the nation, the soul of the Ukrainian people.

It feels trite, as I sit in my office, reflecting of history and studying experimental photography techniques, while a war for survival goes on, but say it I shall, for whatever worth my words have. Слава Украине!