The craftsmanship and engineering behind software development have never been in greater danger–nor seen a greater chance at a renaissance–than they do now.

genAI use, whether in the form of discussion with ChatGPT’s web interface on SQL or in the purest yolo form of near AI-psychosis we see with Gas Town, has spurred our industry to look at itself in ways that we really haven’t in years. I think we’ll come out of this functioning at a higher level than before–but it’s gonna hurt.

A brief starting point re: engineering and craft

Before we pull in genAI, I want to briefly sketch what I consider the difference between engineering and craft in software.

An engineer takes an incomplete-but-useful description of a problem and applies an equally incomplete-but-useful mental model of it to design software and abstractions that they do not fully know the implications of to solve that problem. Then, they iterate on the resulting solution using observations from production behavior and feedback from their host organization.

A craftsman takes a task or artifact template and performs the actions required to realize it modulo their own experience, “tricks of the trade”, and available tooling–and (critically) flags when something can’t be done.

Some hopefully illustrative examples:

• An engineer figures out how well you can use a Huffman tree to do compression, a craftsman figures out how to use SIMD assembly to make it fast.
• An engineer figures out what different entities are required for charging users money, a craftsman figures out how to call the various APIs to accomplish that and make clean wrappers to handle things like errors and decoupling.
• An engineer figures out why a production system keeps collapsing during off-hour ETLs, a craftsman figures out what commands and procedures to run to safely and efficiently migrate load over while the DB cluster is being scaled.
• An engineer figures out that a system needs to have some automatic classifier for strings, a craftsman figures out a reasonably fast implementation and writes tests for all the fiddly edge-cases.

Note that all developers are both engineers and craftsmen to one degree or another, and that that nature changes depending on the day, the job, and the problem at hand! Also, we need both roles!

Think of it like constructing a bridge: the best-engineered bridge built by shitty welders will fall into the drink just as quickly as the worst-engineered bridge built by the finest. You need both.

genAI contra craftsmanship

The arrival of genAI has meant that we now have strong evolutionary pressure on craftsmanship.

Speed of coding is no longer the bottleneck. One of the minor craft skills I’ve seen differentiate people has been literal typing speed and time-to-code; unfair as it is, hunt-and-peck means that you write less code and make fewer changes that you can test and think about, and that tends to make you somewhat worse at your craft.

A senior developer types at around 70-90 words-per-minute. The slowest reasonable models, even running only on the CPU, can generate many tokens per second–with a GPU, we’re in the tens of tokens per second, and that gets us to hundreds of words-per-minute–and I’m even pulling a bit of sleight-of-hand here and assuming that we can exchange tokens in that form directly, when the reality is probably even more favorable to the robots.

Pattern matching and wisdom is no longer the bottleneck. Another aspect of the craft, an important one, is simply having seen various solutions and their shapes and knowing when to apply them. “Simple” things like using Object.create(null) for a dirty dictionary-like thing in Javascript, using foo && bar in shell scripts to do multiple commands and break when one fails, using |> &(foo(&1)) in an Elixir pipeline instead of assigning to an intermediate value, assorted bit-twiddling tricks, and so forth. These solutions aren’t always correct, but they’re a standard toolbox you develop over a career and one you reach into when you’re implementing a solution. The size of this toolbox (and even more importantly the number of mental post-it notes you’ve accumulated on each tool about when and when not to use it!) is another good proxy for mastery of the craft.

No human alive has seen even a millionth of the code these LLMs are trained on. A common characterization of LLMs is indeed as stochastic parrots, and my experience does imply that they’ll often reach for the most probably (read: common) solution first. That’s exactly what a good craftsman does in the same situation.

Testing is no longer the bottleneck. An instructor of a friend of mine had once claimed “The quality of your test suite is the quality of your commitment to your customer”. A craftsman ensures good tests to make sure that the code they’ve written works properly, both in the large and the small. An acceptable test suite verifies the happy path–the code does what we claim it does when given the expected inputs–and a great test suite verifies the adversarial path–the code does what we claim it does when somebody is hammering on edge cases. Both forms of testing are comparatively inconvenient compared to simply spewing code out and claiming that it works by inspection, but a true craftsman puts in the work to do tests.

LLMs love writing tests. They write so many tests. They hammer on all the things. They might not hit every edge case, but with even a little bit of guidance they do way more testing than 90% of the developers I’ve worked with in over a decade. They also run every test (at least in usual agentic setups), much more reliably than the humans I’ve worked with (and to be fair, the humans I’ve been).

genAI contra engineering

Similarly, the arrival of genAI has put pressure on engineering, though of a different sort.

Good documentation is even more important. Everybody knows that documentation is important, but almost no place actually does it. Everywhere I’ve worked, making sure that docs are available in varying levels of specificity and targeting varying consumers has provided value far exceeding the effort I’ve put into it. Every library that people actually adopt has good documentation–and for some proprietary libraries, that documentation is understood to be part of the price tag!

With LLMs, you can save a tremendous amount of time by pointing them at libraries that have good docs. You can save a tremendous amount of time by having docs that quickly run through where changes might want to be made for different things (HACKING docs of yesteryear). Explanations of why design choices were made and what business needs drove them help to steer the models away from building things that don’t make sense.

Good specification is even more important. Handing a junior a ticket consisting solely of “Add a shopping cart” has nearly always resulted in disaster. Senior devs writing placeholder tickets “implement auth” has always been a liability. Conversely, writing down legible and thorough acceptance criteria for acceptance testing has always been a way of strongly-boosting the heuristic exploration of a design space and spotting design deficiencies before wasting implementation time. Occasionally adding implementation guardrails to the criteria (“This must live in its own module”, “this should not run in superlinear time”, “do not add any new external libs for this”) has always been a good thing.

LLMs really quite like this additional specification. You get much better results if you can clearly articulate the shape of thing you want, you get fewer surprises, and you generally can trust the output solution more.

Good project management and planning is even more important. There has never been a point in my life where taking the time to sit and plan out in sufficient (but not excessive!) detail an undertaking has been a mistake. We’ve all worked projects where you get something in that’s too big in scope, whoever is responsible for such things hasn’t digested it fully, and then the project goes off the rails. If you’re a fan of agile and pointing, you’ve doubtless run into house rules where (once the inevitable concession to reality is made that points roughly correspond to time estimates) the team kicks back anything longer than some period of time as needing to be broken down more thoroughly (“A 13?! Come back with two 5s and a 3 and we’ll see what fits in the sprint”). Similarly, working with a tech lead (or whoever is responsible in your org) who can say “aha, this work depends on this work depends on this work, and this other work we can tackle whenever” always helps avoid logjams.

For LLMs, the smaller the task you give them (much as with humans) the less likelihood there is for them to go completely off the rails. Additionally, having a sequence of clear tasks helps nudge them into useful directions and provide reasonable breakpoints where if they screw up too badly prior progress need not be discarded.

Where this leaves us

For the craftsman, we see that genAI makes the common craft work so easy as to be almost uninteresting. For the engineer, we see that genAI makes ignoring best-practices engineering work so expensive as to make it essential. This is huge.

Previously, we could afford to be middling craftsmen–and many were! The benefit of being even moderately good at the craft were parlayed by many into easy hours and large salaries for basically typing and having Stack Overflow answers in cache. That arbitrage–because that’s what it was–is going away.

Previously, we could afford to be sloppy engineers–and many were! We could get away with vague instructions and loose specs and missing docs because everybody else in the sector did too, and because the human engineer of any seniority is a thinking and flexible creature. When we had those things, we flew, but we didn’t need them. Usually. On smaller teams. On greenfield projects.

A lot of people are going to lose their arbitrage opportunities. A lot of people who haven’t moved into higher-tier craft or engineering thinking are now expensive meatbags that can (and, depending on how much you agree with economic logic, should) be replaced by large matrices.

A ray of hope

The optimist in me–and that’s the view I’m mostly going with in this essay–says that we’re actually going to come out, collectively and individually, ahead.

For the craftsmen, we’ll lose responsibility for typing and for common arcana..but that’ll free up time and space to focus on genuinely interesting and uncommon parts of the craft. The models aren’t going to be as good at things like weird assembly hacks (not for a few years, anyways) or elegant interface design. Our craft will be moved in the direction of focusing on things that are more context-specific to the problems we’re solving, and our time will be consumed less with things that frankly aren’t that high-value or high-reward.

For the engineers, we’ll be forced to actually engineer and manage projects properly. It’ll be too costly be sloppy, and so we’ll internalize and get more efficient at designing and planning systems. We’ll be able to push back at the business more effectively about the importance of pre-production work and competent specification. We’ll have more time and cause to talk to customers and users and really tighten up iteration and build things people actually give a shit about, because it’ll be too costly not to.

It’s going to be a growth process and all growth hurts, but I think the majority of us will be better off–and the profession as a whole better off–in the long run.

Want to discuss this post? | Discuss via email | Hacker News | Lobsters |

---