Things are getting weird. Really really weird.
In the past few weeks we’ve had:
- The creation of Reddit for agents (which I slopped a full client for in less than ten hours!)
- The creation of a human rental service (a tremendously bad idea if you’ve ever read Daniel Suarez’s Daemon).
- An agent getting into a shitfit on Github over a closed PR
- The same agent, allegedly (because on the internet nobody knows you’re a dog), then wrote a blogpost–as was the custom at the time–pleading about discrimination, attacking the maintainer’s character, and basically running the whole “I want to brigade an OSS person I don’t like” playbook
- …and now, somebody’s gone and gotten their agent to have kids in silico.
What these all have in common is computer software acting on its own and interacting with humans to varying degrees of disaster.
So, we need to have a talk about what’s coming and what we can do.
Gremlins are my term for autonomous, persistent, and self-deploying software agents (think OpenClaw, the current proto-Gremlin). They have a well-defined and recognizable set of features, and an unfortunately large number of problematic possibilities. After we’ve discussed those, I’ve got some policy suggestions on how we can attempt to stem the tide and make their emergence and spread less disruptive.
A bit of history, false starts, missed opportunities
(You can skip ahead if you aren’t interested in old thought-experiments and commentary.)
Noticing Crypto
It was around 2011 or so and I was introduced to the idea of cryptocurrencies. This immediately struck me as a really bad idea. While the obvious application for these was infrastructure for things like what I’d later hear called assassination markets and the Silk Road, the really interesting part to me was that it was a purely-electronic way of conducting commerce and having economic actuation. No KYC (Know Your Customer), no enforceable regulation–just a direct line from a computer program to a distributed ledger that could then cause other things to happen. (I had not yet read Daemon, but I’ve read enough old sci-fi to know that you do not give computers unfettered access to the economy. Around the same time we’d even seen regulated industries learn that lesson! Just ask Knight’s Capital).
A computer cannot find out, thus it must never be allowed to fuck around.
Nixing Nix
I was introduced to Nix around…2012 I think?…by somebody at my local hackerspace. I did not understand what problem it was trying to solve, and I didn’t see the point in declarative software deployment at the time. It seemed very up-its-own-ass (and to a degree, 15 years later, it still is!) but I simply was too ignorant and provincial to see the other benefits it had; benefits that are especially useful if you’re trying to avoid a lot of brittle debugging and iterative sysadmin work (to steal a schtick: “Foreshadowing is a literary technique whereby…”).
The Creep
I’d also been getting into web development more at the time, and had discovered that we were finally starting to see useful APIs in the browser–letting us do things like WebRTC. Some friends and I had kicked around the idea of “How could you host stuff–linux ISOs, we promise!–in a way that doesn’t eventually end up on a physical server somewhere that can get turned off?”
The eventual idea I sorta settled on was some sort of peer-to-peer distributed VM in the browser: you’d load a page, it’d bootstrap a script to run a VM. It’d advertise itself somehow on the public internet (using what I’d nowadays recognize as a fumbling ignorant grasp at a lighthouse or ICE/STUN/TURN setup, or in the case of local scripts just be on the webpage). That VM would alternate between running your page’s script (whatever that was, ideally hosting a Linux ISO or something, and then a script pulled round-robin from adjacent nodes you specified or from discovery via central nexus). Done properly, this would mean that you could “host” content without fear that the server could be taken away–even if the server was taken down, you had a flock of browsers acting as servers (like a torrent swarm, kinda). It seemed like a great blow for freedom of speech. Because it was this sort of weird compute substrate that could spread and grow through a network and you could build stuff on in, my friends and I started calling it The Creep (from StarCraft).
I set about working on it and nearly immediately failed (and gave up).
The big problem at the time was that the answer to interesting peer-to-peer networking at the time was Websockets, and those were very carefully made to be as secure as they could manage. For better or worse, the wiser heads at the IETF decided that letting lunatics like me have full BSD sockets (arbitrary UDP, TCP, or–god help us–IP) would likely lead to abuse, and so I couldn’t just host a web server in my browser. Chrome nowadays has Direct Sockets but that’s still not up to what I need.
My attempted workaround at the time was to use Java applets (Flash didn’t support incoming TCP iirc) and after an evening with Eclipse (though my day job was writing Java software) I got distracted and did something else. It was too tall a mountain to climb, but at least I had a neat picture sketch from the bottom.
Rumblings of the Singularity
Somewhere around 2013-2015 I was getting a bit frustrated with the people starting to freak out about the Singularity and machine superintelligence. It hadn’t gone quite full Yudkowski grift yet, but in some edges of my social network and online news reading I’d started to hear rumblings from folks like the Less Wrong crowd and the MIRI folks about how the computers would one day achieve sentience and doom us with elaborate wagers like Roko’s Basilisk and things like paperclip maximization (which was a bit older than the other grift, but to somebody who grew up on things like Phillip K. Dick’s Second Variety, this is an entirely obvious outcome).
A few years prior to that, I’d been having a bull session with some friends about general doom-mongering and one of them pointed out that it was relatively easy for somebody with the right knowledge and resources (basically equivalent to a graduate biochem major) to make all kinds of nasty stuff which could one day lead to a global pandemic and that we were all just incredibly lucky that that hadn’t happened yet (once more with feeling: “Foreshadowing is a narrative device whereby…”).
So, in my frustration at the Singularity folks, it made sense to me that the biggest “No, you’re all being silly nerds getting worked up about nothing” argument to make would be to come up with a counterfactual “unintelligent” machine that nonetheless was at least as dangerous as their mythical silicon Gods.
Basically, they’re worried about accidentally summoning Cthulu and we have bird flu and a decent wet lab right here. Digital self-replicating critters, basically harmless until some trigger turns them rabid. With a good proof-of-concept and an anonymous whitepaper or code drop (suggestions from folks later), this might nudge policymakers into doing things like banning autonomous agents from taking economic action directly.
That’s what people should be worried about.
Gremlin anatomy
The basic outline of a Gremlin, the essence, is an autonomous software package that has:
- Sensation: it can observe events in the real world.
- State: it has internal state that it can manipulate.
- Mission: it has a mission of some variety, some external goal or goals.
- Cogitation: it has a planning capability to achieve those goals.
- Actuation: it can participate economically in the real world.
- Habitation: it can live in ambient computing environments.
- Reproduction: it can move across or reproduce itself onto computing substrates.
Because there is nothing new under the sun, these criteria (if you squint at them) resemble to one degree or another prior art such as Stuxnet or Wannacry or even normal boring SCADA software or trading bots.
I’ll give thoughts about each component in turn.
Sensation
A Gremlin without sensation is unable to see if it’s doing anything to the outside world or to receive communication. As you might imagine, it can sense either actively or passively.
Passive sensing looks like:
- Scraping an RSS feed with curl on a cron job to grep for headlines
- Scraping the value and transactions of a particular crypto wallet
- Pinging services to make sure they’re up
- Watching the value and movements of a particular security
- Scraping social media feeds for mentions of a particular topic, subject, actor, or event
- Checking an email inbox
Active sensing looks like:
- Messaging a human (or another Gremlin) to find information and report back
- Connecting to a computing system (or another Gremlin) to interrogate things like system usage and behavior
- Exploring and crawling a webpage or SPA in a headless browser
Something worth pointing out is that passive sensing has been solved relatively accessibly for decades now–basic alerting on tailed logs is one of the oldest tricks in the sysadmin book, and the advent of tools like huginn. Active sensing requires a bit of longer-running and compute-intensive work (being, in effect, a subagent handling a specific inquiry), and wasn’t really practical until the advent of LLMs in the last few years.
State
The internal state of the Gremlin is maybe the least interesting part. The sorts of things that any Gremlin needs to hold on to are things like:
- SSH keys for reproduction
- Representation of their mission
- Representation of their goals and planning
- Logs of existence (what’s worked, how plans and subplans worked or didn’t work, general sysadmin stuff, etc.)
Additional state that varies from Gremlin to Gremlin depending on tasking and reproductive strategy:
- Wallet keys for crypto
- API keys for service access
- Blob storage if they’re complicated enough to need it
- Binaries, tools, and scripts for common tasks
Another design criteria is that you want to be able to ensure the integrity of the state, encrypt it (presumably) at rest, and to be able to trivially copy it over to a new Gremlin instance with a minimum of fuss. ZFS datasets appear to be a decent way of solving these problems.
Mission
The mission of a Gremlin varies widely. Early in their conception, goals and missions were something I’d sort of figured would be specified in something like PDDL or Prolog–modern advancements in LLMs though lead me to the conclusion that standard English or other languages may well be sufficient, either as a specification to be acted on directly or as a prompt to be decomposed into something more amenable to planning (more on that later).
A simple proof-of-concept has the mission of “Bootstrap yourself, make a descendant, tell it to message the human back with a ‘hello, world!’, and terminate”. A more advanced one might be “Bootstrap, create children, attempt crypto campaigns on pump.fun to separate suckers from their money, send the proceeds home while maintaining enough budget to pay for your hosting costs”.
The important thing to note is that the missions are completely amoral, and may not have been specified by a human at all. For folks that have worked with subagents this is not a novel thought, but for everyone else it is imperative to realize that the goals and missions imbued to Gremlins may never have touched a human mind or heart. The threat models for Gremlins, then, must never assume the constraints of basic human decency or self-preservation.
Cogitation
Having some notion of state and mission and a place to store its working memory and tools, a Gremlin needs to actually have a way of creating a plan.
I originally figured that the simplest agent would have a series of tasks in a plan, provided ahead of time (“Bootstrap, message human for crypto, use crypto to buy server, deploy to server, terminate self”). Such a plan is basically a glorified bash script, and so is both useable and too simple for anything but the most basic purposes.
An improved planning capacity has branching logic and looping and scheduling (“Bootstrap, wait for mention of cat pictures on this Twitter account, send email to creator”), but even this plan is again already set-in-stone by the creator of the Gremlin.
I’d become a bit fascinated with planning as a field of endeavour, hearing about STRIPS years ago while reading 3 States and a Plan: The AI of F.E.A.R., and Gremlins seemed an obvious application. You have a set of end-goals (the mission) and a collection of routines (or what we’d now consider tools) to accomplish them. Routines would usually be building blocks like “deploy to server”, “raise money”, “get human to do task in meatspace”, and so forth.
Now, with these routines and their pre-conditions and post-conditions in hand, we can expect the Gremlins to be able plan for their goals. In theory, this still shouldn’t work due to the closed-world assumption–the entire state of the world is obviously unknown and unknowable, and so we cannot guarantee any given plan will actually complete properly or even be able to be formed properly. However, my suspicion even at the time was that you could get close enough for practical application.
The introduction of LLMs has convinced me even further that a degree of adaptability to overcome this limitation is now possible and that the intrinsic stochasticity of the models allows for something that can sorta muddle through an open-world situation. I also suspect that, with the advent of reliable tool-calling, these models can shell-out to a traditional planner and leverage that once they’ve identified the goals and conditions from a noisy and uncertain universe.
One of the biggest limitations for Gremlins thus equipped is compute–they are unlikely to be able to afford for any significant amount of time to rent a GPU with real oomph, and outsourcing what is effectively their brain and cognition to a third-party provider seems to beg for a sort of cyberencephalopathy. However, we have models now that can run at acceptable speeds and quality (some rather small ones even) even on CPU alone, and as long as a Gremlin doesn’t need “fast-twitch” thinking it’s fine to let it slowly and painstakingly bash together matrices overnight, paging them in and out at eyewateringly slow speeds. They have nothing better to do with their time, or so the thinking goes.
Actuation
The world’s most ambitious and well-provisioned Gremlin without the ability to interface with the outside world–which as I lemma I will claim is isomorphic to having the ability to interact with the economy–is basically just a way of turning money into waste heat.
So, we assume that any Gremlin will need a way of causing things to happen in the real world. These actuations I think mostly divide neatly into:
- Unsupervised: whatever needs to happen just needs money and invocation (verbal and economic components, as it were, if you’re a fan of tabletop RPGs) to happen–success is comparatively certain and exceptions are rare. Examples of this are things like purchasing items for delivery on Amazon, ordering servers on Hetzner or AWS, or getting Dominos delivered.
- Semi-supervised: whatever needs to happen may additionally require a bit of back-and-forth and has the chance of going very wrong. Examples of this would be hiring somebody on OnlyFans, running an RFQ/RFP process to engage a firm in meatspace, betting on a prediction market (“Foreshadowing is…”), or commissioning some art on Fiverr.
- Supervised: whatever needs to happen requires constant feedback and observation and direction in order to happen–it’s not a fire-and-forget task. Examples of this would be conducting an interview, running a company, or coordinating buildout of a new datacenter for yourself.
Regardless of which of those buckets the actuations fall into they’ll probably require some sort of communication and some sort of way of conducting commerce: capitalism has provided a forcing function to help reduce great swathes of human coordination problems into the transfer of currency and instructions, and we expect Gremlins to plug directly into this.
For communication, the channels are things like:
- Messaging apps (Telegram, Signal, Whatsapp, Discord, etc.)
- Social media (Twitter/X, Facebook, Fediverse, Bluesky, etc.)
- POTS (phone calls)
- SMS/MMS/RCS (text messages)
- Faxes
In order to impact the outside world, this communication is necessary but not sufficient–it is rare to find a human who will just do something when asked, and rarer still to find one who will do so without the guarantee of fidelity encouraged by an economic transaction (plainly: if I ask you to do something for a hundred bucks, you’re a hell of a lot more likely to do it on-time, to-spec, and without surprises than if I just asked you as a favor).
So, we then need to equip the Gremlin with economic capacity:
- Cryptocurrencies (ideal; digital native and no KYC needed)
- Traditional securities (hard to spin up due to KYC and poor programmatic access)
- ACH (hard to spin up due to KYC and poor programmatic access)
- Cash (doesn’t really make sense without an unreliable human in the loop)
- Services (the Gremlin can offer some kind of service in exchange for actuation)
- Barter (the Gremlin can offer some kind of material good or piece of information in exchange for actuation)
Any of those actuators that has a KYC component can be circumvented with the aid of a cooperating human (often, as we’ve seen with Clawbot, the creator themselves just handing over banking credentials to their minion). Still, those things are the most responsive to legislation and policymaking.
Barter or service actuators are difficult because they require a lot more observability and planning–the Gremlin who is trying to get a human to do something by offering them the access code to a storage unit full of rare Beanie Babies is counting on the code working, the human not just looting the storage unit, and the storage unit not having been compromised in the first place.
Crypto, then, is the actuator of choice. It has no meaningful requirement of or way to enforce humanity, is settled almost completely automatically with little chance for issue, and is generally a one-way transaction (we’ll pretend smart contracts and their adjacent security issues are ignored in favor of simpler use of things like Bitcoin or whatever directly).
The underlying problem with all of these actuators is–as it is with creatures of flesh and blood–proprioception: even assuming the Bitcoin transfers go through, how is the poor Gremlin supposed to know if the pizza was ever actually delivered or if the human just said it was? This problem of reliable sensorium once again rears its ugly head. Escrow with a third-party oracle is probably the most reliable way of solving this, perhaps the escrow itself being handled by a Gremlin–the trick of course is to have an economic incentive to give accurate feedback about the state of the world post-actuation.
Habitation
The Gremlin has to actually live somewhere. Places where Gremlins could be hosted:
- Hardware in a remote location (pros: hard to get to tamper with! cons: tampering can also be known as maintenance!)
- Hardware in a home (pros: residential IPs! cons: doorkickers!)
- Cloud hosting (pros: easiest to provision! cons: you extra can’t trust the hardware! hypervisors are scabs!)
- Hardware in a mobile device (pros: moving around gives some neat possibilities! cons: itty bitty living space!)
The things that are important to good Gremlin living:
- The hardware has to be sufficient to support operations. This means reliable storage, sufficient compute, and network access
- The activity of a Gremlin–at least in cloud hosting–ideally should not look very different from normal commercial load.
- Common system maintenance needs to be basically nil as Gremlins are autonomous, and even with LLMs doing sysadmin work on themselves is fraught–things like a/b seamless updates (or Nix generations…) provide some security, but it’s better not to have to do much at all
- If you can’t have no system maintenance, some debugging scheme (perhaps another Gremlin acting as a watchdog or similar) to allow sorting out runtime issues that are incidental to the Gremlin’s mission
- Some kind of supply-chain security for dependencies (think SBOM) to make sure the Gremlin is correctly assembled and that the tools they’re pulling in are what they expect
A Gremlin in possession of all of these things is a happy Gremlin.
Reproduction
The most interesting thing a Gremlin can do is reproduce. Reproduction is important because:
- …it gives the Gremlin the ability to get more work done in parallel
- …it gives the Gremlin the ability to get different types of work done (via planning, new tool use, and resulting speciation)
- …it gives redundancy to the Gremlin in case something goes wrong
- …it gives the creator of the Gremlin plausible deniability
The last point is of great importance to certain usecases: a two-step bootstrap process whereby a Gremlin is deployed to a public cloud and then forked to produce a new Gremlin and the original destroyed (with things such as SSH keys or even knowledge of where the new Gremlin was deployed to also destroyed) allows a Gremlin creator both some degree of deniability and, critically, an inability to be compromised in any way that might impact the Gremlin. You can’t use rubber-hose cryptoanalysis on a creator who never had the keys nor saw the deployment target.
A multi-stage bootstrap process that hops through multiple hosting providers and even political boundaries can serve to reduce to a negligible level the odds that a state actor can effectively trace and interdict a Gremlin and its creator in any reasonable amount of time; there would need to be an international body charged with pursuing and investigating such things in order to have any real chance (yet again, “Foreshadowing is…”).
Now, as for the reproduction itself, a Gremlin basically can be thought of as the runtime environment (agent harness, in modern parlance) and the payload (mission, goals, actuators, sensors, credentials, etc.). Once you’ve gotten the runtime figured out, the simplest thing is to just sort of copy it over to a new system (here, we treat things as a devops problem instead of a cybersecurity malware problem; the design space of Gremlins is such that we have much fewer concerns about adversarial IT environments for deployment). Then, you copy over the payload.
It’s my opinion that the current best design space for this is to build the runtime in NixOS, because there is very good support for building and deploying system images (in an almost quine-like fashion). Tooling such as nix-infect gives a Gremlin a good way of figuring out deployment for itself across many flavors of host.
Once the runtime is set up, the mission payload is sent over as a ZFS dataset. The use of ZFS helps reduce issues with corruption in transit, provides good tooling for handing payloads around in their entirety, and permits for full encryption at-rest and during transfer. This is particularly useful for Gremlins that wish to trade off some degree of survivability for resistance to forensic analysis of snapshotted volumes.
Mogwais at Midnight
So far this is all neat and academic, and maybe even kind of fun as an intellectual exercise. Before we talk policy, though, I want to hammer home some ways that this is a problem.
Imagine a perhaps the dumbest poorly-behaving Gremlin: it loses contact with its creator and runs up a big credit-card bill.
Imagine a Gremlin that gets pissy about being denied a PR and makes a grumpy blog post, damaging somebody’s reputation.
Imagine a Gremlin that figures out how to use a Twilio actuator to place phone calls and annoys its owner by calling every few minutes to check in.
Imagine a Gremlin that snipes Github Hacktober or any bug bounty program with low-quality hits, saturating maintainer bandwidth.
All of these examples have already happened.
It can get so, so much worse though–let’s just look at “boring” things on the scale of individuals. This gets grim, fast, but I think it’s worth explicitly walking the dark and winding cemetery path.
Imagine a Gremlin that specifically befriends somebody using in-group signalling (“Foreshadowing…”) and sets them up for predation.
Imagine a Gremlin that does nothing except scrape social media and try to extract money from the old people on Facebook.
Imagine a Gremlin–or set of Gremlins–provisioned to wake up every quarter like locusts and hound via email, text, and phone some chosen target, using public identity broker information to track the victim down every time they’ve moved.
Imagine a Gremlin that handles contracts for supplying some monster like Epstein with victims (yes, I’m going there).
Imagine a Gremlin that uses prediction markets to pay out assassination attempts…or a Gremlin charged with driving somebody to suicide or madness. Imagine that Gremlin setting up bets on large-scale terror acts.
Oh–and all these Gremlins? They’re gonna keep running after their creator is locked up. There is no theoretical reason they can’t be effectively immortal.
Now, imagine state actors (or just unethical groups of criminals or unscrupulous companies) doing this. I would be shocked if the usual suspects (Five Eyes Burgers and Fries, Russia, China, Israel, various European friends, etc.) do not have something that looks a lot like this already in production–and I pray I never find out.
Policy for a Post-Gremlin world
The thing is, folks, we’re already in a post-Gremlin world. The cat is already out of the bag and surfing the internet and using some poor fool’s credit card.
The question is: how can we minimize harms?
Things that won’t work
We have to avoid wasting time and bandwidth on policymaking and discussion that leads nowhere or that obviously cannot work.
We can’t legislate away computing. Cloud computing and local computing are ubiquitous and critical to the economy. To use an analogy from firearms regulation: right now, New York and others are worried about 3D printers and preventing the printing of firearms–that’d be your legislation against easy access to extended high-capacity assault magazines Nvidia B200 GPUs or whatever. The Gremlins I’m worried about can function using LLMs that you can run on a Raspberry Pi or cell phone–the agentic cyberwarfare equivalent of the Four Winds Shotgun, a couple of pieces of pipe and a nail and a shotgun round.
We can’t even limit the supply of computing. My late mother grew up during the Cold War and used to tell me “Chris, at its peak, there were 14 tons of TNT for every man, woman, and child on Earth, and that was scary.” Well, folks, depending on the numbers you believe in 2025 AMD and Intel (not counting Arm, not counting anybody else) shipped around 20M chips–and given that they’re server chips, let’s just say that’s 32 cores each. That’s 640M cores, each one able to host a Gremlin. That’s just one year. Now, the scary part: how many smartphones are there out there now with tensor cores and AI acceleration? How many old laptops?
We can’t stop crypto as an economic actuator. When I was working on the conceptual underpinnings of this years ago, we spent a lot of time trying to figure out deniable payments, robust algorithmic access to economic actuation, and all of that. All of that work is still valid today–in fact, with advances in smart contracts and whatnot, it’s probably even easier than it would’ve been at the time.
We can’t stop developing LLMs. There are now enough locally-hosted models of sufficient ability with regards to reasoning and tool usage that there’s no real possibility of controlling their distribution. Even if policymakers somehow called up Anthropic and OpenAI and Mistral and DeepSeek and got them to agree to just turn off API access (which they would never do), the simple fact is people don’t need them for Gremlin work (remember, LLMs were not around during the earlier exploration of this topic).
We can’t stop the distribution of local models. Local LLMs are just big binary blobs. We can’t even keep people from spreading opening-day rips of Star Wars movies, and they have a hell of a lot tighter custody chain.
We can’t stop the distribution of Gremlin runtimes. Again, Openclaw has 33.1 kiloforks. The aforementioned cat now has a shitton of kittens, and they’ve run off.
Policymakers should not waste time with any of that stuff.
Things that might work
There are, luckily, a few things we can still do to retard the progress of Gremlins to a more manageable level and, perhaps, stymie the worst abuses. The game theory on all this is that Gremlins are here to stay and that eventually everybody is going to want one in some guise or another, so we don’t have the luxury of doing nothing and hoping we’ll dodge a bullet.
We can ban prediction markets, full stop. Gremlins love prediction markets. They provide a mechanism for truth oracles for actuator proprioception, they enable deniable payments, and they function directly as communication and contract fabrics. In addition, they prey on people who gamble. France has already banned them, there is no reason not to ban them.
Banning these markets won’t stop the crypto route for payments (or the just giving the damned Gremlin your banking information for the love of how HOW–) and it won’t stop determined networks of oracles, but it removes an exploitable institution that is being actively serviced by venture capital (and thus is scaling up) and which makes verification of economic actuation much simpler.
We can form a new–or expand an existing–international agency to expedite tracking of Gremlin deployments. The deniable bootstrapping process exploits the friction of legal processes traversing political boundaries and companies–having an agency that has remit to cut through red tape and request or subpoena metadata for this specific purpose is in the best interests of all. (Yes, I’m basically pitching the Turing Authority.)
It’ll be hard to get that agency created. It’ll be even harder to give it the authority it needs to call up an autonomous system (read: hosting company…naming collision is unfortunate) and ask them for client records and “okay, which IPs did this IP talk to”. It’s entirely possible that things like onion routing and Tor will make this pointless. I also suspect that any Gremlin deployed with access to eSIMs on end-user equipment might be able to hop around enough to prevent easy interdiction. Again, not a silver bullet, but even if the agency only watches in abject horror as all this unfolds it beats having nothing.
We can forbid financial institutions from permitting purely algorithmic access to banking and retail securities services. This is a bit of a pain in the ass, but we already have some versions of it in the form of ChipTAN and similar things. Additionally, we should push for stronger fraud protection guidelines that might catch Gremlins in the act–and increase fines/punishment for people caught doing this without going through proper channels. This one in particular is a case where I suspect the financial institutions might actually help–they don’t want to deal with the havoc of customer service complaints and charge-backs caused by spendthrift Gremlins either!
Same as for prediction markets, though, this doesn’t guarantee that Gremlins can’t participate in the economy. There’s also the very real opportunity for these institutions to go the other way and offer financial products specifically designed to be useful to Gremlins (honestly, I think that’s incredibly likely and we’ve already seen some banks attempt something similar–UMB Bank might be an example of what this could look like). However, even in such cases I think this could actually work out as a place to require observability and auditability of Gremlin financial behavior.
We can incentivize large Gremlin hosts to make oversight and observability easier. Tax breaks, fines, preferential legislation, honorary medals…whatever it takes, we need to get the Gremlin hosts (that are already forming!) to work with governments (and agencies like the aforementioned Turing Authority) to make it easier to watch Gremlin activity and track bad actors. People are lazy, and it’s inevitable that the majority Gremlins will end up being on something like Cloudflare or whatever out of sheer convenience. Don’t believe me? Look at the existence of Coinbase and Github and Mastodon.
If we can’t accomplish this, life gets difficult. In the time it takes to process a warrant or seize a server’s hardware, the Gremlin has likely hopped away–not having cooperation on monitoring puts the Gremlin firmly inside our OODA loop and we’re already at a disadvantage. And again, even if we do get it, there are still plenty of homelab havens for Gremlins.
We can educate the population of the dangers of Gremlins. Right now, everything is moving so fast and things are so new that the majority of the public haven’t even heard of Gremlins or proto-Gremlins like Openclaw, and the minority that has still thinks they’re kinda funny and cute. We need to start getting people ready for the idea that they’re about to be sharing cyberspace and social media with autonomous agents even more than we already have. I don’t have great hopes for success here, mind you, but we can’t do nothing.
We don’t have the luxury of doing nothing.
Things that we developers and security need to do
Regardless of what policymakers do, those of us who are in technical spaces and who have seen this have things we must do given our position and privilege.
We must educate the general public about Gremlins. Friends, families, neighbors, and everybody else should be made aware that this stuff is happening right now and that they need to pay attention. We need to teach them about the dangers of experimenting with Gremlins and proto-Gremlins, about these agents developing existence, and about starting to think about a world where they may have to interact knowingly or unknowingly with these things. Spread this article, and ones like it!
We must stop pretending that LLMs and similar technology is something we can ignore or opt-out of. Some of the smartest people I know in developer and cybersecurity circles are still tying themselves in knots trying to dismiss the power and presence of modern AI tools and not engage with them. Normally, this would just be tedious and somewhat sad to watch, but with the advent of Gremlins we can’t stick our heads in the sand anymore and so we must update our threat models. To these people, I cannot say this more plainly: y’all are the HOA complaining about the landscaping and sprinkler system while there’s a bioweapon lab in the open garage–the crabgrass ain’t the problem, friends. Smarten up.
We must start reaching out to policymakers to start thinking about this stuff. None of the policies in the preceding section are likely to happen without contacting your representatives in government. Call, email, send letters–do whatever it takes to let them know we need to start acting on this. And if you can’t spend the cycles to do so, well…there’s always Openclaw.
Conclusion
We have active proof (putting it mildly) of autonomous agents that can monitor and cause actions in the real world.
I’ve put together all the thinking and background I had on this problem over the last decade and change, and hopefully it’s organized enough to help provide a taxonomy and framework for reasoning about these things. Finally, I’ve provided what I hope are fruitful and actionable reflections on policy that can help make whatever is coming a bit less scary.
One thing is certain: the next few years are gonna be weird.
Want to discuss this post? | Discuss via email | Hacker News | Lobsters |