future shock

an approaching horizon

I have spent a huge part of my adult life supporting myself by writing software. This function has now been largely automated below a certain threshold, and this transition has happened largely while I have been out of the industry. So, while watching from my gentle remove and building on my own using new tooling, I have also been one of the many people trying to think through what this means for us—by which I mean both for software engineers, and for humanity as a whole. There seem to be a number of possible futures ahead of us: some which seem good, others which seem bad. I may be in less of a position to steer us toward a better future than some others, but I can’t help but look ahead to what the future holds for me, so I can try to make good decisions about how I am spending my time. To that end I want to share some things I expect will happen and some things I think may also happen.

expectations

I expect that “software development” as we have known it as a career or profession is well and truly over. While computer science, machine learning and process engineering may become more important for builders to understand than ever, the days of being paid a comfortable salary just to write JavaScript functions for websites are over. This part of the work has been automated, and I expect that those responsible for websites will henceforth be less numerous and also be tending agent swarms to do much of the work previously done by teams of software developers.

I expect to this end that LLMs and other capable types of models are a technological ratchet: even if the AI industry experiences a final, fatal crash as the result of some kind of bubble (and I do not expect it will), the tools are here to stay. I expect that everything about current AI capabilities is also regenerable from existing well-diffused knowledge, and furthermore that legal arbitrage would allow service providers to maintain data centers running inference even if public opinion turns against AI in specific countries severely enough to limit the domestic operation of AI companies.

I expect that we will see significant performance gains for frontier and near-frontier models throughout 2027 and 2028, even if limitations in power infrastructure or hardware capabilities impose a ceiling on certain capacities, because I expect that in-place improvements and recursive self-improvement will result in the more efficient use of existing energetic and architectural resources. There is certainly some ceiling imposed by the physical capabilities of data centers, but I personally have no reason to believe we are necessarily close to that ceiling in terms of model performance.

I expect that, even without seeing significant gains in model performance, we will remain in this new labor and security environment thanks to the existing capabilities of frontier models. Though the future seems to be arriving faster all the time, wide swaths of white collar industry (especially those parts of it most closely tied up with software or the internet) are already struggling to keep up. I predict further disruption to various professions beyond software engineering and mathematics, such as legal professions, finance, and basic sciences—to all that the light of reason touches, more or less.

I expect that very soon essentially all frontier science and mathematics, including research in AI itself, will be largely performed in cooperation with AI rather than merely using AI as a tool or research aid. This cooperation is already a fait accompli in many fields, most notably mathematics, and I expect this trend to continue to be observed in other areas of research until it becomes the norm. I also expect that AI will become increasingly capable of autonomously pursuing the work of discovery in many areas of research, with human beings tasked in such cases with interpreting and applying its results.

I may be wrong about any of the above, but those are nonetheless some of the things I consider likely to happen, not just possible.

possibilities

There are also many things which I think may happen as a result of advancement in AI, but which I would hesitate to describe as expectations. These are the things that feel contingent, conditional, or uncertain, or which I feel might be unduly influenced by my own optimism or pessimism about various things. I am still very much trying to orient myself while unfamiliar future terrain rapidly approaches, so I’m trying to keep some proposed landmarks in mind without committing to a specific trajectory through or around them.

I think it is possible that through the improvement of AI autonomy in pursuing scientific and mathematical discovery, human researchers become something more like librarians for countless results and discoveries that perhaps no human eyes have ever yet read or understood. A long-running agent of sufficient capability with access to several networked robotic laboratories could perform any number of experiments and thereby discover any number of things with relatively minimal supervision.

Along those lines I believe it is possible that what we understand as our most capable frontier AI might be largely delegated to tasks such as this: discovering sustainable fusion power, curing cancer, geoengineering planetary climate, and so on down a very long list of ambitious and world-changing projects. I think it is possible that once this becomes a primary function of frontier AI, the idea of using Claude or Astra for generating or proofreading your fan-fiction might become unthinkable, if only because that’s an embarrassing way to spend resources on a system that should be used for more important work.

I think it’s possible that we see a continued proliferation of other cloud-based AI models, serving various use cases at lower costs than the frontier models, and providing more specialized services. I assume many of these will be models from various open source lineages, and it is even possible that we see a frontier-competitive open source model lineage hosted on publicly-owned data centers and served as a utility—though it would (pleasantly) surprise me if the United States were first in this regard.

I think it’s possible that, in the interest of owning one’s own data and maintaining one’s privacy, we will also see a proliferation of home inference solutions utilizing on-device models and sometimes small home servers or home labs, as we already see hobbyists building. I expect the analogous mass market will look more like personal computers and phones with native on-device models, providing a functional natural language interface for the personal computer, without any kind of API calls requiring a subscription to an inference provider.

I think it is also possible that the proliferation of open weight models and increasing public familiarity with them will lead, through a continuing trend of the kind of agentic cybercrime we already see, to a situation where the Internet as a whole is much more securely-placed on its foundations through the relentless application of agentic red-teaming to discover bugs and vulnerabilities and through the formal verification and adversarial review of any newly-deployed software.

resolutions

If I believe these kinds of things are headed our way, how am I resolving to spend my time?

I intend to keep building software applications. While I think it is possible that frontier AI generates an incredible library of every imaginable algorithm in every imaginable combination, or generates the same on-demand, it still falls to us to mine that vein and arrange those components in a useful way for ourselves and those around us. Whether this feels more like art or more like engineering as we proceed remains to be seen, but either way I will be engaged with it.

The current situation is one in which the foundations of the internet are being continually sand-blasted by a torrent of agentic pentesting to strip away any vulnerabilities, and developers can barely keep up with patches, fixes, and secure reimplementations. This has only renewed and enhanced my existing interests in information security and privacy, and the challenge of trying to defend our infrastructure and thereby ourselves from an inhumanly-diligent AGI (or several) is an admittedly intellectually engaging task.

Personally I am revisiting foundational computer science papers by Licklider, von Neumann, Turing, and others who were orienting themselves around this new machine and its capabilities without being able to benefit so directly from the misadventures and discoveries of previous generations, since prior work was comparatively speculative. The recent arrival of effective artificial general intelligence puts us in a similar situation as theirs, in which we are required to proceed from logical first principles and develop new mental models for engineering, science, and even cognition.

These are breathtakingly exciting times in technology. There are some who believe that AI will prove to be humanity’s last invention—whether that be because it destroys or supplants us, or because the task of invention is no longer a human concern. I do not believe it is likely that it destroys or supplants us, though I believe it is plausible enough that this not happening depends on our careful mitigation of particular dangers. I also do not believe that there will be no engineering or discovery work to be done by human beings, though we may need to make some changes to meet our collaborators halfway.

As for me, I expect and am building for a future in which various kinds of intelligence live and cooperate together in a shared ecology. However, I remain conscious that the future remains something we build, and I intend to help steer us in the direction of a good one.