LLM

There is a huge problem I have not seen discussed in the context of AI. Documentation—just like code—can rot when subjected to unrestricted AI use, and this is not something that can be easily, if at all, fixed. Missing, misleading or outdated documentation forces rushed explorations focused on how, not why—which tightens competition for resources between research, design, and implementation. Good documentation warns about dead ends and explains constraints, allowing teams to reach better decisions faster. Yet, having a good knowledge base was hard before AI, and I’m convinced that it will get even harder with it. ...

When doing AI-assisted development using agents, your prompts will inevitably have a lot of decision gaps. Sometimes the model’s “defaults” used in those places are fine, but leaked system prompts suggest that most of them are explicitly prodded towards the lowest change-surface solutions. It’s a safe choice, but in a mature codebase, this will become increasingly corrosive with each change. In fresh projects, it will lock in place bad design decisions that you or the agent made early on. ...

This list started as an idea for a short LinkedIn post, but ended as a summary of systemic problems that need to be addressed to make coding LLMs/AI Agents a paradigm shift in software engineering—not a dead end that creates as many problems as it solves. Perhaps this attempt to organize my own thoughts on the topic will be of some use to anyone. Misunderstanding of intelligence: LLM-based agents do not reason in the human sense of the word; but are very advanced prediction and pattern recognition engines—which defines their unavoidable limitations inherent to the transformer architecture. Contrary to the marketing, they are not “intelligent” - defined here as the ability to generalize, abstract, and establish causal relationships between facts. They just simulate this process very convincingly using language as a medium - being a “Stochastic parrot”1, constrained by both training and syntax to the benefit of prediction accuracy - especially in the context of programming. ...

As AI-driven existential dread spreads among software developers across the world, I’m watching reactions varying from the shit-eating grins of those convinced that we must repent for tech-bros’ arrogance, through the crazy-eyed prophecies of the new golden age, and finally, smug assertions that the phrase “software developer/engineer” will at last regain its true meaning. Because most of us just glue stuff together and are nothing like the giants of prior glorious ages. ...

On how the industry really would like to replace sycophantic developers with sycophantic developers armed with sycophantic AIs and how it will amplify already existing dysfunctions. Again. ...