Recently, I needed to extract the structure of a codebase—including all the source code and file names—so I could feed it into an LLM. Instead of writing a script myself, I just asked an LLM to write it for me.

The LLM produced a mostly working solution that required tweaking for edge cases and boundary conditions. I estimated it would take me about an hour to get everything working perfectly. But then I realized, If I have this problem, others must too. A quick search led me to an open-source project that did exactly what I needed—built by someone using an LLM, just like me.

https://github.com/yamadashy/repopack

At first, I figured I’d modify this open-source project to fit my specific use case. I started tweaking the code using an LLM for assistance. But then, I hit a roadblock—getting it to extract function return values was becoming cumbersome. After struggling for a bit, it occurred to me: Why am I even bothering with code? Instead of continuing to modify the code, I just prompted the LLM directly with the repository link. The LLM processed the code and delivered exactly what I needed—no coding required.

This experience made me realize something much larger: What if it’s LLMs all the way down? LLMs might start as productivity enhancers, helping us generate or modify code faster, but what happens when they get smart enough to bypass the need for code entirely? We’re already seeing this in code generation, but it extends beyond that. In areas like data processing, customer support, and content creation, LLMs are automating tasks that once required complex systems or manual effort. The right model with the right prompt can deliver results faster and more efficiently than any custom software.

If you’re building an LLM-based application, you may be one model update away from becoming obsolete. Your competition isn’t just other companies—it could be the next AI model. If you’re working in an area where you’re pushing pixels or mangling bits—no matter how advanced your algorithm—there’s a serious risk that LLM-class AI could replace you.

The future is uncertain, but one thing is clear: it’s safer to overestimate the capabilities of LLMs. So, as you build your product and startup, ask yourself: What if it’s LLMs all the way down? What’s your moat against the next model update