LLMs Will Only Get You So Far

I will admit right off the bat that I am a luddite when it comes to web development. Adopting new technologies opens the door to a whole host of problems:

• Developers going down rabbit holes that they find interesting but keep them distracted from core business needs.
• If the ROI is not as great as promised, which is often the case in my experience, it can erode trust between the tech team and the rest of the company. Their priority items had to wait so long for this?
• If the change is customer-facing it can be disorienting for customers who have been trained for years to use the internet in particular ways (I’m looking at you, horizontal parallax scrolling animations).

LLM’s seemingly have none of these problems. They increase developer output by leaps and bounds, are incredibly helpful for system design, and have brought surprise and delight to billions of everyday people. Your chances of getting funding for a new venture, whether within an existing company or starting from scratch, are basically nil unless you are prepared to talk in exhaustive detail about your plan for using LLM’s.

All of this is fine and dandy for a seductively long time. If you’ve got a good product idea and you’re building your codebase using an LLM from day 1, you can quickly build something that works well enough to get you some funding and customers. The dream of a low / no-code solution feels closer than it has ever been. You’ve just popped the champagne cork…

When it all goes to hell in a handbasket.

LLM-assisted coding is powerful, but hamstrung by context and disposition. LLM’s responses are agreeable and fast, their knowledge vast. Their biggest weakness is you. You don’t know what you don’t know, and most of the time an LLM is not going to fill in those gaps for you. In its efforts to provide you with what you asked for with as little friction as possible, it will ignore the many of the implications of what you are doing.

I recently read this story on LinkedIn:

Someone asked Claude to make you a button on their website that charges a customer money. It worked! Even the tests Claude wrote passed! But it did not handle the scenario familiar to anyone who has used a slow website where a customer clicks a button, nothing seems to happen, then starts banging on that button like they are refreshing the link for Beyoncé tickets. Suddenly that customer has accidentally ordered 19 copies of your product, gotten real mad, gotten their credit card company involved, and created several hangovers worth of headaches for your company across departments.

This is when you (hopefully) realize that to make the big time you need your product to work for all of your customers, all of the time. That your LLM of choice has taken you as far as it can by itself. You need someone who knows all of the things to ask it for, how to tell when it has no idea what it is talking about and is figure out a novel solution on their own, how to train others in doing the same. You need me.

I’ve been working with LLM’s in my own coding, at work and for personal projects, since 2023. Beyond that, I’ve spent most of that same period leading teams of young developers who have come into the industry with LLM use as a first-class qualification for their work. I can look at what they’ve done and smell the BS from a mile away, and then work with them to develop the skills they need to integrate these incredibly powerful tools into their work without losing sight of the more traditional skills of software development. This keeps the BS from appearing again, and leads to happy, productive teams on small budgets.

Schedule a call with me today and let’s talk about setting you up to get as much value as you can from LLM’s without them becoming the factor that limits your scaling.