Nick KI have seen lots of recent discussion that goes something like this:
AI is going to build your whole software product and you need drastically fewer people
I feel like I'm being gaslit every time I read this and worry it makes folks early in their software development journey feel like it's a bad time investment.
Note: there was a long conversation on HN about this post you may want to read here.
I'm a developer myself (2k+ commits 3yrs in a row) and have been working a TON with LLMs at Trieve since they came into vogue 2 years ago. I know most all the "AI engineering" tricks; especially wrt retrieval.
Let me walk through the story of a recent PR:
Trieve's code-base was originally intended to work for a single-tenant application called Arguflow. Due to this, we spent very little time writing a quality API since we thought the only application consuming it would be our own.
Eventually, we became an API-first business and this changed. We made V2 response types for our most commonly used routes which were much cleaner and easier to parse.
Unfortunately, this made our OpenAPI spec have a large number of ENUMS which clutters SDK's generated via the spec (open-source or not). I have known this is a pain point for a long time, but recently the yournextstore team brought it back to the forefront of my attention and I resolved to fix it!
It was fairly fixable thanks to hey-api (by
@mrlubos) being awesome and supporting manually defining routes on top of generated types 🙏.
LLM'S ARE NOT AT ALL CLOSE TO BEING ABLE TO DO THIS KIND OF WORK!!
I had to open our lib.rs file, find the routes with V2 types, look at their response type definitions, remind myself of the name for the V2 response field on the Enum, change that line in each of the offending routes, and finally update imports. Not a lot of changelines, but a lot of domain expertise.
After my PR got approved (see it here), I then needed to publish a new release of our client and PR to yournextstore to get rid of now unnecessary code determining the type between V1 and V2.
- LLMs cannot update the SDK code to V2 response types in the first place
- LLMs cannot successfully configure a CI action such that I won't have to publish myself
- LLMs do not understand the downstream effect of fixing the problem will be updating yournextstore's code
Please stop gaslighting human devs into thinking AI is going to make their expertise irrelevant. We are a long ways out from that reality.
