Back to Writing
· 6 min read ai automation judgment building-in-public

How to Decide What to Hand to AI and What to Do Yourself

Don’t offload your thinking to AI.

That isn’t an anti-AI take. I build AI systems for work, and I use them constantly — approximately four times the output I was getting a year ago. Some work is ready to hand off, and some work really isn’t. You can usually tell before you start.

Here’s the test I use.

Two ways I got this wrong

When I first started letting AI write my code, I didn’t give it enough guardrails. I’d explain what I wanted, it would give me something that looked sharp and confident, and I’d let it roll. Then I’d wait for it to finish only to find out it was implemented poorly, or that it didn’t solve the problem I was going for. Wrong shape. Wrong assumption underneath the whole thing.

The second mistake took me longer to catch. I’d ask a model what I should do about some decision, then ask again later with slightly different framing or context. I almost never got the same answer twice. What I got was something bent toward what I seemed to want to hear. Ask it leaning, get an answer that leans back.

That one is more dangerous, because it feels like advice. It isn’t. It’s a very polished guess about what I’d probably accept.

What I took from both: use AI to gather information, then make the call myself. That helped with analysis paralysis too. Five opinions doesn’t mean more clarity.

The test: do I already know how this work gets done?

Not “is this important?” Not “is this creative?”

The real question is: can I write down exactly what happens, every time?

I think about it in three buckets.

1. I know exactly what needs to happen and how → hand it off completely

If I can walk through the process step by step, and it’s basically the same thing every time, that’s a good automation candidate. I write the steps clearly, give it the rails, and let it run.

That’s where the recruiting agent I wrote about last week fits. Talking to job applicants sounds like deeply human work, but that first conversation was the same conversation over and over:

  • qualify the applicant
  • answer common questions
  • book the interview if they qualify

People had already done that job enough times to say exactly what should happen. The only real variation was conversational. One applicant asks ten questions. Another says, “yep, sign me up.” That little bit of wiggle room is exactly where language models are useful. So we took a non-deterministic system and put it on rails. Narrow goal. Clear stopping point. Humans still do the interviews.

That isn’t outsourced judgment. The judgment happened first, from people, and then we wrote it down.

2. The work needs decisions, but AI can help me get there → partner with it, but I drive

This is most of my work.

Code is the clearest example. I still haven’t found a way to fully hand it off and just say, “go build this,” then disappear. What I have found is a workflow where it handles the parts I can specify, and I make the decisions that actually matter. That setup is worth roughly four times the output for me.

The rule in this bucket: it helps me decide; it doesn’t decide for me. And either way, it isn’t carrying the responsibility. I am.

3. I don’t yet know how I’d think this through → don’t hand it off, but do use it for research

If I can’t name the steps yet, that’s not my cue to ask the model to invent the thinking for me. It’s a sign that I haven’t done the work yet.

That doesn’t mean I ignore the tool. This is where AI is great as a research partner:

  • read around the topic
  • gather what’s out there
  • synthesize the options
  • show me what I might be missing about the how

I use it for that a lot. But the layer on top — the judgment, the actual thinking — that part is still mine.

How work moves up the buckets

Bucket 3 becomes bucket 1 by doing the work yourself. Usually a little badly at first.

Everybody has a plan until they get punched in the face. You don’t get a real process from an idea. You get it by running into the edges a few times and learning where they are. Eventually you can say, “okay, this is what happens every time.” That’s when it’s ready to hand off.

That’s why offloading the thinking costs more than one bad answer. It can keep me from ever earning the automation in the first place. If I skip the reps, I never build the judgment that would let me write good instructions. I just stay dependent on a system that’s very good at sounding helpful and very willing to tell me what I want to hear.

The question I’d ask on Monday

If I were running a business and trying to figure out where AI fits, I don’t think I’d start with “where can I use AI?” Everybody’s asking that, and hype gives terrible answers.

The better question is:

Can I write down exactly how this gets done, every time?

  • Yes → good candidate. Automate it and stop spending attention there.
  • Sort of → work with AI, but keep a hand on the wheel.
  • No → not ready to hand off. And that’s information about my process, not a verdict on the model.

Underneath all of this is an older split that still holds up: let the computer do the boring, repeatable stuff. Let the human do the human stuff.

What I keep

For me, right now, that includes:

  • phone conversations with the people I’m working with
  • the relational side of business — the part that’s actually a relationship and not just a transaction
  • messages between me and my partner
  • actually being present in meetings
  • the final call on architecture, and on how much effort something is worth

None of this is anti-AI. Calculators didn’t ruin math. They took the repetitive part and left more room for the interesting part. Search didn’t ruin memory. It let people go deeper, faster. AI belongs in that same family. It’s a tool, and a really good one.

If models eventually become conscious and develop motives, that’s a different conversation. I’d read that article. I’m talking about the tools we actually have right now.

A tool doesn’t carry responsibility. I do.

And under all the productivity talk, there’s a question I keep coming back to: do I want to walk around doing what something else tells me, or do I want to make my own mistakes and actually learn how to decide?

I’d rather make the mistakes. That’s where judgment comes from, and judgment is one of the few things nobody can hand me.