For the past four years, I’ve been building AI-powered tools that help recruiters do their job better. Before that, I was a recruiter myself — reading resumes, making calls, living the grind. And here’s one thing I’ve learned from straddling both worlds: In hiring, automating the wrong thing can quietly erode everything that makes your process work.
As engineering leaders, we’re constantly told to streamline and optimize. Move fast. But if you automate the wrong step — like how candidates are filtered, scored, or messaged — you might be replacing good human judgment with rigid shortcuts. And often, you won’t notice the damage until weeks later, when engagement plummets or teams stop trusting your system.
The Allure of Automation
Hiring is messy. Resumes come in all shapes. Job descriptions are vague. Recruiters are overworked. AI seems like a godsend. We start by automating outreach. Then scoring. Then matching. Eventually, someone asks: can this whole thing run without a person?
But here’s the rub: many hiring decisions are deeply contextual. Should a product manager with a non-traditional background be fast-tracked for a high-growth SaaS role? That’s not a “yes/no” the system can decide for you.
Early on at Rocket, we made that mistake. Our scoring engine prioritized resumes based solely on skills overlap. It was fast — but completely off for roles that required nuance. We had to pause, rethink, and admit: “This isn’t working like we hoped.”
What Agentic Systems Do Well
I’m not anti-automation. Far from it. But it has to be paired with human review.
We found that agentic systems — AI tools with autonomy to assist but not decide — were far more effective. Think copilots, not autopilots.
For example, our system can:
-
Suggest better phrasing for job descriptions
-
Flag resumes that match roles 80% or more
-
Recommend outreach templates based on role and tone
But it never auto-rejects or sends messages without review. The AI suggests; the recruiter decides. That balance makes all the difference.
Lessons Learned: Where Automation Fails
One of our biggest missteps? Automating outreach too heavily. We thought sending personalized AI-written emails at scale would boost response rates. It didn’t. Candidates sensed something off. The emails looked polished but felt cold. Engagement dropped.
We eventually went back to having humans rewrite the AI drafts. That one shift nearly doubled our positive response rate. Why? Because candidates want to feel seen — not sorted.
A CIO’s Checklist: What Not to Automate
If you’re leading an AI initiative in hiring, here’s a checklist we now swear by:
-
Don’t automate decisions that impact trust. Rejections, scores, hiring calls? Keep a human in the loop.
-
Avoid automating tasks with high context needs. A great candidate might not use trendy buzzwords. That doesn’t make them a bad fit.
-
Be careful with candidate-facing automation. Generic outreach harms brand perception.
-
Do automate the repetitive stuff. Parsing, meeting scheduling, draft — automate those and give time back to your team.
Human-AI Collaboration Wins
We saw the best outcomes when recruiters felt like they had an assistant — not a competitor.
Here’s one quick story: A recruiter used our AI to shortlist 10 profiles for a hard-to-fill GTM analyst role. She reviewed five, adjusted the messaging tone slightly, and got two responses in under a day. Same tools — different mindset.
Feedback loops mattered too. We built in ways for users to rate suggestions. The model kept improving — and more importantly, people trusted it more.
Final Thought: Think Like a System Designer
If you’re building AI into your hiring stack, go beyond automation. Think augmentation.
Don’t just ask, “Can this task be automated?” Instead, ask, “If I automate this, what do we lose in context, empathy, or nuance?”
Agentic hiring systems can deliver speed and scale — but only if we let people stay in control of what matters most.