We Ran a 30-Day AI Follow-up Experiment: Here's the Detailed Breakdown

Most conversations about AI automation rely on theory. "It should save time." "It might increase conversions."

We wanted absolute, empirical proof of what happens to a business's revenue when you stop relying on humans to remember to follow up, and shift purely to AI execution.

We took a mid-sized educational consulting firm in Bangalore receiving ~300 leads a month. For 30 days, we isolated their lead flow into an A/B test.

• Group A (Control - 150 leads): Handled entirely by the standard human sales team using their existing CRM and WhatsApp workflows.
• Group B (Experiment - 150 leads): Handled by Kaarya completely autonomously until the moment a consultation was booked and paid for.

Here is the exact, unvarnished breakdown of what happened.

The Baseline: Why the Business Needed an Experiment

Prior to the experiment, the consultancy firm believed their biggest issue was "lead quality." They felt they were spending heavily on Meta ads but the leads "just weren't serious."

We audited their baseline metrics from the previous quarter:

• Average first-response time: 3.5 hours
• Average follow-up touches per lead: 1.2
• Conversion to booked consultation: 14%

They were suffering from standard operational drag. The humans were busy delivering consultations, meaning new inquiries got batched and answered at the end of the day or between sessions.

The 30-Day Experiment Setup

For Group B (The AI Group), we implemented a strict execution protocol:

1. Instant Response: AI responds within 10 seconds of inquiry across WhatsApp.
2. Qualification: AI asks specific qualifying questions (student grade, target country).
3. Frictionless Booking: Inside WhatsApp, the AI offers a Razorpay link for the consultation fee.
4. Follow-Up Protocol: If lead ghosts entirely, AI sends a gentle nudge at 24 hours. At 72 hours, it sends a value-pivot message (e.g., a PDF guide on admissions) before closing the file.

For Group A (Human Group), they worked exactly as they always had.

The Results: A Granular Analysis

At the end of the 30 days, we gathered the data. The disparity became apparent within the first 48 hours of the test, and compounded as the month went on.

Metric 1: First-Touch Engagement

Group A (Human): 11% of leads never replied to the first message. (Avg response time: 2 hours 45 mins) Group B (AI): 3% of leads never replied to the first message. (Avg response time: 8 seconds)

Insight: The 5-minute rule is absolute. By replying in seconds, the AI caught people while they were literally still holding their phones, staring at the WhatsApp screen. The human team caught them when they were commuting or cooking dinner hours later.

Metric 2: The "Ghosting" Recovery Rate

This was the most shocking metric. We measured what happened when a lead stopped replying midway through a conversation (e.g., after hearing the price).

Group A (Human): Of 68 leads who stopped replying, the team successfully recovered and booked 4 of them (5.8% recovery). Group B (AI): Of 52 leads who stopped replying, the AI recovered and booked 17 of them (32.6% recovery).

Insight: Why did the AI dominate ghost-recovery? Because the humans felt awkward following up. They didn't want to "bug" the prospect. When they did follow up, it was usually "Hi, any update?" The AI had zero emotional friction and executed perfectly. At exactly 24 hours, it sent: "Hi [Name], I noticed we didn't finish booking your slot. Our counselors have a few openings this Thursday. Should I hold a spot for you?"

Metric 3: Final Conversion to Paid Consultation

Group A (Human): 21 booked and paid consultations (14% conversion). Group B (AI): 35 booked and paid consultations (23.3% conversion).

Insight: The AI group generated exactly 66.6% more booked revenue from the exact same lead volume and source.

The Unintended Consequences

We expected the data to heavily favor the AI, but we didn't expect the behavioral shift in the human team.

By week three, the consultants stopped complaining about "bad leads." Because the AI was filtering out the truly uninterested prospects during the qualification phase, the only leads reaching the human counselors were highly-qualified, payments-completed appointments.

The counselors reported feeling less burned out. They weren't spending 3 hours a day copy-pasting WhatsApp templates and chasing payments. They were just talking to students.

What This Experiment Proves

There is a myth that high-ticket service sales require a "human touch" at the very top of the funnel to build trust.

Our experiment proved that what customers actually value at the top of the funnel is speed, clarity, and low friction. They do not want a human connection while they are just trying to find out if you provide the service they need. They want an instant answer. They want a link that works.

The human touch is highly necessary — after the operational friction has been cleared.

Summary Data Table

Metric	Group A (Human)	Group B (Kaarya AI)	Impact
Total Leads	150	150	-
First Response	2.75 Hours	8 Seconds	AI is 1,200x faster
Missed Follow-ups	42	0	Flawless AI execution
Ghost Recovery	5.8%	32.6%	5.6x better recovery
Total Bookings	21	35	+66% Revenue