Hire an AI consultant who can show you three things on the first call: a system that ran for someone, a number it produced, and something that went wrong. Then give them a task small enough to finish in two weeks, with a fixed price and a way to score the result. If all they can show is slides, you are buying a strategy document.
I sell AI engineering, so I have a stake in this. Run the same test on me. If I fail it, don’t hire me.
Why do buyers distrust AI consultants?
Because the failure is easy to hide. A model’s output reads as right whether it is right or not, and somebody has to check it. The person selling you the work should not be the only one grading it.
In a Reddit thread from July 2026, the poster said some consultants seem legit. But “others feel like they learned a few buzzwords, made a ChatGPT prompt pack, and now sell AI transformation strategy for thousands of dollars.” One reply gave a one-line filter: “Easy tell: ask what their last project failed at.” Another: “A useful AI consultant is usually the one who makes the project smaller.”
I think that last one is right. A smaller project is also easier to price and easier to score.
What should an AI consultant show you on the first call?
Three things, in this order.
A system that ran. Not a demo and not a deck. Something in production that you can see in the actual tool. The one we can show you is an AI marketing platform we built for a client, an advertising service where one AI system plans campaigns and generates the pieces, and a person approves every piece before it goes out.
A number from it, with its denominator. On that platform, 10 forbidden prompts produced 0 pieces, and each AI judge was tested on labelled examples before it went live. Another system we built, an email assistant for a PropTech housing platform, scored 22 of 24 on a frozen eval. It is not in production.
A failure. What went wrong, what it cost, what they changed. Ours is in the open: that 22 of 24 leaves two failures to read one by one and too few cases to call a hit rate, and we have no long production record of task success for any single agent. AI agent examples has the whole table. A consultant who has shipped real systems has failures like these. One who answers with “challenges” and no detail may not have shipped one.
Which seven questions expose a bluff?
| Ask | A good answer sounds like | Slow down when |
|---|---|---|
| What did your last AI project get wrong? | A specific failure, its cost, what changed afterwards | Everything went well, or “we hit some challenges” |
| How will we know it works? | A set of real examples with the right answer written next to each, and a score | “We’ll review the outputs together” |
| What will it cost per task once it runs? | A formula with model calls, retries and checks, measured on your examples | A build price and nothing about running it |
| Who checks the model’s output, and who checks the checker? | A person approves, and any model judge was tested on labelled examples first | A model that approves its own work |
| Would a plain workflow be enough here? | They consider it first and tell you if it is | The answer is always an agent |
| Who owns the prompts, the examples and the code, and in whose accounts do they run? | You, in your accounts or ones you approved | Their platform holds them |
| Who will do the work, and can I interview them? | A named person you can talk to before you sign | “Our team will be assigned” |
Two of these need more than a table cell.
The second question is the one that matters. Anthropic’s engineering team writes that “20-50 simple tasks drawn from real failures is a great start” for an eval, which is the set of test cases that scores an AI system. Collecting those can be an afternoon of your own work, not a research project. If a consultant tells you they need months of data first, ask why.
The third one is easy to skip. The build is a one-time cost, but running the agent costs money on every task. I wrote the arithmetic in How much does an AI agent cost?.
How do you test a consultant before you sign?
Pay for a small piece first. Pick one task and write down real examples with the right answer next to each. The consultant builds the simplest thing that could work, runs it on your examples, and reports four things: how many results were approved, how it failed, what an approved result cost, and what production would take.
Check your own examples before you hand them over. Anthropic’s test for a good task is that “two domain experts would independently reach the same pass/fail verdict.” If two people on your team disagree on whether an output is right, no consultant can score it, and that disagreement is the first thing to fix.
This is what our AI Agent Pilot is: USD 5,000, two weeks, one agent on one real use case of yours, with evals and cost per task measured. The AI engineering page has the scope. It tests the working relationship as much as the agent, and you can run the same test with any other firm.
How much does an AI consultant cost?
I won’t print a market range, because a range without a scope tells you nothing. These are our numbers:
- AI Agent Pilot: USD 5,000, fixed, two weeks.
- Discovery, if you’re not sure the use case is ready: USD 7,000, one to four weeks, ending in a roadmap.
- By the hour, for work after that: Senior USD 45 to 50, Lead or Architect USD 55 to 60, Mid 35 to 40, Junior 30.
The pricing page has the whole table.
What should the contract say?
Three lines matter more than the rest. The prompts, the example set and the code should be yours from day one (with us, code and intellectual property are the client’s from the first day). Done should be written as a score on your examples, not as “delivered”. And a named person is accountable for the work. With us, the client interviews the engineer who will do it, and I lead every project end to end.
When should you not hire one at all?
Don’t hire one if nobody on your side can tell a right output from a wrong one, because no consultant can fix that for you. The same goes for a rare task, or for high stakes with nobody checking the output.
A consultant who tells you in week one that a plain workflow is enough has just saved you money. Hire that one.
This week, send the same short brief to two firms: the task, your first ten examples with the answers, and one question, “what will this cost per approved result?” Compare what comes back. The one that asks to see more of your examples before quoting is the one worth a call. If you want us in that comparison, start the AI Agent Pilot: USD 5,000, 2 weeks.
Sources
- Reddit, r/ArtificialInteligence, thread “1uy2izh”, July 16, 2026. I quote the poster and two replies exactly as retrieved on October 11, 2026.
- Anthropic, “Demystifying evals for AI agents”, published January 9, 2026, accessed October 15, 2026.