OrchestriAI
Back to the field guide
AI systemsGuide5 min read

How to evaluate an AI automation proposal before you sign anything

AI automation proposals vary widely in scope and delivery quality. Here's what to ask, what to look for, and the red flags worth investigating before you sign.

By Shariq Riaz

In this guide

AI automation proposals vary widely in scope and delivery quality. Here's what to ask, what to look for, and the red flags worth investigating before you sign.

4 sectionsPractical implementation context5 min read

AI automation providers have very different levels of experience, delivery process, and domain knowledge. Getting proposals is easy. Evaluating whether one fits your actual workflow is harder.

What to ask before you sign

Work they've actually shipped. Ask for relevant production examples, not only mockups or demos. Where confidentiality allows, ask to speak with a client who uses something they built. Ask what failed, how they detected and fixed it, and how the system is monitored now.

How they scope projects. A fixed price can be credible for a narrow, standardized deliverable. Complex custom work usually needs discovery, documented assumptions, acceptance criteria, and a change process before the estimate means much.

What happens when something breaks. Because things break. What's the response time? Is there a maintenance arrangement? Is there documentation that would let someone else step in if needed?

Who actually does the work. Ask who will design, implement, test, and support the system, including whether subcontractors are involved and who remains accountable for delivery.

Red flags

Proposals heavy on AI terminology and light on specifics about how your actual business process will work.

Specific outcome promises, "we'll cut your processing time by 60%", made before any discovery about your current process.

No mention of failure modes, edge cases, or what happens when the automation encounters input it wasn't designed for.

A timeline that is not tied to concrete deliverables, testing, dependencies, and client review.

The right framing

An AI automation project is a collaboration. You know your business. The builder knows the implementation options. Discovery should turn those two perspectives into an agreed scope, measurable acceptance criteria, and a support plan.

A builder who asks good questions, acknowledges what they don't know, and is honest about tradeoffs is more likely to deliver something that works than one who has all the answers before they've heard the questions.

Build notes: questions to ask before you sign

Ask the vendor to show a working example on your own data, not a demo dataset. Ask what happens when a step fails: who is told, what is retried, and what is left half done. Ask who owns the accounts and credentials the system runs under, and get it in writing that you keep access if the relationship ends.

Compare proposals on the same footing. List what each includes: build, hosting, monitoring, fixes after launch and training. A lower price that leaves out monitoring often costs more within a year. Ask for a small paid pilot with a clear success measure, such as hours saved on one named task, before agreeing to a large scope.

Check references that match your situation, in your industry and at a similar size, and ask what went wrong as well as what went well. A vendor who cannot name a problem they fixed has either not done much or is not being straight.

Write your decision down before the final meeting: the problem you expect it to solve, the number you will check, the date you will check it and the condition under which you would stop. Having that on paper keeps the project honest later and protects you from the sunk cost of a build that is already half finished.

Shariq Riaz

Written by

Shariq Riaz

AI Automation Engineer · CPHIMS · PMP · CBAP

11 years in enterprise IT at Fortune 500 companies. Now I build custom AI automations for healthcare, real estate, financial services, and freight forwarding teams.

Free automation checklist

Free automation checklist for your industry

A one-page checklist of the manual work clinics, real estate teams, financial firms and freight forwarders most often automate. Pick your industry and tick what is still done by hand.

Your email comes only to me. No spam, and I never sell it. See the Privacy Policy.

Free automation audit

Tell me your biggest manual task. I'll send back one specific automation for it, free.