How to Integrate AI Into Your Application: Process Flow
Taking an AI feature from idea to production: pick the use case, test models against real examples, add guardrails, pass a security review, and roll out in stages with monitoring.
4 teams15 steps3 decisionsFree download
Free to download. Edit it in Flow Studio to change the teams and steps, ask AI for changes, or export to PowerPoint, Word or draw.io.
Step by step
The same flow as a numbered list, ready for an SOP, a BRD or a ticket.
| # | Step | Who | Next |
|---|---|---|---|
| 1 | AI feature ideaStartProduct · step 2 | Product | step 2 |
| 2 | Define the use case and success measureDocumentOne job, one user, one metric: e.g. cut ticket handling time 20% with answers judged correct 90% of the timeProduct · step 3 | Product | step 3 |
| 3 | Collect real examples with good answersGolden set of 30-100 cases incl. edge cases and things it must refuse; stored in the repoProduct · step 4 | Product | step 4 |
| 4 | Is AI the right tool?DecisionProduct · Yes: step 6, No: step 5 | Product | Yes: step 6, No: step 5 |
| 5 | Use rules insteadEndProduct · Done | Product | Done |
| 6 | Compare models on the examplesScore 2-3 models on quality, latency (p95) and cost per 1k requests; LLM-as-judge plus human spot checksEngineering · step 7 | Engineering | step 7 |
| 7 | Design prompt, data access and toolsVersioned system prompt; retrieval (RAG) for company data; structured output via JSON schema; least-privilege data scopesEngineering · step 8 | Engineering | step 8 |
| 8 | Add guardrailsInput checks for prompt injection, output filters for PII and unsafe content, refusal paths, max tokens, timeoutsEngineering · step 9 | Engineering | step 9 |
| 9 | Security and privacy reviewData flow diagram, vendor DPA and retention terms, no training on customer data, threat model incl. prompt injectionSecurity and legal · step 10 | Security and legal | step 10 |
| 10 | Approved?DecisionSecurity and legal · Yes: step 12, No: step 11 | Security and legal | Yes: step 12, No: step 11 |
| 11 | Fix findingsEngineering · Re-review: step 9 | Engineering | Re-review: step 9 |
| 12 | Release to a pilot group behind a flagFeature flag at 5-10% of users; A/B against the current flow; in-product thumbs up/downOperations · step 13 | Operations | step 13 |
| 13 | Watch quality, cost and errorsWaitDashboards: eval score on sampled traffic, cost per request, error and timeout rate, user feedbackOperations · step 14 | Operations | step 14 |
| 14 | Meets the success measure?DecisionProduct · Yes: step 15, No: improve: step 7 | Product | Yes: step 15, No: improve: step 7 |
| 15 | Roll out to everyoneEndOperations · Done | Operations | Done |
When to use it
Use it when a product team wants to add an AI assistant, summariser, search or writing feature to an existing app, whether the model runs through OpenAI, Anthropic's Claude, Google Gemini, Azure AI Foundry, Amazon Bedrock or an open model you host.
Who's involved
- Product6
- Engineering4
- Security and legal2
- Operations3
Yours works differently?
Describe your own process in a few sentences and Flow Studio draws it, with your teams and your rules.
Describe your processarrow_forwardFor the technical team
The build decisions behind this flow. Each step above also carries its technical detail.
- 01Put model calls behind one internal gateway service: one place for keys, retries, timeouts, logging, cost tracking and switching providers.
- 02Store prompts as versioned files with their eval results, not as strings scattered in code.
- 03Stream responses to the UI and set a hard timeout with a graceful fallback message.
- 04Log inputs, outputs, model, prompt version, latency and token cost per request, with personal data masked.
- 05Rate-limit per user and per account, and cap max output tokens, to protect both cost and abuse.
How to get it right
- 1Write 30 to 100 real examples with the answer you'd accept before choosing a model. That set decides the model, the prompt and when you're ready to ship.
- 2Decide early what the feature must never do (give legal advice, reveal other customers' data, take payments) and test for it, not just for good answers.
- 3Ship behind a feature flag to a small group first and compare against the old way, so you can show it helps before it reaches everyone.
- 4Budget for cost per request from day one. Long prompts and big models add up fast at scale.
Questions
What are the steps to add AI to an existing application?
Pick one clear use case and its success measure, collect real examples, compare a few models on those examples, design the prompt and data access, add guardrails, run a security and privacy review, then roll out behind a flag with monitoring for quality, cost and errors.
Do we need to train our own model?
Usually not. Most product features work well with a hosted model plus a good prompt and your own data passed in at request time (retrieval). Fine-tuning is worth it only when a tested prompt still misses a consistent style or format.
How do we know the AI feature is good enough to launch?
Agree a pass mark on your example set before testing (for example 90% judged correct, zero unsafe answers), measure every model and prompt change against it, and keep the set growing with real failures after launch.
More process flow templates
- RAG pipelineHow retrieval-augmented generation answers questions from your own documents: documents are split, embedded and indexed, then each question pulls the best passages and the model answers with citations.
- AI agent with tools and MCPHow an agentic AI system works through a task: it plans, calls tools through MCP servers with scoped access, asks a person before risky actions, checks results and loops until the goal is met.
- MCP server integrationConnecting an internal system to AI assistants through a Model Context Protocol server: design the tools, secure them, test them, review them and roll them out with monitoring.
- LLM evaluation and releaseHow a team changes a prompt or model safely: run the evaluation suite, compare against the current version, release to a slice of traffic, watch the numbers and roll back fast if needed.