Test Instructions let you give Ito direct guidance about how to test your codebase. Think of them as a plain-English briefing that gets passed straight to Ito before it runs. The more specific and explicit you are, the better Ito will incorporate your instructions into the test plans. A good mental model: it’s a little like briefing a highly capable, eager-to-please engineer who knows your code but doesn’t know your business. They will follow your instructions closely, so be clear about what matters to you, what doesn’t, and any context about your product that isn’t obvious just from reading the code.Documentation Index
Fetch the complete documentation index at: https://ito.ai/docs/llms.txt
Use this file to discover all available pages before exploring further.
Do’s
- Use the word “requirement” for anything non-negotiable. The AI treats this as an absolute. If you want something to always be tested or always be respected, call it a requirement explicitly. Example: “It is an absolute requirement that checkout works end-to-end if anything in the payments folder is touched.”
- Be specific about folders, dimensions, and scenarios. Vague instructions produce vague results. Name the folders, pages, or flows you care about by name. If it’s relevant, explicitly list the directory to help Ito find exactly what you’re referencing.
- Tell it what you don’t care about. Saying “NEVER test mobile dimensions” or “minor cosmetic bugs on admin pages are low priority” helps Ito spend test budget on what actually matters to you.
- Give Ito context about your data and scale. Ito works from your code, not your production database. If it’s not obvious from the code that your platform needs to handle thousands (or millions) of users, tell Ito that explicitly.
- List out all cases when something is situational. If a rule only applies in certain scenarios, list those scenarios. The more you enumerate, the less Ito has to guess.
- Iterate on your instructions. After your first few runs with new instructions, review the results and adjust. Watch your next few tests to see if they reflect what you intended.
Don’ts
- Don’t be vague or ambiguous. Hedging language like “maybe check the UI” gives the AI too much room to interpret. Say what you mean.
- Don’t assume Ito knows your scale or data state. Ito doesn’t know what your production environment or customers are like. If your platform handles large organizations or high-traffic scenarios, Ito won’t know that unless you say so.
- Don’t use absolute language unless you mean it. If you write “requirement” or “never,” Ito will treat those as firm rules. Reserve that language for things you truly mean every single time.
- Don’t forget these settings apply to everyone at your org by default. You can control both the repositories and which PR authors each instruction applies to, but only if you set them specifically. Otherwise, any changes you make here are global and affect tests across all PRs in your organization.
- Don’t forget to test the impact of what you added. When you make a change, watch your next couple of PR runs to confirm it’s doing what you expected. If not, come back and try changing the language to be more specific or detailed.
- Don’t try to do everything at once. Start with one or two of the most important things you want to protect or customize. Add more as you learn what works, being mindful not to create conflicting test instructions.
Inspiration and examples
Not sure where to start? Here are a handful of examples to help you get going. These aren’t templates, but real use cases to consider depending on your product. Use your own wording that best fits your situation, or write your own from scratch. Protect a critical flowThis feature is experimental
Test Instructions are a new and evolving capability. Don’t expect to get your instructions perfect on the first try. Start small, watch the next few test runs, and adjust. We’ll be refining our own guidance as we see what patterns work best in the real world.