# Ito vs Qodo: Generating Tests vs Running the App

Qodo writes tests for your changed code. Ito runs your app and tests the flows. Both target the same gap. Here's which one fits which failure mode.

### Ito

Builds and runs your application on the pull request, drives the affected flows, reports what broke.

### Qodo

Reviews the diff and generates test cases for the changed code. Formerly CodiumAI.

## What each one does on a pull request

Qodo analyses the diff, reviews it conversationally, and generates test cases for the changed code. You get review comments plus a set of tests you can commit. The tests then live in your repository and run in CI forever.

Ito reads the diff to work out which user flows the change affects, builds the application in an isolated container, drives those flows in a browser, and reports what broke with video and logs. Nothing is added to your repository.

Both are trying to close the same gap: the change looked fine, nobody knows if it works.

## The case for generating tests

It is a good case and worth stating without hedging.

**The tests persist.** Run Qodo once, commit the tests, and they protect that code path for as long as it exists. Ito re-runs on every pull request and produces nothing durable. If you value accumulating a safety net over time, generation is the model that does that.

**They run everywhere.** Committed tests run in any CI, on any machine, offline, in a fork. There is no dependency on a vendor being up.

**Coverage on code without a runnable surface.** A parser, a serialisation library, a pure calculation module. These have no user flow to drive. Generated unit tests are the only automated option that reaches them, and Ito has very little to offer here.

**It is cheaper to start.** Qodo has a free tier. There is no environment to configure and no container to make work.

If your gap is "we have no tests," generation addresses that directly and nothing else on this page does.

## The case against generating tests

Two problems, one well known and one not.

### The known one: maintenance

Generated tests are tests. They break when the code changes, they need updating, and the maintenance cost is the same as hand-written tests. The World Quality Report has roughly half of QA leaders naming AI-generated test maintenance as their biggest challenge, which is a striking finding for a technology sold as a labour saver. Generating a thousand tests is fast. Owning a thousand tests is not.

### The less obvious one: generated tests encode current behaviour, including bugs

A test generator reads what the function does and asserts that it does that. If the function has an off-by-one, the generated test asserts the off-by-one result and passes forever. You now have a green suite that actively defends a defect, which is worse than having no test, because the green is load-bearing in someone's decision to merge.

A human writing a test starts from an idea of what the code should do. Generation starts from what it does. That distinction sounds academic until you are looking at a passing suite and a production incident.

There is one case where this is a feature rather than a bug: characterisation tests before a refactor, where you explicitly want to freeze existing behaviour so you can detect any change. Qodo is genuinely good for that.

## Four shapes that get through both static review and generated tests

| Bug | Generated test | Running the app |
| --- | --- | --- |
| Two workers claim the same row — Each operation individually synchronised, the sequence not. No unit test exercises two workers. | Missed | Caught |
| A callback that never fires — The object was created through a path that skips it. The unit test creates it through the normal path. | Missed | Caught |
| A provider renamed a response field — Your code is unchanged. Your mocked test still passes, because the mock still returns the old shape. | Missed | Caught |
| A migration that locks under load — Takes an ACCESS EXCLUSIVE lock. Runs in 40ms against 200 local rows, holds for minutes against 40 million. | Missed | Caught |

The third one is worth dwelling on, because it is the specific failure mode of tested-but-mocked code. The more thoroughly you mock, the more confidently you test your own assumptions rather than reality.

## The case against running the application

Our side of it, stated properly.

### Do not buy Ito if any of these are true

**Your app cannot build unattended.** Most applications almost do. The gap is usually a migration expecting seeded data, an environment variable that only exists in someone's shell, or a service assuming a sibling is up. Budget a day, sometimes a week.

**You want something durable.** Stop paying and you have nothing. Qodo leaves you with tests.

**You need editor-speed feedback.** Building and driving an app is minutes. A unit test is milliseconds.

**You ship a library or SDK.** No runnable surface, essentially nothing to exercise. Qodo is the right tool and we are not.

**You are not on GitHub.** Qodo covers more platforms.

## Side by side

|  | Ito | Qodo |
| --- | --- | --- |
| Approach | Builds and runs the application | Reviews the diff and generates tests |
| Leaves artifacts in your repo | No | Yes, test files |
| Works without a runnable app | No | Yes |
| Catches concurrency and integration failures | Yes | Only if a test happens to exercise them |
| Ongoing maintenance burden | None | Generated tests need upkeep like any test |
| Feedback speed | Minutes | Seconds once written |
| Setup cost | App must build in a container | Minimal |
| Free tier | Trial | Yes |
| Platforms | GitHub | GitHub, GitLab, Bitbucket |
| Pricing | $40/seat/mo, unlimited repos | Free tier, paid plans |

## Which to buy

Buy both if you have both problems, which is common. They do not overlap much and they are not competing for the same seat in the pipeline.

The question that separates them: when something broke last quarter, would a unit test have caught it? If yes, your gap is coverage and Qodo addresses it. If a unit test would have passed while the system was broken, more tests will not help.

## Frequently asked questions

### Is Qodo the same as CodiumAI?

Yes. Qodo is the current name; the product was previously CodiumAI. Older comparisons and documentation use the old name.

### Does Qodo replace writing tests?

It replaces writing the first draft. The tests still need reviewing, because a generated test asserts what the code currently does rather than what it should do, and it still needs maintaining when the code changes.

### Can I use Ito and Qodo together?

Yes, and they are unusually complementary. Qodo builds unit coverage and Ito verifies behaviour at the pull request. They post different signals and do not conflict.

### Does Ito generate tests I can keep?

It attaches a failing test to a finding so you can reproduce it, but it does not build a suite you own and maintain. If accumulating durable coverage is the goal, that is Qodo's model rather than ours.

### Which is cheaper?

Qodo, particularly at the free tier. Ito is $40 per seat per month with unlimited repositories. The comparison changes if you are weighing either against maintaining an E2E suite by hand, which is the cost both are actually displacing.

### Does Qodo run my application?

No. It performs static analysis and generates tests. The tests run in your CI; Qodo itself does not execute your product.