TestChimp vs UI Automation Scripts

A common approach taken for test automation is writing UI automation scripts in frameworks such as Playwright / Selenium / Cypress etc. While they are great for sanity checking the UI layer of your application, using them for comprehensive user journey and related business logic testing has a few key downsides compared to TestChimps’ full stack manual to automation test conversion approach.

No Coding Required

UI automation scripts still require an engineer to manually write the test script simulating the scenario being automated. This requires mastering framework specific syntax and idiomatics of the scripting language - which can introduce a considerable learning curve when it comes to doing anything non-trivial.

TestChimp records the complete payloads observed during manual testing sessions. You can then load them in Test Studio and update specific field values easily in the structured view of a given request. TestChimp automatically suggests variables and assertions to add to the test, making the test creation as simple as clicking a few buttons. You can also create complex assertions by simply right clicking on the fields and adding them as expressions of an assertion as shown below:

More Thorough Assertions

Typically, manually authored scripts tend to have minimal superficial assertions given the manual effort required. For instance, a script author may add an assertion to just verify that the response of an API request is successful (does not error). Now imagine this endpoint is expected to return a list of items in the response (say a list of products to display etc.). If the only assertion added is the response being successful, even if the list returned in the response is empty, the test will keep passing despite the functionality being broken.

In comparison, TestChimp analyzes the actual request and response payloads observed during the test execution to derive a meaningful and comprehensive set of assertions, and suggests them automatically - allowing you to quickly add descriptive set of assertions to your test making them significantly more thorough than manually authored counterparts.

Verifies Every Service Layer

UI automation tests focus on verifying just UI elements whereas TestChimp validates every service layer of your application stack. Typically, this would require writing API tests and multiple integration tests to cover the expected behaviors at each layer. 

Consider for instance, a P2P payments application. When a user sends payment to another user, the UI layer test would only be able to verify  whether the UI screen of user A shows “payment successfully sent”. This doesn’t necessarily mean that everything worked as intended - since as long as the frontend received a successful response, it would show that screen. For thorough verification, you will need to verify that at the backend layer that's handling transactions, the correct recipient (as sent in the frontend request) was indeed sent the payment, the correct amount was debited at the storage layer, the transaction IDs recorded for both the users are consistent etc. None of those data are available at the UI layer for the UI automation script to verify them. 

TestChimp, on the other hand, records interactions at every layer of the stack, and suggests assertions for each layer by analyzing the payloads of interactions with that layer (and across steps in the session). You can easily add assertions verifying behavior deep in the stack by simply right clicking on a payload field and adding as an expression in an assertion without requiring any code.

Faster Execution

Unlike UI automation scripts, TestChimp tests do not require rendering of UI elements - which is usually slow (loading and rendering HTML elements / libraries etc.). This enables TestChimp to execute tests at blazing fast speeds, making them suitable for large test suites that are run frequently - without costing significant resource usages.

Immune from breakages due to UI changes

A common concern with UI based automation is that they break frequently whenever UI is updated, and requires writing the scripts carefully to make them resilient to changes. But UI changes are usually decoupled from the business logic implementations of the backend, and relying on UI automation scripts for testing your systems’ business logic means your tests would break even when the underlying business logic implementation is unchanged - making the tests brittle. UI based testing is also usually flaky due to the non deterministic nature of UI rendering / delays etc. adding a lot of noise to your test execution results. Since TestChimp replays and validates at API layer and beyond, TestChimp tests don’t break with UI changes, and doesn’t suffer the flakiness introduced by UI rendering, resulting in more robust set of test suites.

Tests that mimic user journeys more realistically

UI scripts are hand crafted UI manipulations as authored by a test engineer. In comparison, since TestChimp tests are based off of recorded actual usage sessions, they mimic the user journey being tested more closely (since the test engineer simply has to manually run through the desired user journey as a normal user would). This results in tests that are more meaningful, testing out all the steps and variations that could otherwise be missed out.