top of page

Create Full Stack Tests From Session Recordings

Creating Tests


After the SDKs are installed, you can simply use your web app as usual and TestChimp will record what happens in the entire stack during sessions (A session is active until the web browser is closed). You can then query for specific sessions via the Sessions tab.




Click on the Play button on the session, which will open up the full stack recording replay. Click on the “Create Test…” button. This will open up the session in Test Studio where you can add / remove steps, update request fields etc. from the recorded session. TestChimp will automatically suggest assertions based on the requests / responses as well as variables to be configured. You can define additional assertions covering any layer of your stack and additional variables as needed. Once done, you can save the test. Tests are organized into test suites which help group related tests together.


Editing Tests


Add Step


You can add a step by clicking on the “+” button on the step after which the new step should be added. To add a step to the beginning of the test, click on the “+” button on the first step, and select the “Before” option in the resulting popup.


Remove Step


Click on the Delete button on the corresponding step.


Update Request Field Values


Request field values can be directly updated on the text fields in the structured view. Alternatively, you can specify a generator for the field which can be one of the following:


  • Variable Expression: A mix of constant values and variables referred within {{ }} syntax. Eg: “Bearer {{id_token}}”


  • Vocabulary: Provide a list of values from which the field value should be populated. In different runs, different values from the vocabulary will be selected for the run. This introduces fuzziness to the tests making them more thorough and robust.


  • Script: You can write a JS script that outputs the desired value. You can also explain in plain english the expected output, and get the script automatically written by our ChimpGPT.


Once a test step is edited, you can click on the “Generate Request… ” button to view a sample generated request (to verify everything is as expected). You can select the environment and the endpoint to send it to, and click on Send to execute just that step.


Add Assertions


Assertions represent what you want to "verify" to be behaving as expected in your test. TestChimp provides powerful capabilities around creating versatile assertions.


For instance, you can create assertions that span across multiple steps (eg: make sure the cart id used in add to cart API call is the same as the cart id passed in calculate_shipping_cost step), or even utilize data points recorded within internal execution spans of your tech stack.


An assertion basically consists of a left side expression, a right side expression and an assertion operation. Expressions can be any one of the following:


  • Variable

  • Fixed Value

  • Request / Response field value (at any operation in the call tree of any step)

  • A span attribute value at any operation in the call tree (from the Attributes tab)


To set a field value / span attribute as an expression, click on the action menu at the corresponding field, and select “set as left / right expression”.


You can directly type fixed values as expressions, or use {{ }} notation to type in variables as expressions. The expressions of an assertion can come from different test steps, allowing for cross step verifications.


Define Variables


You can save the value of any field in requests / responses (or span attributes in any operation) as a variable by clicking on the action menu on the field and selecting “Set as Variable…”. The variable will be available for use in all subsequent test steps.


Save Tests


In TestChimp, Test cases are organized into Test Suites. When saving a test, you can select which test suite the test case should be added to. To add to a new test suite, simply type in the name of the new test suite in the test suite selection. 



 


Run Tests



You can execute the created tests either via TestChimp dashboard, or invoke them via TestChimp Data API (from your CI / CD pipeline for instance).


Via TestChimp Dashboard


Go to the Tests tab. Use the filters section to filter into the tests of interest. Click on Run action on the test of interest to run a single test, or select the tests you want to run and click Run Selected Tests button.


The dashboard will be updated with details of the execution progress, and once complete, you can view the test run results by clicking on “View Results”.


 Via API


You can invoke test executions via TestChimp Data API (for example as part of your CI / CD pipeline).


Make a POST request to https://featureservice.testchimp.io/api/run_tests endpoint with the following payload:

{
	“test_case_regex”:”<regex for matching test cases>”,
	// If provided, the test cases in suites matching this regex will be run - else, defaults to all test suites
	“test_suite_regex”:”<regex for matching test suites>”
}

Include the following http headers:

x-testchimp-project-id: <Your TestChimp Project Id>

x-testchimp-api-key-: <Your TestChimp Api Key>


Eg: To run all test cases:


{
	“test_case_regex”:”.*”
}

This will return a response of following format:

{
map<string,string> execution_id_map=1;
string batch_invocation_id=2;
}

Eg:

{
  "batch_invocation_id":"6493-asda-1211-gkaa",
  "execution_id_map": {
    "LoginTestSuite#LoginSuccessTest": "14230-asdfa-d45f2-ddsfs",
    "PaymentTestSuite#MasterCardPaymentTest": "23231-afffa-d45f2-dd41s",
    "LoginTestSuite#LoginSuccessTest": "12630-aghfa-d65f4-1esfs",
    ...
  }
}

You can poll for the results of the tests periodically (eg: every 30s) by making a POST request to https://featureservice.testchimp.io/api/list_test_invocation_results endpoint with the following payload (to poll for each individual test invocations' state):


{
	“invocation_ids”:[<Invocation Ids received in the run tests response>]
}

Or, to get the invocation state of all tests belonging to the current batch invocation, you can query by the batch_invocation_id returned in the run test response:


{
	"batch_invocation_id":"<BATCH_INVOCATION_ID>"
}

Remember to Include the X-testchimp-project-id and X-testchimp-api-key headers for authorization.

bottom of page