mongo/buildscripts/resmokelib/testing/testcases/README.md

TestCases

TestCases extend Python-based unittest.TestCase objects that resmoke can run as different "kinds" of tests.

Supported TestCases

Specify any of the following as the test_kind in your Suite config:

• all_versions_js_test: AllVersionsJSTestCase - Alias for JSTestCase for multiversion passthrough suites.
  • It runs with all combinations of versions of replica sets and sharded clusters. The distinct name is picked up by task generation.
• benchmark_test: BenchmarkTestCase - A Benchmark test to execute.
• bulk_write_cluster_js_test: BulkWriteClusterTestCase - A test to execute with connection data for multiple clusters passed through TestData.
• cpp_integration_test: CPPIntegrationTestCase - A C++ integration test to execute.
• cpp_libfuzzer_test: CPPLibfuzzerTestCase - A C++ libfuzzer test to execute.
• cpp_unit_test: CPPUnitTestCase - A C++ unit test to execute.
• db_test: DBTestCase - A dbtest to execute.
• fsm_workload_test: FSMWorkloadTestCase - A wrapper for several copies of a _SingleFSMWorkloadTestCase to execute.
• js_test: JSTestCase - A wrapper for several copies of a _SingleJSTestCase to execute
  • Around 75% of all suites use the js_test kind. See jstests/README.md for specific guidance.
• json_schema_test: JSONSchemaTestCase - A JSON Schema test to execute.
• magic_restore_js_test: MagicRestoreTestCase - A test to execute for running tests in a try/catch block.
• mongos_test: MongosTestCase - A TestCase which runs a mongos binary with the given parameters.
• multi_stmt_txn_passthrough: MultiStmtTxnTestCase - Test case for multi statement transactions.
• parallel_fsm_workload_test: ParallelFSMWorkloadTestCase - An FSM workload to execute.
• pretty_printer_test: PrettyPrinterTestCase - A pretty printer test to execute.
• py_test: PyTestCase - A python test to execute.
• query_tester_self_test: QueryTesterSelfTestCase - A QueryTester self-test to execute.
• query_tester_server_test: QueryTesterServerTestCase - A QueryTester server test to execute.
• sdam_json_test: SDAMJsonTestCase - Server Discovery and Monitoring JSON test case.
• server_selection_json_test: ServerSelectionJsonTestCase - Server Selection JSON test case.
• sleep_test: SleepTestCase - SleepTestCase class.
• tla_plus_test: TLAPlusTestCase - A TLA+ specification to model-check.

Interfaces

Top level interfaces:

• TestCase - A test case to execute. The run_test method must be implemented.
• ProcessTestCase - Base class for TestCases that executes an external process. The _make_process method must be implemented.

Subclasses:

• JSRunnerFileTestCase - A test case with a static JavaScript runner file to execute.
• MultiClientsTestCase - A wrapper for several copies of a SingleTestCase to execute.
• TestCaseFactory - Convenience interface to initialize and build test cases

Fixture TestCases

These are testcases that are used to coordinate fixture lifecycles via resmoke's internal FixtureTestCaseManager.

NOTE This design does lead to seeing "extra" tests in a run, where a fixture sets up, your N tests are run, and the fixture tears down, so you see N+2 "tests" passing via resmoke.

• FixtureTestCase - Base class for the fixture test cases.
• FixtureSetupTestCase - TestCase for setting up a fixture.
• FixtureTeardownTestCase - TestCase for tearing down a fixture.
• FixtureAbortTestCase - TestCase for killing/aborting a fixture. Intended for use before archiving a failed test.
  • When resmoke detects that a test has failed (and archiving is configured​), it dynamically generates a new FixtureAbortTestCase for immediate execution. This test case sends a SIGABRT to each running mongod process.

Testing TestCases

Self-tests for the testcases themselves can be found in buildscripts/tests/resmokelib/testing/testcases/