38c5aab661
* Extend the data generation Python framework for cost calibration to support data generation for CE testing as follows: - the entry point is ce_generate_data.py, - the configuration of the generated data is in ce_generate_data_settings.py, - all collection data is exported into a single JSON file stored in 'jstests/query_golden/libs/data', and a schema file stored in the same directory * Implement a JS data loader function that also creates all indexes specified in the schema file. * Add a small JS test that shows how to load the generated JSON files into collections.
Cost Model Calibrator
Python virtual environment
The following assumes you are using python from the MongoDB toolchain.
/opt/mongodbtoolchain/v4/bin/python3
Getting started
(mongo-python3) deactivate # only if you have another python env activated
sh> /opt/mongodbtoolchain/v4/bin/python3 -m venv cm # create new env
sh> source cm/bin/activate # activate new env
(cm) python -m pip install -r requirements.txt # install required packages
(cm) python start.py # run the calibrator
(cm) deactivate # back to bash
sh>
Install new packages
(cm) python -m pip install <package_name> # install <package_name>
(cm) python -m pip freeze > requirements.txt # do not forget to update requirements.txt