Commit Graph

184 Commits

Author SHA1 Message Date
Alex Gorrod
9768c2db0b WT-2554 Add initial C test framework. (#2655)
* WT-2554 Add initial C test framework.

With a few sample applications. Need to do further work to cleanup
the same applications and share as much code/style as possible.

* WT-2554 Fixup build errors.

* Fix compiler warning.

* Cleanup

* Ensure each test runs in a different directory.

* Update C test suite names and add copyright.

* Move test code into subdirectories.

* Replace test_util.i with a real utility library now that we have one.

Nothing needs to be an inline function.

Mostly cherry picked from 7c18420.

* Use WiredTiger getopt in C test suite for platform portability.

* Add a header comment to each test case.

* Add a C test suite entry auto generator

* Style, KNF

* Implement review feedback.

* Build test/utility library on Windows

* Add comment to script.  Fix a printf.
2016-05-23 13:13:54 -04:00
Keith Bostic
2437ec597d WT-2615 Enabling checkpoints in test/format leads to reduced concurrency (#2720)
Don't queue threads for checkpoint operations, just skip the scheduled
checkpoint operation if a checkpoint is already in progress.

Don't wait for backups to complete if scheduled to do a named checkpoint,
do an unnamed checkpoint instead.

Fix a bug: we were scheduling named checkpoints 80% of the time instead
of the documented 20%; change that number to 5%, named checkpoints aren't
worth testing that heavily.

Don't bother avoiding scheduling two named checkpoints in a row, it's
not worth the effort.
2016-05-05 09:25:01 +10:00
Keith Bostic
b59476e03d WT-2600: clean up test program includes (#2708)
Clean up test program #includes and add generic handling to memory allocation failures.

Now the test programs include <wt_internal.h>, the local includes
aren't necessary, and it makes lint sad.
2016-05-02 16:05:08 +10:00
Keith Bostic
f9a525e493 WT-2103: add incremental backup testing to format
Add support for doing incremental backups.
2016-04-14 12:30:34 -04:00
Keith Bostic
2a71e4b928 WT-2103: add incremental backup testing to format
Change the backup code to use the new __wt_copy_and_sync() code instead
of shell commands, get better testing on that code.

Rename "BACKUP2" to "BACKUP_COPY".
2016-04-13 10:39:42 -04:00
Keith Bostic
d99e86d43a WT-2535: Add an explicit test for transactions reading their writes
When we switched cursor semantics to insert/update by default, we more
exactly match Berkeley DB's semantics, simplify some code, not-found is
no longer an expected return.
2016-04-12 10:11:28 -04:00
Keith Bostic
7d4e2ba7c2 WT-2535: Add an explicit test for lost updates
If the isolation level is snapshot, track the operations and verify them
when terminating the transaction.
2016-04-10 15:16:53 -04:00
Keith Bostic
aeeca6ce6d WT-2535: Add an explicit test for lost updates
Rework underlying operation routines to return WT_NOTFOUND, allows the
caller to make better decisions about the cursor's position.  In
general, don't rollback the transaction unless we get a rollback return
and we're actually in a transaction, continue in the current transaction
in the WT_NOTFOUND case.
2016-04-09 13:48:38 -04:00
Keith Bostic
cf91012882 WT-2535: Add an explicit test for lost updates
Switch to manipulating WT_ITEMs instead of individual buffers, it makes
some things in the ops code easier.
Print key-counts with u64's, not u32's.
2016-04-09 09:57:42 -04:00
Keith Bostic
03a47af3ed WT-2535: Add an explicit test for lost updates
Change key_gen/val_gen to take a void * function argument, avoids a
bunch of random casting when using WT_ITEMs to hold keys/values.
2016-04-08 18:28:20 -04:00
Keith Bostic
13028c7430 WT-2535: Add an explicit test for lost updates
Add a configuration option for the frequency of explicit transactions,
we want to run with both many and no explicit transactions.
2016-04-07 18:15:04 -04:00
Susan LoVerso
20b1a6ed80 WT-2497 Create second backup copy. 2016-03-17 10:39:32 -04:00
Keith Bostic
525b9a6f30 WT-2422: multiple definitions of custom die function
Define the custom die function once per program.
2016-02-22 11:03:47 -05:00
Alex Gorrod
2b4cd8de33 WT-2405 Switch new testing error handler to be shared in test util 2016-02-19 01:54:23 +00:00
Alex Gorrod
ec9a451025 WT-2405 Switch test/format to use shared testutil_die 2016-02-19 01:44:58 +00:00
Keith Bostic
ca48eaa794 WT-2405: test utility error handling.
Add a check routine to make simple error handling easier.
2016-02-18 08:58:39 -05:00
Keith Bostic
41a08af673 WT-2361: column-store starting record number error
Add a "quiet" configuration command, so it's possible to keep format quiet
from the CONFIG file.
2016-02-02 07:23:08 -05:00
Keith Bostic
12be963758 WT-2352: Allow build and test without requiring lz4
Remove support for the bzip2 compression engine.
2016-01-28 12:44:09 -05:00
Keith Bostic
e2ba70597a WT-2287: WT_SESSION.rebalance
Use a static buffer for the wiredtiger_open configuration, that way we
don't have to free it to make clang LeakSanitizer happy.
2016-01-19 20:33:22 -05:00
Keith Bostic
2fbec8cc81 Merge branch 'develop' into wt-2287-rebalance 2016-01-19 09:59:03 -05:00
Keith Bostic
5a84febed7 WT-2247, WT-2312: variable-length column-store in-memory page splits
Review comments, get rid of incorrect/unnecessary const declarations.
2016-01-19 08:35:51 -05:00
Keith Bostic
f19f26b60f Merge branch 'develop' into wt-2247-column-store-splits 2016-01-19 08:19:20 -05:00
Keith Bostic
a6f27c99a5 Merge branch 'develop' into wt-2287-rebalance 2016-01-01 16:51:17 -05:00
Keith Bostic
344a714c3d Upgrade copyright notices from 2015 to 2016. 2016-01-01 16:37:39 -05:00
Keith Bostic
4a4439a792 Copy the configuration string, they get free'd when the run ends. 2015-12-31 21:17:15 -05:00
Keith Bostic
8e31d56d32 error: initialization from incompatible pointer type
[-Werror=incompatible-pointer-types]
2015-12-31 15:28:28 -05:00
Keith Bostic
f73041042f error: assignment discards "const" qualifier from pointer target type
[-Werror=discarded-qualifiers]
2015-12-31 15:11:01 -05:00
Keith Bostic
1df86bfb7a WT-2287: Add support for a WT_SESSION.rebalance method that rebuilds the
underlying btree, correcting any imbalance.
2015-12-17 12:29:08 -05:00
Keith Bostic
23d5f86dc0 WT-2258: Add configuration support for a direct_io=(data) option. 2015-12-02 18:05:38 -05:00
Keith Bostic
add119923b WT-2178: add support to format for in-memory runs. 2015-11-04 10:21:40 -05:00
Keith Bostic
939c7210db lint. 2015-10-19 09:26:07 -04:00
Keith Bostic
3332588532 Add simple strdup/malloc wrappers so we don't have to check for error
returns.
2015-08-07 09:36:36 -04:00
Keith Bostic
1b4e888edb Merge branch 'develop' into wt-1967-evict-any 2015-07-28 20:02:03 -04:00
Michael Cahill
7063f8b6e8 SERVER-19340 Avoid type aliasing in the random number generator. 2015-07-27 12:07:31 +10:00
Keith Bostic
5d0f39b1c9 Clean up error handling around records not being found; if we have an
open cursor that found the record once, it should find it again, it's
an error if we've lost that record.
2015-07-21 14:50:28 -04:00
Keith Bostic
1c0ee83087 WT-1967: creating a lookaside table to hold updates which are no longer
useful, but which still might be required by current readers.

When reconciling pages with updates which still might be read by existing
snapshot readers, write the updates into a lookaside table.  When those
pages are read back into the cache, any still-relevant updates are read
back from the lookaside table and attached to the page.

Add a new on-disk page-header flag, WT_PAGE_LAS_UPDATE, set when a page
has records in the look-aside table.

Remove the __wt_evict_page() code, replace with calls to __wt_evict(),
and add testing in reconciliation to only write committed updates.  As
part of that change, remove the WT_ISO_EVICTION transaction isolation
configuration, it's no longer used (WT_ISO_READ_UNCOMMITTED is used
instead).

Delete the WT_PAGE_MODIFY.rec_max_txn field (no longer needed after
reconciliation is changed to return EBUSY if a page wasn't marked
clean, replacing the test of WT_PAGE_MODIFY.rec_max_txn __evict_review).

Delete the test for all updates being globally visible when a page is
discarded from the cache, that's no longer true. Delete the
WT_SESSION_DISCARD_FORCE flag, used to turn off that check when forcibly
discarding a page.
2015-07-19 10:58:10 -04:00
Keith Bostic
d42029bc23 WT-1967: Add support for a long-running, isolation=snapshot, reader
thread.
2015-07-14 11:05:06 -04:00
Alex Gorrod
51d328b67e Add ability to turn verify and salvage off in test/format.
This can be handy when reproducing failures that aren't uncovered
with verify/salvage, since it decreases the runtime for a single pass.
2015-06-30 01:29:43 +00:00
Keith Bostic
bb986cf409 Avoid read/write races in the PRNG: switch to using an 8B value instead
of a two, 4B value array, declare the value volatile to avoid having the
compiler alter the read/write path.

Add a check if a PRNG component ever goes to zero and reset it.
2015-05-30 11:53:52 -04:00
Keith Bostic
abdcc19a24 Merge pull request #1975 from wiredtiger/log-compress-stress
Add encryption and log compression to stress testing
2015-05-18 12:37:52 -04:00
Don Anderson
42e6b560d9 Added simple 'rotn' encryption, to be used on 30% of runs. WT-1471 2015-05-17 14:23:37 -04:00
Don Anderson
6cbbbb1f67 Added logging compression, some refactoring related to logging. WT-1471. 2015-05-14 16:31:20 -04:00
David Hows
21148870aa Change to __wt_exists 2015-05-12 13:41:33 +10:00
Keith Bostic
78d623472c Merge branch 'develop' into lz4-compress-destsize 2015-05-11 10:38:26 -04:00
Keith Bostic
dcedd7fa7e Single-thread failure so we don't close the global file handles twice. 2015-05-10 10:42:30 -04:00
Keith Bostic
8210599a4e Don't call fclose multiple times, clear the FILE handle to make sure
it's only once.
2015-05-10 10:38:01 -04:00
Keith Bostic
ca7a3de5eb Rework test/format to use the WiredTiger RNG, not rand.
Give every thread its own RNG state so there's no competition for the
cache lines.

Initialize the random-number generator early so we catch all possible
configuration uses.

Remove duplicates of WiredTiger's macros.
2015-05-09 11:15:59 -04:00
David Hows
daad9bf267 Revert format to old die 2015-05-07 16:57:04 +10:00
David Hows
29477b0ac5 Add testutils to all tests 2015-05-05 14:51:23 +10:00
Keith Bostic
1025b9fa76 Add LZ4 raw support based on the new LZ4_compress_destSize API, making
it the default (LZ4 block mode is now lz4-noraw, to match zlib).

This implementation mostly works, but...

It's not backward compatible because we're storing more information in
the destination buffer than before (two 4B values instead of a one 8B
value). The 2.5.3 release is the only release that had LZ4 support, so
I don't expect this to be a problem. We could be backward compatible,
but not without a fair amount of pain.

This code sizes decompression buffers incorrectly. The WiredTiger btree
code sizes the decompression buffer based on the final in-memory size
of the data, which isn't correct for data compressed with LZ4's raw
compression function which may have compressed more data than the final
in-memory size.
2015-05-03 11:51:21 -04:00