Commit Graph

702 Commits

Author SHA1 Message Date
Alex Gorrod
9768c2db0b WT-2554 Add initial C test framework. (#2655)
* WT-2554 Add initial C test framework.

With a few sample applications. Need to do further work to cleanup
the same applications and share as much code/style as possible.

* WT-2554 Fixup build errors.

* Fix compiler warning.

* Cleanup

* Ensure each test runs in a different directory.

* Update C test suite names and add copyright.

* Move test code into subdirectories.

* Replace test_util.i with a real utility library now that we have one.

Nothing needs to be an inline function.

Mostly cherry picked from 7c18420.

* Use WiredTiger getopt in C test suite for platform portability.

* Add a header comment to each test case.

* Add a C test suite entry auto generator

* Style, KNF

* Implement review feedback.

* Build test/utility library on Windows

* Add comment to script.  Fix a printf.
2016-05-23 13:13:54 -04:00
sueloverso
9a6329b560 WT-2632 Tolerate EBUSY when a checkpoint cursor is open. (#2734)
* WT-2632 Tolerate EBUSY when a checkpoint cursor is open.

* Tolerate EBUSY in general.
2016-05-19 14:44:15 -04:00
Keith Bostic
0893547e78 WT-2628 Reconciliation can return without unlocking the page lock (#2728)
Don't clean up reconciliation structures before they've been initialized.

Fix two paths where scratch buffers could be allocated and fail to be
released.
2016-05-16 11:13:05 -04:00
Keith Bostic
2437ec597d WT-2615 Enabling checkpoints in test/format leads to reduced concurrency (#2720)
Don't queue threads for checkpoint operations, just skip the scheduled
checkpoint operation if a checkpoint is already in progress.

Don't wait for backups to complete if scheduled to do a named checkpoint,
do an unnamed checkpoint instead.

Fix a bug: we were scheduling named checkpoints 80% of the time instead
of the documented 20%; change that number to 5%, named checkpoints aren't
worth testing that heavily.

Don't bother avoiding scheduling two named checkpoints in a row, it's
not worth the effort.
2016-05-05 09:25:01 +10:00
Keith Bostic
b59476e03d WT-2600: clean up test program includes (#2708)
Clean up test program #includes and add generic handling to memory allocation failures.

Now the test programs include <wt_internal.h>, the local includes
aren't necessary, and it makes lint sad.
2016-05-02 16:05:08 +10:00
Keith Bostic
5be0124f3b WT-2574: format doesn't free all allocated configure memory
I never remember that booleans don't take "on" or "off" arguments,
change it so I don't have to.
2016-04-22 16:32:56 -04:00
Keith Bostic
1a013c1e91 WT-2574: format doesn't free all allocated configure memory
Rework format configuration to use config_single() more, that way we
discard allocated memory when in-memory configurations turn off previous
configurations.
2016-04-22 16:09:34 -04:00
Michael Cahill
3a6488a12d Merge pull request #2684 from wiredtiger/wt-2574-free-dup-config
WT-2574 Ensure test/format doesn't leak memory
2016-04-22 14:02:24 +10:00
Michael Cahill
7c17fc21a2 WT-2553 Fix implicit size_t conversion in test/format.
test/format/config.c:415:16: error: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'uint32_t' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
2016-04-22 12:52:52 +10:00
Alex Gorrod
cda4840c4b Clear pointer after freeing memory. 2016-04-22 01:50:07 +00:00
Alex Gorrod
c899852fea WT-2574 Ensure test/format doesn't leak memory
It can if an option is specified in the configuration file and
the command line.
2016-04-22 01:24:51 +00:00
sueloverso
2391a26f9c Merge pull request #2680 from wiredtiger/wt-2574-free-config-mem
WT-2574: format doesn't free all allocated configure memory
2016-04-21 13:35:16 -04:00
Keith Bostic
7f489b91ef WT-2572: don't select an in-memory format run if incompatible options configured
Don't configure in-memory if LSM is the data-source.
2016-04-21 11:30:30 -04:00
Keith Bostic
027992b979 WT-2574: format doesn't free all allocated configure memory
Create a new routine that frees all allocated configuration memory at
the end of the run, separate from the function that clears temporary
settings before each new run.
2016-04-21 11:20:35 -04:00
Keith Bostic
9288c08316 WT-2572: recovery tests require logging, must turn off in-memory configuration
A more general fix -- don't configure in-memory at all, if there's a
specific configuration of an incompatble feature, like logging.
2016-04-21 10:59:23 -04:00
Keith Bostic
4544a68228 WT-2572: recovery tests require logging, must turn off in-memory configuration 2016-04-21 10:45:47 -04:00
Michael Cahill
fc1233bb50 Merge pull request #2656 from wiredtiger/wt-2223
WT-2223 WT-2553 in_memory documentation and stress testing
2016-04-21 09:48:25 +10:00
Keith Bostic
79543f9021 WT-2223 WT-2553 in_memory documentation and stress testing
Revert a change making the record-number fields uint32_t's, it's
messier than what was there before, leave them as uint64_t's.
2016-04-19 13:38:19 -04:00
Keith Bostic
561741d581 WT-2223 WT-2553 in_memory documentation and stress testing
Changes to allow running in-memory mode format, automatically; 1 in 20
randomly selected runs are in-memory.

Use 2x the initial load's data size as the minimum cache size. That
appears to always be enough for the column-store initial load, and is
usually enough for the row-store initial load. When not enough, we reset
the number of keys to the number we were able to load, and also set the
insert percentage to a low value and the delete percentage to a higher
value, in order to give ourselves some room in the cache so the run can
reasonably continue.

There is another major change: previous versions of this code used 6-8
times the initial load's data size worth of cache, because we were
creating a relatively large number of huge keys (those are roughly 80K
byte keys). The reason for that was we only generate 1000 random key
lengths (and then select among those lengths), and with a million keys
selecting from 1000 key lengths, we were creating huge keys too often.
This code no longer does that, and both keys and values now select huge
key/value sizes only once in every 2500 key/value pairs.
2016-04-19 13:18:49 -04:00
Susan LoVerso
e607ca1719 WT-2557 Need to truncate logs before closing backup cursor. 2016-04-19 11:14:42 -04:00
Keith Bostic
7d4e2cf5ca Merge branch 'develop' into wt-2223 2016-04-19 08:46:47 -04:00
Keith Bostic
e6b2dff81f WT-2223: Add stress testing for in-memory
WT_CACHE_FULL requires transaction rollback.
2016-04-18 12:29:30 -04:00
Keith Bostic
0409f6d53d Merge branch 'develop' into wt-2223 2016-04-18 12:18:17 -04:00
Keith Bostic
c5bac1fb3c WT-2557: format test program should discard log files after incremental backup
KNF
2016-04-16 08:33:58 -04:00
Keith Bostic
3bec9a933e WT-2557: format test program should discard log files after incremental backup 2016-04-16 08:32:51 -04:00
Keith Bostic
833db3433a WT-2103: add incremental backup testing to format
Don't do incremental backups if archival logging is configured.
2016-04-15 15:45:51 -04:00
Keith Bostic
39a9629e82 WT-2223: Add stress testing for in-memory
Change test/format to handle WT_CACHE_FULL returns.
2016-04-15 14:28:05 -04:00
Keith Bostic
8ba8a77187 WT-2103: add incremental backup testing to format
Fix a comment.
2016-04-15 12:11:57 -04:00
Keith Bostic
f9a525e493 WT-2103: add incremental backup testing to format
Add support for doing incremental backups.
2016-04-14 12:30:34 -04:00
Keith Bostic
9eacbca4f9 WT-2103: add incremental backup testing to format
Instead of copying the sources twice for hot backup, (they could
change), copy the sources once, then copy the copy to somewhere
else.
2016-04-14 08:39:46 -04:00
Keith Bostic
2a71e4b928 WT-2103: add incremental backup testing to format
Change the backup code to use the new __wt_copy_and_sync() code instead
of shell commands, get better testing on that code.

Rename "BACKUP2" to "BACKUP_COPY".
2016-04-13 10:39:42 -04:00
Keith Bostic
bc966e4e85 WT-2103: add incremental backup testing to format
The "stop threads" condition was never being reset, so we only did
backups, compaction and a long-running transaction on the first chunk
of work, not on any subsequent chunks.
2016-04-13 10:22:01 -04:00
Keith Bostic
d99e86d43a WT-2535: Add an explicit test for transactions reading their writes
When we switched cursor semantics to insert/update by default, we more
exactly match Berkeley DB's semantics, simplify some code, not-found is
no longer an expected return.
2016-04-12 10:11:28 -04:00
Keith Bostic
4890b92d51 WT-2535: Add an explicit test for lost updates
If we see WT_ROLLBACK in the snapshot test code, return it so we abort
the transaction.
2016-04-10 17:47:39 -04:00
Keith Bostic
4f84ad8b8c WT-2535: Add an explicit test for lost updates
Rework the code to display a snapshot isolation mistake, to better
handle column-store output.
2016-04-10 15:33:21 -04:00
Keith Bostic
127c0d2dc1 WT-2535: Add an explicit test for lost updates
Unused label.
2016-04-10 15:21:03 -04:00
Keith Bostic
7d4e2ba7c2 WT-2535: Add an explicit test for lost updates
If the isolation level is snapshot, track the operations and verify them
when terminating the transaction.
2016-04-10 15:16:53 -04:00
Keith Bostic
a3750a0160 WT-2535: Add an explicit test for lost updates
Reset the external API handle when reopening the connection.
2016-04-10 15:16:33 -04:00
Keith Bostic
0bd8c960c5 WT-2535: Add an explicit test for lost updates
Pull the value buffers out of the working routines and into the caller,
minor cleanup that makes the next set of changes simpler.
2016-04-09 14:24:54 -04:00
Keith Bostic
aeeca6ce6d WT-2535: Add an explicit test for lost updates
Rework underlying operation routines to return WT_NOTFOUND, allows the
caller to make better decisions about the cursor's position.  In
general, don't rollback the transaction unless we get a rollback return
and we're actually in a transaction, continue in the current transaction
in the WT_NOTFOUND case.
2016-04-09 13:48:38 -04:00
Keith Bostic
cf91012882 WT-2535: Add an explicit test for lost updates
Switch to manipulating WT_ITEMs instead of individual buffers, it makes
some things in the ops code easier.
Print key-counts with u64's, not u32's.
2016-04-09 09:57:42 -04:00
Keith Bostic
03a47af3ed WT-2535: Add an explicit test for lost updates
Change key_gen/val_gen to take a void * function argument, avoids a
bunch of random casting when using WT_ITEMs to hold keys/values.
2016-04-08 18:28:20 -04:00
Keith Bostic
33e2ceb566 WT-2535: Add an explicit test for lost updates
Switch from "int ret" to WT_DECL_RET everywhere.
2016-04-08 17:38:52 -04:00
Keith Bostic
ad25271a65 WT-2535: Add an explicit test for lost updates
Don't rollback transactions unless read returns WT_ROLLBACK, not
finding a searched-for key isn't reason to rollback.
2016-04-08 17:35:56 -04:00
Keith Bostic
ab42dd2acd WT-2535: Add an explicit test for lost updates
Replace a test and shift a chunk of code to a lesser indentation
level for clarity, no real change.
2016-04-08 17:30:52 -04:00
Keith Bostic
6449a81e40 WT-2535: Add an explicit test for lost updates
Instead of setting the isolation level each time a session is created,
set it every time a new transaction is begun.
2016-04-08 17:27:00 -04:00
Keith Bostic
019c0aa4d8 WT-2535: Add an explicit test for lost updates
Switch a few true/false integers to a boolean type.
2016-04-08 17:06:19 -04:00
Keith Bostic
13028c7430 WT-2535: Add an explicit test for lost updates
Add a configuration option for the frequency of explicit transactions,
we want to run with both many and no explicit transactions.
2016-04-07 18:15:04 -04:00
Susan LoVerso
20b1a6ed80 WT-2497 Create second backup copy. 2016-03-17 10:39:32 -04:00
Keith Bostic
cc0d34b093 WT-2471: review WiredTiger "int" printf formats 2016-03-11 16:02:41 -05:00