Increase the pre-alloc of WT_EXT/WT_SIZE structures from 5/2 to 5/5 in
the write/free paths, and from 100/10 to 250/250 in the checkpoint path.
Turn off cache limits: checkpoints can take tens of thousands of WT_EXT
structures and hundreds of WT_SIZE structures, we want to cache
everything until the checkpoint completes. Add a cleanup call after the
checkpoint so we don't tie down megabytes of memory between checkpoints.
Split the previous pre-allocate function into two parts, it's simpler
as a function to pre-allocate entries and a separate function to take
an entry for use.
Remove the tracking of "memory allocations while locked"; there are
extent lists we read while not holding any locks, and they can be
really, really large requiring the allocation of tens of thousands of
WT_EXT structures, they make the statistic meaningless.
Change compaction to attempt compaction any time it looks likely we can
recover the last 10% of the file, and, for now, to only attempt to
recover that last 10% of the file.
Don't rewrite a block if there isn't a useful available block in the
first 90% of the file. (Previously, compacting the whole file in one
pass could end up re-writing blocks to available blocks at the end of
the file which isn't useful, and, in some cases, even extending the
file.
The reason we're only compacting the last 10% of the file is that we
will need to checkpoint periodically otherwise re-written blocks, which
should be the new, available blocks for compaction, won't be considered
because they're still in use. Doing that automatically will be the next
set of changes.
Change compaction to use first-fit allocation instead of best-fit so
we're always copying to the beginning of the file if at all possible.
Add WT_SESSION::create "block_allocation" configuration string, allows
applications to configure first-fit. Change test/format to use the
first-fit algorithm 20% of the time.
Remove the WT_SESSION.compact "trigger" configuration string, it's no
longer used.
Rename WT_SESSION::create "allocation_size" configuration string to
"block_allocation_size" to match "block_allocation" and
"block_compressor" configuration strings.
Inline the block extension search functions, they get called a lot.
Add "compact" verbose debugging string.
Fix a bug where a spinlock could be left held by compaction.
ignores some fields, its intent is to clear specific fields between
stat calls, not clear the whole structure. Add clear functionality
to the stat initialization function so it can be used to refresh stat
structures.
Change the name of the stat "clear" function to "refresh" so nobody
makes this mistake in the future (but don't drive that change out to the
application-level, "clear" is still the right way for the application
programmer to think about this).
Change the LSM and the table statistics aggregation code to always call
the statistics structure's init function, that will clear any previous
values.
