Compare commits

...

27 Commits
2.1.1 ... 2.1.2

Author SHA1 Message Date
Alex Gorrod
5c3db7cc42 Update NEWS file for 2.1.2 release 2014-03-28 17:22:56 +11:00
Alex Gorrod
10f6b5eb31 Cut release 2.1.2 2014-03-28 17:13:04 +11:00
Alex Gorrod
e7b41e6450 lint
Originally fixed as part of larger change:
cc25faad1f
2014-03-28 17:07:30 +11:00
Alex Gorrod
33413809c4 Bump version tag to 2.1.2 on develop. 2014-03-28 17:06:18 +11:00
Keith Bostic
a73bf83c14 error: comparison between signed and unsigned integer expressions
[-Werror=sign-compare]
2014-03-28 16:38:15 +11:00
Alex Gorrod
b762b6287e Rename use of WT_READGEN_OLDEST to WT_READ_GEN_OLDEST.
Since the variable name is different on this branch.
2014-03-28 16:22:38 +11:00
Michael Cahill
c05ae226e8 Fix a comment: finish the sentence. 2014-03-28 16:16:33 +11:00
Michael Cahill
08a9394ddf Before merging extent lists, check if one is bigger than the other, and walk through the smaller one. This requires that we track sizes for the temporary avail list we build during checkpoints, otherwise we can't make the size tracking match up after the merge. 2014-03-28 16:16:24 +11:00
Keith Bostic
cebc9aa4e7 Inline the block extension function, do a fast check for available
bytes before doing a search.
2014-03-28 16:16:14 +11:00
Keith Bostic
cb789a39f3 whitespace 2014-03-28 16:16:03 +11:00
Keith Bostic
758ebdbe40 *shrug* Don't put an if/else inside the for loop, it's not needed and
it may help the compiler figure out the code motion.

Assert we never append an entry that's in the wrong place.
2014-03-28 16:15:50 +11:00
Keith Bostic
cce1701ba4 Add a fast path for file extensions so we don't walk the list of
allocations for workloads where we already know we want to either
extend, or add an element after, the last element.  Ref #924.
2014-03-28 16:15:38 +11:00
Keith Bostic
3fe22060a1 Don't maintain the per-size skiplist unless it's an element on the live
"avail" list.  The problem is we're holding the live system's spinlock
when updating the per-size skiplist, and that appears to be a bottleneck.
Reference #924.
2014-03-28 16:15:17 +11:00
Alex Gorrod
dcc693a38e Short circuit cleaning up reconciliation boundary structures.
The list of structures can grow very large, and cleaning up on each
reconciliation is a performance bottle neck.
2014-03-28 16:15:08 +11:00
Michael Cahill
bfda2844d2 If we underflow a page size during a merge, that should not be fatal: just limit the memory size transferred to the original page size. 2014-03-28 16:03:36 +11:00
Alex Gorrod
97a553b90b Update sharec_cache configuration again.
Remove enabled option, set default value for name to empty, and require
that a name is provided if any shared_cache configuration is provided.

Also update upgrading documentation and tests.
2014-03-28 16:03:25 +11:00
Alex Gorrod
402c34fb64 typo 2014-03-28 16:02:06 +11:00
Alex Gorrod
809838c4e2 Update shared cache test to match configuration changes. 2014-03-28 16:01:54 +11:00
Alex Gorrod
68c782ca6c Add upgrade doc for shared_cache config changes.
Also fine tune when we return an error in shared cache configuration parsing.
2014-03-28 16:01:43 +11:00
Alex Gorrod
44f3e099f4 Update shared cache configuration options:
* Change enable to enabled to match log setting.
* Return an error if there is a shared_cache configuration string
  but it's not enabled and not explicit cache_size configuration.
2014-03-28 16:01:30 +11:00
Michael Cahill
3d94b015fa If a page is awaiting forced eviction, and LRU eviction fails, don't bump the read generation. 2014-03-28 16:01:14 +11:00
Michael Cahill
36573ef3c7 Take a copy of the update transaction ID in reconciliation in case we race with a rollback. Go straight to aggressive eviction once the cache gets stuck.
refs #894
2014-03-28 16:00:57 +11:00
Alex Gorrod
be98ccec72 Merge branch 'develop' 2014-03-04 16:04:48 +11:00
Alex Gorrod
006f6b7afe Merge branch 'develop' 2014-02-04 14:30:31 +11:00
Alex Gorrod
064ba0af87 Merge branch 'develop' 2013-12-12 14:59:33 +11:00
Alex Gorrod
f06f36655f Merge branch 'develop' 2013-11-19 11:53:22 +11:00
Michael Cahill
e1c54ea983 Bump the release date in the README file. 2013-10-10 09:51:27 +11:00
23 changed files with 287 additions and 148 deletions

14
NEWS
View File

@@ -1,3 +1,17 @@
WiredTiger release 2.1.2, 2014-03-27
------------------------------------
The WiredTiger 2.1.2 release contains performance enhancements and bug fixes.
Significant changes include:
Update the configuration settings for shared_cache to make the distinction
between cache_size and shared_cache less confusing. See upgrading
documentation for more information.
Various performance enhancements to improve the performance of checkpoints.
Fix a bug that could cause a hang with small caches under heavy load. [#894]
WiredTiger release 2.1.1, 2014-03-04
------------------------------------

6
README
View File

@@ -1,6 +1,6 @@
WiredTiger 2.1.1: (March 4, 2014)
WiredTiger 2.1.2: (March 28, 2014)
This is version 2.1.1 of WiredTiger.
This is version 2.1.2 of WiredTiger.
WiredTiger release packages and documentation can be found at:
@@ -9,7 +9,7 @@ WiredTiger release packages and documentation can be found at:
Information on configuring, building and installing WiredTiger can be
found at:
http://source.wiredtiger.com/2.1.1/install.html
http://source.wiredtiger.com/2.1.2/install.html
WiredTiger licensing information can be found at:

View File

@@ -1,6 +1,6 @@
WIREDTIGER_VERSION_MAJOR=2
WIREDTIGER_VERSION_MINOR=1
WIREDTIGER_VERSION_PATCH=1
WIREDTIGER_VERSION_PATCH=2
WIREDTIGER_VERSION="$WIREDTIGER_VERSION_MAJOR.$WIREDTIGER_VERSION_MINOR.$WIREDTIGER_VERSION_PATCH"
WIREDTIGER_RELEASE_DATE=`date "+%B %e, %Y"`

View File

@@ -2,8 +2,8 @@ dnl build by dist/s_version
VERSION_MAJOR=2
VERSION_MINOR=1
VERSION_PATCH=1
VERSION_STRING='"WiredTiger 2.1.1: (March 4, 2014)"'
VERSION_PATCH=2
VERSION_STRING='"WiredTiger 2.1.2: (March 28, 2014)"'
AC_SUBST(VERSION_MAJOR)
AC_SUBST(VERSION_MINOR)

View File

@@ -1,2 +1,2 @@
dnl WiredTiger product version for AC_INIT. Maintained by dist/s_version
2.1.1
2.1.2

5
dist/api_data.py vendored
View File

@@ -300,9 +300,6 @@ connection_runtime_config = [
shared cache configuration options. A database should configure
either a cache_size or a shared_cache not both''',
type='category', subconfig=[
Config('enable', 'false', r'''
whether the connection is using a shared cache''',
type='boolean'),
Config('chunk', '10MB', r'''
the granularity that a shared cache is redistributed''',
min='1MB', max='10TB'),
@@ -310,7 +307,7 @@ connection_runtime_config = [
amount of cache this database is guaranteed to have
available from the shared cache. This setting is per
database. Defaults to the chunk size''', type='int'),
Config('name', 'pool', r'''
Config('name', '', r'''
name of a cache that is shared between databases'''),
Config('size', '500MB', r'''
maximum memory to allocate for the shared cache. Setting

View File

@@ -25,11 +25,12 @@ __wt_block_ckpt_init(
ci->root_offset = WT_BLOCK_INVALID_OFFSET;
WT_RET(__wt_block_extlist_init(session, &ci->alloc, name, "alloc"));
WT_RET(__wt_block_extlist_init(session, &ci->avail, name, "avail"));
WT_RET(__wt_block_extlist_init(session, &ci->discard, name, "discard"));
WT_RET(__wt_block_extlist_init(session, &ci->alloc, name, "alloc", 0));
WT_RET(__wt_block_extlist_init(session, &ci->avail, name, "avail", 1));
WT_RET(__wt_block_extlist_init(
session, &ci->ckpt_avail, name, "ckpt_avail"));
session, &ci->discard, name, "discard", 0));
WT_RET(__wt_block_extlist_init(
session, &ci->ckpt_avail, name, "ckpt_avail", 1));
return (0);
}
@@ -365,7 +366,7 @@ __ckpt_process(
*/
__wt_block_extlist_free(session, &ci->ckpt_avail);
WT_RET(__wt_block_extlist_init(
session, &ci->ckpt_avail, "live", "ckpt_avail"));
session, &ci->ckpt_avail, "live", "ckpt_avail", 1));
/*
* We've allocated our last page, update the checkpoint size. We need
@@ -568,10 +569,11 @@ live_update:
* avail list alone.
*/
__wt_block_extlist_free(session, &ci->alloc);
WT_ERR(__wt_block_extlist_init(session, &ci->alloc, "live", "alloc"));
WT_ERR(__wt_block_extlist_init(
session, &ci->alloc, "live", "alloc", 0));
__wt_block_extlist_free(session, &ci->discard);
WT_ERR(
__wt_block_extlist_init(session, &ci->discard, "live", "discard"));
WT_ERR(__wt_block_extlist_init(
session, &ci->discard, "live", "discard", 0));
#ifdef HAVE_DIAGNOSTIC
/*

View File

@@ -7,12 +7,38 @@
#include "wt_internal.h"
static int __block_append(WT_SESSION_IMPL *, WT_EXTLIST *, off_t, off_t);
static int __block_ext_overlap(WT_SESSION_IMPL *,
WT_BLOCK *, WT_EXTLIST *, WT_EXT **, WT_EXTLIST *, WT_EXT **);
static int __block_extlist_dump(
WT_SESSION_IMPL *, const char *, WT_EXTLIST *, int);
static int __block_merge(WT_SESSION_IMPL *, WT_EXTLIST *, off_t, off_t);
/*
* __block_off_srch_last --
* Return the last element in the list, along with a stack for appending.
*/
static inline WT_EXT *
__block_off_srch_last(WT_EXT **head, WT_EXT ***stack)
{
WT_EXT **extp, *last;
int i;
last = NULL; /* The list may be empty */
/*
* Start at the highest skip level, then go as far as possible at each
* level before stepping down to the next.
*/
for (i = WT_SKIP_MAXDEPTH - 1, extp = &head[i]; i >= 0;)
if (*extp != NULL) {
last = *extp;
extp = &(*extp)->next[i];
} else
stack[i--] = extp--;
return (last);
}
/*
* __block_off_srch --
* Search a by-offset skiplist (either the primary by-offset list, or the
@@ -127,30 +153,6 @@ __block_off_srch_pair(
}
}
/*
* __block_extlist_last --
* Return the last extent in the skiplist.
*/
static inline WT_EXT *
__block_extlist_last(WT_EXT **head)
{
WT_EXT *ext, **extp;
int i;
ext = NULL;
for (i = WT_SKIP_MAXDEPTH - 1, extp = &head[i]; i >= 0;) {
if (*extp == NULL) {
--i;
--extp;
continue;
}
ext = *extp;
extp = &(*extp)->next[i];
}
return (ext);
}
/*
* __block_ext_insert --
* Insert an extent into an extent list.
@@ -166,17 +168,34 @@ __block_ext_insert(WT_SESSION_IMPL *session, WT_EXTLIST *el, WT_EXT *ext)
* If we are inserting a new size onto the size skiplist, we'll need a
* new WT_SIZE structure for that skiplist.
*/
__block_size_srch(el->sz, ext->size, sstack);
szp = *sstack[0];
if (szp == NULL || szp->size != ext->size) {
WT_RET(__wt_block_size_alloc(session, &szp));
szp->size = ext->size;
szp->depth = ext->depth;
if (el->track_size) {
__block_size_srch(el->sz, ext->size, sstack);
szp = *sstack[0];
if (szp == NULL || szp->size != ext->size) {
WT_RET(__wt_block_size_alloc(session, &szp));
szp->size = ext->size;
szp->depth = ext->depth;
for (i = 0; i < ext->depth; ++i) {
szp->next[i] = *sstack[i];
*sstack[i] = szp;
}
}
/*
* Insert the new WT_EXT structure into the size element's
* offset skiplist.
*/
__block_off_srch(szp->off, ext->off, astack, 1);
for (i = 0; i < ext->depth; ++i) {
szp->next[i] = *sstack[i];
*sstack[i] = szp;
ext->next[i + ext->depth] = *astack[i];
*astack[i] = ext;
}
}
#ifdef HAVE_DIAGNOSTIC
if (!el->track_size)
for (i = 0; i < ext->depth; ++i)
ext->next[i + ext->depth] = NULL;
#endif
/* Insert the new WT_EXT structure into the offset skiplist. */
__block_off_srch(el->off, ext->off, astack, 0);
@@ -185,16 +204,6 @@ __block_ext_insert(WT_SESSION_IMPL *session, WT_EXTLIST *el, WT_EXT *ext)
*astack[i] = ext;
}
/*
* Insert the new WT_EXT structure into the size element's offset
* skiplist.
*/
__block_off_srch(szp->off, ext->off, astack, 1);
for (i = 0; i < ext->depth; ++i) {
ext->next[i + ext->depth] = *astack[i];
*astack[i] = ext;
}
++el->entries;
el->bytes += (uint64_t)ext->size;
@@ -306,21 +315,32 @@ __block_off_remove(
* Find and remove the record from the size's offset skiplist; if that
* empties the by-size skiplist entry, remove it as well.
*/
__block_size_srch(el->sz, ext->size, sstack);
szp = *sstack[0];
if (szp == NULL || szp->size != ext->size)
return (EINVAL);
__block_off_srch(szp->off, off, astack, 1);
ext = *astack[0];
if (ext == NULL || ext->off != off)
goto corrupt;
for (i = 0; i < ext->depth; ++i)
*astack[i] = ext->next[i + ext->depth];
if (szp->off[0] == NULL) {
for (i = 0; i < szp->depth; ++i)
*sstack[i] = szp->next[i];
__wt_block_size_free(session, szp);
if (el->track_size) {
__block_size_srch(el->sz, ext->size, sstack);
szp = *sstack[0];
if (szp == NULL || szp->size != ext->size)
return (EINVAL);
__block_off_srch(szp->off, off, astack, 1);
ext = *astack[0];
if (ext == NULL || ext->off != off)
goto corrupt;
for (i = 0; i < ext->depth; ++i)
*astack[i] = ext->next[i + ext->depth];
if (szp->off[0] == NULL) {
for (i = 0; i < szp->depth; ++i)
*sstack[i] = szp->next[i];
__wt_block_size_free(session, szp);
}
}
#ifdef HAVE_DIAGNOSTIC
if (!el->track_size) {
int not_null;
for (i = 0, not_null = 0; i < ext->depth; ++i)
if (ext->next[i + ext->depth] != NULL)
not_null = 1;
WT_ASSERT(session, not_null == 0);
}
#endif
--el->entries;
el->bytes -= (uint64_t)ext->size;
@@ -407,7 +427,7 @@ __wt_block_off_remove_overlap(
* __block_extend --
* Extend the file to allocate space.
*/
static int
static inline int
__block_extend(
WT_SESSION_IMPL *session, WT_BLOCK *block, off_t *offp, off_t size)
{
@@ -458,6 +478,9 @@ __wt_block_alloc(
WT_EXT *ext, **estack[WT_SKIP_MAXDEPTH];
WT_SIZE *szp, **sstack[WT_SKIP_MAXDEPTH];
/* Assert we're maintaining the by-size skiplist. */
WT_ASSERT(session, block->live.avail.track_size != 0);
WT_STAT_FAST_DATA_INCR(session, block_alloc);
if (size % block->allocsize != 0)
WT_RET_MSG(session, EINVAL,
@@ -477,17 +500,19 @@ __wt_block_alloc(
*
* If we don't have anything big enough, extend the file.
*/
if (block->live.avail.bytes < (uint64_t)size)
goto append;
if (block->allocfirst) {
if (!__block_first_srch(block->live.avail.off, size, estack)) {
WT_RET(__block_extend(session, block, offp, size));
goto done;
}
if (!__block_first_srch(block->live.avail.off, size, estack))
goto append;
ext = *estack[0];
} else {
__block_size_srch(block->live.avail.sz, size, sstack);
if ((szp = *sstack[0]) == NULL) {
WT_RET(__block_extend(session, block, offp, size));
goto done;
append: WT_RET(__block_extend(session, block, offp, size));
WT_RET(__block_append(
session, &block->live.alloc, *offp, (off_t)size));
return (0);
}
/* Take the first record. */
@@ -519,7 +544,7 @@ __wt_block_alloc(
__wt_block_ext_free(session, ext);
}
done: /* Add the newly allocated extent to the list of allocations. */
/* Add the newly allocated extent to the list of allocations. */
WT_RET(__block_merge(session, &block->live.alloc, *offp, (off_t)size));
return (0);
}
@@ -855,15 +880,79 @@ int
__wt_block_extlist_merge(WT_SESSION_IMPL *session, WT_EXTLIST *a, WT_EXTLIST *b)
{
WT_EXT *ext;
WT_EXTLIST tmp;
u_int i;
WT_VERBOSE_RET(session, block, "merging %s into %s", a->name, b->name);
/*
* Sometimes the list we are merging is much bigger than the other: if
* so, swap the lists around to reduce the amount of work we need to do
* during the merge. The size lists have to match as well, so this is
* only possible if both lists are tracking sizes, or neither are.
*/
if (a->track_size == b->track_size && a->entries > b->entries) {
tmp = *a;
a->bytes = b->bytes;
b->bytes = tmp.bytes;
a->entries = b->entries;
b->entries = tmp.entries;
for (i = 0; i < WT_SKIP_MAXDEPTH; i++) {
a->off[i] = b->off[i];
b->off[i] = tmp.off[i];
a->sz[i] = b->sz[i];
b->sz[i] = tmp.sz[i];
}
}
WT_EXT_FOREACH(ext, a->off)
WT_RET(__block_merge(session, b, ext->off, ext->size));
return (0);
}
/*
* __block_append --
* Append a new entry to the allocation list.
*/
static int
__block_append(WT_SESSION_IMPL *session, WT_EXTLIST *el, off_t off, off_t size)
{
WT_EXT *ext, **astack[WT_SKIP_MAXDEPTH];
u_int i;
WT_ASSERT(session, el->track_size == 0);
/*
* Identical to __block_merge, when we know the file is being extended,
* that is, the information is either going to be used to extend the
* last object on the list, or become a new object ending the list.
*
* First, get a stack for the last object in the skiplist, then check
* for a simple extension. If that doesn't work, allocate a new list
* structure, and append it.
*/
ext = __block_off_srch_last(el->off, astack);
if (ext != NULL && ext->off + ext->size == off)
ext->size += size;
else {
/* Assert we're appending to the list. */
WT_ASSERT(session, ext == NULL || ext->off + ext->size < off);
WT_RET(__wt_block_ext_alloc(session, &ext));
ext->off = off;
ext->size = size;
for (i = 0; i < ext->depth; ++i)
*astack[i] = ext;
++el->entries;
}
el->bytes += (uint64_t)size;
return (0);
}
/*
* __wt_block_insert_ext --
* Insert an extent into an extent list, merging if possible.
@@ -1167,7 +1256,7 @@ int
__wt_block_extlist_truncate(
WT_SESSION_IMPL *session, WT_BLOCK *block, WT_EXTLIST *el)
{
WT_EXT *ext;
WT_EXT *ext, **astack[WT_SKIP_MAXDEPTH];
WT_FH *fh;
off_t size;
@@ -1177,7 +1266,7 @@ __wt_block_extlist_truncate(
* Check if the last available extent is at the end of the file, and if
* so, truncate the file and discard the extent.
*/
if ((ext = __block_extlist_last(el->off)) == NULL)
if ((ext = __block_off_srch_last(el->off, astack)) == NULL)
return (0);
if (ext->off + ext->size != fh->size)
return (0);
@@ -1205,7 +1294,7 @@ __wt_block_extlist_truncate(
*/
int
__wt_block_extlist_init(WT_SESSION_IMPL *session,
WT_EXTLIST *el, const char *name, const char *extname)
WT_EXTLIST *el, const char *name, const char *extname, int track_size)
{
char buf[128];
@@ -1214,6 +1303,7 @@ __wt_block_extlist_init(WT_SESSION_IMPL *session,
WT_RET(__wt_strdup(session, buf, &el->name));
el->offset = WT_BLOCK_INVALID_OFFSET;
el->track_size = track_size;
return (0);
}

View File

@@ -45,14 +45,20 @@ __block_ext_alloc(WT_SESSION_IMPL *session, WT_EXT **extp)
int
__wt_block_ext_alloc(WT_SESSION_IMPL *session, WT_EXT **extp)
{
WT_EXT *ext;
WT_BLOCK_MGR_SESSION *bms;
u_int i;
bms = session->block_manager;
/* Return a WT_EXT structure for use from a cached list. */
if (bms != NULL && bms->ext_cache != NULL) {
(*extp) = bms->ext_cache;
bms->ext_cache = bms->ext_cache->next[0];
ext = bms->ext_cache;
bms->ext_cache = ext->next[0];
/* Clear any left-over references. */
for (i = 0; i < ext->depth; ++i)
ext->next[i] = ext->next[i + ext->depth] = NULL;
/*
* The count is advisory to minimize our exposure to bugs, but
@@ -60,6 +66,8 @@ __wt_block_ext_alloc(WT_SESSION_IMPL *session, WT_EXT **extp)
*/
if (bms->ext_cache_cnt > 0)
--bms->ext_cache_cnt;
*extp = ext;
return (0);
}

View File

@@ -88,7 +88,7 @@ __wt_block_verify_start(
* set of checkpoints.
*/
WT_RET(__wt_block_extlist_init(
session, &block->verify_alloc, "verify", "alloc"));
session, &block->verify_alloc, "verify", "alloc", 0));
/*
* The only checkpoint avail list we care about is the last one written;

View File

@@ -229,7 +229,7 @@ __evict_worker(WT_SESSION_IMPL *session)
/* Check to see if the eviction server should run. */
if (bytes_inuse > (cache->eviction_target * bytes_max) / 100)
flags = (loop > 10) ?
flags = (F_ISSET(cache, WT_EVICT_STUCK) || loop > 10) ?
WT_EVICT_PASS_AGGRESSIVE : WT_EVICT_PASS_ALL;
else if (dirty_inuse >
(cache->eviction_dirty_target * bytes_max) / 100)
@@ -1191,7 +1191,8 @@ __wt_evict_lru_page(WT_SESSION_IMPL *session, int is_app)
* the page and some other thread may have evicted it by the time we
* look at it.
*/
page->read_gen = __wt_cache_read_gen_set(session);
if (page->read_gen != WT_READ_GEN_OLDEST)
page->read_gen = __wt_cache_read_gen_set(session);
WT_WITH_BTREE(session, btree, ret = __wt_evict_page(session, page));

View File

@@ -143,10 +143,10 @@ __merge_unlock(WT_PAGE *page)
* __wt_cache_page_inmem_incr.
*/
static void
__merge_transfer_footprint(WT_SESSION_IMPL *session,
WT_PAGE *newpage, WT_PAGE *oldpage, size_t size)
__merge_transfer_footprint(WT_PAGE *newpage, WT_PAGE *oldpage, size_t size)
{
WT_ASSERT(session, size < oldpage->memory_footprint);
if (oldpage->memory_footprint < size)
size = oldpage->memory_footprint;
oldpage->memory_footprint -= size;
newpage->memory_footprint += size;
}
@@ -171,13 +171,13 @@ __merge_switch_page(WT_PAGE *parent, WT_REF *ref, WT_VISIT_STATE *state)
newref = state->ref++;
if (ref->addr != NULL)
__merge_transfer_footprint(state->session, state->page,
parent, sizeof(WT_ADDR) + ((WT_ADDR *)ref->addr)->size);
__merge_transfer_footprint(state->page, parent,
sizeof(WT_ADDR) + ((WT_ADDR *)ref->addr)->size);
if (parent->type == WT_PAGE_ROW_INT &&
(ikey = __wt_ref_key_instantiated(ref)) != NULL)
__merge_transfer_footprint(state->session,
state->page, parent, sizeof(WT_IKEY) + ikey->size);
__merge_transfer_footprint(state->page, parent,
sizeof(WT_IKEY) + ikey->size);
if (ref->state == WT_REF_LOCKED) {
child = ref->page;

View File

@@ -487,12 +487,14 @@ __rec_write_init(
}
/*
* Clean up any pre-existing boundary structures: almost none of this
* should be necessary (already_compressed is the notable exception),
* but it's cheap.
* Clean up any used boundary structures. It is worth short-circuiting
* the traversal because sometimes the boundary list can grow to over
* a million entries. The only field we really need to clear is
* already_compressed, but let's be paranoid.
*/
if (r->bnd != NULL)
for (bnd = r->bnd, i = 0; i < r->bnd_entries; ++bnd, ++i) {
for (bnd = r->bnd, i = 0;
i < r->bnd_entries && bnd->entries > 0; ++bnd, ++i) {
bnd->start = NULL;
bnd->recno = 0;
bnd->entries = 0;

View File

@@ -25,7 +25,6 @@ static const WT_CONFIG_CHECK confchk_connection_open_session[] = {
static const WT_CONFIG_CHECK confchk_shared_cache_subconfigs[] = {
{ "chunk", "int", "min=1MB,max=10TB", NULL },
{ "enable", "boolean", NULL, NULL },
{ "name", "string", NULL, NULL },
{ "reserve", "int", NULL, NULL },
{ "size", "int", "min=1MB,max=10TB", NULL },
@@ -322,8 +321,7 @@ static const WT_CONFIG_ENTRY config_entries[] = {
{ "connection.reconfigure",
"cache_size=100MB,error_prefix=,eviction_dirty_target=80,"
"eviction_target=80,eviction_trigger=95,shared_cache=(chunk=10MB,"
"enable=0,name=pool,reserve=0,size=500MB),statistics=none,"
"verbose=",
"name=,reserve=0,size=500MB),statistics=none,verbose=",
confchk_connection_reconfigure
},
{ "cursor.close",
@@ -432,8 +430,8 @@ static const WT_CONFIG_ENTRY config_entries[] = {
"eviction_dirty_target=80,eviction_target=80,eviction_trigger=95,"
"extensions=,file_extend=,hazard_max=1000,log=(archive=,enabled=0"
",file_max=100MB,path=\"\"),lsm_merge=,mmap=,multiprocess=0,"
"session_max=100,shared_cache=(chunk=10MB,enable=0,name=pool,"
"reserve=0,size=500MB),statistics=none,"
"session_max=100,shared_cache=(chunk=10MB,name=,reserve=0,"
"size=500MB),statistics=none,"
"statistics_log=(path=\"WiredTigerStat.%d.%H\",sources=,"
"timestamp=\"%b %d %H:%M:%S\",wait=0),transaction_sync=fsync,"
"use_environment_priv=0,verbose=",

View File

@@ -47,13 +47,26 @@ __wt_conn_cache_pool_config(WT_SESSION_IMPL *session, const char **cfg)
if (F_ISSET(conn, WT_CONN_CACHE_POOL))
reconfiguring = 1;
else {
/* Only setup if a shared cache was explicitly configured. */
WT_RET(__wt_config_gets(
session, cfg, "shared_cache.enable", &cval));
if (!cval.val)
return (0);
WT_RET_NOTFOUND_OK(
WT_RET(
__wt_config_gets(session, cfg, "shared_cache.name", &cval));
if (cval.len == 0) {
/*
* Tell the user if they configured some shared cache
* settings, but didn't enable it by naming it.
*/
if (__wt_config_gets(session,
&cfg[1], "shared_cache", &cval) != WT_NOTFOUND)
WT_RET_MSG(session, EINVAL,
"Shared cache configuration requires a "
"pool name");
return (0);
}
if (__wt_config_gets(session,
&cfg[1], "cache_size", &cval) != WT_NOTFOUND)
WT_RET_MSG(session, EINVAL,
"Only one of cache_size and shared_cache can be "
"in the configuration");
/*
* NOTE: The allocations made when configuring and opening a
* cache pool don't really belong to the connection that

View File

@@ -6,9 +6,9 @@ WiredTiger is an high performance, scalable, production quality, NoSQL,
@section releases Releases
<table>
@row{<b>WiredTiger 2.1.1</b> (current),
<a href="releases/wiredtiger-2.1.1.tar.bz2"><b>[Release package]</b></a>,
<a href="2.1.1/index.html"><b>[Documentation]</b></a>}
@row{<b>WiredTiger 2.1.2</b> (current),
<a href="releases/wiredtiger-2.1.2.tar.bz2"><b>[Release package]</b></a>,
<a href="2.1.2/index.html"><b>[Documentation]</b></a>}
@row{<b>WiredTiger 1.6.6</b> (previous),
<a href="releases/wiredtiger-1.6.6.tar.bz2"><b>[Release package]</b></a>,
<a href="1.6.6/index.html"><b>[Documentation]</b></a>}

View File

@@ -1,5 +1,18 @@
/*! @page upgrading Upgrading WiredTiger applications
@section version_212 Upgrading to Version 2.1.2
<dl>
<dt>::wiredtiger_open shared_cache configuration changes</dt>
<dd>
In the 2.1.2 release of WiredTiger the ::wiredtiger_open \c shared_cache
configuration option group have changed. The option that was named \c enable is
no longer available. To enable a \c shared_cache it is compulsory to name
the pool being shared.
We are now also enforcing that only one of \c cache_size and \c shared_cache
are specified in the ::wiredtiger_open configuration string.
</dd>
@section version_211 Upgrading to Version 2.1.1
<dl>

View File

@@ -20,23 +20,24 @@
* alloc: the extents allocated in this checkpoint
* avail: the extents available for allocation
* discard: the extents freed in this checkpoint
* Each of the extent lists is based on two skiplists: first, a by-offset list
* linking WT_EXT elements and sorted by file offset (low-to-high), second, a
* by-size list linking WT_SIZE elements and sorted by chunk size (low-to-high).
* Additionally, each WT_SIZE element on the by-size has a skiplist of its
* own, linking WT_EXT elements and sorted by file offset (low-to-high). This
* list has an entry for extents of a particular size.
* The trickiness is that each individual WT_EXT element appears on two
* skiplists. In order to minimize allocation calls, we allocate a single
* array of WT_EXT pointers at the end of the WT_EXT structure, for both
* skiplists, and store the depth of the skiplist in the WT_EXT structure.
* The skiplist entries for the offset skiplist start at WT_EXT.next[0] and
* the entries for the size skiplist start at WT_EXT.next[WT_EXT.depth].
*
* XXX
* We maintain the per-size skiplists for the alloc and discard extent lists,
* but there's no reason for that, the avail list is the only list we search
* by size.
* An extent list is based on two skiplists: first, a by-offset list linking
* WT_EXT elements and sorted by file offset (low-to-high), second, a by-size
* list linking WT_SIZE elements and sorted by chunk size (low-to-high).
*
* Additionally, each WT_SIZE element on the by-size has a skiplist of its own,
* linking WT_EXT elements and sorted by file offset (low-to-high). This list
* has an entry for extents of a particular size.
*
* The trickiness is each individual WT_EXT element appears on two skiplists.
* In order to minimize allocation calls, we allocate a single array of WT_EXT
* pointers at the end of the WT_EXT structure, for both skiplists, and store
* the depth of the skiplist in the WT_EXT structure. The skiplist entries for
* the offset skiplist start at WT_EXT.next[0] and the entries for the size
* skiplist start at WT_EXT.next[WT_EXT.depth].
*
* One final complication: we only maintain the per-size skiplist for the avail
* list, the alloc and discard extent lists are not searched based on size.
*/
/*
@@ -52,6 +53,8 @@ struct __wt_extlist {
off_t offset; /* Written extent offset */
uint32_t cksum, size; /* Written extent cksum, size */
int track_size; /* Maintain per-size skiplist */
WT_EXT *off[WT_SKIP_MAXDEPTH]; /* Size/offset skiplists */
WT_SIZE *sz[WT_SKIP_MAXDEPTH];
};

View File

@@ -112,7 +112,8 @@ extern int __wt_block_extlist_truncate( WT_SESSION_IMPL *session,
extern int __wt_block_extlist_init(WT_SESSION_IMPL *session,
WT_EXTLIST *el,
const char *name,
const char *extname);
const char *extname,
int track_size);
extern void __wt_block_extlist_free(WT_SESSION_IMPL *session, WT_EXTLIST *el);
extern int __wt_block_map( WT_SESSION_IMPL *session,
WT_BLOCK *block,

View File

@@ -175,6 +175,7 @@ __wt_txn_read_skip(
WT_SESSION_IMPL *session, WT_UPDATE *upd, uint64_t *max_txn, int *skipp)
{
WT_UPDATE *first_upd;
uint64_t txnid;
/*
* Track the largest transaction ID on this page. We store this in the
@@ -190,11 +191,11 @@ __wt_txn_read_skip(
*/
*skipp = 0;
for (first_upd = NULL; upd != NULL; upd = upd->next)
if (upd->txnid != WT_TXN_ABORTED) {
if (TXNID_LT(*max_txn, upd->txnid))
*max_txn = upd->txnid;
if ((txnid = upd->txnid) != WT_TXN_ABORTED) {
if (TXNID_LT(*max_txn, txnid))
*max_txn = txnid;
if (first_upd == NULL) {
if (__wt_txn_visible(session, upd->txnid))
if (__wt_txn_visible(session, txnid))
first_upd = upd;
else
*skipp = 1;

View File

@@ -1236,8 +1236,6 @@ struct __wt_connection {
* @config{shared_cache = (, shared cache configuration options. A
* database should configure either a cache_size or a shared_cache not
* both., a set of related configuration options defined below.}
* @config{&nbsp;&nbsp;&nbsp;&nbsp;enable, whether the connection is
* using a shared cache., a boolean flag; default \c false.}
* @config{&nbsp;&nbsp;&nbsp;&nbsp;chunk, the granularity that a shared
* cache is redistributed., an integer between 1MB and 10TB; default \c
* 10MB.}
@@ -1246,7 +1244,7 @@ struct __wt_connection {
* setting is per database. Defaults to the chunk size., an integer;
* default \c 0.}
* @config{&nbsp;&nbsp;&nbsp;&nbsp;name, name of a cache
* that is shared between databases., a string; default \c pool.}
* that is shared between databases., a string; default empty.}
* @config{&nbsp;&nbsp;&nbsp;&nbsp;size, maximum memory to allocate for
* the shared cache. Setting this will update the value if one is
* already set., an integer between 1MB and 10TB; default \c 500MB.}
@@ -1549,15 +1547,13 @@ struct __wt_connection {
* @config{shared_cache = (, shared cache configuration options. A database
* should configure either a cache_size or a shared_cache not both., a set of
* related configuration options defined below.}
* @config{&nbsp;&nbsp;&nbsp;&nbsp;enable, whether the connection is using a
* shared cache., a boolean flag; default \c false.}
* @config{&nbsp;&nbsp;&nbsp;&nbsp;chunk, the granularity that a shared cache is
* redistributed., an integer between 1MB and 10TB; default \c 10MB.}
* @config{&nbsp;&nbsp;&nbsp;&nbsp;reserve, amount of cache this database is
* guaranteed to have available from the shared cache. This setting is per
* database. Defaults to the chunk size., an integer; default \c 0.}
* @config{&nbsp;&nbsp;&nbsp;&nbsp;name, name of a cache that is shared between
* databases., a string; default \c pool.}
* databases., a string; default empty.}
* @config{&nbsp;&nbsp;&nbsp;&nbsp;size,
* maximum memory to allocate for the shared cache. Setting this will update
* the value if one is already set., an integer between 1MB and 10TB; default \c

View File

@@ -32,7 +32,7 @@ import wiredtiger, wttest
class test_reconfig(wttest.WiredTigerTestCase):
def test_reconfig_shared_cache(self):
self.conn.reconfigure("shared_cache=(enable=true,size=300M)")
self.conn.reconfigure("shared_cache=(name=pool,size=300M)")
def test_reconfig_statistics(self):
self.conn.reconfigure("statistics=(all)")

View File

@@ -63,7 +63,7 @@ class test_shared_cache(wttest.WiredTigerTestCase):
def openConnections(
self,
connections,
pool_opts = ',shared_cache=(enable=true,name=pool,size=200M,chunk=10M,reserve=30M),',
pool_opts = ',shared_cache=(name=pool,size=200M,chunk=10M,reserve=30M),',
extra_opts = '',
add=0):
if add == 0:
@@ -186,7 +186,7 @@ class test_shared_cache(wttest.WiredTigerTestCase):
# Test default config values
def test_shared_cache09(self):
nops = 1000
self.openConnections(['WT_TEST1', 'WT_TEST2'], pool_opts=',shared_cache=(enable=true,name=pool,size=200M)')
self.openConnections(['WT_TEST1', 'WT_TEST2'], pool_opts=',shared_cache=(name=pool,size=200M)')
for sess in self.sessions:
sess.create(self.uri, "key_format=S,value_format=S")
@@ -203,13 +203,13 @@ class test_shared_cache(wttest.WiredTigerTestCase):
self.add_records(sess, 0, nops)
connection = self.conns[0]
connection.reconfigure("shared_cache=(enable=true,size=300M)")
connection.reconfigure("shared_cache=(name=pool,size=300M)")
self.closeConnections()
# Test default config values
def test_shared_cache11(self):
nops = 1000
self.openConnections(['WT_TEST1', 'WT_TEST2'], pool_opts=',shared_cache=(enable=true)')
self.openConnections(['WT_TEST1', 'WT_TEST2'], pool_opts=',shared_cache=(name=pool)')
for sess in self.sessions:
sess.create(self.uri, "key_format=S,value_format=S")