xuemingqiang/mongo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
dce3b942e4e22fcd4c2e8ebb5a0cdbbf1e2f3567
mongo/dist/stat_data.py

479 lines
27 KiB
Python
Raw Normal View History

Add support for aggregating data-source statistics, currently only used by the LSM tree. Add support for automatically updating the list of statistics that are to be "scaled per second" by statlog.py.
2013-04-30 19:57:37 -04:00
# Auto-generate statistics #defines, with initialization, clear and aggregate
# functions.
Move statistics strings into their own class file in dist so we can add config information. Specifically, stat fields can now be marked "perm", which means they aren't cleared by the stat clear method. This means cache allocation information can be moved into the stat structures and displayed normally. This also means statistics are required in the system (although it would probably be pretty easy to only maintain "perm" stat fields instead of all stat fields if we ever need a smaller footprint). --HG-- branch : keith
2010-01-25 04:58:51 +11:00
#
Add support for aggregating data-source statistics, currently only used by the LSM tree. Add support for automatically updating the list of statistics that are to be "scaled per second" by statlog.py.
2013-04-30 19:57:37 -04:00
# NOTE: Statistics reports show individual objects as operations per second.
# All objects where that does not make sense should have the word 'currently'
# or the phrase 'in the cache' in their text description, for example, 'files
# currently open'.
Add prefixes to all stat descriptions. #1313
2014-10-27 11:51:43 -04:00
# NOTE: All statistics descriptions must have a prefix string followed by ':'.
Add reminder comment about statistics heuristic text. #518
2013-04-29 11:18:30 -04:00
#
async_max_queue: collection statistic, aggregate doesn't apply, don't clear. cache_eviction_maximum_page_size: collection statistic, aggregate doesn't apply, don't clear. various txn_checkpoint_XXX statistics: collection statistic, aggregate doesn't apply various Btree maximum size statistics: aggregate the maximum value, rather than doing no aggregation at all. lsm_chunk_count: aggregate the count of LSM chunks. Add syntax checking code to stat.py to disallow aggregation flags for connection-level statistics. Reference #1742.
2015-03-30 09:52:34 -04:00
# Data-source statistics are normally aggregated across the set of underlying
Fix a comment typo.
2015-11-29 09:25:28 -05:00
# objects. Additional optional configuration flags are available:
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
# max_aggregate Take the maximum value when aggregating statistics
WT-2029, improve scalability of statistics Instead of using per-thread arrays of counters (see #2102), use arrays of statistics structures. I started with the scalable_stats branch, so this incorporates most of the commits on that branch, but the diff is against the develop branch to make it easier to review. Specifically, this includes a9675d0: There are only 3 statistics we maintain atomically: the count of open cursors and the split-stash bytes and object counts. Move them to WT_CONNECTION_IMPL fields, and copy them into the statistics on demand, this allows us to remove support for atomically maintaining statistics values. And 6bda287: Get rid of the WT_CONN_{STAT,STAT_GET,STAT_SET} macros, instead maintain a few additional fields in the WT_CONNECTION_IMPL structure (checkpoint min/max, recent and total time). And 7590930: Use a prime number of buckets rather than assuming a good hash (Reference Sedgewick, Algorithms in C, "Hash Functions"). And 63a2082: I'm seeing a roughly 20% advantage of using a loop to clear the stats values, vs. a memset to clear the whole padded chunk, and I'd rather err on the side of writing fewer cache lines, anyway. Additionally: Now we have arrays of statistics structures, separate out each statistic value's description into a separate array, a struct of strings. We could access it directly, and maybe should, but I'm using a function all at the moment. Now the statistic's descriptions are separate from the statistics values, we no longer need macros to access the fields, in general we can simply use the stat structure reference and the field name and set the int64_t directly. This includes removal of the WT_STATS typedef. Change the LSM code to not maintain a local data-source statistics structure in each LSM-tree structure -- that allows us to remove all of the macros to update a single stats structure, and I don't think it complicates things that much. The changes to the LSM statistics code should be closely reviewed.
2015-08-15 11:51:12 -04:00
# no_clear Value not cleared when statistics cleared
# no_scale Don't scale value per second in the logging tool script
whitespace
2016-03-02 04:04:43 +00:00
# size Used by timeseries tool, indicates value is a byte count
WT-2029, improve scalability of statistics Instead of using per-thread arrays of counters (see #2102), use arrays of statistics structures. I started with the scalable_stats branch, so this incorporates most of the commits on that branch, but the diff is against the develop branch to make it easier to review. Specifically, this includes a9675d0: There are only 3 statistics we maintain atomically: the count of open cursors and the split-stash bytes and object counts. Move them to WT_CONNECTION_IMPL fields, and copy them into the statistics on demand, this allows us to remove support for atomically maintaining statistics values. And 6bda287: Get rid of the WT_CONN_{STAT,STAT_GET,STAT_SET} macros, instead maintain a few additional fields in the WT_CONNECTION_IMPL structure (checkpoint min/max, recent and total time). And 7590930: Use a prime number of buckets rather than assuming a good hash (Reference Sedgewick, Algorithms in C, "Hash Functions"). And 63a2082: I'm seeing a roughly 20% advantage of using a loop to clear the stats values, vs. a memset to clear the whole padded chunk, and I'd rather err on the side of writing fewer cache lines, anyway. Additionally: Now we have arrays of statistics structures, separate out each statistic value's description into a separate array, a struct of strings. We could access it directly, and maybe should, but I'm using a function all at the moment. Now the statistic's descriptions are separate from the statistics values, we no longer need macros to access the fields, in general we can simply use the stat structure reference and the field name and set the int64_t directly. This includes removal of the WT_STATS typedef. Change the LSM code to not maintain a local data-source statistics structure in each LSM-tree structure -- that allows us to remove all of the macros to update a single stats structure, and I don't think it complicates things that much. The changes to the LSM statistics code should be closely reviewed.
2015-08-15 11:51:12 -04:00
#
The no_clear flag doesn't imply no_scale (at least, it doesn't imply it at this level, no_clear doesn't cause an entry to be output in the no_scale list), and all entries with no_clear also list no_scale.
2015-11-29 15:10:57 -05:00
# The no_clear and no_scale flags are normally always set together (values that
# are maintained over time are normally not scaled per second).
Move statistics strings into their own class file in dist so we can add config information. Specifically, stat fields can now be marked "perm", which means they aren't cleared by the stat clear method. This means cache allocation information can be moved into the stat structures and displayed normally. This also means statistics are required in the system (although it would probably be pretty easy to only maintain "perm" stat fields instead of all stat fields if we ever need a smaller footprint). --HG-- branch : keith
2010-01-25 04:58:51 +11:00
Add a "size of checkpoint" statistic. Merge statistics from file and LSM sources into a "data source" statistic structure. Rename and regroup some shared stastistics. Make statistics constant names upper case. Add a helper to the Python API to lookup in a cursor in a simple expression. closes #232
2012-11-16 21:02:27 +11:00
from operator import attrgetter
Add prefixes to all stat descriptions. #1313
2014-10-27 11:51:43 -04:00
import sys
Add a "size of checkpoint" statistic. Merge statistics from file and LSM sources into a "data source" statistic structure. Rename and regroup some shared stastistics. Make statistics constant names upper case. Add a helper to the Python API to lookup in a cursor in a simple expression. closes #232
2012-11-16 21:02:27 +11:00
Move statistics strings into their own class file in dist so we can add config information. Specifically, stat fields can now be marked "perm", which means they aren't cleared by the stat clear method. This means cache allocation information can be moved into the stat structures and displayed normally. This also means statistics are required in the system (although it would probably be pretty easy to only maintain "perm" stat fields instead of all stat fields if we ever need a smaller footprint). --HG-- branch : keith
2010-01-25 04:58:51 +11:00
class Stat:
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
def __init__(self, name, tag, desc, flags=''):
Change the Python scripts to follow Python indenting standards (4 space indents, spaces only).
2014-12-03 20:43:47 -05:00
self.name = name
self.desc = tag + ': ' + desc
self.flags = flags
Move statistics strings into their own class file in dist so we can add config information. Specifically, stat fields can now be marked "perm", which means they aren't cleared by the stat clear method. This means cache allocation information can be moved into the stat structures and displayed normally. This also means statistics are required in the system (although it would probably be pretty easy to only maintain "perm" stat fields instead of all stat fields if we ever need a smaller footprint). --HG-- branch : keith
2010-01-25 04:58:51 +11:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
def __cmp__(self, other):
Change the Python scripts to follow Python indenting standards (4 space indents, spaces only).
2014-12-03 20:43:47 -05:00
return cmp(self.desc.lower(), other.desc.lower())
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
class AsyncStat(Stat):
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
prefix = 'async'
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
def __init__(self, name, desc, flags=''):
Change the Python scripts to follow Python indenting standards (4 space indents, spaces only).
2014-12-03 20:43:47 -05:00
Stat.__init__(self, name, AsyncStat.prefix, desc, flags)
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
class BlockStat(Stat):
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
prefix = 'block-manager'
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
def __init__(self, name, desc, flags=''):
Change the Python scripts to follow Python indenting standards (4 space indents, spaces only).
2014-12-03 20:43:47 -05:00
Stat.__init__(self, name, BlockStat.prefix, desc, flags)
Use the "btree" tag consistently (before we had both "Btree" and "btree").
2014-11-03 11:31:52 -05:00
class BtreeStat(Stat):
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
prefix = 'btree'
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
def __init__(self, name, desc, flags=''):
Change the Python scripts to follow Python indenting standards (4 space indents, spaces only).
2014-12-03 20:43:47 -05:00
Stat.__init__(self, name, BtreeStat.prefix, desc, flags)
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
class CacheStat(Stat):
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
prefix = 'cache'
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
def __init__(self, name, desc, flags=''):
Change the Python scripts to follow Python indenting standards (4 space indents, spaces only).
2014-12-03 20:43:47 -05:00
Stat.__init__(self, name, CacheStat.prefix, desc, flags)
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
class CompressStat(Stat):
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
prefix = 'compression'
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
def __init__(self, name, desc, flags=''):
Change the Python scripts to follow Python indenting standards (4 space indents, spaces only).
2014-12-03 20:43:47 -05:00
Stat.__init__(self, name, CompressStat.prefix, desc, flags)
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
class ConnStat(Stat):
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
prefix = 'connection'
def __init__(self, name, desc, flags=''):
Change the Python scripts to follow Python indenting standards (4 space indents, spaces only).
2014-12-03 20:43:47 -05:00
Stat.__init__(self, name, ConnStat.prefix, desc, flags)
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
class CursorStat(Stat):
prefix = 'cursor'
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
def __init__(self, name, desc, flags=''):
Change the Python scripts to follow Python indenting standards (4 space indents, spaces only).
2014-12-03 20:43:47 -05:00
Stat.__init__(self, name, CursorStat.prefix, desc, flags)
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
class DhandleStat(Stat):
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
prefix = 'data-handle'
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
def __init__(self, name, desc, flags=''):
Change the Python scripts to follow Python indenting standards (4 space indents, spaces only).
2014-12-03 20:43:47 -05:00
Stat.__init__(self, name, DhandleStat.prefix, desc, flags)
WT-1315. Support statistics on join cursors. Statistics for these are a bit different than for a data source or a connection: 1) The statistics are simple (currently only 3), but range over sets of indices, so traversing the stats is a two level operation. Added a private callback API to advance to the next index when the stats have been exhausted. 2) cursors are used as single threaded operations, so arrays of stats is to avoid thread conflicts is not needed. Minor change: the stats_desc callback changed to return an int, and added an arg to provide some context. In the join cursor case, the returned desc is not a static string, it needs to be created and managed.
2015-11-17 10:34:11 -05:00
class JoinStat(Stat):
prefix = '' # prefix is inserted dynamically
def __init__(self, name, desc, flags=''):
Stat.__init__(self, name, JoinStat.prefix, desc, flags)
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
class LogStat(Stat):
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
prefix = 'log'
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
def __init__(self, name, desc, flags=''):
Change the Python scripts to follow Python indenting standards (4 space indents, spaces only).
2014-12-03 20:43:47 -05:00
Stat.__init__(self, name, LogStat.prefix, desc, flags)
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
class LSMStat(Stat):
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
prefix = 'LSM'
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
def __init__(self, name, desc, flags=''):
Change the Python scripts to follow Python indenting standards (4 space indents, spaces only).
2014-12-03 20:43:47 -05:00
Stat.__init__(self, name, LSMStat.prefix, desc, flags)
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
class RecStat(Stat):
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
prefix = 'reconciliation'
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
def __init__(self, name, desc, flags=''):
Change the Python scripts to follow Python indenting standards (4 space indents, spaces only).
2014-12-03 20:43:47 -05:00
Stat.__init__(self, name, RecStat.prefix, desc, flags)
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
class SessionStat(Stat):
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
prefix = 'session'
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
def __init__(self, name, desc, flags=''):
Change the Python scripts to follow Python indenting standards (4 space indents, spaces only).
2014-12-03 20:43:47 -05:00
Stat.__init__(self, name, SessionStat.prefix, desc, flags)
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
class TxnStat(Stat):
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
prefix = 'transaction'
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
def __init__(self, name, desc, flags=''):
Change the Python scripts to follow Python indenting standards (4 space indents, spaces only).
2014-12-03 20:43:47 -05:00
Stat.__init__(self, name, TxnStat.prefix, desc, flags)
Add separate statistics for pages yielding because the WT_REF isn't available.
2014-12-19 09:19:20 -05:00
class YieldStat(Stat):
prefix = 'thread-yield'
def __init__(self, name, desc, flags=''):
Stat.__init__(self, name, YieldStat.prefix, desc, flags)
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
##########################################
# Groupings of useful statistics:
# A pre-defined dictionary containing the group name as the key and the
# list of prefix tags that comprise that group.
##########################################
groups = {}
groups['cursor'] = [CursorStat.prefix, SessionStat.prefix]
groups['evict'] = [CacheStat.prefix, ConnStat.prefix, BlockStat.prefix]
groups['lsm'] = [LSMStat.prefix, TxnStat.prefix]
groups['memory'] = [CacheStat.prefix, ConnStat.prefix, RecStat.prefix]
groups['system'] = [ConnStat.prefix, DhandleStat.prefix, SessionStat.prefix]
Add a "clear" option when closing a statistics cursor. --HG-- rename : dist/stat_class.py => dist/stat_data.py
2011-07-29 16:20:59 +10:00
Move statistics strings into their own class file in dist so we can add config information. Specifically, stat fields can now be marked "perm", which means they aren't cleared by the stat clear method. This means cache allocation information can be moved into the stat structures and displayed normally. This also means statistics are required in the system (although it would probably be pretty easy to only maintain "perm" stat fields instead of all stat fields if we ever need a smaller footprint). --HG-- branch : keith
2010-01-25 04:58:51 +11:00
##########################################
Connect the old Btree statistics code to the statistics cursors: if you create a file: cursor, you can configure it as a "statistics" cursor if you want to display statistics for a particular file, or you can open "statistics" cursor if you want to display statistics for the system in general. Move to two stat groups: WT_CONNECTION and WT_BTREE (the old cache stats moved into the connection stats), the old btree file stats moved into the btree stats, I deleted the file handle stats. Delete the db/db_stat file, the parts of it that are still used moved to btree/bt_stat.c. Rename "stat" cursors to be "statistics" cursors. Turn the Python dist/stat_class.py Stat class into a list, I couldn't get dictionaries to sort by value for some reason.
2011-07-28 16:05:39 -04:00
# CONNECTION statistics
Move statistics strings into their own class file in dist so we can add config information. Specifically, stat fields can now be marked "perm", which means they aren't cleared by the stat clear method. This means cache allocation information can be moved into the stat structures and displayed normally. This also means statistics are required in the system (although it would probably be pretty easy to only maintain "perm" stat fields instead of all stat fields if we ever need a smaller footprint). --HG-- branch : keith
2010-01-25 04:58:51 +11:00
##########################################
Rename __wt_conn_stats to __wt_connection_stats so the sorting is not locale-dependent.
2011-08-26 15:34:51 +10:00
connection_stats = [
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# System statistics
##########################################
WT-2318 Add use of auto adjusting condition for some log threads.
2016-03-04 17:13:31 -05:00
ConnStat('cond_auto_wait', 'auto adjusting condition wait calls'),
ConnStat('cond_auto_wait_reset', 'auto adjusting condition resets'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
ConnStat('cond_wait', 'pthread mutex condition wait calls'),
ConnStat('file_open', 'files currently open', 'no_clear,no_scale'),
ConnStat('memory_allocation', 'memory allocations'),
ConnStat('memory_free', 'memory frees'),
ConnStat('memory_grow', 'memory re-allocations'),
ConnStat('read_io', 'total read I/Os'),
ConnStat('rwlock_read', 'pthread mutex shared lock read-lock calls'),
ConnStat('rwlock_write', 'pthread mutex shared lock write-lock calls'),
ConnStat('write_io', 'total write I/Os'),
Review the statistics: rename the statistics names for consistency and so they sort together, rename cursor operations to match the actual cursor method names, map connection cache statistics to/from data-source cache statistics, clean up the separation between the btree and block-manager statistics (mostly get rid of "file" anything, it's now either "block" or "btree"), move some minor btree eviction statistics gathering from bt_evict.c to rec_evict.c (it's still messy in a couple of places, but it's now all in rec_evict.c).
2012-12-05 15:09:12 +00:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# Async API statistics
##########################################
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
AsyncStat('async_alloc_race', 'number of allocation state races'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
AsyncStat('async_alloc_view', 'number of operation slots viewed for allocation'),
WT-2448 Add no_scale flag into a couple of statistics.
2016-03-07 10:04:12 +11:00
AsyncStat('async_cur_queue', 'current work queue length', 'no_scale'),
Get rid of some abbreviations in the async statistics messages.
2014-11-03 12:00:34 -05:00
AsyncStat('async_flush', 'number of flush calls'),
AsyncStat('async_full', 'number of times operation allocation failed'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
AsyncStat('async_max_queue', 'maximum work queue length', 'no_clear,no_scale'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
AsyncStat('async_nowork', 'number of times worker found no work'),
Get rid of some abbreviations in the async statistics messages.
2014-11-03 12:00:34 -05:00
AsyncStat('async_op_alloc', 'total allocations'),
AsyncStat('async_op_compact', 'total compact calls'),
AsyncStat('async_op_insert', 'total insert calls'),
AsyncStat('async_op_remove', 'total remove calls'),
AsyncStat('async_op_search', 'total search calls'),
AsyncStat('async_op_update', 'total update calls'),
First bits of async code
2014-03-24 16:17:27 -04:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# Block manager statistics
##########################################
WT-2438 Add size identifier to statistics based on timeseries classification.
2016-03-01 06:34:53 +00:00
BlockStat('block_byte_map_read', 'mapped bytes read', 'size'),
BlockStat('block_byte_read', 'bytes read', 'size'),
BlockStat('block_byte_write', 'bytes written', 'size'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
BlockStat('block_map_read', 'mapped blocks read'),
BlockStat('block_preload', 'blocks pre-loaded'),
BlockStat('block_read', 'blocks read'),
BlockStat('block_write', 'blocks written'),
Review the statistics: rename the statistics names for consistency and so they sort together, rename cursor operations to match the actual cursor method names, map connection cache statistics to/from data-source cache statistics, clean up the separation between the btree and block-manager statistics (mostly get rid of "file" anything, it's now either "block" or "btree"), move some minor btree eviction statistics gathering from bt_evict.c to rec_evict.c (it's still messy in a couple of places, but it's now all in rec_evict.c).
2012-12-05 15:09:12 +00:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# Cache and eviction statistics
##########################################
WT-2438 Add size identifier to statistics based on timeseries classification.
2016-03-01 06:34:53 +00:00
CacheStat('cache_bytes_dirty', 'tracked dirty bytes in the cache', 'no_clear,no_scale,size'),
CacheStat('cache_bytes_internal', 'tracked bytes belonging to internal pages in the cache', 'no_clear,no_scale,size'),
CacheStat('cache_bytes_inuse', 'bytes currently in the cache', 'no_clear,no_scale,size'),
CacheStat('cache_bytes_leaf', 'tracked bytes belonging to leaf pages in the cache', 'no_clear,no_scale,size'),
CacheStat('cache_bytes_max', 'maximum bytes configured', 'no_clear,no_scale,size'),
CacheStat('cache_bytes_overflow', 'tracked bytes belonging to overflow pages in the cache', 'no_clear,no_scale,size'),
CacheStat('cache_bytes_read', 'bytes read into cache', 'size'),
CacheStat('cache_bytes_write', 'bytes written from cache', 'size'),
WT-2429 Add a statistic to track eviction aggressive mode.
2016-02-25 03:53:48 +00:00
CacheStat('cache_eviction_aggressive_set', 'eviction currently operating in aggressive mode', 'no_clear,no_scale'),
Add a statistic to count how much eviction application threads are doing.
2014-12-02 10:31:52 +11:00
CacheStat('cache_eviction_app', 'pages evicted by application threads'),
Sort the statistics in each group.
2015-08-07 08:22:52 -04:00
CacheStat('cache_eviction_checkpoint', 'checkpoint blocked page eviction'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
CacheStat('cache_eviction_clean', 'unmodified pages evicted'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
CacheStat('cache_eviction_deepen', 'page split during eviction deepened the tree'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
CacheStat('cache_eviction_dirty', 'modified pages evicted'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
CacheStat('cache_eviction_fail', 'pages selected for eviction unable to be evicted'),
CacheStat('cache_eviction_force', 'pages evicted because they exceeded the in-memory maximum'),
CacheStat('cache_eviction_force_delete', 'pages evicted because they had chains of deleted items'),
CacheStat('cache_eviction_force_fail', 'failed eviction of pages that exceeded the in-memory maximum'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
CacheStat('cache_eviction_hazard', 'hazard pointer blocked page eviction'),
CacheStat('cache_eviction_internal', 'internal pages evicted'),
WT-2438 Add size identifier to statistics based on timeseries classification.
2016-03-01 06:34:53 +00:00
CacheStat('cache_eviction_maximum_page_size', 'maximum page size at eviction', 'no_clear,no_scale,size'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
CacheStat('cache_eviction_queue_empty', 'eviction server candidate queue empty when topping up'),
CacheStat('cache_eviction_queue_not_empty', 'eviction server candidate queue not empty when topping up'),
CacheStat('cache_eviction_server_evicting', 'eviction server evicting pages'),
CacheStat('cache_eviction_server_not_evicting', 'eviction server populating queue, but not evicting pages'),
CacheStat('cache_eviction_slow', 'eviction server unable to reach eviction goal'),
CacheStat('cache_eviction_split_internal', 'internal pages split during eviction'),
WT-2182: when internal pages grow large enough, split them into their parents instead of deepening the tree at that point. Includes: Move the internal page split check out of __split_parent(), instead have __split_parent's callers call a new function __split_parent_climb(), that starts at the first page and walks up the tree splitting internal pages until it reaches the root. It's using lock-coupling, as we should be able to discard each level's lock as we go. Add a new function __split_internal() that splits an internal page into its parent, called to split any internal page other than the root. It looks a lot like __split_deepen() (the root split function), but does a split-right of the internal page: split the contents of the internal page into a bunch of newly allocated internal pages, then insert that new set of pages into the original page's parent, replacing the original page's WT_REF at that level. Add a new statistic so we can distinguish between splitting leaf pages, splitting internal pages, and splitting the root (deepening the tree). I changed a statistic's message, so this change needs approval, it may not be backward compatible.
2015-10-29 12:17:33 -04:00
CacheStat('cache_eviction_split_leaf', 'leaf pages split during eviction'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
CacheStat('cache_eviction_walk', 'pages walked for eviction'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
CacheStat('cache_eviction_worker_evicting', 'eviction worker thread evicting pages'),
Add code to split a too-large insert list onto a new page as part of reconciling/evicting a page, to catch append-only workloads and keep from making threads wait for the reconciliation/eviction.
2014-11-30 10:56:21 -05:00
CacheStat('cache_inmem_split', 'in-memory page splits'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
CacheStat('cache_inmem_splittable', 'in-memory page passed criteria to be split'),
Remove "cursor" from the lookaside table statistics' names.
2015-08-26 12:03:17 -04:00
CacheStat('cache_lookaside_insert', 'lookaside table insert calls'),
CacheStat('cache_lookaside_remove', 'lookaside table remove calls'),
Add the cache_overhead configuration string to allow applications to configure their cache overhead.
2015-01-22 18:08:15 -05:00
CacheStat('cache_overhead', 'percentage overhead', 'no_clear,no_scale'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
CacheStat('cache_pages_dirty', 'tracked dirty pages in the cache', 'no_clear,no_scale'),
CacheStat('cache_pages_inuse', 'pages currently held in the cache', 'no_clear,no_scale'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
CacheStat('cache_read', 'pages read into cache'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
CacheStat('cache_read_lookaside', 'pages read into cache requiring lookaside entries'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
CacheStat('cache_write', 'pages written from cache'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
CacheStat('cache_write_lookaside', 'page written requiring lookaside records'),
CacheStat('cache_write_restore', 'pages written requiring in-memory restoration'),
Review the statistics: rename the statistics names for consistency and so they sort together, rename cursor operations to match the actual cursor method names, map connection cache statistics to/from data-source cache statistics, clean up the separation between the btree and block-manager statistics (mostly get rid of "file" anything, it's now either "block" or "btree"), move some minor btree eviction statistics gathering from bt_evict.c to rec_evict.c (it's still messy in a couple of places, but it's now all in rec_evict.c).
2012-12-05 15:09:12 +00:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# Dhandle statistics
##########################################
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
DhandleStat('dh_conn_handle_count', 'connection data handles currently active', 'no_clear,no_scale'),
DhandleStat('dh_session_handles', 'session dhandles swept'),
DhandleStat('dh_session_sweeps', 'session sweep attempts'),
WT-1481 Further regularize sweep stat naming.
2015-08-12 19:46:02 +10:00
DhandleStat('dh_sweep_close', 'connection sweep dhandles closed'),
DhandleStat('dh_sweep_ref', 'connection sweep candidate became referenced'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
DhandleStat('dh_sweep_remove', 'connection sweep dhandles removed from hash list'),
WT-1481 Further regularize sweep stat naming.
2015-08-12 19:46:02 +10:00
DhandleStat('dh_sweep_tod', 'connection sweep time-of-death sets'),
DhandleStat('dh_sweeps', 'connection sweeps'),
Periodically close session and connection dhandles that are no longer used. (#586)
2013-08-13 13:28:54 -04:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# Logging statistics
##########################################
WT-2438 Add size identifier to statistics based on timeseries classification.
2016-03-01 06:34:53 +00:00
LogStat('log_buffer_size', 'total log buffer size', 'no_clear,no_scale,size'),
LogStat('log_bytes_payload', 'log bytes of payload data', 'size'),
LogStat('log_bytes_written', 'log bytes written', 'size'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
LogStat('log_close_yields', 'yields waiting for previous log file close'),
WT-2438 Add size identifier to statistics based on timeseries classification.
2016-03-01 06:34:53 +00:00
LogStat('log_compress_len', 'total size of compressed records', 'size'),
LogStat('log_compress_mem', 'total in-memory size of compressed records', 'size'),
Don't compress if the record is already smaller than the minimum allocation size. #1359
2014-12-09 12:17:41 -05:00
LogStat('log_compress_small', 'log records too small to compress'),
Added log compression stats. refs #1359.
2014-12-04 09:57:18 -05:00
LogStat('log_compress_write_fails', 'log records not compressed'),
Sort the statistics in each group.
2015-08-07 08:22:52 -04:00
LogStat('log_compress_writes', 'log records compressed'),
WT-2104 Use 'operations' instead of 'calls' for consistency.
2015-09-15 14:06:24 -04:00
LogStat('log_flush', 'log flush operations'),
WT-2318 Add use of auto adjusting condition for some log threads.
2016-03-04 17:13:31 -05:00
LogStat('log_force_write', 'log force write operations'),
LogStat('log_force_write_skip', 'log force write operations skipped'),
WT-2438 Add size identifier to statistics based on timeseries classification.
2016-03-01 06:34:53 +00:00
LogStat('log_max_filesize', 'maximum log file size', 'no_clear,no_scale,size'),
Change prepare to prealloc and make it a value rather than boolean config.
2014-11-25 14:25:27 -05:00
LogStat('log_prealloc_files', 'pre-allocated log files prepared'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
LogStat('log_prealloc_max', 'number of pre-allocated log files to create', 'no_clear,no_scale'),
LogStat('log_prealloc_missed', 'pre-allocated log files not ready and missed'),
Change prepare to prealloc and make it a value rather than boolean config.
2014-11-25 14:25:27 -05:00
LogStat('log_prealloc_used', 'pre-allocated log files used'),
Add some stats. #1683
2015-02-23 08:51:45 -05:00
LogStat('log_release_write_lsn', 'log release advances write LSN'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
LogStat('log_scan_records', 'records processed by log scan'),
LogStat('log_scan_rereads', 'log scan records requiring two reads'),
LogStat('log_scans', 'log scan operations'),
Sort the statistics in each group.
2015-08-07 08:22:52 -04:00
LogStat('log_slot_closes', 'consolidated slot closures'),
WT-1989 First pass at 2nd patch integration.
2015-07-15 12:03:59 -04:00
LogStat('log_slot_coalesced', 'written slots coalesced'),
WT-2438 Add size identifier to statistics based on timeseries classification.
2016-03-01 06:34:53 +00:00
LogStat('log_slot_consolidated', 'logging bytes consolidated', 'size'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
LogStat('log_slot_joins', 'consolidated slot joins'),
LogStat('log_slot_races', 'consolidated slot join races'),
WT-2075 Add stat and a busy retry so that a blocked thread can progress.
2015-09-10 17:21:32 -04:00
LogStat('log_slot_switch_busy', 'busy returns attempting to switch slots'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
LogStat('log_slot_transitions', 'consolidated slot join transitions'),
WT-2094 Add unbuffered statistic.
2015-09-09 12:45:43 -04:00
LogStat('log_slot_unbuffered', 'consolidated slot unbuffered writes'),
Sort the statistics in each group.
2015-08-07 08:22:52 -04:00
LogStat('log_sync', 'log sync operations'),
LogStat('log_sync_dir', 'log sync_dir operations'),
LogStat('log_write_lsn', 'log server thread advances write LSN'),
WT-2318 Add use of auto adjusting condition for some log threads.
2016-03-04 17:13:31 -05:00
LogStat('log_write_lsn_skip', 'log server thread write LSN walk skipped'),
Sort the statistics in each group.
2015-08-07 08:22:52 -04:00
LogStat('log_writes', 'log write operations'),
WT-2151 Add zero-fill stat. Check correct flags field.
2015-09-29 13:51:43 -04:00
LogStat('log_zero_fills', 'log files manually zero-filled'),
Start logging implementation
2013-05-09 15:50:16 -04:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# Reconciliation statistics
##########################################
WT-2218 Add connection level stat for number of deleted pages.
2015-11-17 14:21:40 -05:00
RecStat('rec_page_delete', 'pages deleted'),
WT-2218 Add fast-path delete stat.
2015-11-18 15:17:13 -05:00
RecStat('rec_page_delete_fast', 'fast-path pages deleted'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
RecStat('rec_pages', 'page reconciliation calls'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
RecStat('rec_pages_eviction', 'page reconciliation calls for eviction'),
WT-2438 Add size identifier to statistics based on timeseries classification.
2016-03-01 06:34:53 +00:00
RecStat('rec_split_stashed_bytes', 'split bytes currently awaiting free', 'no_clear,no_scale,size'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
RecStat('rec_split_stashed_objects', 'split objects currently awaiting free', 'no_clear,no_scale'),
Add some basic reconciliation stats to the connection stats.
2013-02-18 16:34:00 -05:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# Transaction statistics
##########################################
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
TxnStat('txn_begin', 'transaction begins'),
TxnStat('txn_checkpoint', 'transaction checkpoints'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
TxnStat('txn_checkpoint_generation', 'transaction checkpoint generation', 'no_clear,no_scale'),
TxnStat('txn_checkpoint_running', 'transaction checkpoint currently running', 'no_clear,no_scale'),
TxnStat('txn_checkpoint_time_max', 'transaction checkpoint max time (msecs)', 'no_clear,no_scale'),
TxnStat('txn_checkpoint_time_min', 'transaction checkpoint min time (msecs)', 'no_clear,no_scale'),
TxnStat('txn_checkpoint_time_recent', 'transaction checkpoint most recent time (msecs)', 'no_clear,no_scale'),
TxnStat('txn_checkpoint_time_total', 'transaction checkpoint total time (msecs)', 'no_clear,no_scale'),
Sort the statistics in each group.
2015-08-07 08:22:52 -04:00
TxnStat('txn_commit', 'transactions committed'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
TxnStat('txn_fail_cache', 'transaction failures due to cache overflow'),
TxnStat('txn_pinned_checkpoint_range', 'transaction range of IDs currently pinned by a checkpoint', 'no_clear,no_scale'),
TxnStat('txn_pinned_range', 'transaction range of IDs currently pinned', 'no_clear,no_scale'),
TxnStat('txn_pinned_snapshot_range', 'transaction range of IDs currently pinned by named snapshots', 'no_clear,no_scale'),
TxnStat('txn_rollback', 'transactions rolled back'),
WT-2222 Fixup snapshot statistics re: review comments.
2015-11-19 01:02:59 +00:00
TxnStat('txn_snapshots_created', 'number of named snapshots created'),
TxnStat('txn_snapshots_dropped', 'number of named snapshots dropped'),
Sort the statistics in each group.
2015-08-07 08:22:52 -04:00
TxnStat('txn_sync', 'transaction sync calls'),
Add a connection-level statistic for btree cursor ops and an LSM "rows merged" counter.
2013-04-23 12:28:05 -04:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# LSM statistics
##########################################
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
LSMStat('lsm_checkpoint_throttle', 'sleep for LSM checkpoint throttle'),
LSMStat('lsm_merge_throttle', 'sleep for LSM merge throttle'),
LSMStat('lsm_rows_merged', 'rows merged in an LSM tree'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
LSMStat('lsm_work_queue_app', 'application work units currently queued', 'no_clear,no_scale'),
LSMStat('lsm_work_queue_manager', 'merge work units currently queued', 'no_clear,no_scale'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
LSMStat('lsm_work_queue_max', 'tree queue hit maximum'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
LSMStat('lsm_work_queue_switch', 'switch work units currently queued', 'no_clear,no_scale'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
LSMStat('lsm_work_units_created', 'tree maintenance operations scheduled'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
LSMStat('lsm_work_units_discarded', 'tree maintenance operations discarded'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
LSMStat('lsm_work_units_done', 'tree maintenance operations executed'),
Add a connection-level statistic for btree cursor ops and an LSM "rows merged" counter.
2013-04-23 12:28:05 -04:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# Session operations
##########################################
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
SessionStat('session_cursor_open', 'open cursor count', 'no_clear,no_scale'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
SessionStat('session_open', 'open session count', 'no_clear,no_scale'),
Add support for open-cursor count statistics in both the connection and data-handles. This requires atomic update versions of a couple of the stat macros.
2013-04-28 11:19:16 -04:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
# Total cursor operations
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
CursorStat('cursor_create', 'cursor create calls'),
CursorStat('cursor_insert', 'cursor insert calls'),
CursorStat('cursor_next', 'cursor next calls'),
CursorStat('cursor_prev', 'cursor prev calls'),
CursorStat('cursor_remove', 'cursor remove calls'),
CursorStat('cursor_reset', 'cursor reset calls'),
WT-2046 - Change wording of counter
2015-08-19 14:06:30 +10:00
CursorStat('cursor_restart', 'cursor restarted searches'),
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
CursorStat('cursor_search', 'cursor search calls'),
CursorStat('cursor_search_near', 'cursor search near calls'),
WT-2218 Add truncate stats
2015-11-16 13:12:25 -05:00
CursorStat('cursor_truncate', 'truncate calls'),
First pass at adding groupings for stats. #1365
2014-11-26 15:11:22 -05:00
CursorStat('cursor_update', 'cursor update calls'),
Add separate statistics for pages yielding because the WT_REF isn't available.
2014-12-19 09:19:20 -05:00
##########################################
# Yield statistics
##########################################
Add sleep backoff if waiting for page too long. SERVER-16269
2014-12-19 16:59:34 +00:00
YieldStat('page_busy_blocked', 'page acquire busy blocked'),
YieldStat('page_forcible_evict_blocked', 'page acquire eviction blocked'),
busy-work: sort the stat options, make the stat options units match other stat option strings, sleep_cnt and wait_cnt should be unsigned, initialize wait_cnt to 0 and increment it in the if/test for clarity, rename page_in_sleep to page_sleep, matches other yield fields.
2014-12-19 12:27:27 -05:00
YieldStat('page_locked_blocked', 'page acquire locked blocked'),
YieldStat('page_read_blocked', 'page acquire read blocked'),
YieldStat('page_sleep', 'page acquire time sleeping (usecs)'),
Connect the old Btree statistics code to the statistics cursors: if you create a file: cursor, you can configure it as a "statistics" cursor if you want to display statistics for a particular file, or you can open "statistics" cursor if you want to display statistics for the system in general. Move to two stat groups: WT_CONNECTION and WT_BTREE (the old cache stats moved into the connection stats), the old btree file stats moved into the btree stats, I deleted the file handle stats. Delete the db/db_stat file, the parts of it that are still used moved to btree/bt_stat.c. Rename "stat" cursors to be "statistics" cursors. Turn the Python dist/stat_class.py Stat class into a list, I couldn't get dictionaries to sort by value for some reason.
2011-07-28 16:05:39 -04:00
]
Move statistics strings into their own class file in dist so we can add config information. Specifically, stat fields can now be marked "perm", which means they aren't cleared by the stat clear method. This means cache allocation information can be moved into the stat structures and displayed normally. This also means statistics are required in the system (although it would probably be pretty easy to only maintain "perm" stat fields instead of all stat fields if we ever need a smaller footprint). --HG-- branch : keith
2010-01-25 04:58:51 +11:00
WT-2428 Add a "json" mode to statistics logging. This generates statistics in a format that is compatible with MongoDB so that we can share tools for analysis.
2016-02-25 11:50:55 +11:00
connection_stats = sorted(connection_stats, key=attrgetter('desc'))
Add a "size of checkpoint" statistic. Merge statistics from file and LSM sources into a "data source" statistic structure. Rename and regroup some shared stastistics. Make statistics constant names upper case. Add a helper to the Python API to lookup in a cursor in a simple expression. closes #232
2012-11-16 21:02:27 +11:00
Restructure the cache code, and split it into multiple files. There's now an I/O server that does all of the reading for the system, and the linked lists of pages on each hash bucket has been replaced with an array that's grown on demand. --HG-- branch : keith
2010-03-26 00:44:21 +11:00
##########################################
Add a "size of checkpoint" statistic. Merge statistics from file and LSM sources into a "data source" statistic structure. Rename and regroup some shared stastistics. Make statistics constant names upper case. Add a helper to the Python API to lookup in a cursor in a simple expression. closes #232
2012-11-16 21:02:27 +11:00
# Data source statistics
Restructure the cache code, and split it into multiple files. There's now an I/O server that does all of the reading for the system, and the linked lists of pages on each hash bucket has been replaced with an array that's grown on demand. --HG-- branch : keith
2010-03-26 00:44:21 +11:00
##########################################
Add a "size of checkpoint" statistic. Merge statistics from file and LSM sources into a "data source" statistic structure. Rename and regroup some shared stastistics. Make statistics constant names upper case. Add a helper to the Python API to lookup in a cursor in a simple expression. closes #232
2012-11-16 21:02:27 +11:00
dsrc_stats = [
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# Session operations
##########################################
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
SessionStat('session_compact', 'object compaction'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
SessionStat('session_cursor_open', 'open cursor count', 'no_clear,no_scale'),
Add a connection-level statistic for btree cursor ops and an LSM "rows merged" counter.
2013-04-23 12:28:05 -04:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# Cursor operations
##########################################
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
CursorStat('cursor_create', 'create calls'),
CursorStat('cursor_insert', 'insert calls'),
CursorStat('cursor_insert_bulk', 'bulk-loaded cursor-insert calls'),
WT-2438 Add size identifier to statistics based on timeseries classification.
2016-03-01 06:34:53 +00:00
CursorStat('cursor_insert_bytes', 'cursor-insert key and value bytes inserted', 'size'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
CursorStat('cursor_next', 'next calls'),
CursorStat('cursor_prev', 'prev calls'),
CursorStat('cursor_remove', 'remove calls'),
WT-2438 Add size identifier to statistics based on timeseries classification.
2016-03-01 06:34:53 +00:00
CursorStat('cursor_remove_bytes', 'cursor-remove key bytes removed', 'size'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
CursorStat('cursor_reset', 'reset calls'),
WT-2046 - Change wording of counter
2015-08-19 14:06:30 +10:00
CursorStat('cursor_restart', 'restarted searches'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
CursorStat('cursor_search', 'search calls'),
CursorStat('cursor_search_near', 'search near calls'),
WT-2218 Add truncate stats
2015-11-16 13:12:25 -05:00
CursorStat('cursor_truncate', 'truncate calls'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
CursorStat('cursor_update', 'update calls'),
WT-2438 Add size identifier to statistics based on timeseries classification.
2016-03-01 06:34:53 +00:00
CursorStat('cursor_update_bytes', 'cursor-update value bytes updated', 'size'),
Maintain bytes read/written as well as I/O counts, reference #381.
2012-11-23 17:46:34 +00:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# Btree statistics
##########################################
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
BtreeStat('btree_checkpoint_generation', 'btree checkpoint generation', 'no_clear,no_scale'),
BtreeStat('btree_column_deleted', 'column-store variable-size deleted values', 'no_scale'),
BtreeStat('btree_column_fix', 'column-store fixed-size leaf pages', 'no_scale'),
BtreeStat('btree_column_internal', 'column-store internal pages', 'no_scale'),
BtreeStat('btree_column_rle', 'column-store variable-size RLE encoded values', 'no_scale'),
BtreeStat('btree_column_variable', 'column-store variable-size leaf pages', 'no_scale'),
Use the "btree" tag consistently (before we had both "Btree" and "btree").
2014-11-03 11:31:52 -05:00
BtreeStat('btree_compact_rewrite', 'pages rewritten by compaction'),
BtreeStat('btree_entries', 'number of key/value pairs', 'no_scale'),
WT-2438 Add size identifier to other statistics as required.
2016-03-01 06:38:02 +00:00
BtreeStat('btree_fixed_len', 'fixed-record size', 'max_aggregate,no_scale,size'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
BtreeStat('btree_maximum_depth', 'maximum tree depth', 'max_aggregate,no_scale'),
WT-2438 Add size identifier to other statistics as required.
2016-03-01 06:38:02 +00:00
BtreeStat('btree_maxintlkey', 'maximum internal page key size', 'max_aggregate,no_scale,size'),
BtreeStat('btree_maxintlpage', 'maximum internal page size', 'max_aggregate,no_scale,size'),
BtreeStat('btree_maxleafkey', 'maximum leaf page key size', 'max_aggregate,no_scale,size'),
BtreeStat('btree_maxleafpage', 'maximum leaf page size', 'max_aggregate,no_scale,size'),
BtreeStat('btree_maxleafvalue', 'maximum leaf page value size', 'max_aggregate,no_scale,size'),
Use the "btree" tag consistently (before we had both "Btree" and "btree").
2014-11-03 11:31:52 -05:00
BtreeStat('btree_overflow', 'overflow pages', 'no_scale'),
BtreeStat('btree_row_internal', 'row-store internal pages', 'no_scale'),
BtreeStat('btree_row_leaf', 'row-store leaf pages', 'no_scale'),
Review the statistics: rename the statistics names for consistency and so they sort together, rename cursor operations to match the actual cursor method names, map connection cache statistics to/from data-source cache statistics, clean up the separation between the btree and block-manager statistics (mostly get rid of "file" anything, it's now either "block" or "btree"), move some minor btree eviction statistics gathering from bt_evict.c to rec_evict.c (it's still messy in a couple of places, but it's now all in rec_evict.c).
2012-12-05 15:09:12 +00:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# LSM statistics
##########################################
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
LSMStat('bloom_count', 'bloom filters in the LSM tree', 'no_scale'),
LSMStat('bloom_false_positive', 'bloom filter false positives'),
LSMStat('bloom_hit', 'bloom filter hits'),
LSMStat('bloom_miss', 'bloom filter misses'),
LSMStat('bloom_page_evict', 'bloom filter pages evicted from cache'),
LSMStat('bloom_page_read', 'bloom filter pages read into cache'),
WT-2438 Add size identifier to other statistics as required.
2016-03-01 06:38:02 +00:00
LSMStat('bloom_size', 'total size of bloom filters', 'no_scale,size'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
LSMStat('lsm_checkpoint_throttle', 'sleep for LSM checkpoint throttle'),
async_max_queue: collection statistic, aggregate doesn't apply, don't clear. cache_eviction_maximum_page_size: collection statistic, aggregate doesn't apply, don't clear. various txn_checkpoint_XXX statistics: collection statistic, aggregate doesn't apply various Btree maximum size statistics: aggregate the maximum value, rather than doing no aggregation at all. lsm_chunk_count: aggregate the count of LSM chunks. Add syntax checking code to stat.py to disallow aggregation flags for connection-level statistics. Reference #1742.
2015-03-30 09:52:34 -04:00
LSMStat('lsm_chunk_count', 'chunks in the LSM tree', 'no_scale'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
LSMStat('lsm_generation_max', 'highest merge generation in the LSM tree', 'max_aggregate,no_scale'),
LSMStat('lsm_lookup_no_bloom', 'queries that could have benefited from a Bloom filter that did not exist'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
LSMStat('lsm_merge_throttle', 'sleep for LSM merge throttle'),
Maintain bytes read/written as well as I/O counts, reference #381.
2012-11-23 17:46:34 +00:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# Block manager statistics
##########################################
WT-2438 Add size identifier to other statistics as required.
2016-03-01 06:38:02 +00:00
BlockStat('allocation_size', 'file allocation unit size', 'max_aggregate,no_scale,size'),
Sort the statistics in each group.
2015-08-07 08:22:52 -04:00
BlockStat('block_alloc', 'blocks allocated'),
WT-2438 Add size identifier to statistics based on timeseries classification.
2016-03-01 06:34:53 +00:00
BlockStat('block_checkpoint_size', 'checkpoint size', 'no_scale,size'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
BlockStat('block_extension', 'allocations requiring file extension'),
BlockStat('block_free', 'blocks freed'),
WT-2060: Replace remaining uses of no_aggregate with max_aggregate and remove support for no_aggregate; in the remaining no_aggregate fields (btree_fixed_len, allocation_size, block_magic, block_major, and block_minor), the maximum value is at least as good as the last one we find (and in some cases, such as the file allocation size, arguably better). The reason is because no_aggregate simply read from slot 0, and that assumes there's a WT_STAT_XXX_SET call in the code that updates this sp ecific statistic, and we have no way to enforce that, so don't try. This will be marginally slower because we're checking more fields on statistics calls, but I can't imagine it will matter.
2015-11-30 11:04:05 -05:00
BlockStat('block_magic', 'file magic number', 'max_aggregate,no_scale'),
BlockStat('block_major', 'file major version number', 'max_aggregate,no_scale'),
BlockStat('block_minor', 'minor version number', 'max_aggregate,no_scale'),
WT-2448 Add no_scale flag into a couple of statistics.
2016-03-07 10:04:12 +11:00
BlockStat('block_reuse_bytes', 'file bytes available for reuse', 'no_scale,size'),
WT-2438 Add size identifier to statistics based on timeseries classification.
2016-03-01 06:34:53 +00:00
BlockStat('block_size', 'file size in bytes', 'no_scale,size'),
Review the statistics: rename the statistics names for consistency and so they sort together, rename cursor operations to match the actual cursor method names, map connection cache statistics to/from data-source cache statistics, clean up the separation between the btree and block-manager statistics (mostly get rid of "file" anything, it's now either "block" or "btree"), move some minor btree eviction statistics gathering from bt_evict.c to rec_evict.c (it's still messy in a couple of places, but it's now all in rec_evict.c).
2012-12-05 15:09:12 +00:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# Cache and eviction statistics
##########################################
WT-2438 Add size identifier to statistics based on timeseries classification.
2016-03-01 06:34:53 +00:00
CacheStat('cache_bytes_read', 'bytes read into cache', 'size'),
CacheStat('cache_bytes_write', 'bytes written from cache', 'size'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
CacheStat('cache_eviction_checkpoint', 'checkpoint blocked page eviction'),
Track splits during eviction by data-source as well as by connection. Don't double-count in-memory splits (we're incrementing cache_eviction_split in the underlying split-parent routine, not in the caller, so it's counting both normal and in-memory splits). Instead, cache_eviction_split is normal eviction splits, cache_inmem_split is in-memory splits,
2015-02-09 14:04:02 -05:00
CacheStat('cache_eviction_clean', 'unmodified pages evicted'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
CacheStat('cache_eviction_deepen', 'page split during eviction deepened the tree'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
CacheStat('cache_eviction_dirty', 'modified pages evicted'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
CacheStat('cache_eviction_fail', 'data source pages selected for eviction unable to be evicted'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
CacheStat('cache_eviction_hazard', 'hazard pointer blocked page eviction'),
CacheStat('cache_eviction_internal', 'internal pages evicted'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
CacheStat('cache_eviction_split_internal', 'internal pages split during eviction'),
WT-2182: when internal pages grow large enough, split them into their parents instead of deepening the tree at that point. Includes: Move the internal page split check out of __split_parent(), instead have __split_parent's callers call a new function __split_parent_climb(), that starts at the first page and walks up the tree splitting internal pages until it reaches the root. It's using lock-coupling, as we should be able to discard each level's lock as we go. Add a new function __split_internal() that splits an internal page into its parent, called to split any internal page other than the root. It looks a lot like __split_deepen() (the root split function), but does a split-right of the internal page: split the contents of the internal page into a bunch of newly allocated internal pages, then insert that new set of pages into the original page's parent, replacing the original page's WT_REF at that level. Add a new statistic so we can distinguish between splitting leaf pages, splitting internal pages, and splitting the root (deepening the tree). I changed a statistic's message, so this change needs approval, it may not be backward compatible.
2015-10-29 12:17:33 -04:00
CacheStat('cache_eviction_split_leaf', 'leaf pages split during eviction'),
Add code to split a too-large insert list onto a new page as part of reconciling/evicting a page, to catch append-only workloads and keep from making threads wait for the reconciliation/eviction.
2014-11-30 10:56:21 -05:00
CacheStat('cache_inmem_split', 'in-memory page splits'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
CacheStat('cache_inmem_splittable', 'in-memory page passed criteria to be split'),
CacheStat('cache_overflow_value', 'overflow values cached in memory', 'no_scale'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
CacheStat('cache_read', 'pages read into cache'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
CacheStat('cache_read_lookaside', 'pages read into cache requiring lookaside entries'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
CacheStat('cache_read_overflow', 'overflow pages read into cache'),
CacheStat('cache_write', 'pages written from cache'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
CacheStat('cache_write_lookaside', 'page written requiring lookaside records'),
CacheStat('cache_write_restore', 'pages written requiring in-memory restoration'),
Review the statistics: rename the statistics names for consistency and so they sort together, rename cursor operations to match the actual cursor method names, map connection cache statistics to/from data-source cache statistics, clean up the separation between the btree and block-manager statistics (mostly get rid of "file" anything, it's now either "block" or "btree"), move some minor btree eviction statistics gathering from bt_evict.c to rec_evict.c (it's still messy in a couple of places, but it's now all in rec_evict.c).
2012-12-05 15:09:12 +00:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# Compression statistics
##########################################
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
CompressStat('compress_raw_fail', 'raw compression call failed, no additional data available'),
CompressStat('compress_raw_fail_temporary', 'raw compression call failed, additional data available'),
Sort the statistics in each group.
2015-08-07 08:22:52 -04:00
CompressStat('compress_raw_ok', 'raw compression call succeeded'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
CompressStat('compress_read', 'compressed pages read'),
CompressStat('compress_write', 'compressed pages written'),
CompressStat('compress_write_fail', 'page written failed to compress'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
CompressStat('compress_write_too_small', 'page written was too small to compress'),
Minor clean up and enhancement for the reconciliation statistics, add a set of compression statistics, both to the data-source statistics.
2013-01-19 11:05:05 +00:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# Reconciliation statistics
##########################################
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
RecStat('rec_dictionary', 'dictionary matches'),
Sort the statistics in each group.
2015-08-07 08:22:52 -04:00
RecStat('rec_multiblock_internal', 'internal page multi-block writes'),
RecStat('rec_multiblock_leaf', 'leaf page multi-block writes'),
Quit trying to make stat_data.py fit into 80 columns, now requires a wide window to edit (join broken lines, sort all entries).
2015-11-29 09:17:21 -05:00
RecStat('rec_multiblock_max', 'maximum blocks required for a page', 'max_aggregate,no_scale'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
RecStat('rec_overflow_key_internal', 'internal-page overflow keys'),
RecStat('rec_overflow_key_leaf', 'leaf-page overflow keys'),
RecStat('rec_overflow_value', 'overflow values written'),
RecStat('rec_page_delete', 'pages deleted'),
WT-2218 Add fast-path delete stat.
2015-11-18 15:17:13 -05:00
RecStat('rec_page_delete_fast', 'fast-path pages deleted'),
Sort the statistics in each group.
2015-08-07 08:22:52 -04:00
RecStat('rec_page_match', 'page checksum matches'),
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
RecStat('rec_pages', 'page reconciliation calls'),
RecStat('rec_pages_eviction', 'page reconciliation calls for eviction'),
WT-2438 Add size identifier to statistics based on timeseries classification.
2016-03-01 06:34:53 +00:00
RecStat('rec_prefix_compression', 'leaf page key bytes discarded using prefix compression', 'size'),
RecStat('rec_suffix_compression', 'internal page key bytes discarded using suffix compression', 'size'),
Review the statistics: rename the statistics names for consistency and so they sort together, rename cursor operations to match the actual cursor method names, map connection cache statistics to/from data-source cache statistics, clean up the separation between the btree and block-manager statistics (mostly get rid of "file" anything, it's now either "block" or "btree"), move some minor btree eviction statistics gathering from bt_evict.c to rec_evict.c (it's still messy in a couple of places, but it's now all in rec_evict.c).
2012-12-05 15:09:12 +00:00
Fix whitespace in dist/stat_data.py
2014-10-28 15:25:57 +11:00
##########################################
# Transaction statistics
##########################################
Create a set of subclasses to ensure we can't forget prefixes or create inconsistent ones. This change exactly matches the previous output, I want to make sure it's clear where I've changed the output vs. where I'm changing how we create that output. Reference #1313.
2014-11-03 11:28:19 -05:00
TxnStat('txn_update_conflict', 'update conflicts'),
Add a new statistics cursor type "statistics:lsm". Update ex_stat.c to demonstrate usage. Update ex_test_perf.c to have a statistics thread.
2012-10-08 17:23:39 +11:00
]
WT-2428 Add a "json" mode to statistics logging. This generates statistics in a format that is compatible with MongoDB so that we can share tools for analysis.
2016-02-25 11:50:55 +11:00
dsrc_stats = sorted(dsrc_stats, key=attrgetter('desc'))
WT-1315. Support statistics on join cursors. Statistics for these are a bit different than for a data source or a connection: 1) The statistics are simple (currently only 3), but range over sets of indices, so traversing the stats is a two level operation. Added a private callback API to advance to the next index when the stats have been exhausted. 2) cursors are used as single threaded operations, so arrays of stats is to avoid thread conflicts is not needed. Minor change: the stats_desc callback changed to return an int, and added an arg to provide some context. In the join cursor case, the returned desc is not a static string, it needs to be created and managed.
2015-11-17 10:34:11 -05:00
##########################################
# Cursor Join statistics
##########################################
join_stats = [
JoinStat('accesses', 'accesses'),
JoinStat('actual_count', 'actual count of items'),
JoinStat('bloom_false_positive', 'bloom filter false positives'),
]
WT-2428 Add a "json" mode to statistics logging. This generates statistics in a format that is compatible with MongoDB so that we can share tools for analysis.
2016-02-25 11:50:55 +11:00
join_stats = sorted(join_stats, key=attrgetter('desc'))
Reference in New Issue Copy Permalink