xuemingqiang/mongo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
17,817 Commits 160 Branches 1,474 Tags
dce3b942e4e22fcd4c2e8ebb5a0cdbbf1e2f3567
Commit Graph

372 Commits

Author SHA1 Message Date
Alex Gorrod
bf32b9f6dd WT-2375 Update reverse integer collator to use streaming pack API. 2016-03-14 03:29:25 +00:00
Michael Cahill
7b40740393 Merge branch 'develop' into wt-2375-collator-tests 2016-03-14 13:33:44 +11:00
Alex Gorrod
6c3446725a WT-2376 Consistently name snappy byteswap routine. 2016-02-22 00:26:49 +00:00
Keith Bostic
9905036c70 WT-2376: Modules should compile without including wt_internal.h header file 
Don't include <wt_internal.h> in the compression functions: inline the
necessary swap routines, they're pretty simple and there's unlikely to
be any performance issue here, we're swapping only a very small amount
of information per compression/decompression.
 2016-02-21 11:39:28 -05:00
Don Anderson
3544728eb8 WT-2375 Use WT_EXTENSION API to unpack, for portability. 
In addition, various naming cleanups and error path simplification.
 2016-02-02 14:43:35 -05:00
Don Anderson
faf504846b WT-2375 Whitespace and spelling. 2016-02-01 22:55:46 -05:00
David Hows
47cbbc2b05 WT-2372 - wt_internal.h should always be the first include, when needed 2016-02-02 14:43:52 +11:00
Don Anderson
9b87a2f121 WT-2375 Added tests using revint, a reverse integer collator for testing. 
Revint demonstrates how collators are expected to decode non-trivial key
formats and work in situations with duplicate keys.  The new test uses
both a collator and an extractor.
 2016-02-01 22:34:40 -05:00
Michael Cahill
7bd982093b Merge branch 'develop' into WT-60 2016-01-29 12:36:27 +11:00
Keith Bostic
12be963758 WT-2352: Allow build and test without requiring lz4 
Remove support for the bzip2 compression engine.
 2016-01-28 12:44:09 -05:00
Keith Bostic
1a72dbfb0d WT-60: big endian support 
Stylistic, lint cleanups.
 2016-01-28 08:49:35 -05:00
Keith Bostic
1d0447e18a WT-60: big endian support 
Add lz4 and snappy compression support.
 2016-01-28 04:19:17 -05:00
Keith Bostic
344a714c3d Upgrade copyright notices from 2015 to 2016. 2016-01-01 16:37:39 -05:00
Michael Cahill
982c5862fb Fix some OS X clang warnings. 
ext/extractors/csv/csv_extractor.c:155:25: error: implicit conversion loses integer precision: 'long' to 'int' [-Werror,-Wshorten-64-to-32]
src/cursor/cur_join.c:1022:16: error: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'u_int' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
 2015-11-27 09:20:25 +11:00
Keith Bostic
bc1301ad7b WT-2234: Coverity 1339897: Resource leaks (RESOURCE_LEAK) 2015-11-25 08:57:55 -05:00
Don Anderson
4a7f74260e WT-1315. Remove dependency from extension to WT library, it's not portable. 2015-11-04 13:13:59 -05:00
Don Anderson
3507971e82 WT-1315. Modified csv_extractor arguments to allow either string or int 
columns to be extracted.  This is needed for comprehensive join cursor testing.
 2015-11-03 13:31:59 -05:00
Keith Bostic
1bdccb84fd WT-1822 
Coverity CID #1320624, resource leak.
 2015-08-28 09:05:36 -04:00
Keith Bostic
12044d22cc s_all run of: "style.py" resulted in: 
../ext/encryptors/rotn/rotn_encrypt.c:421: missing comment for rotn_configure

ext/encryptors/rotn/rotn_encrypt.c  99  Warning 534: Ignoring return
value of function '__wt_extension_api::err_printf' (compare with line
105, file src/include/wiredtiger_ext.h)
 2015-08-26 08:47:13 -04:00
Michael Cahill
0d1129df17 Fix a build warning on OS X in the CSV extension. 
../../../../ext/extractors/csv/csv_extractor.c:131:29: error: implicit conversion loses integer precision: 'long' to 'int' [-Werror,-Wshorten-64-to-32]
	csv_extractor->field_num = field_num;
 2015-08-26 22:34:37 +10:00
Don Anderson
0ef8cd2349 WT-1822. Add test to ensure that error codes returned by decrypt 
during a wiredtiger_open are returned by wiredtiger_open.  This
requires a special config argument to be passed to the rotn encryptor.
 2015-08-24 21:18:05 -04:00
Keith Bostic
896797a8a4 Whitespace: 3 spaces after a period in a comment. 
Sorry, but it just bugged me.
 2015-06-14 09:59:37 -04:00
David Hows
44f69e90e5 Add fix for C2054 to lz4_compress 2015-05-29 16:50:53 +10:00
Alex Gorrod
fceaaa105e Fix gcc 4.7 compiler warning. 2015-05-22 11:53:48 +10:00
Don Anderson
126ede1190 WT-1940. Fixed unpack call to use extension API. 2015-05-21 10:23:25 -04:00
Don Anderson
c730ba868a WT-1940. Modify CSV extractor to copy the result string safely. 2015-05-20 22:40:43 -04:00
Don Anderson
9c5f7c4327 WT-1940. Added extractor tests and a small fix for customized extractors. 2015-05-20 14:19:22 -04:00
Michael Cahill
77163ad484 Avoid islower / isupper in the ROTN code: it behaves in unexpected ways on various systems, particularly if the locale is not "C". 
refs WT-1822
 2015-05-14 15:27:24 +10:00
Don Anderson
c6b3183209 Handle potential error condition from bzip. 2015-05-13 14:38:44 -04:00
Michael Cahill
9880d7563f Merge branch 'develop' into encryption-api 
Conflicts:
	ext/compressors/lz4/lz4_compress.c
 2015-05-12 14:47:28 +10:00
Keith Bostic
08c4bf822a Replace a large paragraph on why we allocate extension structures instead 
of making them static with a single line -- the big paragraph shows up in
the documentation.
 2015-05-08 10:13:12 -04:00
Michael Cahill
5f56cf0a52 Fix some minor build issues with encryption: don't commit Makefile.in, avoid some signed / unsigned warnings. 2015-05-07 11:53:27 +10:00
Don Anderson
c48d5f94c7 In rotn, remove unneeded checks and fixed return value for presize(). WT-1822. 2015-05-05 10:35:42 -04:00
Don Anderson
c48f02327b Removed references to random(). WT-1822. 2015-05-04 23:29:26 -04:00
Keith Bostic
7b5c155b2c Alex's review, src_len is used. 2015-05-04 12:21:24 -04:00
Keith Bostic
5e7a31152c Alex's review, beef up the decompress comment explaining the bounce 
buffer.
 2015-05-04 12:19:53 -04:00
Keith Bostic
9cf7ad5ed5 Document that prefix.unused can be used as a mode flag in the future. 2015-05-04 12:03:45 -04:00
Keith Bostic
9b13791f82 It seems possible the destination length is smaller than the target 
page size if there's not much data to write, use the smaller of the
two sizes as the LZ4 targetDestSize value.
 2015-05-04 11:56:09 -04:00
Keith Bostic
fafd177bae Alex's review, switch to more intuitive field names in the prefix 
structure.
 2015-05-04 11:43:05 -04:00
Susan LoVerso
9888eff271 Merge branch 'develop' into encryption-api 
Conflicts:
	src/docs/command-line.dox
 2015-05-04 11:07:38 -04:00
Keith Bostic
9976ddd0b4 Allocate a scratch buffer as necessary during decompression, solves the 
problem of our caller not knowing the true length of the decompressed
bytes and so not allocating a big enough buffer for decompression.

Fix a bug, we weren't using raw compression's "extra" argument as part
of the target buffer size calculation.
 2015-05-03 15:28:18 -04:00
Keith Bostic
059aee7010 Fix a typo, was never returning success from raw compression. 2015-05-03 14:27:36 -04:00
Keith Bostic
a560965c41 Rewrite the binary search function (I took zlib's, but zlib's search 
is fundamentally different, LZ4's search has different needs.)
 2015-05-03 14:06:58 -04:00
Keith Bostic
1025b9fa76 Add LZ4 raw support based on the new LZ4_compress_destSize API, making 
it the default (LZ4 block mode is now lz4-noraw, to match zlib).

This implementation mostly works, but...

It's not backward compatible because we're storing more information in
the destination buffer than before (two 4B values instead of a one 8B
value). The 2.5.3 release is the only release that had LZ4 support, so
I don't expect this to be a problem. We could be backward compatible,
but not without a fair amount of pain.

This code sizes decompression buffers incorrectly. The WiredTiger btree
code sizes the decompression buffer based on the final in-memory size
of the data, which isn't correct for data compressed with LZ4's raw
compression function which may have compressed more data than the final
in-memory size.
 2015-05-03 11:51:21 -04:00
Keith Bostic
4119c22f29 Minor KNF, whitespace cleanup. 2015-05-03 11:25:15 -04:00
Keith Bostic
7342715b92 KNF 2015-05-02 13:25:12 -04:00
Keith Bostic
a93953b356 add missing () on return call. 2015-05-02 13:23:29 -04:00
Keith Bostic
1adec13515 Remove "wt_" prefix from local functions, they're not adding 
any additional information.
 2015-05-02 13:22:18 -04:00
Keith Bostic
7c00702d1f Move the find_slot function near the compress-raw function, its only 
caller.
 2015-05-02 13:20:13 -04:00
Keith Bostic
a6f9efd710 KNF, lint. 2015-04-29 16:27:29 -04:00