The updates_in_heterogeneous_repl_set.js test wants to call stepUp()
on each of its replica set members to ensure that each one gets an
opportunity to handle inserts and updates. However, we found that the
stepUp()-triggered election was occasionally failing, because even
though a call to awaitReplication() ensured that all members were
caught up on their oplogs, some members were not aware that their
peers were caught up.
The new awaitNodesAgreeOnOpTime() function uses an assert.soon() to
wait until all the members are caught up on their peers' status, so
that the election always succeeds.
With the new UpdateNodes class hierarchy, there are two code paths for
applying an update to a document that have slightly different
semantics. The order of fields in the resulting document can vary
depending on which code path is used to apply an update. A difference
in ordering between documents in a replica set is considered a
"mismatch," so we need to ensure that secondaries always apply updates
using the same update system that the primary uses.
When an update executes as part of the application of an oplog entry,
the update is now allowed to have a $v field, which allows it to
specify which semantics were used by the operation that we are
replicating by applying the entry. When the primary uses the new
semantics (because it is a 3.6 mongod with featureCompatibilityVersion
set to 3.6), it includes {$v: 1} in the oplog's update document to
indicate that the secondary should apply with the newer 'UpdateNode'
semantics.
There are two other places where we need this behavior:
1) In role_graph_update.cpp, where the handleOplogUpdate observer
needs to update its in-memory BSON representation of a role to
reflect an update in the admin database and
2) in the applyOps command, which is used for testing how oplog
entries get applied.
Both these code paths set the fromOplogApplication flag, which
replaces the old fromReplication flag, and they also gain behavior
that used to be exclusive to oplog applications from
replication. (Specifically, they skip update validation checks, which
should have already passed before the oplog entry was created.)
My original implementation of the $jsonSchema ban included a Status
error messaged that assumed the only reason to disallow $jsonSchema is
because of a featureCompatibilityVersion < 3.6. That assumption will
not be true, so this patch replaces that error with a generic
error. Code paths that include the $jsonSchema check because of the
featureCompatibilityVersion now rewrites this generic error with an
error that includes a link to the relevant documentation.
There is a ticket open to enable $jsonSchema in more places:
SERVER-30987.
If a user updates a 3.4 node to 3.6 but keeps the feature
compatibility version at 3.4, it should remain possible to be in a
replica set with 3.4 nodes or even to downgrade back to 3.4. To that
end, we ensure that is not possible to create a collection validator
with a $jsonSchema expression unless the feature compatibility version
is 3.6. A $jsonSchema validator would not replicate correctly to a 3.4
node, and its existence in the database would prevent a downgrade to
3.4.
This patch ensures collections have UUIDs when
featureCompatibilityVersion is 3.6 and collections do not have UUIDs
when featureCompatibilityVersion is 3.4 (or below).
Unrevert 4b222edf45 and
Fix random base64 generation in test helper
The good news is that the changes to base64::decode()
did exactly what they were intended to do.
The bad news is that data_generators.js was producing
invalid base64 sequences when (length % 4) == 1.
When deleting the donor range after migrating a chunk off of a shard,
the range deleter will schedule the deletion at some time in the future,
according to a server parameter orphanCleanupDelaySecs, which defaults
to 900, or 15 minutes. It does not delay range deletions preparatory to
migrating a range in, and does not put off deleting the donor range if
the moveChunk command has set the option _waitForDelete.
The file jstests/sharding/write_commands_sharding_state.js had CR (0x0D)
line endings, which made the patch fail lint. The substantive changes
in the file were to add "_waitForDelete" options to the moveChunk
commands.
