MongoDB - Replication and rollback
One of the test exams to get the MongoDB Developer certification was about the Replication. The scenary was:
Three node running with a delay of two hours.
- Node 1 (Primary): Goes down for an hour, after a write operation is initiated.
- Node 2 (Secondary): Is elected primary when Node 1 goes down. Node 2 doesn't have the write operation, because write operation has not yet returned at the time of the failure.
- Node 3 (Secondary): is lagging
Note: All writes to the database are issued with w=majority and j=1.
Well...the question was: "Will there be a rollback of data on Node 1 when it comes back up?"
How we read from the oficial mongoDB documentation
"A rollback reverts write operations on a former primary when the member rejoins its replica set after a failover. A rollback is necessary only if the primary had accepted write operations that the secondaries had not successfully replicated before the primary stepped down. When the primary rejoins the set as a secondary, it reverts, or “rolls back,” its write operations to maintain database consistency with the other members.
MongoDB attempts to avoid rollbacks, which should be rare. When a rollback does occur, it is often the result of a network partition. Secondaries that can not keep up with the throughput of operations on the former primary, increase the size and impact of the rollback.
A rollback does not occur if the write operations replicate to another member of the replica set before the primary steps down and if that member remains available and accessible to a majority of the replica set."
So the answer is that it depends on whether Node 2 has processed the write.
Note: A mongod instance will not rollback more than 300 megabytes of data. If your system must rollback more than 300 megabytes, you must manually intervene to recover the data. If this is the case, the following line will appear in your mongod log:
[replica set sync] replSet syncThread: 13410 replSet too much data to roll back
Finally, this is an excellent post that explains how to implement step by step the replication.