Project 2: Raft Revisions and Replicated Shell

This project has two components. First, you will revise the code you wrote for Project 1 based on the feedback you received in code reviews. Second, you will add a persistent log to your Raft implementation and use the replicated log to build a simple replicated state machine that executes shell commands.

Code revisions

Modify and refactor your Raft implementation based on the feedback you received from code reviews. I expect you to fix all of the issues that I raised in my code review (check with me if you don't think it is possible or reasonable to deal with some of my requests). My overall review comments usually include a few larger tasks as well. You should also incorporate most or all of the suggestions you received from other students. If you get conflicting comments from different reviewers, you can decide how to handle them (but check with me before going against any of my suggestions). You will need to make many small changes, such as improving comments, but most of you will also need to make significant changes to some of your interfaces, which will have more global ramifications.

Your overall goal should be to make your code as clean and simple as possible. Now that you've gotten feedback and had a chance to hear how other groups solved the same problems, you should have lots of ideas for how to improve your design. For example, you should refactor your abstractions to provide the simplest possible interfaces to your classes, with the best possible information hiding. Also, make your code as easy to read as possible; to do this you will need good documentation as well as a clean class structure. For this project, it is fine to use ideas from other projects.

Before refactoring your existing code, I recommend doing at least a rough design for the new features described below: their needs may suggest additional changes to make in the structure of your code.

Create a file changes in the top-level directory of your repository, which briefly describes the most significant ways in which you improved your Project 1 code for Project 2 (what you changed, at a high level, and why you did it). This file can be relatively short (10-20 lines): it's intended to make it easier for reviewers to find the most important changes.

Replicated shell

In addition to revising your code based on the Project 1 reviews, you will also add new features to your Raft implementation; this will give you an opportunity to apply some of the design lessons you learned during Project 1. In this project you will create a simple replicated shell: when a user types shell commands in a client application, they will be forwarded to the Raft state machine, replicated in all of the logs, and then executed as shell commands on each of the Raft servers. The new code for this project must include the following:

A mechanism for storing log entries in one or more files. The contents of each log entry will be a string of any length. In addition to the string, you must also store a term number with each log entry. You should assume that logs could grow too large to store entirely in memory. Your design must be efficient enough to support update rates of 100-1000 new entries per second (for example, it is not OK to rewrite the entire log every time a new entry is added). You may not use an existing database system or storage system for this: you must build your functionality on the basic C/C++ file I/O facilities such as fopen, std::iostream, or std::fstream. If you are not sure whether it's OK to use a particular library in your implementation, check with me.
Leaders must replicate their log entries to followers as described in the Raft paper. All log entries must be safely stored in a file before a response is generated for an AppendEntries request. Use log information to implement the election restriction described in Section 5.4.1 of the Raft paper. Your mechanism for log replication must be efficient enough to support a fairly high volume of log messages.
Leaders must determine when log entries are committed, and they must propagate commitIndex to followers as described in the Raft paper.
Connect the command execution mechanism with the log to implement a replicated shell. Before the leader executes a shell command, it must add the command to its log as a new entry and replicate that log entry on the other servers. Once a command has been committed, the leader can execute it and return the result to the client. Followers must also execute shell commands once they have been committed. The leader must be able to handle multiple commands that have been received but not yet executed. Log entries must be executed in order according to their log indices, and only one command should execute at a time.
Each server must store persistent information about which log entries it has executed. When a server restarts after a crash, it must execute any committed entries that were not executed before the crash, but it must not reexecute commands that already executed. If a server crashes while in the middle of executing a command (or before recording its completion), you may assume that the command did not execute at all (dealing with this situation properly is part of linearizability, which is beyond the scope of this project).

Additional Notes and Requirements

This project must be implemented in C++ using the "-Wall -Werror" compiler options.
Before starting on Project 2, merge your project1 branch back into the master branch of your repository. You can do this by closing the pull request for Project 1 (select the "Merge commit" option). Do not delete the Project 1 pull request; just close it.
Create a new branch, project2, and do all of your work for this project on that branch.
For the main class that implements your log storage, create an initial version of the file that contains top-level declarations and interface comments, but no method bodies. Commit this version of the file as a separate commit on the project2 branch, and tag that commit commentsBeforeCode2. Make sure that the message for this commit also includes the name of the file. I recommend that you write comments before code for all your files, but I will only require it for this one file. To add a tag to the current commit, invoke the following commands:
```
git tag -a commentsBeforeCode2
git push origin commentsBeforeCode2
```
You can test your replicated shell using simple shell commands such as "echo foo", "echo foo bar > test.out" and "cat test.out". Since each server runs in a different working directory, you should be able to see a separate copy of the file appear in each server's directory.
You do not need to implement the mechanisms described in the Raft paper for cluster membership changes, log compaction, snapshots, or full linearizability. The replicated shell you implement will not be a perfect replicated state machine: some shell commands, such as "pwd" or "hostname", may behave differently on different servers, and it is possible for a shell command to get executed multiple times if a server crashes at the wrong time.

Submitting Your Project

To submit your project, push all of your changes to GitHub on the project2 branch and then create a pull request. The base for the pull request should be your master branch (which now contains all of your work up through Project 1) and the comparison branch should be the head of your project2 branch. Use "Project 2" as the title for your pull request. If your project is not completely functional at the time you submit, describe what is and isn't working in the comments for the pull request. Don't forget to include a changes file as described above.

Late Days

If you are planning to use late days for this project (or any project) please send me an email before the project deadline so that I know how many late days you plan to use.