Welcome to the blog of Can Sar, a Stanford CS major. This blog is made up of my thoughts on Computer Science and the computer industry, as well as ever exciting tales from my life.
Walking back from ISIS yesterday I realized the importance of Mazières LBFS paper to my research. The LBFS associates hashes with file chunks, and before a client requests a chunk over the network, it asks the server for the hash of that chunk. If the client realizes it already has this sector (since many files share chunks), it is done and has saved bandwidth, otherwise it requests the sector, thereby taking twice as much roundtrip time.
By selectively using this scheme for some pages/sectors and not using it for others, we can trade latency for bandwidth to some degree. This seems very important for transfer of both memory and files. One case this would probably improve significantly is the time when a program is loaded, where bandwidth is the biggest problem, since we are trying to transfer many pages as quickly as possible. On the other hand, when communicating between remote processes, latency is of course far more important, since we are trying to minimize the time processes are waiting for others rather than doing useful work.
I think that this will be a very interesting technique, that I will strongly consider while thinking about the design of my system.
I also just saw that the source to LBFS is available here, which might be very interesting. I will of course not be implementing anything of this sort for my project, since it would be far too much work that I would rather devote to other aspects of the project. I am also wondering how exactly, IP issues with GPL code works. If I read it and then go on to write a related system in the future, but never consult the actual code in the future, would this be objectionable? From my understanding, and some of the issues that have come up in the recent fatherhood of Linux discussions, I think that it is completely OK, but I will inquire about this in the future.
(References)