This week at work we’ve been experimenting with MogileFS (link) which is a “filing system” capable of storing/retrieving many files across many hosts. It’s not a true file system, meaning that you can’t mount it and look at it with “ls” or “cat” files, etc. but using a custom perl module you can put files in and take them back out.

Some interesting facts about it: It’s based on HTTP GET/PUT so I believe it would be a good complement to other systems we have at work (without going into too much detail, it’s no surprise to anyone that Shutterfly receives lots of picture files from users and stores them for later printing :) Also, the MogileFS tracker module takes care of ensuring that files are replicated like you want; for example, if you want 2 copies of each file living on different nodes, after the initial upload the file will be duplicated appropriately. In the event of failure of one of the nodes, other nodes that hold the same data as the lost one will duplicate the items again to ensure replication is maintained.
Rest of this is probably boring to many