Is there something like Syncthing that works like a RAID system? I'd like to use multiple computers in different locations and use them to backup and expand my storage size.
Basically redundant but distributed storage with N node fail tolerance. Preferably interfaceable via a "/mnt/cloud" directory so I can use it from my laptop and easily write backup scripts and store data into it.
If this existed I'd have something big to work on setting up next week.
This is an interesting idea, but I don't think it could work the way you are describing it (like RAID). Remember the limitations of RAID, and then add intermittent failures and huge latency to the communication between the "drives". It would not be able to remain coherent in real time.
Now, I think you could definitely set up a system that could handle the distributed data like this and allow you to access it, but I am skeptical that you could construct such a system that would be able to read/write/update/delete files on the fly and retain N-node failure tolerance at all times. You would probably have to allow the system to replicate data slowly.
There is also the problem of network splits. When they rejoin, how do they resolve conflicts and such?
into the filesystem and another server would be reading through everything and processing it (not in real time, but as a batch system).
It's ok for things to be left over for collection at the next cycle.
Also netsplits aren't much of an issue if you're largely just doing writes and reads on seperate files. It only becomes a problem for short-term and currently processed files (that are being read and written to in real time).
This is more of a long-term cold storage system and not nodes talking to each other.
TL;DR
"It's not a bug it's a feature"
Patient: "Doctor, how do I stop the pain I get from doing this"
Doctor: "You stop doing that"
You set up one introducer, where storage nodes and clients meet and exchange each other's address. After that your clients can use the storage as one big single, end-to-end encrypted, resilient space where N nodes out of M can fail without any effect on your data.
Is it possible to mount Tahoe-LAFS as a file system? If it is, then it's exactly what I'm looking for.
Edit: Also, can you say "this group of nodes count's as a single location" for failure protection? So if I have two locations where I store servers and then 10 other one off data collection locations can I say "I want you to treat these datacenters as one node since they are very likely to fail together if they fail".
I read the FAQ and this is apperently an asked question! I'm not surprised because I think many people are thinking of doing the type of thing I want to do.
Here it is directly from the Q&A...
" Q12: If I had 3 locations each with 5 storage nodes, could I configure the grid to ensure a file is written to each location so that I could handle all servers at a particular location going down? "
" A: Not directly. We have a wiki page and some tickets (linked from the wiki page) about this but it's deeper than it looks and we haven't come to a conclusion on how to build it.
The current system will try to distribute the shares as widely as possible, using a different pseudo-random permutation for each file, but it is completely unaware of server properties like "location". If you have more free servers than shares, it will only put one share on any given server, but you might wind up with more shares in one location than the others.
For example, if you have 15 servers in three locations A:1/2/3/4/5, B:6/7/8/9/10, C:11/12/13/14/15, and use the default 3-of-10 encoding, your worst case is winding up with shares on 1/2/3/4/5/6/7/8/9/10, and not use location C at all. The most likely case is that you'll wind up with 3 or 4 shares in each location, but there's nothing in the system to enforce that: it's just shuffling all the servers into a ring, starting at 0, and assigning shares to servers around and around the ring until all the shares have a home.
The possible distributions of shares into locations (A, B, C) are:
It's not exactly a filesystem, but git-annex allows you to configure multiple nodes, and to defined the minimum number of copies each file should have (globally or per file type). Using the daemon (assistant), the nodes will automatically copy files from other nodes until the number is reached.
That seems to be focused on throughput. I'm focused on how much data I can store into the system and if it's redundant so long as N nodes don't die (and if they do I'd like it to attempt to reorganize the network so they system keeps working with N node failure).
I don' t think so. My experience with it is non-existent, but I read up on it before and now again, and self-healing and replication was pretty high on the feature list. Anyway, what you describes is a distributed/clustered (fault-tolerant) file system. Tahoe-LAFS as mentioned by jabl is one of those, https://en.wikipedia.org/wiki/List_of_file_systems#Distribut... has a list.
Basically redundant but distributed storage with N node fail tolerance. Preferably interfaceable via a "/mnt/cloud" directory so I can use it from my laptop and easily write backup scripts and store data into it.
If this existed I'd have something big to work on setting up next week.