35d ago
So is there a server program to partner this with? Something that acts as a torrent file builder, tracker and simple file server for the torrent? I can imagine in a large org you could store a gigantic quantity of public data on a server that creates a torrent whenever the data changes, serves the.torrent file over http and also acts as a tracker. You could wrap the FUSE client in some basic logic that detects newer torrents on the server and reloads/remounts.

Many moons ago I created a Linux distribution for a bank. It was based on Ubuntu NetBoot with minimal packages for their branch desktop. As the branches were serverless, the distro was self-seeding. You could walk into a building with one of them and use it to build hundreds of clones in a pretty short time. All you needed was wake-on-lan and PXE configured on the switches. The new clones could also act as seeds. Under the hood it served a custom Ubuntu repo on nginx and ran tftp/inetd and wackamole (which used libspread, neither have been maintained for years). Once a machine got built, it pulled a torrent off the "server" and added it to transmission. Once that was completed the machine could also act as a seed, so it would start up wackamole, inetd, nginx, tracker etc. At first you seed 10 machines reliably, but once they were all up, you could wake machines in greater numbers. Across hundreds of bank branches I deployed the image onto 8000 machines in a few weeks (most of the delays due to change control and staged rollout plan). Actually the hardest part was getting the first seed downloaded to the branches via their old Linux build, and using one of them to act as a seed machine. That was done in 350+ branches, over inadequate network connections (some were 256kbps ISDN)

This tool should be upgrade to use Bittorrent v2 new functions.

Especially merkle hash trees which enable :

- per-file hash trees - directory structure

I dream of having a BTFS that will fix my "damaged" media files. E.g. ones I media shift, if my disk was scratched and portions are missing, or if the codec options I picked suck, it could download the "damaged" portions of my media and fix it seamlessly.
Submitting because I'm surprised why this isn't used more... couldn't we build a virtualmachine/OS's as an overlay on BTFS? Seems like an interesting direction.
This is the perfect client for accessing Internet Archive content! Each IA item automatically has a torrent that has IA's web seeds. Try Big Buck Bunny:

btfs mountpoint


BTFS – mount any .torrent file or magnet link as directory - - June 2020 (121 comments)

BitTorrent file system - - Jan 2016 (33 comments)

Or even better store data as an sqlite file that is full-text-search indexed. Then you can full-text search the torrent on demand:
the top comment <> by saurik (yes, that one) on the previous 121 comment thread back in 2020 sums up my feelings about the situation: BTFS is a "one CID at a time" version of IPFS

I do think IPFS is awesome, but is going to take some major advances in at least 3 areas before it becomes something usable day-to-day:

1. not running a local node proxy (I hear that Brave has some built-in WebTorrent support, so maybe that's the path, but since I don't use Brave I can't say whether they are "WebTorrent in name only" or what

2. related to that, the swarm/peer resolution latency suffers in the same way that "web3 crypto tomfoolery" does, and that latency makes "browsing" feel like the old 14.4k modem days

3. IPFS is absolutely fantastic for infrequently changing but super popular content, e.g. wikipedia, game releases, MDN content, etc, but is a super PITA to replace "tip" or "main" (if one thinks of browsing a git repo) with the "updated" version since (to the best of my knowledge) the only way folks have to resolve that newest CID is IPNS and DNS is never, ever going to be a "well, that's a good mechanism and surely doesn't contribute to one of the N things any outage always involves"

I'm aware that I have spent an inordinate amount of words talking about a filesystem other than the one you submitted, but unlike BTFS, which I would never install, I think that those who click on this and are interested in the idea of BTFS may enjoy reading further into IPFS, but should bear in mind my opinion of its current shortcomings

anacrolix has a fuse driver since 2013. I'm in the early stages of removing it. There are WebDAV, 3rd party FUSE, and HTTP wrappers of the client all doing similar things: serving magnet links, infohashes, and torrent files like an immutable filesystem. BitTorrent v2 support is currently in master.
Pretty cool on the surface, but 2 things I can think of right now that mostly kills it for me (and maybe many others). 1) the things one usually use Bittorrent for tend to need the complete files to be all there to be useful, and internet speeds is a limiter in that regard. 2) seeder count tends to break down pretty quickly, as people delete or even just move the files elsewhere for whatever reason, so availability falls.
I found it might be quite useful for huge LLMs since those are now hosted on BitTorrent. Of course it is not going to be as practical as IPFS since IPFS is content addressable and easier to do random access.
Couldn't help but think it would be epic if it went the other way - you throw files into a folder, it's set as read-only and then share that as a torrent.
Here's a similar idea for block devices:

It lets you create a block device (/dev/nbd0) backed by a torrent. Blocks are fetched in the background, or on demand (by prioritizing blocks according to recent read requests).

In practice it works - you can even boot a VM from it - but it's quite slow unless you have lots of seeds. There's a danger, particularly with VMs, that you can hit time outs waiting for a block to be read, unless you adjust some guest kernel parameters.

There are some bootable examples in that page if you want to try it.

I've thought about using this for my media server in the past, but in the end I ran into to many issues trying to automate it. Then there's all the normal issues, slow downloads can wreck havoc on some programs expecting the whole thing to be there and I couldn't move files around without breaking connections. It was interesting to mess with, but in the end I just decided it would be a fun challenge to write my own in zig so I could have something "easy" to hack on in the future.
Cool concept. I assume that it seeds if and while the files are present on your device? Tried to read the manpage but unformatted manpage markdown on a phone was too difficult to read.
I did something similar some years ago,

I had no idea what I was doing, most of the hard work IS done by the torrent-stream node package

For a long time I've been thinking there should be a P2P alternative to the snap package store. This looks like it's already got 90% of the work on the technical side to implement it.
No security at all?
so is this like a Dropbox alternative using the bittorrent protocol?