102 lines
3.7 KiB
Markdown
102 lines
3.7 KiB
Markdown
# btcli
|
|
|
|
An application intended for use with the BitTorrent protocol.
|
|
|
|
Implements bencode, `.torrent` parsing, HTTP tracker announces, the peer
|
|
wire protocol, and a piece downloader that verifies every piece before
|
|
writing it.
|
|
|
|
## Install
|
|
|
|
```sh
|
|
go install github.com/zg/btcli/cmd/btcli@latest
|
|
```
|
|
|
|
## Use
|
|
|
|
```sh
|
|
btcli file.torrent # download into the current directory
|
|
btcli -o ~/Downloads file.torrent
|
|
btcli -info file.torrent # show what the torrent describes
|
|
btcli -verify file.torrent # check data already on disk
|
|
```
|
|
|
|
```
|
|
$ btcli -info ubuntu.torrent
|
|
name: ubuntu-24.04.iso
|
|
info hash: 74a5f8a80e75a1b72995e0004f6928ce4d2f54a4
|
|
total size: 5.70 GiB
|
|
piece length: 512.00 KiB
|
|
pieces: 11673
|
|
trackers:
|
|
https://torrent.ubuntu.com/announce
|
|
```
|
|
|
|
## Packages
|
|
|
|
| package | role |
|
|
|---|---|
|
|
| `bencode` | the encoding used by torrents and trackers |
|
|
| `metainfo` | `.torrent` parsing and the info hash |
|
|
| `tracker` | HTTP announce and peer list parsing |
|
|
| `peer` | handshake, wire messages, bitfields |
|
|
| `download` | piece scheduling, verification, file writing |
|
|
|
|
## Design notes
|
|
|
|
**Bencode dictionary keys must be sorted, and it matters.** The info hash is
|
|
the SHA-1 of the *re-encoded* info dictionary, so an encoder that emits keys
|
|
in Go's randomised map order produces a different hash on every run — and
|
|
one no tracker or peer recognises. The encoder sorts; a test re-encodes the
|
|
same dictionary fifty times and asserts the bytes are identical. The decoder
|
|
also *rejects* unsorted keys and non-canonical integers (`i03e`, `i-0e`)
|
|
rather than accepting input it could not reproduce.
|
|
|
|
**A torrent is untrusted input.** A file path of `..` would let a torrent
|
|
write outside the download directory — the "zip slip" traversal. Path
|
|
components are rejected at parse time, and `FullPath` re-checks that the
|
|
resolved path stays under the root. Both have tests.
|
|
|
|
**So is a peer.** `ParsePiece` validates the index, offset and block length
|
|
against the piece actually requested before copying anything: an unchecked
|
|
copy into a fixed buffer is a straightforward overflow. Message lengths are
|
|
capped, because a peer can claim any length it likes and a 4 GiB claim would
|
|
otherwise force a 4 GiB allocation. A test feeds it wrong indexes, huge
|
|
offsets and overflowing blocks, and asserts the destination buffer is never
|
|
touched.
|
|
|
|
**Every piece is verified before it is written.** A peer can send anything;
|
|
without the SHA-1 check the file is silently corrupted. A test runs a
|
|
deliberately corrupt seeder and asserts the download never reports success.
|
|
|
|
**Requests are pipelined.** Up to five block requests stay in flight per
|
|
peer. One at a time would pay a full round trip per 16 KiB block, which
|
|
makes throughput latency-bound rather than bandwidth-bound.
|
|
|
|
**Work is returned to the queue, not lost.** A peer that lacks a piece, or
|
|
fails mid-transfer, puts the work back for another peer rather than
|
|
abandoning it.
|
|
|
|
**Multi-file torrents are one byte stream.** A piece can straddle a file
|
|
boundary, so the writer maps stream offsets onto files and splits a write
|
|
across two of them where needed. Getting this wrong corrupts both files at
|
|
the seam, so it has a dedicated test.
|
|
|
|
## Tests
|
|
|
|
The download path is tested end to end against a **real seeder over a real
|
|
socket** — handshake, bitfield, interest, pipelined requests, blocks, hash
|
|
verification and file writing all have to be correct for the test to pass.
|
|
Variants of that seeder serve corrupt data or stall entirely, to check that
|
|
verification rejects the first and timeouts handle the second.
|
|
|
|
```sh
|
|
go test -race ./...
|
|
```
|
|
|
|
## Not implemented
|
|
|
|
UDP trackers, DHT, peer exchange, magnet links, and uploading beyond
|
|
announcing `have`. Tracker support is HTTP/HTTPS only, and an unsupported
|
|
scheme is reported rather than silently skipped.
|