BSV - Backup, Synchronization, Versioning

Bsv is a tool to perform backups, file/directory synchronization between devices and light versioning. It's architecture is inspired from git, but focus on backup first, synchronization second and as a byproduct also allow versioning.

Bsv is in a very early stage of development and is not production ready yet.

Development setup

Bsv currently only supports Python >= 3.11. Once you have a compatible Python version, just fetch the code and install bsv in editable mode in a virtual env:

git clone https://git.draklia.net/draklaw/pybsv.git
cd pybsv
python -m venv venv
. venv/bin/activate
pip install -e .[test]

This will automatically install the dependencies (including pytest). Happy hacking !

Roadmap

v0.0.1 - Simple CAS + insert/remove files manually

Basic features. Naïve CAS implementation that store everything in a single big file with no option for removing objects. Supports a single "local" repository.

• Simple CAS implementation (it's OK if it's naïve).
• Content-based chunking to store files.
• bsv init command to initialize bsv.
• bsv info print useful information bsv configuration.
• bsv log show the history of snapshots.
• bsv show <digest> show the object digest.
• bsv ls <bsv-path> list files in a bsv directory.
• bsv mkdir <bsv-path> create a directory in bsv directly.
• bsv add [-r] <fs-path> <bsv-path> copy files from the filesystem to bsv.
• bsv get [-r] <bsv-path> <fs-path> copy files from bsv to the filesystem.
• bsv rm [-r] <bsv-path> create a directory in bsv directly.

v0.0.2 - File map + snapshots

Add support for mapping files from BSV virtual file system to the actual file system. Add snapshot and restore commands.

• bsv map list mappings between bsv paths and filesystem paths.
• bsv map add <bsv-path> <fs-path> add a mapping.
• bsv map remove <bsv-path> <fs-path> remove a mapping.
• bsv snapshot capture a snapshot, i.e. ensure that mapped files in the repositories match what is on the filesystem.
• bsv restore <snapshot> <fs-path> update files on the filesystem to the version captured by snapshot.

v0.0.3 - Multiple repository

Support multiple repository. Repository can be configured to store only metadata (typically for the local repository) or everything.

• Support repositories that store only metadata.
• bsv repo [-v] list all known repositories.
• bsv repo create <name> <destination> create a new repository.
• bsv repo add [<name>] <destination> add an already existing repository.
• bsv repo remove <name>|<destination> remove a repository.
• bsv fetch [<name>] fetch latest metadata from known repositories.
• bsv sync similar to snapshot + fetch + restore: Fetch latest changes from the repositories and update the filesystem to match. In case of conflict (file changed both in the repositories and locally), performs a snapshot first to ensure all conflicting versions are backed'up, then use some conflict-resolution strategy and warn the user.

v0.0.4 - Proper CAS

• Safe concurrent access (e.g. when several devices use a shared repository).
• Support removing objects.
• Garbage collection (remove unreferenced objects).
• Use garbage collection to keep metadata-only repository clean.

v0.0.5 - Some extra features

• bsv tag <name> [<snapshot>] [-m <message>] set/update a tag (an alias to a specific snapshot).
• Support for symlinks.

Later

• bsv watch starts a daemon that watch changes in mapped directories and automatically create snapshots.
• bsv http starts an http server that expose an API + an interface to manipulate BSV. Allow to list files, explore history, download and upload files...
• Bsv protocol + client/server
• Custom rules for repository to select what must be stored or not.
  • Create sensible rules for backup (keep a lot of recent versions, less for older versions).
• Add object set support (a kind of object that simply store a collection of objects). Can be used as tag.
• Add mail object ?