S3QL is a file system that stores all its data online. It supports Amazon S3, Google Storage, and OpenStack and effectively provides you with a hard disk of dynamic, infinite capacity that can be accessed from any computer with Internet access. S3QL provides a standard, full featured Unix file system that is conceptually indistinguishable from any local file system. Additional features include compression, encryption, data de-duplication, immutable trees, and snapshotting, which make it especially suitable for online backup and archiving. The design favors simplicity and elegance over performance and feature-creep. Care has been taken to make the source code as readable and serviceable as possible. Solid error detection, error handling, and extensive automated test cases are provided.
|Tags||FUSE Compression File system backup storage encrypted backup encryption de-duplication|
|Operating Systems||Linux FreeBSD Mac OS X|
Please note that S3QL has moved from Google Code to Bitbucket. The new project URL is https://bitbucket.org/nikratio/s3ql/overview
Release Notes: Another race condition that could lead to mount.s3ql crashing with `ValueError: I/O operation on closed file` was fixed. S3QL now stores multiple copies of the master encryption key to allow recovery if the backend loses the object holding the primary copy. A problem with automatic cache size detection was fixed (mount.s3ql was treating bytes as kilobytes). An "AttributeError in LegacyDecryptDecompressFilter" crash when reading objects written by old S3QL versions was fixed. A problem with umount.s3ql giving a strange error when the mountpoint is still in use was fixed.
Release Notes: Various changes were made.
Release Notes: Fixes a crash when statvfs() is called on an almost empty filesystem (which may happen automatically when using some desktop environments, resulting in an immediate crash after mounting an empty filesystem). Fixes another race condition that could lead to the (apparent) vanishing of the file system metadata when the mount.s3ql is killed at an unfortunate time.
Release Notes: This release removes a race condition which resulted in a crash when a file was deleted while the upload was in progress and the upload was then restarted because of a connection problem, and another race condition which could lead to the (apparent) vanishing of the filesystem metadata when the mount.s3ql was killed at an unfortunate time. Specifying bzip2 as the compression algorithm for mount.s3ql no longer gives an error. S3QL programs no longer create an empty file in the default log file location if a custom log file is specified with --log.
Release Notes: Attempts to upgrade a filesystem that is already at the newest revision no longer render the local metadata copy unusable. Google storage bucket names containing dots no longer cause problems with SSL certificate validation. Fixes a crash when statvfs() is called on an almost empty file system. There is a new "swiftks" backend for OpenStack/Swift servers. Eliminates a race condition that could lead to mount.s3ql crashing with a "KeyError in collections/abc.py". Fixes a crash that could occur when trying to clean up after receiving a connection or server error when uploading an object.