Projects / S3QL

S3QL

S3QL is a file system that stores all its data online. It supports Amazon S3, Google Storage, and OpenStack and effectively provides you with a hard disk of dynamic, infinite capacity that can be accessed from any computer with Internet access. S3QL provides a standard, full featured Unix file system that is conceptually indistinguishable from any local file system. Additional features include compression, encryption, data de-duplication, immutable trees, and snapshotting, which make it especially suitable for online backup and archiving. The design favors simplicity and elegance over performance and feature-creep. Care has been taken to make the source code as readable and serviceable as possible. Solid error detection, error handling, and extensive automated test cases are provided.

Tags
Licenses
Operating Systems
Implementation

RSS Last announcement

Moved to BitBucket 05 Jan 2014

Please note that S3QL has moved from Google Code to Bitbucket. The new project URL is https://bitbucket.org/nikratio/s3ql/overview

RSS Recent releases

  •  30 Mar 2014 16:50

    Release Notes: There are no changes in S3QL itself. The S3QL 2.8 tarball accidentally included a copy of the Python dugong module; this has been fixed.

    •  15 Mar 2014 02:17

      Release Notes: Fixes various problems with using a proxy server. Sending SIGUSR1 to mount.s3ql now generates a stack trace (debugging feature). When passing --installed to the test runner, S3QL commands are now loaded from $PATH instead of the packages bin/ directory. The runtest.py script now comes with the correct shebang (i.e. it can now be called as "./runtests.py" instead of "python3 runtests.py"). S3QL now requires the Python Dugong module. A filesystem hang when all upload threads encountered unexpected backend problems has been fixed.

      •  17 Dec 2013 11:34

        Release Notes: Another race condition that could lead to mount.s3ql crashing with `ValueError: I/O operation on closed file` was fixed. S3QL now stores multiple copies of the master encryption key to allow recovery if the backend loses the object holding the primary copy. A problem with automatic cache size detection was fixed (mount.s3ql was treating bytes as kilobytes). An "AttributeError in LegacyDecryptDecompressFilter" crash when reading objects written by old S3QL versions was fixed. A problem with umount.s3ql giving a strange error when the mountpoint is still in use was fixed.

        •  05 Dec 2013 00:55

          Release Notes: Various changes were made.

          •  27 Oct 2013 22:40

            Release Notes: Fixes a crash when statvfs() is called on an almost empty filesystem (which may happen automatically when using some desktop environments, resulting in an immediate crash after mounting an empty filesystem). Fixes another race condition that could lead to the (apparent) vanishing of the file system metadata when the mount.s3ql is killed at an unfortunate time.

            Screenshot

            Project Spotlight

            prodatum

            A patch editor for the E-MU Proteus 2000 sample-based synthesizer family.

            Screenshot

            Project Spotlight

            TurnKey Ruby On Rails Appliance

            A Ruby on Rails appliance that is easy to use and lightweight.