S3QL
S3QL is a file system that stores all its data online. It supports Amazon S3, Google Storage, and OpenStack and effectively provides you with a hard disk of dynamic, infinite capacity that can be accessed from any computer with Internet access. S3QL provides a standard, full featured Unix file system that is conceptually indistinguishable from any local file system. Additional features include compression, encryption, data de-duplication, immutable trees, and snapshotting, which make it especially suitable for online backup and archiving. The design favors simplicity and elegance over performance and feature-creep. Care has been taken to make the source code as readable and serviceable as possible. Solid error detection, error handling, and extensive automated test cases are provided.
| Tags | FUSE Compression File system backup storage encrypted backup encryption de-duplication |
|---|---|
| Licenses | GPLv3 |
| Operating Systems | Linux |
| Implementation | C Python |
Recent releases
- 10 Apr 2013 22:24
Release Notes: Reduced CPU consumption when S3QL is idle. This release automatically retries requests when S3 reports an "internal error" (this functionality was accidentally broken in 1.13). It fixes a hang when using s3qlrm to delete a directory with a very large number of sub-directories.
- 24 Feb 2013 19:42
Release Notes: This release fixes a bug that caused malformed metadata to be written out when using recent eglibc versions. It fixes issues when trying to access a directory at the root of the filesystem that has recently been removed with s3qlrm. Ostensible ACL support has been removed. This release better handles buckets with invalid DNS names. It will retry if DNS is not available.
- 04 Sep 2012 14:32
Release Notes: The file system is now more verbose about how to remedy the situation when attempting to upgrade file systems needing fsck. fsck now detects and fixes directory entries which do not have an entry type stored in their mode field. There were various small bugfixes and code improvements.
- 04 May 2012 20:44
Release Notes: This release fixes a crash when using the S3 backend (regression introduced in release 1.11), increases the minimum reported filesystem size to 1 TB, and works around a df bug so size is reported properly.
- 29 Apr 2012 23:26
Release Notes: Metadata about stored objects is now split into multiple fields, so no field is longer than 256 bytes. (This makes S3QL compatible with a wider range of storage providers.) The SWIFT backend now retries if it receives 5xx error codes from the remote server, and no longer fails when attempting to upload data with an expired auth token.
Nikratio
24 Jul 2010 16:37
Heartbeat
- Popularity Score
- 529.27
- Vitality Score
- 26.69
- Subscriptions
- 47
