Opendedup is a project that develops a deduplication based filesystem for Windows and Linux called SDFS. SDFS is designed to support the unique needs of virtual environments and supports enhanced functionality for VMWare, Xen, and KVM. It can deduplicate a petabyte or more of data. It supports over 3TB per gigabyte of memory at a 128k chunk size. It can perform deduplication/reduplication at a line speed of 250 MB/s or more. VMware support: it works with vms, and can deduplicate at 4k block sizes. This is required to deduplicate Virtual Machines effectively Deduplicated data can be stored locally, on the network across multiple nodes, or in the cloud. The filesystem can deduplicate inline or periodically based on needs. This can be changed on the fly. There is support for file or folder level snapshots.
|Tags||Linux Compression volume management network attached storage storage Windows port Windows support|
|Operating Systems||Linux (64 Bit) Windows (32 bit)|
New SDFS performance numbers are out and have greatly improved in 1.0.5. SDFS now provides over 10 Gb/s for most work loads. Check it at http://ope...
Release Notes: This release fixes unique block deletion for filebasedchunkstore, volume freezing due to thread deadlock, and Azure bucket name not being honored. It enhances read performance by 40% under NFS through better locking techniques and file caching, and better file level locking under concurrent I/O should allow for lower latency multi write to the same file. It adds block caching for the S3, Azure, and FileChunkStore and defaults to 10MB. This should improve performance for multiple writes. This option can be configured during volume creation using the --chunk-store-read-cache option.
Release Notes: This version includes an updated Windows port, better garbage collection, and better performance. It also fixes replication, garbage collection, and threading bugs.
Release Notes: This release adds major enhancements to volume replication.
Release Notes: This release has many major enhancements including Azure deduplicated storage backend and better volume replication. It also includes murmur3 hashing, which can improve performance by up to 40%.
Release Notes: Fixes in this release focused on replication and snapshots. The ability to compact a volume offline was added.