From Fedora Project Wiki
No edit summary
No edit summary
Line 1: Line 1:
{{admon/important | Comments and Explanations | The page source contains comments providing guidance to fill out each section.  They are invisible when viewing this page.  To read it, choose the "edit" link.<br/> '''Copy the source to a ''new page'' before making changes!  DO NOT EDIT THIS TEMPLATE FOR YOUR FEATURE.'''}}
{{admon/important | Set a Page Watch| Make sure you click ''watch'' on your new page so that you are notified of changes to it by others, including the Feature Wrangler}}
{{admon/note | All sections of this template are required for review by FESCo.  If any sections are empty it will not be reviewed }}
{{admon/note | All sections of this template are required for review by FESCo.  If any sections are empty it will not be reviewed }}
<!-- All fields on this form are required to be accepted by FESCo.
We also request that you maintain the same order of sections so that all of the feature pages are uniform.  -->
<!-- The actual name of your feature page should look something like: Features/YourFeatureName.  This keeps all features in the same namespace -->


= Feature Name =
= Feature Name =
Line 18: Line 8:


== Owner ==
== Owner ==
<!--This should link to your home wiki page so we know who you are-->
* Name: [[User:drunkahol| Duncan Innes]]
* Name: [[User:drunkahol| Duncan Innes]]


Line 31: Line 20:


== Detailed Description ==
== Detailed Description ==
<!-- Expand on the summary, if appropriate. A couple sentences suffices to explain the goal, but the more details you can provide the better. -->
Data deduplication is often used for backup purposes and for virtual machine image storage. lessfs can determine if data is redundant by calculating an unique (192 bit) tiger hash of each block of data that is written. When lessfs has determined that a block of data needs to be stored it first compresses the block with LZO or QUICKLZ compression. The combination of these two techniques results in a very high overall compression rate for many types of data. Multimedia files like mp3, avi or jpg files can not be compressed by lessfs when they are only stored once on the filesystem.
 
http://www.lessfs.com/wordpress/?page_id=50


== Benefit to Fedora ==
== Benefit to Fedora ==
Line 64: Line 55:


== Documentation ==
== Documentation ==
<!-- Is there upstream documentation on this feature, or notes you have written yourself?  Link to that material here so other interested developers can get involved. -->
* http://www.lessfs.com/wordpress/
* http://www.lessfs.com/wordpress/


Line 73: Line 63:


== Comments and Discussion ==
== Comments and Discussion ==
* See [[Talk:Features/YourFeatureName]] <!-- This adds a link to the "discussion" tab associated with your page.  This provides the ability to have ongoing comments or conversation without bogging down the main feature page -->
* See [[Talk:Features/LessFS]]





Revision as of 13:55, 12 November 2010

Note.png
All sections of this template are required for review by FESCo. If any sections are empty it will not be reviewed

Feature Name

LessFS

Summary

LessFS is a filesystem deduplication project. The aim is to reduce disk usage where filesystem blocks are identical by only storing 1 block and using pointers to the original block for copies. This method of storage is becoming popular in Enterprise solutions for reducing disk backups and minimising virtual machine storage in particular.

Owner

  • Email: duncan AT innes DOT net

Current status

  • Targeted release: ?
  • Last updated: 2010-11-12
  • Percentage of completion: 0%


Detailed Description

Data deduplication is often used for backup purposes and for virtual machine image storage. lessfs can determine if data is redundant by calculating an unique (192 bit) tiger hash of each block of data that is written. When lessfs has determined that a block of data needs to be stored it first compresses the block with LZO or QUICKLZ compression. The combination of these two techniques results in a very high overall compression rate for many types of data. Multimedia files like mp3, avi or jpg files can not be compressed by lessfs when they are only stored once on the filesystem.

http://www.lessfs.com/wordpress/?page_id=50

Benefit to Fedora

This will bring an as yet unavailable enterprise tool to Fedora. Storage is becoming the biggest consumer of energy in the datacentre. De-duplication will help bring that power and cost requirement down.

Scope

Attempts have been made to package as an RPM but seem to have stalled.

How To Test

User Experience

De-duplication will be noticeable to target users by greatly reducing the disk space requirements for backups to disk and for virtual machine storage. Greater reductions are seen where many images/backups share a common data set.

Dependencies

Contingency Plan

None necessary - this is a new feature and does not change any current part of Fedora

Documentation

Release Notes

Comments and Discussion