From Fedora Project Wiki
Project Sponsor
Name: Mike McGrath
Fedora Account Name: mmcgrath
Group: Infrastructure
Infrastructure Sponsor: mmcgrath
Secondary Contact info
Name: Huzaifa Sidhpurwala
Fedora Account Name: huzaifas
Group: Infrastructure
Name: Allen Kistler
Fedora Account Name: akistler
Group: Infrastructure
Name: Keiran Smith
Fedora Account Name: affix
Group: Infrastructure
Project Info
Project Name: Search Engine Enhancement
Target Audience: All users of Fedora web sites
Expiration/Delivery Date (required): F13
Description/Summary: Fedora needs a search engine[1]
Requirements:
- Crawl the web sites (wiki and non-wiki)
- Search the web sites (wiki and non-wiki)
Preferences:
- Python-based (no Java)
- Programmable keywords to have control over what pages get displayed for certain keywords
- XML or library interface so other applications can use it
Project plan (Detailed):
- Investigate and evaluate existing open source search engines
- Select candidate software
- Create public test instance of candidate software
- Test for functionality, performance, and impact (re-evaluate, if necessary)
- Create capacity and deployment plans
- Deploy
Specific resources needed
- Public Test for testing candidate software
- Permanent home(s) for deployment
- Web server(s)
- Database server(s)
Software Investigation and Evaluation
- HtdigSearch [2]
- Huzaifa (in progress)
- SphinxSearch [3]
- Huzaifa (in progress)
- Ferret
- Ruby port of Lucene
- Gonzui [4] (specializes in source code search)
- written in Ruby
- not actively maintained
- KinoSearch
- Perl port of Lucene
- Namazu [5]
- written in Perl
- OpenFTS [6]
- Not suitable
- written in Perl or TCL on top of PostgreSQL
- Python interface available
- not actively maintained
- Plucene
- Perl port of Lucene
- DataparkSearch [7]
- Not suitable
- written in C
- Egothor [8]
- Not suitable
- written in Java
- Grub [9]
- Not suitable
- written in C#
- ht://dig [10]
- Not suitable
- written in C++
- not actively maintained
- Indri [11]
- Not suitable
- written in C/C++
- Isearch [12]
- Not suitable
- written in C++
- Lucene [13]
- Not suitable
- originally in Java, ported to others
- Perl ports are Plucene and KinoSearch; Ruby port is Ferret
- see Lucene Implementations [14]
- mnoGoSearch [15]
- Not suitable
- written in C
- MWSearch [16]
- Not suitable
- Requires EzMwLucene (Java, not desirable) to be running on the servers to be searched
- EzMwLucene is wiki-only, therefore MWSearch is wiki-only
- Nutch [17]
- Not suitable
- written in Java
- based on Lucene
- RigorousSearch [18]
- Not suitable
- Crawls the MediaWiki database, not the web site. It doesn't work for non-MediaWiki web sites, including any non-wiki web site.
- Sphinx [19]
- Not suitable
- written in C++
- Swish-e [20]
- Not suitable
- written in C
- Swish++ is a rewrite in C++
- Terrier (TERabyte RetrIEveR) [21]
- Not suitable
- written in Java
- Xapian [22]
- Not suitable
- written in C++
- YaCy [23]
- Not suitable
- written in C
- Zettair [24]
- Not suitable
- written in C
- ↑ "Fedora Search Engine". Infrastructure/Tickets. https://fedorahosted.org/fedora-infrastructure/ticket/1055.
- ↑ "HtdigSearch Extension". MediaWiki. https://secure.wikimedia.org/wikipedia/mediawiki/wiki/Extension:HtdigSearch.
- ↑ "SphinxSearch Extension". MediaWiki. https://secure.wikimedia.org/wikipedia/mediawiki/wiki/Extension:SphinxSearch.
- ↑ "Gonzui". SourceForge. http://gonzui.sourceforge.net/.
- ↑ "Namazu". Namazu Project. http://www.namazu.org/.
- ↑ "OpenFTS". SourceForge. http://openfts.sourceforge.net/.
- ↑ "DataparkSearch". DataparkSearch. http://www.dataparksearch.org/.
- ↑ "Egothor". Egothor. http://www.egothor.org/.
- ↑ "Grub". Wikia, Inc.. http://grub.org/.
- ↑ "ht://Dig". The ht://Dig Group. http://www.htdig.org/.
- ↑ "Indri". The Lemur Project. http://www.lemurproject.org/indri/.
- ↑ "Isearch". Isite. http://isite.awcubed.com/.
- ↑ "Lucene". Apache Software Foundation. http://lucene.apache.org/.
- ↑ "Lucene Implementations". Apache Software Foundation. http://wiki.apache.org/lucene-java/LuceneImplementations.
- ↑ "mnoGoSearch". LavTech. http://www.mnogosearch.org/.
- ↑ "MWSearch Extension". MediaWiki. https://secure.wikimedia.org/wikipedia/mediawiki/wiki/Extension:MWSearch.
- ↑ "Nutch". Apache Software Foundation. http://lucene.apache.org/nutch/.
- ↑ "RigorousSearch Extension". MediaWiki. https://secure.wikimedia.org/wikipedia/mediawiki/wiki/Extension:RigorousSearch.
- ↑ "Sphinx". Sphinx Technologies. http://sphinxsearch.com/.
- ↑ "Swish-e". Swish-e. http://swish-e.org/.
- ↑ "Terrier". Terrier Project. http://ir.dcs.gla.ac.uk/terrier/.
- ↑ "Xapian". Xapian Project. http://xapian.org/.
- ↑ "YaCy". Karlsruhe Institute of Technology. http://yacy.net/.
- ↑ "Zettair". Search Engine Group, Royal Melbourne Institute of Technology. http://www.seg.rmit.edu.au/zettair/.
Public Testing
<tbd>
Deployment Plan
<tbd>