A new weighting scheme and discriminative approach for information retrieval in static and dynamic document collections

Ibrahim, Osman A. S. and Landa-Silva, Dario (2014) A new weighting scheme and discriminative approach for information retrieval in static and dynamic document collections. In: 14th UK Workshop on Computational Intelligence (UKCI2014), 8-10 September 2014, Bradford, West Yorkshire, UK.

[img]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (342kB) | Preview

Abstract

This paper introduces a new weighting scheme in information retrieval. It also proposes using the document centroid as a threshold for normalizing documents in a document collection. Document centroid normalization helps to achieve more effective information retrieval as it enables good discrimination between documents. In the context of a machine learning application, namely unsupervised document indexing and retrieval, we compared the effectiveness of the proposed weighting scheme to the 'Term Frequency - Inverse Document Frequency' or TF-IDF, which is commonly used and considered as one of the best existing weighting schemes. The paper shows how the document centroid is used to remove less significant weights from documents and how this helps to achieve better retrieval effectiveness. Most of the existing weighting schemes in information retrieval research assume that the whole document collection is static. The results presented in this paper show that the proposed weighting scheme can produce higher retrieval effectiveness compared with the TF-IDF weighting scheme, in both static and dynamic document collections. The results also show the variation in information retrieval effectiveness that is achieved for static and dynamic document collections by using a specific weighting scheme. This type of comparison has not been presented in the literature before.

Item Type: Conference or Workshop Item (Paper)
Additional Information: © 2014 IEEE. Published in: 2014 14th UK Workshop on Computational Intelligence (UKCI): Student Central Lecture Theatre SC0.51, University of Bradford, Bradford, West Yorkshire, UK, 8-10 September 2014 / editors Daniel Neagu, Mariam Kiran, Paul Trundle. [Piscataway, N.J.] :IEEE,c2014. ISBN 9781479955381.
Keywords: information retrieval
Schools/Departments: University of Nottingham UK Campus > Faculty of Science > School of Computer Science
Related URLs:
URLURL Type
UNSPECIFIEDOrganisation
Depositing User: Landa-Silva, Dario
Date Deposited: 22 Jan 2016 10:02
Last Modified: 15 Sep 2016 16:04
URI: http://eprints.nottingham.ac.uk/id/eprint/31329

Actions (Archive Staff Only)

Edit View Edit View