Type

Conference Proceedings

Authors

Alan F. Smeaton
Neil O'Hare
Paul Ferguson
James Lanagan

Subjects

Computer Science

Topics
computer software social networks blogging artificial intelligence information retrieval interactive computer systems noise removal blogs

Coping with noise in a real-world weblog crawler and retrieval system (2010)

Abstract In this paper we examine the effects of noise when creating a real-world weblog corpus for information retrieval. We focus on the DiffPost (Lee et al. 2008) approach to noise removal from blog pages, examining the difficulties encountered when crawling the blogosphere during the creation of a real-world corpus of blog pages. We introduce and evaluate a number of enhancements to the original DiffPost approach in order to increase the robustness of the algorithm. We then extend DiffPost by looking at the anchor-text to text ratio, and dis- cover that the time-interval between crawls is more impor- tant to the successful application of noise-removal algorithms within the blog context, than any additional improvements to the removal algorithm itself.
Collections Ireland -> Dublin City University -> Status = Published
Ireland -> Dublin City University -> DCU Faculties and Centres = Research Initiatives and Centres
Ireland -> Dublin City University -> Publication Type = Conference or Workshop Item
Ireland -> Dublin City University -> DCU Faculties and Centres = DCU Faculties and Schools: Faculty of Engineering and Computing: School of Computing
Ireland -> Dublin City University -> Subject = Computer Science: Interactive computer systems
Ireland -> Dublin City University -> Subject = Computer Science: Artificial intelligence
Ireland -> Dublin City University -> Subject = Computer Science
Ireland -> Dublin City University -> DCU Faculties and Centres = DCU Faculties and Schools
Ireland -> Dublin City University -> DCU Faculties and Centres = Research Initiatives and Centres: CLARITY: The Centre for Sensor Web Technologies
Ireland -> Dublin City University -> Subject = Computer Science: Information retrieval
Ireland -> Dublin City University -> DCU Faculties and Centres = DCU Faculties and Schools: Faculty of Engineering and Computing
Ireland -> Dublin City University -> Subject = Computer Science: Computer software

Full list of authors on original publication

Alan F. Smeaton, Neil O'Hare, Paul Ferguson, James Lanagan

Experts in our system

1
Alan F. Smeaton
Dublin City University
Total Publications: 492