DESIGN AND IMPLEMENTATION OF A QUESTION ANSWERING SYSTEM USING CLASSIFICATION BASED MODEL

Abstract

Identification and classification of question on an academic question answering site has been a crucial task because the entire answer extraction process relies on finding the correct question type and hence the correct answer type. In this thesis, a model for identifying, extracting and classifying questions in order to enhance high quality answer in an online forum is developed. One of the major issues in question extraction and classification in forum is the restriction on the number categories considered such as Who, What, Where, Which, Why and How which are not sufficient to capture all possible questions. This research considered a number of parameters focused on context based spam detection for removal of spams in online forum in order to enhance question identification and classification. Fuzzy logic algorithm was applied to combine these heuristics parameters to determine the presence of spam. Part of speech tagging was applied to the extracted sentences from blog to get the structure and further analyse on the structure based on the presence and position of predefined question tags to determine if they should be extracted as questions. Using tags for analysis eliminates dealing with issues like case sensitivity, grammatical construction and synonyms. Question classification was carried out with Naïve Bayes and semantic relationship between extracted questions was achieved with cosine similarity model. The result was obtained from 1000 dataset files crawled from ResearchGate, each file represents a group and a group contains one or more sentences. A total of 24,911 sentences were obtained and the number of questions extracted is dependent on the value of the question tag occurrence. Experiments were conducted on dataset crawled from Research Gate and result shows that the proposed model can identify, classify and extract questions from Community Question Answering (CQA). The evaluation of the system and the performance show that the system outperforms other system in term of identifying and classifying all questions in the forum and extending question identification from previous research.