[Talk Summary 10] Parse Tree Fragmentation of Ungrammatical Sentences

Huma Hashemi, ISP graduate student, University of Pittsburgh had a talk about "Parse Tree Fragmentation of Ungrammatical Sentences" on Friday, 2016/11/18. She presented an evaluation of Parser Robustness for ungrammatical sentences.

Huma started the talk by giving a introduction about natural language processing (NLP) that brings about a motivation for her proposal. One of the most challenging issues that NPL has to deal with is "noisier" texts such as English-as-a-second language and machine translation. For many NLP applications that requires a parser, the sentences may not be well-formed, for instance, information extraction, question answering and summarization systems. Therefore, to build a good NLP application, a parser should be able to parse ungrammatical sentences.
 
Huma's research focuses on answering the question "how much parser's performance degrades when deal with grammar mistake?" and evaluation of a parser on ungrammatical sentences.  There are three main contributions of Huma's work presented in the talk:

  • a metric and methodology for evaluating ungrammatical sentences without referring to a gold standard corpus;
  • a quantitative comparison of parser accuracy of leading dependency parsers on ungrammatical sentences; this may help practitioners to select an appropriate parser for their applications; and 
  • a suite of robustness analyses for the parsers on specific kinds of problems in the ungrammatical sentences; this may help developers to im- prove parser robustness in the future.

Huma ran an experiment on the 8 well-known parsers (Malt, Mate, MST, SNN, SyntaxNet, Turbo, Tweebo, and Yara) using two datasets, which are (1) Penn Treebank (news data) 5000 sentences, and (2) Tweebank (twitter data) 1000 tweets. She concluded that parsers indeed have different responses to ungrammatical sentences of various types. If the input data contains text more similar to tweets (e.g. containing URLs and emoticons), Malt or Turbo parser may be good choices. If the input data is more similar to the machine translation outputs; SyntaxNet, Malt, Tweebo and Turbo parser are good options.

11/18/2016
5317 Sennott Square
University of Pittsburgh

Comments

Post a Comment

Popular posts from this blog

Personalized Access to Scientific Publications: from Recommendation to Explanation

Dario De Nart, Felice Ferrara, and Carlo Tasso ( UMAP-2013 ) http://link.springer.com/chapter/10.1007/978-3-642-38844-6_26 In the paper “Personalized Access to Scientific Publications: from Recommendation to Explanation”, Dario De Nart et al. present a Recommender and Explanation System (RES) that is able to recommend scientific publication and show explanation to users.  In order to do recommendation, Dario De Nart et al., firstly, use the The Dikpe Keyphrase extraction algorithm introduced by Pudota (2010) to extract keyphrases from papers. Each keyphrase has a weight called keyphraseness that reveals the several lexical and statistical indicators exploited in the extraction process. Higher is the keyphraseness, more relevant is the KP. For each paper in the collection, they represent it as a conceptual graph. Each node in the graph is a term broken down from a KP. Two nodes are connected when they appear in the same KP. For instance, the KP “document retrieval” is split into tw
Read more

[Talk Summary 2] Dynamic Information Retrieval Model

Image
In this seminar, prof essor Grace Hui Yang presents a dynamic information retrieval model which helps users explore the information space in order to find out which documents are relevant and which aren't, satisfying their information need. The dynamic IR task aims to find relevant documents for a session of multiple queries. It happens when information needs are complex, vague, evolving, often containing multiple subtopics. A dynamic system is one which changes or adapts over time, based on a sequence of events. While static IR does not learn directly from users and the parameters are updated periodically, dynamic IR is developed from interactive IR which exploits users' feedback to give recommendation or improve the search result or optimize ranking, which is called dynamic ranking principle. But interactive IR just treat interaction independently and response to immediate feedback. Dynamic IR tries to optimize over all interaction, uses it for long term gain and models
Read more

FolkTrails: Interpreting Navigation Behavior in a Social Tagging System

Thomas Niebler, Martin Becker, Daniel Zoller, Stephan Doerfel, & Andreas Hotho ( CIKM-2016 ) http://dl.acm.org/citation.cfm?doid=2983323.2983686 In the paper “FolkTrails: Interpreting Navigation Behavior in a Social Tagging System”, Niebler et al. present an investigate navigation trails in the popular scholarly social tagging system BibSonomy. They tried to understand users’ dynamic browsing behaviors on this kind of systems, how individual users behave differently on navigation, and whether the semantic nature of the underlying folksonomy can help to explain navigation. Firstly, Niebler et al. study different hypotheses about the navigational user behavior in Bibsonomy, a social bookmarking system. They test these hypotheses by using HypTrails, an approach utilizing first-order Markov chain models and Bayesian inference for expressing and comparing hypotheses about human trails. These hypotheses include: Uniform hypothesis: serves as a baseline, users randomly pick a l
Read more