Projects | Anh Totti Nguyen
David Seunghyun Yoon
Description: A Dataset for Phrase Understanding and Semantic Search
Phrase in Context is a curated benchmark for phrase understanding and semantic search, consisting of three tasks of increasing difficulty: Phrase Similarity (PS), Phrase Retrieval (PR) and Phrase Sense Disambiguation (PSD). The datasets are annotated by 13 linguistic experts on Upwork and verified by two groups: ~1000 AMT crowdworkers and another set of 5 linguistic experts. PiC benchmark is distributed under CC-BY-NC 4.0 .
PS is a binary classification task with the goal of predicting whether two multi-word noun phrases are semantically similar or not given the same context sentence. This dataset contains ~56K pairs of two phrases along with their contexts used for disambiguation, since two phrases only sometimes are not enough for semantic comparison. Around 28K positive examples were annotated by linguistic experts on Upwork.com while the other 28K negative examples were created by randomly replacing 50% of the phrase token
Explore PS Download PS
Projects | Anh Totti Nguyen
David Seunghyun Yoon