SemEval-2010 Word Sense Induction & Disambiguation Task
Download Trial Data & Task Description
The trial data & task description files can be downloaded by clicking here.
Download Training, testing datasets & evaluation scripts
- Evaluation scripts & keys available here.
- Training available here.
- Testing available here.
- WSI systems results avalaible here.
Testing Dataset Information
The testing dataset is part of the OntoNotes (Hovy et al, 2006). Each test instance consisted of a maximum of three sentences. The texts come from various news sources including the Wall Street Journal, CNN, ABC and others.
Verbs' Testing Dataset Description Information
|
Lemma | Instances | ITA | Senses |
accommodate.v | 12 | 0.75 | 3 |
sniff.v | 15 | 0.93 | 3 |
cheat.v | 16 | 0.81 | 2 |
presume.v | 16 | 0.81 | 2 |
reap.v | 16 | 0.94 | 2 |
haunt.v | 17 | 0.82 | 2 |
cultivate.v | 17 | 0.82 | 4 |
frame.v | 19 | 0.89 | 4 |
level.v | 20 | 0.75 | 4 |
regain.v | 20 | 0.9 | 2 |
bow.v | 22 | 0.82 | 5 |
root.v | 23 | 0.78 | 4 |
shave.v | 26 | 0.96 | 2 |
owe.v | 29 | 0.9 | 3 |
analyze.v | 29 | 0.9 | 2 |
swim.v | 31 | 0.9 | 2 |
mount.v | 32 | 0.94 | 5 |
signal.v | 34 | 0.91 | 2 |
assemble.v | 37 | 0.81 | 2 |
assert.v | 37 | 0.81 | 3 |
straighten.v | 37 | 0.84 | 3 |
deploy.v | 40 | 0.78 | 2 |
expose.v | 41 | 0.9 | 2 |
swear.v | 44 | 0.98 | 5 |
weigh.v | 46 | 0.98 | 6 |
pour.v | 47 | 0.89 | 4 |
separate.v | 51 | 0.9 | 2 |
relax.v | 53 | 0.87 | 3 |
divide.v | 58 | 0.91 | 5 |
slow.v | 59 | 0.9 | 2 |
appeal.v | 66 | 0.85 | 4 |
commit.v | 71 | 0.9 | 3 |
pursue.v | 73 | 0.92 | 2 |
observe.v | 76 | 0.78 | 4 |
conclude.v | 76 | 0.8 | 4 |
figure.v | 78 | 0.81 | 5 |
stick.v | 79 | 0.8 | 4 |
question.v | 82 | 0.8 | 2 |
violate.v | 83 | 0.96 | 2 |
defend.v | 94 | 0.91 | 2 |
lay.v | 107 | 0.77 | 6 |
reveal.v | 122 | 0.88 | 2 |
apply.v | 123 | 0.93 | 4 |
insist.v | 124 | 0.87 | 2 |
deny.v | 133 | 0.86 | 3 |
introduce.v | 142 | 0.87 | 3 |
operate.v | 190 | 0.81 | 2 |
lie.v | 208 | 0.97 | 4 |
wait.v | 346 | 0.97 | 2 |
happen.v | 581 | 0.97 | 4 |
|
Nouns' Test Dataset Information
|
Lemma | Instances | ITA | Senses |
access.n | 48 | 1 | 8 |
accounting.n | 31 | 0.94 | 5 |
address.n | 37 | 0.92 | 10 |
air.n | 174 | 0.89 | 8 |
body.n | 190 | 0.89 | 10 |
camp.n | 33 | 1 | 8 |
campaign.n | 148 | 0.88 | 5 |
cell.n | 84 | 0.99 | 8 |
challenge.n | 72 | 0.89 | 7 |
chip.n | 112 | 0.93 | 13 |
class.n | 132 | 1 | 9 |
commission.n | 50 | 0.9 | 7 |
community.n | 189 | 1 | 8 |
dealer.n | 67 | 0.94 | 5 |
display.n | 40 | 0.97 | 6 |
edge.n | 32 | 0.97 | 6 |
entry.n | 45 | 1 | 6 |
failure.n | 66 | 1 | 7 |
field.n | 155 | 0.88 | 11 |
flight.n | 107 | 1 | 11 |
foundation.n | 52 | 0.9 | 8 |
function.n | 35 | 1 | 7 |
gap.n | 51 | 1 | 5 |
gas.n | 123 | 0.98 | 6 |
guarantee.n | 58 | 1 | 5 |
house.n | 162 | 0.91 | 14 |
idea.n | 200 | 1 | 7 |
innovation.n | 33 | 0.91 | 3 |
legislation.n | 70 | 1 | 4 |
margin.n | 60 | 1 | 6 |
mark.n | 70 | 0.96 | 9 |
market.n | 865 | 0.78 | 7 |
mind.n | 111 | 1 | 7 |
moment.n | 143 | 0.99 | 6 |
movement.n | 63 | 0.94 | 7 |
note.n | 96 | 0.93 | 8 |
office.n | 332 | 1 | 7 |
officer.n | 187 | 0.97 | 4 |
origin.n | 23 | 1 | 5 |
park.n | 43 | 0.93 | 7 |
promotion.n | 27 | 1 | 5 |
rally.n | 46 | 1 | 5 |
reputation.n | 28 | 1 | 4 |
road.n | 138 | 0.89 | 6 |
screen.n | 28 | 0.93 | 10 |
shape.n | 46 | 0.85 | 6 |
speed.n | 52 | 0.87 | 9 |
television.n | 161 | 1 | 4 |
threat.n | 140 | 0.98 | 4 |
tour.n | 30 | 1 | 5 |
|
Acknowledgements
We gratefully acknowledge the support of the EU FP7 INDECT project, Grant No. 218086, the National Science Foundation Grant NSF-0715078, Consistent Criteria for Word Sense Disambiguation, and the GALE program of the Defense Advanced Research Projects Agency, Contract No. HR0011-06-C-0022, a subcontract from the BBN-AGILE Team.
Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. Ontonotes: the 90% solution. In Proceedings of NAACL, Companion Volume: Short Papers on XX, pages 57-60. ACL.