Additive Compositionality of Word Vectors

Yeon Seonwoo, Sungjoon Park, Dongkwan Kim, and Alice Oh. Workshop on Noisy User-generated Text at EMNLP (EMNLP W-NUT), 2019

Additive compositionality of word embedding models has been studied from empirical and theoretical perspectives. Existing research on justifying additive compositionality of existing word embedding models requires a rather strong assumption of uniform word distribution. In this paper, we relax that assumption and propose more realistic conditions for proving additive compositionality, and we develop a novel word and sub-word embedding model that satisfies additive compositionality under those conditions. We then empirically show our model’s improved semantic representation performance on word similarity and noisy sentence similarity.

Leave a Comment