A Latent Space Approach to Dynamic Embedding of Co-occurrence Data (2007)

Purnamrita Sarkar, Sajid Siddiqi, Geoff Gordon


We consider dynamic co-occurrence data, such as author-word links in papers published in successive years of the same conference. For static co-occurrence data, researchers often seek an embedding of the entities (authors and words) into a low-dimensional Euclidean space. We generalize a recent static co-occurrence model, the CODE model of Globerson et al. (2004), to the dynamic setting: we seek coordinates for each entity at each time step. The coordinates can change with time to explain new observations, but since large changes are improbable, we can exploit data at previous and subsequent steps to find a better explanation for current observations. To make inference tractable, we show how to approximate our observation model with a Gaussian distribution, allowing the use of a Kalman filter for tractable inference. The result is the first algorithm for dynamic embedding of co-occurrence data which provides distributional information for its coordinate estimates. We demonstrate our model both on synthetic data and on author-word data from the NIPS corpus, showing that it produces intuitively reasonable embeddings. We also provide evidence for the usefulness of our model by its performance on an author-prediction task.

Full text

Download (application/pdf, 269.3 kB)

Approximate BibTeX Entry

    Year = {2007},
    Booktitle = {Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics (AI-STATS)},
    Author = { Purnamrita Sarkar, Sajid Siddiqi, Geoff Gordon },
    Title = {A Latent Space Approach to Dynamic Embedding of Co-occurrence Data}

Copyright 2010, Carnegie Mellon University, Auton Lab. All Rights Reserved.