Title: Latent Maximum Entropy Principle for Statistical Language Modeling

Authors: Shaojun Wang, Ronald Rosenfeld, Yunxin Zhao

Abstract: In this paper, we describe a unified probabilistic framework for statistical language modeling, latent maximum entropy principle. The salient feature of this approach is that the hidden causal hierarchical dependency structure can be encoded into the statistical model in a principled way by mixtures of exponential families with a rich expressive power. We first show the problem formulation, solution, and certain convergence properties. We then discribe how to use this machine learning technique to model various aspects of natural language such as syntactic structure of sentence, semantic information in document. Finally, we draw a conclusion and point out future research directions.

a01sw128.ps a01sw128.pdf