M. Studeny and J. Vejnarova: The multiinformation function as a tool for measuring stochastic dependence. In Learning in Graphical Models (M. I. Jordan ed.) Kluwer, Dordrecht 1998, pp. 261-298.

Given a collection of random variables X(i) where i belongs to a finite nonempty set N, the corresponding multiinformation function ascribes (to every subset A of N) the relative entropy of the joint distribution of X(A) (that is, the random vector composed of the variables X(i) where i belongs to A) with respect to the product of distributions of individual random variables X(i) through i in A. We argue that it is a useful tool for solving the problems concerning stochastic (conditional) dependence and independence (at least in discrete case). First, it makes possible to express the conditional mutual information between X(A) and X(B) given X(C) (for every disjoint subsets A,B,C of N) which can be considered as a good measure of conditional stochastic depedence. Second, one can introduce reasonable measures of dependence of level r among variables of X(A) (where A is a subset of N and 0 < r < card A) which are expressible by means of the multiinformation function. Third, it enables one to derive theoretical results on (nonexistence of an) axiomatic characterization of stochastic conditional independence models.

In fact, the last part of the paper is nothing but a didactive proof of the result from the paper:
M. Studeny: Conditional independence relations have no finite complete characterization. In Information Theory, Statistical Decision Functions and Random Processes. Transactions of the 11th Prague Conference vol. B (S. Kubik, J.A. Visek eds.), Kluwer, Dordrecht - Boston - London (also Academia, Prague) 1992, pp. 377-396.

AMS classification 94A17, 68T30 (03B30, 62H05)

conditional mutual information
relative entropy
level-specific measures of dependence
(regular) inference rule
perfect rule

A pdf copy (converted postscript version) (366kB) is available.

The results of the paper has a connection to the following works: