Potential conditional mutual information: Estimators and properties
Author and article information
Abstract
The conditional mutual information I(X;Y|Z) measures the average information that X and Y contain about each other given Z. This is an important primitive in many learning problems including conditional independence testing, graphical model inference, causal strength estimation and time-series problems. In several applications, it is desirable to have a functional purely of the conditional distribution pY|X,Z rather than of the joint distribution pX,Y,Z. We define the potential conditional mutual information as the conditional mutual information calculated with a modified joint distribution pY|X,ZqX,Z, where qX,Z is a potential distribution, fixed airport. We develop K nearest neighbor based estimators for this functional, employing importance sampling, and a coupling trick, and prove the finite k consistency of such an estimator. We demonstrate that the estimator has excellent practical performance and show an application in dynamical system inference.
Cite this as
2017. Potential conditional mutual information: Estimators and properties. PeerJ Preprints 5:e3345v1 https://doi.org/10.7287/peerj.preprints.3345v1Author comment
This is a preprint submission to PeerJ Preprints.
Sections
Additional Information
Competing Interests
The authors declare that they have no competing interests.
Author Contributions
Arman Rahimzamani analyzed the data, contributed reagents/materials/analysis tools, wrote the paper, prepared figures and/or tables, performed the computation work, reviewed drafts of the paper, designed the mathematical model and algorithm..
Sreeram Kannan analyzed the data, contributed reagents/materials/analysis tools, wrote the paper, reviewed drafts of the paper, designed the mathematical model and algorithm..
Data Deposition
The following information was supplied regarding data availability:
Please contact the authors in case you need access to the code.
Funding
This work was supported in part by NSF Career award (grant 1651236) and NIH award number R01HG008164. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
