The tau-path test for monotone association in an unspecified subpopulation: Application to chemogenomic data mining

In data mining and other settings, there is sometimes a need to identify relationships between variables when the relationship may hold only over a subset of the observations available. For example, expression of a particular gene may cause resistance to an anticancer drug, but only over certain types of cancer cell-lines. It may not be known in advance which types of cancer cell-lines (e.g., estrogen-regulated, newly differentiated, central nervous system) employ such a method of resistance. This situation differs from the usual setting in which partial correlations are estimated conditional on a known selection, such as the value of another variable. For any pair of variables of interest, the goal is to test if these are associated in some unspecified subpopulation that is represented by a subsample of the data we have available.Previous approaches rely heavily on bivariate normal assumptions, which are not easily adapted to non-linear association. We have tried several parametric and non-parametric approaches, and for both inferential and computational reasons have chosen to present a procedure based on a sequential development of Kendall’s tau measure of monotone association. The sequence is achieved by reordering observations so that the sample tau coefficients {Tk}{Tk} for the first k=2,…,nk=2,…,n of the nn observations form a monotone decreasing path, ending at Kendall’s tau coefficient TnTn. Boundaries are constructed so that 95% of the paths remain within the boundaries under the null hypothesis of independence. A boundary crossing at any point kk is evidence of a stronger than expected association amongst a subpopulation represented by the kk observations involved. The method is used to screen for association between gene expression and compound activity amongst types of cancer cell-lines in the NCI-60 database. More generally, the method may be used to deconvolve a mixture of absolutely continuous bivariate distributions with the same margins but differing in strength of association.We prove that a particular method of reordering the observations is optimal against any other ordering for simultaneously identifying the highest ττ association in subsets of size k(k=2,…,n). Furthermore, assuming a subpopulation of kk, we present a way of quantifying how likely any observation is to be in that subpopulation.