Multivariate Gaussian Feature Selection.

We concentrate our research activities on the multivariate feature selection, which is one important part of many machine learning tasks. In partucular, Linear Discriminant Analysis [1] belongs to the state-of-the-art methods for the multivariate analysis. From the theoretical point of view, it is the well-known fact that LDA is best suitable in the case the features are Gaussian distributed. In the theoretical part of the presented paper, we analyse the properties of the multivariate discriminant analysis with respect to the feature selection. In this context, we consider a binary supervised learning task and assume that the features are Gaussian distributed. The discriminant analysis solves the mentioned supervised learning task by maximising of the discriminant value, calculated for the linear combination of the features. The initial LDA solution a 2 Rd is considered for all given features from the feature space X  Rd. The corresponding discriminant is calculated by the formula: d(a; x1, . . . , xd) := (μ+ − μ−)2 2+ + 2− , where μ+/− are projected class means and 2 +/− are projected class variances (with respect to a). We proof several propositions with the aim to find subsets of the features having higher discriminant value as original d(a; x1, . . . , xd). For the suitability in the real world settings, here we are interested in fast searching for such subsets. The performance of the mentioned propositions is examined experimentally on datasets from UCI repository [2]. Several application scenarien will be discussed and tested on the datasets. In addition, tests show that the performance can be achieved also in the case the features are not Gaussian distributed.