A new approach to kernel based data analysis algorithms

Kernel based methods (KBMs) [1, 2] are arguably the best data analysis technique currently available [3, 4]. Unlike Neural Networks in which, besides a global minimum, several local minima exist, a Kernel based fitting/classifying problem is a convex optimization problem with a single minimum. However, finding this minimum (and in doing so yielding optimal parameters of a given observational model) in practice requires the manipulation, such as inversion, of large matrices. This has been challenging even when the number of data points is just over a few thousands [5][6].

The well established direct methods for updating, or inverting huge matrices fail due to the expense of a large increase in core-memory storage and CPU-time, even for moderately-sized systems. The root of the problem is that direct methods have O(N²) core memory storage requirements and the CPU-time scales as O(N³), where N is the dimension of the matrix (the number of data points, here). Despite the advances in computer power, "conventional" computers can only solve relatively small problems (N ≈ 10⁴ to 10⁵).

Another outstanding drawback of the KBMs is how to choose the appropriate kernel function for a given data set [4].

In this paper we would like to propose a computationally efficient training scheme for KBMs for obtaining the global minimum. We also present a systematic approach to selecting the appropriate kernel functions. Some preliminary results on chemical data sets will be illustrated.