David Windridge

In multimodal information fusion domains, such as remote sensing, it is not uncommon to encounter objects with one or more missing modality for which combination cannot be performed. This is particularly problematic for kernel-based fusion, where objects themselves define the embedding space, making conventional methods for dealing with missing modality information (such as mean-substitution) inapplicable. However, by interpreting the aggregate of disjoint training sets as a complete data set with missing inter-modality kernel measurements to be filled in by appropriately chosen substitutes, a novel kernel-based technique, the neutral-point method is derived. Missing modalities are thus substituted in manner that is unbiased with regards to the overall classification. Critically, unlike conventional missing-data substitution methods, explicit calculation of neutral points may be omitted by virtue of their implicit incorporation within the in the SVM training framework.

Experiments based on the publicly available Biosecure DS2 multimodal data set show that the SVM-NPS approach achieves very good generalization performance since the method is, in structural terms, a kernel-based analog of the well-known sum rule combination scheme exhibiting similar error-cancelling behaviour.