Abstract

The literature reports a wide variety of attribution methods for explaining the predictions made by time series classification (TSC) algorithms. These post-hoc explanation methods span from model-specific to agnostic procedures that operate at different granularity levels. Despite their relative success, they often fail to generate sparse explanations, thereby increasing the cognitive overhead for experts seeking to isolate the most relevant features associated with model performance. Another limitation of segment-based explanation methods for TSC problems is that they do not allow any expert intervention. In this paper, we present an agnostic explainer termed Sparseness-Optimized Feature Importance (SOFI) that can be used to explain the predictions generated by any black-box TSC model. In practice, the explanation takes the form of a ranking of time series segments whose cumulative perturbation leads to fast degradation in model performance. Those segments should ideally be provided or defined by experts to ensure that explanations are meaningful and aligned with domain knowledge. As a second contribution, we mathematically demonstrate that under the modularity assumption, the optimal segment ranking associated with SOFI is unique. If the modularity assumption is dropped, we prove that multiple segment importance rankings lead to the optimal model performance degradation. In our experiments, we study the effect of different strategies for computing time series segments and perturbation operators on the explanation results. Simulation results show that SOFI generates explanations that are equally robust, up to 16 times more faithful, and 1.4 times sparser than those generated by the state-of-the-art explainers used for comparison.