Three sources of classification error

1. Bayes or Indistinguishability Error: The error due to overlapping densities. This error is an inherent property of the problem and can never be eliminated;
2. Model Error: The error due to having an incorrect model. This error can only be eliminated if the designer specifies a model that includes the true model which generated the data. Designers generally choose the model based on knowledge of the problem domain rather than on the subsequent estimation method, and thus the model error in maximum-likelihood and Bayes methods rarely differ.
3. Estimation Error: The error arisinig from the fact that the parameters are estimated from a finite sample. This error can best be reduced by increasing training data.

These three points are from "Pattern Classification" by Duda. From my point of view, the second point is the most common. Actually, it is about model structure choice or learning. The third one, parameter learning, works on the outcome of second point normally.