Download Machine Learning

Here, the sign function converts anything positive into a 1 and anything negative into a
–1 (zero remains 0).
Boosting Code
There is example code in …/opencv/samples/c/letter_recog.cpp that shows how to use
boosting, random trees and back-propagation (aka multilayer perception, MLP). The
code for boosting is similar to the code for decision trees but with its own control
parameters:
struct CvBoostParams : public CvDTreeParams {
int
boost_type;
// CvBoost:: DISCRETE, REAL, LOGIT, GENTLE
int
weak_count;
// How many classifiers
int
split_criteria;
// CvBoost:: DEFAULT, GINI, MISCLASS, SQERR
double weight_trim_rate;
CvBoostParams();
CvBoostParams(
int
boost_type,
int
weak_count,
double
weight_trim_rate,
int
max_depth,
bool
use_surrogates,
const float* priors
);
};
In CvDTreeParams, boost_type selects one of the four boosting algorithms listed previously. The split_criteria is one of the following.
• CvBoost :: DEFAULT (use the default for the particular boosting method)
• CvBoost :: GINI (default option for real AdaBoost)
• CvBoost :: MISCLASS (default option for discrete AdaBoost)
• CvBoost :: SQERR (least-square error; only option available for LogitBoost and gentle
AdaBoost)
The last parameter, weight_trim_rate, is for computational savings and is used as described next. As training goes on, many data points become unimportant. That is,
the weight Dt(i) for the ith data point becomes very small. The weight_trim_rate is a
threshold between 0 and 1 (inclusive) that is implicitly used to throw away some training samples in a given boosting iteration. For example, suppose weight_trim_rate is set
to 0.95. This means that samples with summary weight 1.0–0.95 = 0.05 (5%) do not
participate in the next iteration of training. Note the words “next iteration”. The samples are not discarded forever. When the next weak classifier is trained, the weights are
computed for all samples and so some previously insignificant samples may be returned
back to the next training set. To turn this functionality off, set the weight_trim_rate
value to 0.
Observe that CvBoostParams{} inherits from CvDTreeParams{}, so we may set other parameters that are related to decision trees. In particular, if we are dealing with features
498 |
Chapter 13: Machine Learning