Download BinClass: A Software Package for Classifying Binary Vectors User's

minimizing the information lost due to the joining of the classes. We dene
the information content of the class Cj , rst by taking one column
(
tij > tj ? tij ; hij = tij
tij tj ? tij ; hij = tj ? tij ;
where hij is the largest of the number of the zero and one bits in the ith
column of the class Cj . The information content of the class Cj is obtained
by summing hij over the columns.
d
X
hj =
i=1
hij :
(18)
Now if we take two classes, Cj and Cj , the information content of their union
is
(
(tij + tij ) > (tj + tj ) ? (tij + tij ); hij[j = tij + tij
(tij + tij ) (tj + tj ) ? (tij + tij ); hij[j = (tj + tj ) ? (tij + tij )
0
0
0
0
0
0
0
0
0
0
0
d
X
hj[j =
0
i=1
hij[j
0
(19)
0
These two equations (18) and (19) gives us a way to dene the loss of the
information due to the joining
(20)
hj + hj ? hj[j ;
0
0
and the natural way is to join the classes that produce the smallest loss of
information. The basic idea of parsimony is not very far away from using
stochastic complexity as a tree forming critereon. They both deal with the
concept of the information content. Parsimony has been used for quite a
long time now. In the early years computers could not perform oating
point arithmetic very well, and thus the parsimony was dened in a way that
it can be implemented using integer arithmetic.
Distortion minimizing tree
Let is recall the formula (4) for class distortion. Similarly to parsimony, we
dene the class distortion of the union of the two classes Cj and Cj by
X
I (Cj[j ) = t +1 t
(21)
(x; Cj [ Cj );
j
j x2Cj [Cj
0
0
0
0
where
0
(x(l) ; Cj [ Cj ) =
0
d
X
17
i=1
jxi(l) ? aij[j j:
0
(22)