milk.classifiers
Class MINND

java.lang.Object
  |
  +--milk.classifiers.MIClassifier
        |
        +--milk.classifiers.MINND

All Implemented Interfaces:

java.lang.Cloneable, weka.core.OptionHandler, java.io.Serializable

public class MINND

extends MIClassifier

implements weka.core.OptionHandler

0657.591B Dissertation Multiple-Instance Nearest Neighbour with Distribution learner .

It uses gradient descent to find the weight for each dimension of each exeamplar from the starting point of 1.0. In order to avoid overfitting, it uses mean-square function (i.e. the Euclidean distance) to search for the weights.
It then uses the weights to cleanse the training data. After that it searches for the weights again from the starting points of the weights searched before.
Finally it uses the most updated weights to cleanse the test exemplar and then finds the nearest neighbour of the test exemplar using partly-weighted Kullback distance. But the variances in the Kullback distance are the ones before cleansing.

Details see the 591 dissertation of Xin Xu.

See Also:

Serialized Form

Field Summary

protected double[]	m_Class The class label of each exemplar
protected int	m_Dimension The dimension of each exemplar, i.e.
protected double[][]	m_Mean The mean for each attribute of each exemplar
protected int	m_Neighbour The number of nearest neighbour for prediction
protected int	m_NumClasses The number of class labels in the data
protected double	m_Rate The learning rate in the gradient descent
protected double[][]	m_Variance The variance for each attribute of each exemplar
protected double[]	m_Weights The weight of each exemplar

Constructor Summary

MINND()

Method Summary

void	buildClassifier(Exemplars exs) As normal Nearest Neighbour algorithm does, it's lazy and simply records the exemplar information (i.e.
double	classifyExemplar(Exemplar e) Use Kullback Leibler distance to find the nearest neighbours of the given exemplar.
Exemplar	cleanse(Exemplar before) Cleanse the given exemplar according to the valid and noise data statistics
void	findWeights(int row, double[][] mean) Use gradient descent to distort the MU parameter for the exemplar.
java.lang.String[]	getOptions() Gets the current settings of the Classifier.
double	kullback(double[] mu1, double[] mu2, double[] var1, double[] var2, int pos) This function calculates the Kullback Leibler distance between two normal distributions.
java.util.Enumeration	listOptions() Returns an enumeration describing the available options Valid options are:
static void	main(java.lang.String[] args) Main method for testing.
Exemplar	preprocess(Exemplars data, int pos) Pre-process the given exemplar according to the other exemplars in the given exemplars.
void	setOptions(java.lang.String[] options) Parses a given list of options.
double	target(double[] x, double[][] X, int rowpos, double[] Y) Compute the target function to minimize in gradient descent The formula is: 1/2*sum[i=1..p](f(X, Xi)-var(Y, Yi))^2

Methods inherited from class milk.classifiers.MIClassifier

distributionForExemplar, forName, makeCopies

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

m_Neighbour

protected int m_Neighbour

The number of nearest neighbour for prediction

m_Mean

protected double[][] m_Mean

The mean for each attribute of each exemplar

m_Variance

protected double[][] m_Variance

The variance for each attribute of each exemplar

m_Dimension

protected int m_Dimension

The dimension of each exemplar, i.e. (numAttributes-2)

m_Class

protected double[] m_Class

The class label of each exemplar

m_NumClasses

protected int m_NumClasses

The number of class labels in the data

m_Weights

protected double[] m_Weights

The weight of each exemplar

m_Rate

protected double m_Rate

The learning rate in the gradient descent

Constructor Detail

MINND

public MINND()

Method Detail

buildClassifier

public void buildClassifier(Exemplars exs)
                     throws java.lang.Exception

As normal Nearest Neighbour algorithm does, it's lazy and simply records the exemplar information (i.e. mean and variance for each dimension of each exemplar and their classes) when building the model. There is actually no need to store the exemplars themselves.

Specified by:

buildClassifier in class MIClassifier

Parameters:

exs - the training exemplars

Throws:

if - the model cannot be built properly

java.lang.Exception - if the classifier has not been generated successfully

preprocess

public Exemplar preprocess(Exemplars data,
                           int pos)
                    throws java.lang.Exception

Pre-process the given exemplar according to the other exemplars in the given exemplars. It also updates noise data statistics.

Parameters:

data - the whole exemplars

pos - the position of given exemplar in data

Returns:

the processed exemplar

Throws:

if - the returned exemplar is wrong

java.lang.Exception

findWeights

public void findWeights(int row,
                        double[][] mean)

Use gradient descent to distort the MU parameter for the exemplar. The exemplar can be in the specified row in the given matrix, which has numExemplar rows and numDimension columns; or not in the matrix.

Parameters:

row - the given row index

Returns:

the result after gradient descent

target

public double target(double[] x,
                     double[][] X,
                     int rowpos,
                     double[] Y)

Compute the target function to minimize in gradient descent The formula is:
1/2*sum[i=1..p](f(X, Xi)-var(Y, Yi))^2

where p is the number of exemplars and Y is the class label. In the case of X=MU, f() is the Euclidean distance between two exemplars together with the related weights and var() is sqrt(numDimension)*(Y-Yi) where Y-Yi is either 0 (when Y==Yi) or 1 (Y!=Yi)

Parameters:

x - the weights of the exemplar in question

rowpos - row index of x in X

Y - the observed class label

Returns:

the result of the target function

classifyExemplar

public double classifyExemplar(Exemplar e)
                        throws java.lang.Exception

Use Kullback Leibler distance to find the nearest neighbours of the given exemplar. It also uses K-Nearest Neighbour algorithm to classify the test exemplar

Overrides:

classifyExemplar in class MIClassifier

Parameters:

e - the instance to be classified

Returns:

the classification

Throws:

java.lang.Exception - if the exemplar could not be classified successfully

cleanse

public Exemplar cleanse(Exemplar before)
                 throws java.lang.Exception

Cleanse the given exemplar according to the valid and noise data statistics

Parameters:

before - the given exemplar

Returns:

the processed exemplar

Throws:

if - the returned exemplar is wrong

java.lang.Exception

kullback

public double kullback(double[] mu1,
                       double[] mu2,
                       double[] var1,
                       double[] var2,
                       int pos)

This function calculates the Kullback Leibler distance between two normal distributions. This distance is always positive. Kullback Leibler distance = integral{f(X)ln(f(X)/g(X))} Note that X is a vector. Since we assume dimensions are independent f(X)(g(X) the same) is actually the product of normal density functions of each dimensions. Also note that it should be log2 instead of (ln) in the formula, but we use (ln) simply for computational convenience. The result is as follows, suppose there are P dimensions, and f(X) is the first distribution and g(X) is the second: Kullback = sum[1..P](ln(SIGMA2/SIGMA1)) + sum[1..P](SIGMA1^2 / (2*(SIGMA2^2))) + sum[1..P]((MU1-MU2)^2 / (2*(SIGMA2^2))) - P/2

Parameters:

mu1 - mu of the first normal distribution

mu2 - mu of the second normal distribution

var1 - variance(SIGMA^2) of the first normal distribution

var2 - variance(SIGMA^2) of the second normal distribution

Returns:

the Kullback distance of two distributions

listOptions

public java.util.Enumeration listOptions()

Returns an enumeration describing the available options Valid options are:

-K number
Set number of nearest neighbour used for prediction (Default: 1)

-S number
Set number of nearest neighbour instances used for cleansing the training data (Default: 1)

-E number
Set number of nearest neighbour exemplars used for cleansing the testing data (Default: 1)

Specified by:

listOptions in interface weka.core.OptionHandler

Returns:

an enumeration of all the available options

setOptions

public void setOptions(java.lang.String[] options)
                throws java.lang.Exception

Parses a given list of options.

Specified by:

setOptions in interface weka.core.OptionHandler

Parameters:

options - the list of options as an array of strings

Throws:

java.lang.Exception - if an option is not supported

getOptions

public java.lang.String[] getOptions()

Gets the current settings of the Classifier.

Specified by:

getOptions in interface weka.core.OptionHandler

Returns:

an array of strings suitable for passing to setOptions

main

public static void main(java.lang.String[] args)

Main method for testing.

Parameters:

args - the options for the classifier