MATLAB Central Newsreader - Sample Correlation Distance Implementation

Sample Correlation Distance Implementation

Ahmad Ammari — Tue, 31 Mar 2009 12:53:01 +0000

I am trying to implement a function that computes the sample correlation distance between column vectors given as a matrix (pos). The function is supposed to return a distance matrix that shows the distances between every column vector pair in the matrix. I got the equation of the sample correlation distance from the statistics toolbox. You can take a look at it in the definition of the pdist function in the Matlab help.

The code of my function is as follows:

function d = corrdist(pos)
[rows,cols] = size(pos);
d = zeros(cols,cols);
for i=1:cols
  for j=1:(i-1)
    d(i,j) = calc_distance(pos(:,i),pos(:,j));
  end
end
d = d + d';
function d = calc_distance(v1,v2)
v1bar = mean(v1);
v2bar = mean(v2);
v1diff = (v1 - v1bar);
v2diff = (v2 - v2bar);
v1difftr = v1diff';
v2difftr = v2diff';
d = 1 - ((v1diff*v2difftr)/(sqrt(v1diff*v1difftr)*sqrt(v2diff*v2difftr)));

My problem is that I get the following error when trying to test the function:

Warning: Matrix is singular to working precision.
> In corrdist>calc_distance at 51
  In corrdist at 34
??? Subscripted assignment dimension mismatch.

Error in ==> corrdist at 34
    d(i,j) = calc_distance(pos(:,i),pos(:,j));

I tried to test the code inside calc_distance(v1,v2) in the Matlab command line and it worked fine. So why is that error and how to solve it?

Thanks in advance.

Re: Sample Correlation Distance Implementation

Roger Stafford — Thu, 02 Apr 2009 00:36:01 +0000

"Ahmad Ammari" <ammari@acm.org> wrote in message <gqt3nd$64k$1@fred.mathworks.com>...
> I am trying to implement a function that computes the sample correlation distance between column vectors given as a matrix (pos). The function is supposed to return a distance matrix that shows the distances between every column vector pair in the matrix. I got the equation of the sample correlation distance from the statistics toolbox. You can take a look at it in the definition of the pdist function in the Matlab help.
>
> The code of my function is as follows:
>
> function d = corrdist(pos)
> [rows,cols] = size(pos);
> d = zeros(cols,cols);
> for i=1:cols
> for j=1:(i-1)
> d(i,j) = calc_distance(pos(:,i),pos(:,j));
> end
> end
> d = d + d';
> function d = calc_distance(v1,v2)
> v1bar = mean(v1);
> v2bar = mean(v2);
> v1diff = (v1 - v1bar);
> v2diff = (v2 - v2bar);
> v1difftr = v1diff';
> v2difftr = v2diff';
> d = 1 - ((v1diff*v2difftr)/(sqrt(v1diff*v1difftr)*sqrt(v2diff*v2difftr)));
>
> My problem is that I get the following error when trying to test the function:
>
> Warning: Matrix is singular to working precision.
> > In corrdist>calc_distance at 51
> In corrdist at 34
> ??? Subscripted assignment dimension mismatch.
>
> Error in ==> corrdist at 34
> d(i,j) = calc_distance(pos(:,i),pos(:,j));
>
> I tried to test the code inside calc_distance(v1,v2) in the Matlab command line and it worked fine. So why is that error and how to solve it?
>
> Thanks in advance.

Your 'calc_distance' function appears to be designed to accept two row vector inputs. With two column vectors for v1 and v2 the quantity "v1diff*v2difftr" would give you a square matrix, not a scalar. When you attempt to find its square root it is not surprising that this matrix is found to be singular; in fact it is so singular it would have a rank of only one! You need to have "v2difftr*v1diff" instead if you are going to use column vector inputs.

Why is it you aren't using 'pdist' itself with the 'correlation distance' metric? It seems to be designed to do just your kind of problem, except that it expects to find distances between the rows of its input matrix, not its columns. You need only do a transpose to adapt to that.

Roger Stafford

Re: Sample Correlation Distance Implementation

Ahmad Ammari — Wed, 08 Apr 2009 11:22:01 +0000

"Roger Stafford" <ellieandrogerxyzzy@mindspring.com.invalid> wrote in message > Your 'calc_distance' function appears to be designed to accept two row vector inputs. With two column vectors for v1 and v2 the quantity "v1diff*v2difftr" would give you a square matrix, not a scalar. When you attempt to find its square root it is not surprising that this matrix is found to be singular; in fact it is so singular it would have a rank of only one! You need to have "v2difftr*v1diff" instead if you are going to use column vector inputs.

Thanks Roger. You are right and I solved the problem by using the dot(v1,v2) function to get the dot product of the two vectors instead of directly multiplying them.

> Why is it you aren't using 'pdist' itself with the 'correlation distance' metric? It seems to be designed to do just your kind of problem, except that it expects to find distances between the rows of its input matrix, not its columns. You need only do a transpose to adapt to that.

Yes I know I could have used the pdist function with the 'correlation' value for the distance parameter but I just wanted to implement the original equation of the sample correlation function so my function can run even if I do not have the statistics toolbox in Matlab installed.
And I have to consider columns not rows as vectors in my implementation because this is how the functions in the neural networks toolbox in Matlab consider vectors to be.