John D'Errico

I just added one more download myself. But that does not mean it is of any value at all. I had to download it to know that it is worthless, even as a homework assignment.

There is no help in this code, and it is extremely specific to an apparent homework problem for this particular student.

This submission is a script that will clear your matlab workspace. In order to use it, you must edit the code, hard coding in the equations. (I'm sorry, but while that is how students seem to use matlab, it is simply poor coding style once you start to use matlab in any serious way.)

The only thing that is of any value here are the internal comments. I do like that aspect, but nothing else that I found.

I like Adam's idea, and this is easily enough put into the code.

More digging shows that the behavior Christophe finds is a function of rounding, and of floating point arithmetic in general. But it is not something that I can make consolidator robust to, since variations at the least significant bit level will always cause problems in such a code.

This choice of a tolerance made by Christophe forces matlab/consolidator to perform a comparison between floating point numbers. With the tolerance set to exactly the difference between consecutive terms in the set provided, in some cases there MUST be a failure. PLEASE read this document:

http://docs.sun.com/source/806-3568/ncg_goldberg.html

The use of floating point arithmetic in MATLAB causes this to fail. Here, using a version of consolidator with a subtly different internal test, I get the result that Christophe did:

consolidator([1,2,3,3.01,6]',[],[],1)
ans =
                    1.5
                    3
                    3.01
                    6

Yet now change the tolerance by only an infinitesimal amount, and we can get yet a different set of rounding results.

consolidator([1,2,3,3.01,6]',[],[],1-10*eps)
ans =
                    1
                    2.5
                    3.01
                    6

consolidator([1,2,3,3.01,6]',[],[],.9999999999999)
ans =
                    1
                    2
                    3.005
                    6

Again, these differences arise because of floating point arithmetic and the use of a tolerance that is so close to the stride between members of the set. This is not something that I can change, fix, repair, or code in a better way, because if I did make a change then some other set of data would cause the same problems.

I will argue that this is what I call the transitivity problem. When you specify a tolerance of 1, how is consolidator to resolve the set [1 2 3]? Are 1 and 2 to be lumped together? Or 2 and 3? Clearly, each of those pairs are the same to within a tolerance of 1. Yet we cannot lump them all into a single group, because 1 and 3 are not within the specified tolerance. Or should we? We might very logically argue to aggregate them down to any of these sets:

[1, 2, 3]
[1.5, 3]
[1, 2.5]
[1.5, 2.5]
[2]

The point is, beware of tests that compare floating point numbers. And beware of forcing code to make those tests. You can (and will) see virtually random results from doing so.

Finally, avoid use of a tolerance that is so close to the stride between elements of the set to be resolved. Consolidator is not designed to be a clustering tool, but to be a tool that will combine replicate values together and to survive small amounts of noise in the data. The tolerance allows minor variations in the numbers to be thus combined. If you try to use consolidator to cluster numbers together, it might succeed, but you can trip it up. And no matter what, the transitivity problem is important, and is not capable of resolution in an unambiguous manner, for ALL sets of data.

John

the new setting "'smoothness',[xsmooth ysmooth]" is a Godsend. Many Thanks

I like Adam's idea, and this is easily enough put into the code.