Thread Subject:

MATLAB Programming Contest: April 5-12, 2006

The next MATLAB Programming Contest will start tomorrow at 9 a.m. EST
and run through Wednesday, April 12 at 5 p.m. EST. Topics and rules
will be posted on the morning of the contest:
   
 <http://www.mathworks.com/contest/>
   
As in recent contests, there will be three stages, darkness,
twilight, and daylight. During darkness, both the code and scores for
all entries will be hidden. In twilight, the scores will be visible
but the code will still be hidden. When daylight arrives, all scores
and code are visible. We'll also be holding several mid-contest prize
giveaways of MATLAB bags, caps, and other good stuff, so don't wait
until the last minute to participate.
   
Please use this thread to talk about the contest, strategies, or to
ask related questions. If you have an "administrative" type of
question that you don't feel applies to anyone else, e-mail us at
contest@mathworks.com.

We've also started a Contest Blog where we will post the latest news,
so be sure to check it out.

 <http://blogs.mathworks.com/contest>

Good luck everyone!
   
- The MATLAB Contest Team -

Min Poh wrote:
>
>
> The next MATLAB Programming Contest ...

The rules say that there are a limited number of moves, but I did not
see what the limit is. Is it limited only by the execution time, or
is there a hard-coded limit?

the cyclist wrote:
>
>
> The rules say that there are a limited number of moves, but I did
> not
> see what the limit is. Is it limited only by the execution time,
> or
> is there a hard-coded limit?

Hi Cyclist,

in the contest rules under 'Developing Your Entry' it states the
following:

"The main routine is solver.m.

 moves = solver(board, nMoves)

where board is the matrix representing the blocks in your initial
game configuration, and nMoves is the number of moves you are
allowed. The return matrix moves can be no longer than nMoves rows."

Therefore each board has a specified number of maximum allowed moves.
I hope this answers your question.

Good luck,
Eric Sampson

Eric Sampson wrote:
>
>
> the cyclist wrote:
>>
>>
>> The rules say that there are a limited number of moves, but I
did
>> not
>> see what the limit is. Is it limited only by the execution
time,
>> or
>> is there a hard-coded limit?
>
> Hi Cyclist,
>
> in the contest rules under 'Developing Your Entry' it states the
> following:
>
> "The main routine is solver.m.
>
> moves = solver(board, nMoves)
>
> where board is the matrix representing the blocks in your initial
> game configuration, and nMoves is the number of moves you are
> allowed. The return matrix moves can be no longer than nMoves
> rows."
>
> Therefore each board has a specified number of maximum allowed
> moves.
> I hope this answers your question.

Yeah, I missed that. Thanks.

Does the blockbusterGUI accept illegal moves? I am able to bust a
single block, which shouldn't work, if I understood the rules
correctly.

By the way, it's very observer-friendly of the contest team to
provide such interactive and viewing-functions. That way, people who
don't have the time to participate can at least follow what is going
on.

Heinrich

Heinrich Acker wrote:
>
>
> Does the blockbusterGUI accept illegal moves? I am able to bust a
> single block, which shouldn't work, if I understood the rules
> correctly.
>
> By the way, it's very observer-friendly of the contest team to
> provide such interactive and viewing-functions. That way, people
> who
> don't have the time to participate can at least follow what is
> going
> on.
>
> Heinrich

Heinrich:

You are right, in the GUI it is allowed to pop single blocks, we'll
update the zip file to reflect what the rules state.

Lucio
- The MATLAB Contest Team -

Can you confirm that the supplied mfiles are not supposed to work on
Matlab6 or earlier? I'm getting lots of errors...

 Undefined function or variable 'N'.
Error in ==> C:\WINDOWS\Desktop\matlabbb\blockbuster\grade.m
(findNeighbors)
On line 60 ==> if ~N(i,j);

and

??? Undefined function or variable 'hf'.

Error in ==>
C:\WINDOWS\Desktop\matlabbb\blockbuster\blockBusterGUI.m (myRelease)
On line 45 ==> buttonType = get(hf,'SelectionType');

in the GUI

j wrote:
>
>
> Can you confirm that the supplied mfiles are not supposed to work
> on
> Matlab6 or earlier? I'm getting lots of errors...
With some slight modifications (mainly to grade), I managed to let it
work on Matlab5.2. (I didn't try to "downgrade" the GUI-function.)
Stijn

thanks Stijn,
i too just needed to add an if then:
        if exist('N'),
in grade.m to make it work but i was hoping someone had fixed the
gui...

Hi all:

For those who found blockBusterGUI useful, we have included in the
zip a variation in which you are allowed to pass a list of moves, now
you can visualize how your solver steps through a given problem.

Hope it helps.

Lucio
- The MATLAB Contest Team -

Can we use the functions you provide for the contest (e.g. grade) ?

How will the contest software handle code written on older matlab
versions?
 
I'm using MatLab 6.1, and I already noticed some differences to the
current version by looking at the functions provided for the contest
(e.g., logical 'and': old '&' vs. new '&&').
 
Best,
dooya

Steve, you're welcome to use "grade", but you'll need to include it
as a subfunction.

I've been peeking behind the curtain at the entries we've received so
far - very impressive!

I'd like to remind everyone that some functions are disallowed and
will cause your entry to fail, including ERROR, PAUSE, and IMAGE.
See the rules for details. If you submitted an entry that uses any
of these, you still have two hours to resubmit them before the end of
Darkness.

This contest is lots of fun. It's easy to get into but leaves lots of
room for improvement. Great job Mathworks team!
 
I do have one minor quibble though. The start/stop times for some
stages are 12 AM or 12 PM and it's tough to figure out which one is
noon and which is midnight. It would be more clear to pick a time
that is not exactly 12:00, like 11:59 or 11:00.
 
Steve

i am using matlab 6.5 and could work around the incompatibility by
declaring each error generating variable as global.

not a very nice way, but it works.

j wrote:
>
>
> thanks Stijn,
> i too just needed to add an if then:
> if exist('N'),
> in grade.m to make it work but i was hoping someone had fixed the
> gui...

volkan

Thanks for pointing this out, ESB. Adriano submitted the exact same
entry about 60 times. I marked the 27 that were left in the queue as
"failed" so we won't waste time scoring them. I'll also send him a
friendly e-mail.

Matthew Simoneau wrote:
>
>
> I've been peeking behind the curtain at the entries we've received
> so
> far - very impressive!
>
> I'd like to remind everyone that some functions are disallowed and
> will cause your entry to fail, including ERROR, PAUSE, and IMAGE.
> See the rules for details. If you submitted an entry that uses any
> of these, you still have two hours to resubmit them before the end
> of
> Darkness.

But the grade function calls ERROR (upon errors...).

Is this alright?

You're right, eigenvector. You'll also need to comment out the ERROR
lines in GRADE.

During the darkness stage I submitted a few entries that are
essentially identical. All failed except one. Passing code included
the line:

warning off MATLAB:conversionToLogical

My code invokes 'logical' in a loop and I am guessing my remaining
entries timed out as a result of the time it takes to generate those
thousands of warnings.

Can someone please clarify the situation on the test suite regarding
the warnings.

volkan

Min Poh wrote:
> The next MATLAB Programming Contest will start tomorrow at > 9
a.m. EST
> and run through Wednesday, April 12 at 5 p.m. EST.

Is this a new spot in the calendar for the contest? I don't recall
it competing with my tax returns in past years. ;-)

volkan, your analysis sounds right to me. Any display to the command
line is a huge drag on the entry. This is true in MATLAB in general
and even more true because of the particulars of the contest
environment.

I concur. I'm currently at the top because I've take this to the
logical endgame and removed all the whitespaces making the code
completely unreadable. But if you notice my comment (and the name)
for the RottenEggs code, I think this is totally against the spirit
of the competition and would like to think there's someway to avoid
it.

As a side note, I was the first to modify Markus's code for minor
speed improvements, which I think IS within the spirit of the
competition and I give him full credit for a very efficient
algorithm. I just made incremental improvements to his
implementation.

Markus wrote:
>
>
> Hi!
>
> Some people try to obfuscate code by replacing variable names by
> unrecognizable words like "zzz1009" and "zzz1048", as DrSuess did
> with his submission called "scrambled eggs" (his code is here: <http://www.mathworks.com/contest/blockbuster.cgi/view_submission.html?id=27039)>.
> he code is clearly based on my winning submission of the twilight
> phase (or an improved modification of it).
>
> I would propose that you withdraw such entries from the list. You
> write in the Contest FAQ:
>
> "One of the goals behind the MATLAB Contest is to learn more about
> MATLAB programming. Viewing and modifying other entries, as well as
> having your own entries be modified, is a great way to learn how to
> write better code."
>
> This goal is clearly interfered by such obfuscation methods.
>
> Regards
> Markus

Code obfuscation rears its ugly head in some of our contests. Here
is some commentary on how it played out in a previous contest:

 <http://www.mathworks.com/contest/gerrymander/evolution.html#23>

My advice is to simply ignore any obfuscated entry. Alan, your
"Faster2?" gets the same result as the obfuscated leader and is only
slightly slower. One person's obfuscated code simply cannot keep up
with the ingenuity of the whole crowd.

If obfuscation becomes a real problem, we could certainly consider
some sort of rule. Something along the line of "obfuscated code
cannot win a prize". I worry, though, about where you draw the line
between code that is obfuscated vs poorly documented.

I think the best you can do about this is to ignore it. The
obfuscated code clearly won't go anywhere. It is leading for the
moment, but neither Dr. Seuss, nor anyone else will be able to
develop it any further. There are still tons of ways to improve the
score.

Anders wrote:
>
>
> It's very frustrating to make solutions run on my Matlab 6.5.
> Nested
> functions inheriting variables and new uses of sort and find have
> to
> be converted. I'd better get an upgrade before next time.

I'm using matlab 6.1 and some of my entries get medium scores when
running on my system, but perform really poor in the real contest. I
still don't know whether this is caused by the runcontest.m don't
working correctly on my matlab 6.1 or my code don't working on
matlab 2006. Since I make use of sort and find, I already checked the
online manuals but did not figure out the differences to the old
usage under matlab 6.1.

Well, it's probably driven me out of the contest. Between obfuscation
and people testing code in the contest by submitting multiple copies
of the same / almost identical code there seems little room for the
"spirit" of the contest. If people don't want to share their code,
tehn they're doing this for the wrong reasons.

I've deobfuscated version of the current leader (A001 from Hunter)
it can be found under
 <http://www.mathworks.com/contest/blockbuster.cgi/view_submission.html?id=27269>

IMHO obfuscated code is going to win this contest. For some strange
reason obfuscated code has a much better runtime (arround 4-5 sec for
the current entries).
There is something fishy here with matlab since there shouldn't be
any major difference in the resulting p-code. So in a way this
contest is sucessfull we are learning a lot about matlab :)

Is there any explanation for the speed improvement of the first
rand-line? That such a line can change results if further down the
code random numbers are used, I fully understand. But (luckily)
giving the same result, but being much faster by adding the use of
random numbers, is very strange. Or am I missing something?

Are there some jokes in the current JIT compiler?

Matthew Simoneau wrote:
>
>
> Thanks for pointing this out, ESB. Adriano submitted the exact
> same
> entry about 60 times. I marked the 27 that were left in the queue
> as
> "failed" so we won't waste time scoring them. I'll also send him a
> friendly e-mail.

The same thing is happening again, this time by Don Antonio. Is it
possible to have the queue check for diffs from the top few leaders
and automatically remove entries that don't contain any changes?

Imre Polik wrote:
>
>
> I think the best you can do about this is to ignore it.

The obfuscated code seems pretty firmly entrenched, at least at the
moment. But if every new entry could try to add a little white space
or sensible structure to the code, or demystify a variable name,
maybe we could evolve back to unobfuscated code.

Of course, nature dictates that destruction is easier than
construction, so we also have to hope that someone else does not come
along and obfuscate to take the lead again.

the cyclist wrote:
> The obfuscated code seems pretty firmly entrenched, at least at the
> moment. But if every new entry could try to add a little white
> space
> or sensible structure to the code, or demystify a variable name,
> maybe we could evolve back to unobfuscated code.
That doesn't seem to help, probably because it seems repeatable that
the "unobfuscated code" does run faster. There are some changes that
were "unhidden", but afterwards they are hidden again. And this code
appearently gets on top easy.(???)
Stijn

Thanks and congratulations Jim Mikola for getting on top with cleaned
code.

There's a lot of copies of the same solution posted now.

notweak = notweak 2 = notweak 3 = no tweak either 3 = no tweak either
4 = no tweak either 5 and so on...

It's time for succesfull results with swapping. It should be worth
working on that, rather than further tuning or obfuscting. Further
tuning should be done later...

Stijn Helsen wrote:
> I suppose that for example Jan Langer doesn't like the tolerance on
> CPU times, and submits manymany identical codes. That people are
> "seduced" to do that in "critical time", I understand it, but just
> during "non-critical times", I don't. Current operating systems
> (combined with Matlab) are not
> realtime-allways-behaving-similar-in-time, that doesn't have to be
> invented again...

Actually, i was hoping that this is a critical time, namely the big
sunday push. i hope the contest team will announce it later today.
but we will see.
Jan

Hi,

My entries keep failing with the following message:

"Error using ==> error Not enough input arguments."

Now, I tried them on two different machines locally (using different
versions of Matlab) and everything is ok. But when I submit I get
crashed. Absolutely exasperating.

Perhaps somebody woule kindly suggest a hint as to where I am
goofing...

Apparently this is related to the funciton ERROR but it's disabled...
so what's going on?

Thanks!

In article <ef2fb57.42@webx.raydaftYaTP>, eigenvector <felixg@tx.technion.ac.il> wrote:

> Hi,
>
> My entries keep failing with the following message:
>
> "Error using ==> error Not enough input arguments."
>
> Now, I tried them on two different machines locally (using different
> versions of Matlab) and everything is ok. But when I submit I get
> crashed. Absolutely exasperating.
>
> Perhaps somebody woule kindly suggest a hint as to where I am
> goofing...
>
> Apparently this is related to the funciton ERROR but it's disabled...
> so what's going on?
>
> Thanks!

PLEASE DO NOT CONTINUOUSLY POST THIS SAME QUESTION
MULTIPLE TIMES!

It has already been suggested that you are not using
the correct arguments for error.

You should verify if one of these systems may have
its own function called error. Try this:

  which error -all

If all else fails, learn to use the debugger.

  help dbstop

Knowing what is going on is impossible, since you
have not told us enough information. Are all systems
at the same release level of matlab? Have you tried
any of the things I've suggested here?

HTH,
John D'Errico


--
The best material model of a cat is another, or preferably the same, cat.
A. Rosenblueth, Philosophy of Science, 1945

Those who can't laugh at themselves leave the job to others.
Anonymous

I just tried to deobfuscate the 28454 entry and compared it to 28428.
It turns out that they are code-wise equivalent. So the obfuscation
causes a difference of 4 seconds. In case someone is interested in
the deobfus-script, I can send it.

Now, is there going to be a big Sunday push or not?
Jan

eigenvector wrote:
>
>
> Hi,
>
> My entries keep failing with the following message:
>
> "Error using ==> error Not enough input arguments."
>
> Now, I tried them on two different machines locally (using
> different
> versions of Matlab) and everything is ok. But when I submit I get
> crashed. Absolutely exasperating.
>
> Perhaps somebody woule kindly suggest a hint as to where I am
> goofing...
>
> Apparently this is related to the funciton ERROR but it's
> disabled...
> so what's going on?
>
> Thanks!

Hello,

I noticed your problem yesterday, but I didn't say anything in hopes
someone more knowledgeable would help you figure it out. I am
guessing here, but it might have something to do with version 2006a's
way of treating percent signs. Is it possible that in the new
release, you can open a comment and close it on the same line by
using two percent signs?

My advice is to remove all the lines with the 'error' function
entirely to be safe, but if you want to leave it in, it couldn't hurt
to try deleting (or adding another) one of the percent signs.

Good luck!

Good idea, Jan. I'll announce the Sunday Push now.

John D'Errico wrote:
>
>
> In article <ef2fb57.42@webx.raydaftYaTP>, eigenvector
> <felixg@tx.technion.ac.il> wrote:
>
>> Hi,
>>
>> My entries keep failing with the following message:
>>
>> "Error using ==> error Not enough input arguments."
>>
>> Now, I tried them on two different machines locally (using
> different
>> versions of Matlab) and everything is ok. But when I submit I
get
>> crashed. Absolutely exasperating.
>>
>> Perhaps somebody woule kindly suggest a hint as to where I am
>> goofing...
>>
>> Apparently this is related to the funciton ERROR but it's
> disabled...
>> so what's going on?
>>
>> Thanks!
>
> PLEASE DO NOT CONTINUOUSLY POST THIS SAME QUESTION
> MULTIPLE TIMES!
>
> It has already been suggested that you are not using
> the correct arguments for error.
>
> You should verify if one of these systems may have
> its own function called error. Try this:
>
> which error -all
>
> If all else fails, learn to use the debugger.
>
> help dbstop
>
> Knowing what is going on is impossible, since you
> have not told us enough information. Are all systems
> at the same release level of matlab? Have you tried
> any of the things I've suggested here?
>
> HTH,
> John D'Errico

Hello John,

(1) About the multiple posting - sorry, my bad. I got confused with
the threads and thought my message didn't appear, if that's some lame
sort of excuse. Anyway, my bad.

(2) Now, I was referring to the MATLAB contest that is now running on
where I do not have access to the machine that is running my entries
(naturally) and so can't run which on it. Thanks for the suggestion,
though. (On my own machine right here the built-in error function is
called).

Regards,
Felix.

P.S.
Just for the record, I do happen to know how to use the debugger :-).
Alas, I find it a tad difficult to debug code on somebody else's
machine out there to which I have zilch access... :-)

eigenvector wrote:
  
> Hi,
>
> My entries keep failing with the following message:
>
> "Error using ==> error Not enough input arguments."

I have the same problem.

Code runs happily on the test set that was distributed with the
contest, under 7.1.0.246(R14) SP3 on XP SP2. I am making no explicit
use of a function called error, so I imagine that an error arises
when my code is run on the contest problems. These things happen ;-)

However, something then seems to go wrong with the error handling,
leading to a meta-error? It would be nice to have a more informative
message.

Regards,
Robert Macrae

Jan Langer wrote:
>
>
> I just tried to deobfuscate the 28454 entry and compared it to
> 28428.
> It turns out that they are code-wise equivalent. So the obfuscation
>
> causes a difference of 4 seconds. In case someone is interested in
> the deobfus-script, I can send it.

I am not sure the difference is due to the obfuscation. For example,
"pinguin-socken" (the leader as I write this) and "1-2-3 Allerlei"
are identical code, both of them obfuscated. They have a time
difference of over 4.2 seconds.

That penguin may be tough to knock off the iceberg.

1. "spirit of collaboration" vs "spirit of competition" - this is a
competition right? Otherwise, anyone up for a round of cum-ba-ya?
2. Let submitters elect to keep their entry in twilight as long as
they want.
3. Eliminate "time" as part of the score. As long as the entry
doesn't time out let low result win. Pick a shorter time out period
too - like 1-minute or 30 seconds.
4. Since folks seem to tune the algorithm for the test-suite, just
give everyone the test-suite before hand. You still need the right
"signature" (set of moves) for your entry to execute properly
regardless of how you develop it. The ContestTeam could still create
and provide a new test-suite mid-way thru the contest to check
algorithm development vs tweaked solutions issues.

-Trying to remain anonymous

On 2006-04-09 14:52 eigenvector said the following:
> Hi,
>=20
> My entries keep failing with the following message:
>=20
> "Error using =3D=3D> error Not enough input arguments."
>=20
> Now, I tried them on two different machines locally (using different
> versions of Matlab) and everything is ok. But when I submit I get
> crashed. Absolutely exasperating.
>=20
> Perhaps somebody woule kindly suggest a hint as to where I am
> goofing...
>=20
> Apparently this is related to the funciton ERROR but it's disabled...
> so what's going on?
>=20
> Thanks!

You=B4re not allowed to use error() in the contest. From the contest home=
=20
page:

  The following are prohibited:

     * MEX-files
     * Java commands or object creation
     * eval, feval, assignin, etc.
     * inline, function handles, anonymous functions, etc.
     * computer, ispc, which, exit, more, edit, inmem, etc.
     * Shell escape such as !, dos, unix
     * Handle Graphics commands
     * ActiveX commands
     * File I/O commands
     * Debugging commands
     * Printing commands
     * Simulink commands
     * Benchmark commands such as tic, toc, flops, clock, pause
     * error, lasterr, etc.
     * persistent

/PB

PB wrote:
>
>
> On 2006-04-09 14:52 eigenvector said the following:
>> Hi,
>>
>> My entries keep failing with the following message:
>>
>> "Error using ==> error Not enough input arguments."
>>
>> Now, I tried them on two different machines locally (using
> different
>> versions of Matlab) and everything is ok. But when I submit I
get
>> crashed. Absolutely exasperating.
>>
>> Perhaps somebody woule kindly suggest a hint as to where I am
>> goofing...
>>
>> Apparently this is related to the funciton ERROR but it's
> disabled...
>> so what's going on?
>>
>> Thanks!
>
> You´re not allowed to use error() in the contest. From the contest
> home
> page:
>
> The following are prohibited:
>
> * MEX-files
> * Java commands or object creation
> * eval, feval, assignin, etc.
> * inline, function handles, anonymous functions, etc.
> * computer, ispc, which, exit, more, edit, inmem, etc.
> * Shell escape such as !, dos, unix
> * Handle Graphics commands
> * ActiveX commands
> * File I/O commands
> * Debugging commands
> * Printing commands
> * Simulink commands
> * Benchmark commands such as tic, toc, flops, clock, pause
> * error, lasterr, etc.
> * persistent
>
> /PB
>

But man, I'm not using ERROR! That's why I'm so baffed.

But in the code that I saw, some output function were used (disp,
sprintf). Try removing them. If an error occurs there, error can be
called without calling it explicitely.
The fact that no error occurs "at home" can also have to do with the
testsuite, so that different parts are called, or with other (wrong)
values.

Maybe this doesn't help, but.....

So, I've introduced another parameter for you to play with, this is
starting to bore me. :)

Dr Suess wrote:
>
>
> 1. "spirit of collaboration" vs "spirit of competition" - this is
> a
> competition right? Otherwise, anyone up for a round of cum-ba-ya?
> ....
It is a competition, but with certain rules and goals. One of the
goals is collaboration and improving other codes. The darkness and
twilight zones are added to give people time to develop their own
algorithms without being tempted too much for just tweaking others
codes.
Regarding the timing. The timing cost is added for good reasons.
Your proposal to reduce maximum time would make it impossible to test
not optimised algorithms, that can be improved for efficiency. So,
the final program has to be fast enough (otherwise the timing cost is
much too high), but at least you can see the result.
All this to say, The timing cost is a good thing. The only sad thing
is the high tolerance in timing. This is something that is discussed
every contest, but "it's computerlife"...

Robert Macrae wrote:
>
>
> eigenvector wrote:
>
>> Hi,
>>
>> My entries keep failing with the following message:
>>
>> "Error using ==> error Not enough input arguments."
>
> I have the same problem.
>
> Code runs happily on the test set that was distributed with the
> contest, under 7.1.0.246(R14) SP3 on XP SP2. I am making no
> explicit
> use of a function called error, so I imagine that an error arises
> when my code is run on the contest problems. These things happen
> ;-)
>
> However, something then seems to go wrong with the error handling,
> leading to a meta-error? It would be nice to have a more
> informative
> message.
>
> Regards,
> Robert Macrae

Any chance of this, Matthew? :-)

Thanks.

Stijn Helsen wrote:
>
>
> But in the code that I saw, some output function were used (disp,
> sprintf). Try removing them. If an error occurs there, error can
> be
> called without calling it explicitely.
> The fact that no error occurs "at home" can also have to do with
> the
> testsuite, so that different parts are called, or with other
> (wrong)
> values.
>
> Maybe this doesn't help, but.....

It didn't help. But thanks anyway.

Dr Suess wrote:

> 4. Since folks seem to tune the algorithm for the test-suite, just
> give everyone the test-suite before hand. You still need the right
> "signature" (set of moves) for your entry to execute properly
> regardless of how you develop it. The ContestTeam could still
> create
> and provide a new test-suite mid-way thru the contest to check
> algorithm development vs tweaked solutions issues.

I don't think this is a good idea. It is quite easy to write a code
that identifies the problems in the test set and outputs the
precomputed moves. This would take fractions of a second, and provide
a very good solution. Of course, such codes could be disqualified,
but there are more clever ways around it.

eigenvector and Robert Macrae, I did some investigation and this "Not
enough input arguments" error turned out to be spurious. I
introduced a bug a couple days ago while tightening up our security.
I just fixed it and am currently re-scoring the affected entries.
I'm sorry the frustration this caused you both.

Matthew Simoneau wrote:
>
>
> eigenvector and Robert Macrae, I did some investigation and this
> "Not
> enough input arguments" error turned out to be spurious. I
> introduced a bug a couple days ago while tightening up our
> security.
> I just fixed it and am currently re-scoring the affected entries.
> I'm sorry the frustration this caused you both.

Thanks a lot, Matthew!

Dr Suess wrote:

> 4. Since folks seem to tune the algorithm for the test-suite, just
> give everyone the test-suite before hand.
...
> The ContestTeam could still
> create
> and provide a new test-suite mid-way thru the contest to check
> algorithm development vs tweaked solutions issues.

As Stijn pointed out the first part wouldn't work, but I wonder
whether a variant of the second might? Hiding some information could
reduce the power of tweaking.

Say run entries twice, once against the complete test-suite and once
against a fixed subsample. Publish only the scores of the reduced
sample as soon as the code is run. When you come to a competition
milestone, publish full scores for all entries received to that
point.

A negative result would be a flurry of entries approaching each
milestone as the more competitive take the opportunity to tweak, but
this would be balanced by an absence of tweaking between milestones
-- in-between no-one is certain which is actually on top. There
should also be little point to obfusticating code as it would all be
uncompetitive by the next milestone.

Robert Macrae

Matthew Simoneau wrote:

> "Not enough input arguments" error turned out to
> be spurious.

Much appreciated.

Robert Macrae

Anders wrote:
>
> Has anyone written a good unobfuscator yet?
I thought I've read somewhere that someone proposed to give his
version of unobfuscator. I've made one myself (undocumented and in
Dutch). Interested?
Stijn

> I thought I've read somewhere that someone proposed to
> give his version of unobfuscator.

This was Jan Langer, message 46.

If anyone would like to submit their deobfuscator to the File
Exchange, we can be sure to publish it right away. Just drop me an
e-mail after you submit it and I'll push it out.

On 2006-04-09 22:41 eigenvector said the following:
> PB wrote:
>>
>> On 2006-04-09 14:52 eigenvector said the following:
>>> Hi,
>>>
>>> My entries keep failing with the following message:
>>>
>>> "Error using =3D=3D> error Not enough input arguments."
>>>
>>> Now, I tried them on two different machines locally (using
>> different
>>> versions of Matlab) and everything is ok. But when I submit I
> get
>>> crashed. Absolutely exasperating.
>>>
>>> Perhaps somebody woule kindly suggest a hint as to where I am
>>> goofing...
>>>
>>> Apparently this is related to the funciton ERROR but it's
>> disabled...
>>> so what's going on?
>>>
>>> Thanks!
>> You=B4re not allowed to use error() in the contest. From the contest
>> home
>> page:
>>
>> The following are prohibited:
>>
>> * MEX-files
>> * Java commands or object creation
>> * eval, feval, assignin, etc.
>> * inline, function handles, anonymous functions, etc.
>> * computer, ispc, which, exit, more, edit, inmem, etc.
>> * Shell escape such as !, dos, unix
>> * Handle Graphics commands
>> * ActiveX commands
>> * File I/O commands
>> * Debugging commands
>> * Printing commands
>> * Simulink commands
>> * Benchmark commands such as tic, toc, flops, clock, pause
>> * error, lasterr, etc.
>> * persistent
>>
>> /PB
>>
>=20
> But man, I'm not using ERROR! That's why I'm so baffed.

Then I suppose I didn=B4t read your post careful enough, sorry about that=
=2E

/PB

When I try to run the top entry, I get the following error:
 Undefined command/function 'mode'.
 
My Matlab is 7.0(R14) and it doesn't seem to have this function. Is
there any way to get around this?
 
Thanks

Vijay Lax wrote:
>
>
> When I try to run the top entry, I get the following error:
> Undefined command/function 'mode'.
>
> My Matlab is 7.0(R14) and it doesn't seem to have this function. Is
>
> there any way to get around this?
>
> Thanks

Maybe a problem with the installation? It should be in

[your matlab dir]\toolbox\matlab\datafun\mode.m

Stijn, I added the most recent blog posting back to the top of the
Queue and Top 20 page. I also removed grade.m and rescored the
effected entries. Thanks.

I have a couple more contest ideas to throw on the pile:

1. Queue summary

On the "Queue and Top 20 web page", it would be nice to have a
summary of the current queue. Maybe just a line at the top of the
page:

   11 entries in queue, 13 minute delay

The delay time is time between current time and when the last item on
the queue was submitted. You could even get fancy and add mid-contest
challenge stats:

               *Queue* *Length* *Time*
                 Total 11 13 min
   1000 char challenge 3 4 min

2. Twilight/Daylight adjustments

Like many other posters, I also support a longer twilight period. I
think 2 days is probably enough. Then, I propose a twilight/daylight
hybrid where each day the top 20 entries are made visible. This lets
everyone share code and ideas, but would help minimize tweak wars
because everyone's actions are not immediately visible. Finally, full
daylight could be the last day (or 2, max) of the contest and
tweaking would reign supreme...

Steve

Steve wrote:
>
> the queue was submitted. You could even get fancy and add
> mid-contest
> challenge stats:
>
> *Queue* *Length* *Time*
> Total 11 13 min
> 1000 char challenge 3 4 min
>

That was supposed to be a nicely formatted 3x3 table but it didn't
work out...

Mihi wrote:
> So in the end I went back to the search for better clustering
> algorithms (some of my ideas only make very good guesses on the
> size
> of a cluster but are faster than the one currently used, maybe they
> will result in a better trade-off between points and runtime)
Maybe to look for some "almost same results" further than one step
(so keep a list of the xxx best combinations, and look deeper if they
are close to each other)?

Min Poh wrote:
>
>
> The next MATLAB Programming Contest ...

Will we be seeing some of the usual summary plots on the statistics
page this contest?

Rick St.Pierre wrote:
> Seems to me that this problem is ripe for some image analysis
> solution, but that is definitely not my area of expertise.
Not mine either, but I'm affraid that time is too critical here for
much smarter ideas. Maybe except when some characteristics can be
seen easy, so that special algorithms work only on special problems.
A possibility is looking to the amount of different values. With
binary "images", more can be done, I suppose.

Hi Contest_Guy

runcontest will try to grade all the boards in the testsuite, most
likely you are getting this warning in other thanthe first board.

Lucio

Contest_Guy wrote:
>
>
> I am just doing some preliminary test from the .zip files I
> downloaded and I am looking at the first board from the testsuite.
> I
> use for moves in the solver
>
> 'moves[1 1 0]'
>
> and I get the following error when I run 'runcontest':
>
> 'Warning: You cannot pop single blocks.
>> In C:\MATLAB6p5\work\Matcon13\blockbuster\grade.m at line 49
> In C:\MATLAB6p5\work\Matcon13\blockbuster\runcontest.m at line
> 42'
>
> It should not say that I cannot pop single blocks as I checked the
> data for the first board. There is a '3' in position (1,1) followed
> by a '3' in position (2,1) and according to the rules I am allowed
> to
> pop two or more contiguous blocks in a N-E-W-S direction. Can you
> please tell me what's going on?

I set the code in 'runcontest' to only examine the 1st board in the
test suite; in other words I set the "for" loops for both

(1) '% Run the submission for each problem in the suite.'
(2) '% Grade all answers.'

to

'for i=1:1'

Again, I have checked the data of the first matrix, namely that of
'testsuite(1).boards' and found that a pop was indeed valid with the
move

'moves = [1 1 0]'

And, if the move is valid, the updated score seems to be wrong; i
checked the sum of the first matrix and it summed to be 1343. With a
pop in the indicated position, there are two other blocks in the
south direction with a value of '3'. There are three '3's total with
a pop score of nine. A pop in this location should then result in a
score of 1343 - (3*3) = 1334. However, the score indicated is 1343.

Also, just to clarify, I am assuming both kinds of moves (pops &
swaps) can be done anywhere on the board and not just on the bottom
most line (the last row of the matrix).

  
Lucio wrote:
>
>
> Hi Contest_Guy
>
> runcontest will try to grade all the boards in the testsuite, most
> likely you are getting this warning in other thanthe first board.
>
> Lucio
>
> Contest_Guy wrote:
>>
>>
>> I am just doing some preliminary test from the .zip files I
>> downloaded and I am looking at the first board from the
> testsuite.
>> I
>> use for moves in the solver
>>
>> 'moves[1 1 0]'
>>
>> and I get the following error when I run 'runcontest':
>>
>> 'Warning: You cannot pop single blocks.
>>> In C:\MATLAB6p5\work\Matcon13\blockbuster\grade.m at line
49
>> In C:\MATLAB6p5\work\Matcon13\blockbuster\runcontest.m at line
>> 42'
>>
>> It should not say that I cannot pop single blocks as I checked
> the
>> data for the first board. There is a '3' in position (1,1)
> followed
>> by a '3' in position (2,1) and according to the rules I am
> allowed
>> to
>> pop two or more contiguous blocks in a N-E-W-S direction. Can
you
>> please tell me what's going on?

Stijn Helsen wrote:
> Robert Macrae wrote:
>> 1) Search for singletons that have 3 neighbours that are
>> different colour to it but same to each other.
>>
> And doesn't help much indeed

My experience with most new tacks 8-/

Robert

Lucio:

Check the Blog comments from yesterday. I posted a 16 move solution
there last night.

Lucio wrote:
>
>
> Marc Laetzel took my challenge in the mid-contest analysis and beat
> me clearing the board in only 17 moves.
>
> Well done Marc
>
>>Hi Lucio,
>>
>>Even better:
>>
>>moves = [ ...
>>19 6 2; ...
>>20 6 0; ...
>>20 6 0; ...
>>20 7 0; ...
>>20 7 0; ...
>> 9 7 0; ...
>>10 7 0; ...
>>11 8 0; ...
>>11 8 0; ...
>>13 8 0; ...
>>14 9 0; ...
>>20 8 0; ...
>>16 7 0; ...
>>17 7 0; ...
>>18 8 0; ...
>>20 4 0; ...
>>20 4 0; ...
>>]
>>
>>Regards
>>
>
>>Marc

Cobus Potgieter wrote:
>
>
> It seems that Don Antonio has plunged us back into darkness! Surely
> some limit should be imposed on the number of entries any single
> person can have in the queue at a specific time.

I agree and propose a queue limit of two per user (IP even?)

I just saw that it seems that there is a minimal result following (in
the exponential graph) a linear relationship
(Rmax=exp(-0.036*t+2.9)). But, the minimum is already lower!!
This doesn't mean anything, but, ..., you have to do anything while
waiting for getting results, and not having any smart ideas anymore...

David Jones, your poptastic 20 was one second late! ;)

Good contest. Somehow I hope there's a new algorithm in that 700+
gueue, blowing away all our parameters.

We've gotta go with the timestamp. Sorry, David. The one that snuck
in one second isn't valid, but we'll score it anyway just to see.

Ack! I didn't know it ended at 5PM .. damn work! Guess I should
read more closely. :)

Oh well, all my work on CalculateMoves goes out the window. I had
about a 10% speed improvement using diff() for finding like elements.
 I'll post it if anyone is interested.

Thanks for a great contest! Best of luck to those of you still in
the queue!!!

Rick.

Mihi wrote:
>
> I've posted a new clustering algorithm for the inner loop, which is
> faster then the one which is used by 99.99% of the submission. But
> the general layout of the outer loop and the first level clustering
> +
> move processing is still the same (the much tweaked on which
> originally came from markus).
>
> As far as I can see my "new" algorithm was able to take the top of
> the “Best Result” competition. Now I can only wait if the other
> submissions of mine, which where more tuned towards the result vs.
> runtime sweet spot, are able to beat the tweak fest that broke out
> 1h
> before the deadline.

Congratulations Mihi for taking the "Best Result" out of my hands.
I should have the low score I got shortly after midnight could not
hang on until the end.

It will be impressive if the same speedier inner loop wins the main
prize. (it is 6:38pm now with several submissions from Mihi still in
the queue)

-- David Jones

Mihi wrote:
>
> I'm definitly interested :)

I put it at:
 <http://home.comcast.net/~rickstpierre/CalculateMoves.m>

No sense taking up space here.

Great job snagging the low result! It will be interesting to see how
it finishes up.

Rick.

Rick St.Pierre wrote:
>
>
> Mihi wrote:
>>
>> I'm definitly interested :)
>
> I put it at:
> <http://home.comcast.net/~rickstpierre/CalculateMoves.m>
>
> No sense taking up space here.
>
> Great job snagging the low result! It will be interesting to see
> how
> it finishes up.
>
> Rick.
We also rewrote that part of the algorithm, got a 25% speed increase.
Anyone that's interested can have a look at the Chuck Norris series
of entries which are the unobfuscated versions of the sirron kcuhC
entries.

Its horribly frustrating to lose to a tweaked version of your own
entry.:-(( But hey, thats the game :-)

Cheers
Hannes & Cobus

Mihi, I am interesting in seeing the new inner loop that you have
implemented. Could you post a well-readable version here? Now that
the contest is over my motivation for clicking through the
submissions and decrypting code has abruptly vanished.

David Jones wrote:
>
> For the next contest, it would be nice to have 2 days of twilight
so there would be a greater diversity of algorithm development

Seconded.

Robert

Yi Cao wrote:
>
>
> Can anyone explain why by changing nthroot ( from sirroN kcuhC 4)
> to
> .^ the winning entry was able to reduce result by more than 100
> points and at at the same time reduce CPU time by 0.4 seconds? I
> can
> undstand reducing CPU time, but connot explain why it also improved
> result. Will nthroot and .^ give different numerical solution?
>
> Yi

Take a look "edit nthroot" .. they do some error correction which
will change the result slightly (as well as increase compuatation
time).

Regarding the twilight phase, I think it should be extended to at
least 2 days (maybe 3). Not only does it give people more time to
come up with good solutions, it allows more people to get involved.

Few other suggestions:
1. Have twilight end sometime on Saturday. This gives a bit of time
for us 40+ hour workers to work on a solution. :)

2. While the Usenet thread is, well, useful, having a contest web
forum may allow for more discussion. At the very least, it would
allow different topics to be in separate threads, plus I'd guess more
people are familiar with them over Usenet.

Rick.

Yi Cao wrote:
>
>
> Can anyone explain why by changing nthroot ( from sirroN kcuhC 4)
> to
> .^ the winning entry was able to reduce result by more than 100
> points and at at the same time reduce CPU time by 0.4 seconds? I
> can
> undstand reducing CPU time, but connot explain why it also improved
> result. Will nthroot and .^ give different numerical solution?
The change in result must come indeed from numerical calculation
differences (calculation error compensation). During the contest
there were more things like that (with even more simpel
calculations). I think now mainly about the topic of the
depth_factor. Replacing them by the values divided by 10, and then
remove that division later when the values are used, changed the
result!! That must also have to do with rounding errors!

Yi Cao wrote:
> Thanks for the reply. I have checked the difference between these
> two. For block number from 1 to 10, the difference is only about
> e-14
> to e-15. It is interesting to know such small difference will cause
> result so different. The difference does have positive effect for
> some cases but also has negative effect for other cases. Overall,
> .^
> solution beated nthroot with the particular parameter set. I did
> remember during some stage of the contest, nthroot has beated .^
> with
> different parameter set.
>
> Yi
One reason it could cause a different result is a block of 2 4's and
a block of 4 2's should have the same score (8). However, with these
slight differences, one has a slightly larger score than the other
and is therefore preferred.

Rick.

Yi Cao wrote:
>
>
> Can anyone explain why by changing nthroot ( from sirroN kcuhC 4)
> to
> .^ the winning entry was able to reduce result by more than 100
> points and at at the same time reduce CPU time by 0.4 seconds? I
> can
> undstand reducing CPU time, but connot explain why it also improved
> result. Will nthroot and .^ give different numerical solution?

I was the one who replaced "nthroot" with the function "a37". ("utr"
is a pseudonym of my pseudonym.) I did this substitution in the
leaders as the deadline approached, and also in several entries in
the queue. I was quite lucky to have selected Hannes and Cobus'
obfuscated "sirroN kcuhC 4" as one of those entries.

The very first time that nthroot(x,n) appeared in a leading entry, I
submitted the obvious tweak x.^(1/n), as did several others. I was
perplexed that it gave different (and worse) results. (Yi Cao
mentioned this phenomenon earlier in this thread.)

I dug into the nthroot source code, and saw that it does one
iteration of Newton's method. I figured that a stripped-down version
of nthroot would be a good tweak to save for later in the contest.

My first try at tweaking "sirroN kcuhC 4", which was called "typ",
actually failed. My stripped-down version was a bit too naked,
because it returned NaN's where x==0. That caused problems elsewhere
in the code.

"typ2" had the version of a37 that was purely x.^(1/n), which was
winning because I got:

(1) the speedup of Hannes and Cobus' algorithmic improvement

(2) the weird benefit of obfuscation [still waiting for that
explanation, MathWorks!]

(3) the speed of x.^(1/n) over nthroot

(4) the blindingly lucky difference in results between those two
functions

Thanks for another fun contest, Mathworks.

Mihi wrote:
>
> The human readable version of my entries came in at place 20 <http://www.mathworks.com/contest/blockbuster.cgi/view_submission.html?id=31882>
> and is almost as fast as the obfuscated ones (or am I to stupid to
> obfuscate properly for matlab? :) ).
>
Mihi, I found a bug of your above mentioned code. By changing to the
parameter set to match the winning entry, your code will go into an
infinite loop for case 22 of the test suit. You may wish to look at
it.

Yi

Mihi wrote:
> The human readable version of my entries came in at place 20 <http://www.mathworks.com/contest/blockbuster.cgi/view_submission.html?id=31882>

So the algorithm is still the same, you just saved some processing
time, right?

The bug is only in the human readable version :) I found it as well,
it happens in rare cases when there are cycles in the list of
clusters. I did a quick fix in the obfuscated versions, which is just
counting the number of steps and stops after 15.

Yi Cao wrote:
>
>
> Mihi wrote:
>>
>> The human readable version of my entries came in at place 20 <http://www.mathworks.com/contest/blockbuster.cgi/view_submission.html?id=31882>
>> and is almost as fast as the obfuscated ones (or am I to stupid
> to
>> obfuscate properly for matlab? :) ).
>>
> Mihi, I found a bug of your above mentioned code. By changing to
> the
> parameter set to match the winning entry, your code will go into an
> infinite loop for case 22 of the test suit. You may wish to look at
> it.
>
> Yi

I agree we should have 2 days of twilight in the next contest
to increase the opportunity for more diverse and sophisticated
algorithm development.

I don't think we need to limit the entries per person, since our
friend Don Antonio didn't gain much from his voluminous submissions.
I think we also saw Tim submit under two different aliases (the
cyclist and utr), so it would be very difficult for the Contest Team
to enforce this kind of rule.

I do think it would be good to have a rule that any entry that is
IDENTICAL to the leading entry will be disqualified without even
being executed. (and this would be quick easy to check). You could
indicate this the same way that Timeout's are shown, so the author
would know what happened.

I think obfuscation is part of the contest, and it is interesting how
it comes and goes at various points. Trying to make a rule against
it would be very difficult to do right.
As far as I can tell, Tim was able to tweak some obfuscated code very
quickly to take the prize, so I don't think obfuscation is as big an
obstacle as it initially seems. Tools may already exist to re-format
MATLAB code to reveal its structure, and after that it is just a
matter of variable names. Especially when you are already familiar
with the key algorithms from earlier in the competition, it may not
be that hard to figure out.

-- David Jones

Mihi wrote:
>
> Last but not least there are some things I would like to see in
> future matlab contests:
>
> 1. longer twilight phase, since this is usually the phase where the
> major algorithmic ideas are developed.
>
> 2. a limit of entries one person can post in a certain timeframe
> (lets say 10/hour), which may be lifted on the last day to give the
> tweakers some time to play around :)
>
> 3. clear rules on obfuscation or at least a default obfuscator

I disagree to David, who was one of the spammers, in some points!

> I don't think we need to limit the entries per person, since our
> friend Don Antonio didn't gain much from his voluminous
> submissions.

Even if he didn't gain anything, the queue was sometimes overfull and
as well the list of submissions.

> I think we also saw Tim submit under two different aliases (the
> cyclist and utr), so it would be very difficult for the Contest
> Team to enforce this kind of rule.

But if they track the submitting IP adress it would at least mean
much more effort to do spamming. Or if you have to copy a number from
an image as spam prevention, even Don Antonio would get weak hands
after his first hundred submissions.
 
> I think obfuscation is part of the contest, and it is interesting
> how
> it comes and goes at various points. Trying to make a

But it definitely decreases the fun a contest makes. It is about
programming and algorithms, not about decrypting code, right?

> matter of variable names. Especially when you are already familiar
> with the key algorithms from earlier in the competition, it may not
> be that hard to figure out.

But how to get familiar if you start later into the contest and
submissions 1 to 263 are obfuscated??
 
All scripts should run through an obfuscator before being evaluated,
so that there is no positive effect on computation time.

The contest team should disqualify all obfuscated versions, at least
those that are on top when a phase ends. In the next contest, I mean.

David Jones wrote:
>
>
>
> I do think it would be good to have a rule that any entry that is
> IDENTICAL to the leading entry will be disqualified without even
> being executed. (and this would be quick easy to check). You could
> indicate this the same way that Timeout's are shown, so the author
> would know what happened.
>
> I think obfuscation is part of the contest, and it is interesting
> how
> it comes and goes at various points. Trying to make a rule against
> it would be very difficult to do right.
> As far as I can tell, Tim was able to tweak some obfuscated code
> very
> quickly to take the prize, so I don't think obfuscation is as big
> an
> obstacle as it initially seems. Tools may already exist to
> re-format
> MATLAB code to reveal its structure, and after that it is just a
> matter of variable names. Especially when you are already familiar
> with the key algorithms from earlier in the competition, it may not
> be that hard to figure out.
>

But how about the combination of the two, i.e. obfuscation of the top
entry without any change? I think this should be disallowed.
Otherwise, it is just too easy for obfuscation to get on top. Then
the consequence is that everyone has to submit obfuscated code to
avoid disadvantage. If it is technically difficult to rule out
obfuscation submission, we realy should make code in the queue and
top n entries unreadable for certain period. This solution should
helps both points, i.e. identical submission and obfuscation of top
entry without any change.

Yi

As suggested earlier in another thread by someone else, I think one
solution that automatically eliminates or reduces the impact of code
obfuscation, obssesive parameter tuning and fear of early code
submission
but yet keeps the spirit of the contest as being a collaborative work

to optimize the algorithm and the code is to have multiple twilight
phases (perhaps 2days-3days and 2days).

I also wish there were more contests, perhaps one in each quarter :)

Thank you the Matlab contest team for the great work.

Alan Chalker wrote:
> I agree it'd be nice to have contests more than
> semi-annually, but I suspect that is asking too
> much of The MathWorks team.

You're right Alan that it's hard for us to do them much more often,
but we definitely enjoy hosting them twice a year. Someday I can
imagine letting people set up their own contests whenever they want,
but it might be a while before we get there.

Lots of good ideas here. We probably will modulate the twilight time
one way or another in the next contest. I want to float two ideas to
see what people think.

One is to introduce subjective voting of some kind. You know how some
picture sites have a button that says "This picture has objectionable
content"? We could have one of those, and if n people from n
different IP addresses (choose appropriate value of n) clicked on
"This entry has been maliciously obfuscated" we could yank it. That
gets us (the contest team) out of the business of passing judgment on
code.

The other idea is less well thought out, but suppose one phase of the
contest involved letting people submit new tests to the test suite
instead of solutions. That might prevent (or delay) the onset of
hard-wired algorithm brittleness. There are obvious problems with
simply submitting tests; clearly anyone with knowledge of the tests
can use that information unfairly. At the same time, modifying the
test suite would definitely exercise the entries in a new way. So I'm
hoping that others will have some useful twists on this idea.

-Ned Gulley.
The Contest Team

Yi Cao wrote:
> What do you exactly mean? Could you point to some IDs for
> reference?
> Thanks.
>
> Yi
I've tried it a couple of times. I can't find them all, but if you
compare 30242 and 30264. Except for some microscopic changes, and
the processboard which isn't done after the last step, the only
difference is in the beginning, the values of depth_factor, but with
the same "principle values". And the result is different.
Any cause except rounding errors?

Rick St.Pierre wrote:
>
>
> Yi Cao wrote:
>> Thanks for the reply. I have checked the difference between
these
>> two. For block number from 1 to 10, the difference is only
about
>> e-14
>> to e-15. It is interesting to know such small difference will
> cause
>> result so different. The difference does have positive effect
.

In my opinion the best way to stop people stop spamming the queue was
suggested by Matthew in comment to the blog:

\begin{quote}
 We were kicking around the idea of only scoring one entry per
contestant on each pass through the queue. That is, we’d only score
one entry from each author until everyone waiting has had one of
their entry scored. The drawbacks are that it is a little tough to
explain and it would encourage contestants to submit under different
names.
\end{quote}

The possible drawback he sees might be eliminated by awarding an
additional price to the contestant who makes the biggest cumulative
improvement to the top score over the whole curse of the contest.
(However, it might be difficult to find a way of weighting
improvements relatively to contest phase - improvements are much
greater at the beginning of the contest.)

Submitting entries under different names would decrease ones chances
to win such an award.

Another good idea I've read somewhere here is to run entries multiple
times instead of one single run and to score the mean performance.
This would make spamming identical unneccessary at least for entries
depending on random numbers.

Finally I would like to thank the contest team as well as the other
contestants - this contest (actually, my first) was really fun!

Ned:

I'm glad you are thinking about making changes. A few comments:

-The 'voting' on obscufated entries to remove is an interesting
thought, but unfortunately ripe for abuse. You suggested tracking it
by IP address. According to the statistics page, there were 175
different usernames used in this contest (albeit some are the same
person with different aliases). To be effective, 'n' would need to
be some fraction of that, let's say 25-50. Unfortunately, many
people have easy access to 25-50 IP addresses with a little bit of
work, be it through forcing a DHCP renewal on a commercial broadband
provider, to different computers in a university computer lab, to
using open-proxies, to using co-workers computers after hours. So
someone could malicously remove entries from the queue. Perhaps a
final 'gut-check' by one of your staff once an entry is flagged for
removal would address this. I also think that a similar IP-based
thing in the submission queue might be somewhat prone to the same
issues.

-I like the test-suite submission idea. Perhaps one way to deal with
it is to have a new 'pre-darkness' time frame during which the
problem rules and test suite are released but the testing queue
remains closed. During that time people could play with the problem
and test suite and then submit to the contest team suggestions for
additional tests. Your team would make a judgement somehow on
whether to include them when the queue opens up, but wouldn't
necessarily need to notify us until after the contest what was
included in the queue. Or perhaps you solicit them throughout the
contest but don't add them until certain milestones are met.

-An alternative idea regarding the test-suite is to only actively
test against a random subset of the suite in any given run. This
would make 'tweaking' to the suite a lot harder since we wouldn't
know if the score changes were due to code changes or suite changes.
At each major milestone, you could stop and have the top X entries in
the rankings run against the full suite to get the official scores.

-I'd also like to make a suggestion regarding the timing variations.
They aren't controllable or repeatable to a certain degree. I suggest
you round those results off to something about the uncertainity,
maybe a tenth or a fifth of a second. Maybe as an alternative or as
a tie-breaker you could use something like the total calls column in
the profiler, with the assumption that less total calls is more
efficient code.

-One final suggestion, what about making the contest a full 'day'
cycle. Start with darkness, then a 'dawn/twilight' phase, then
daylight, then 'dusk/twilight', and finally darkness again. We
essentially already have the last 2 steps compressed into a few hours
at the end due to the queue spamming, but by drawing them out,
flooding the queue at the end wouldn't be needed. The contest also
won't come down to whomever lucked out and choose the correct entry
to tweak in the last few minutes the queue was open.

Ned Gulley wrote:
>
>
> Alan Chalker wrote:
>> I agree it'd be nice to have contests more than
>> semi-annually, but I suspect that is asking too
>> much of The MathWorks team.
>
> You're right Alan that it's hard for us to do them much more often,
> but we definitely enjoy hosting them twice a year. Someday I can
> imagine letting people set up their own contests whenever they
> want,
> but it might be a while before we get there.
>
> Lots of good ideas here. We probably will modulate the twilight
> time
> one way or another in the next contest. I want to float two ideas
> to
> see what people think.
>
> One is to introduce subjective voting of some kind. You know how
> some
> picture sites have a button that says "This picture has
> objectionable
> content"? We could have one of those, and if n people from n
> different IP addresses (choose appropriate value of n) clicked on
> "This entry has been maliciously obfuscated" we could yank it. That
> gets us (the contest team) out of the business of passing judgment
> on
> code.
>
> The other idea is less well thought out, but suppose one phase of
> the
> contest involved letting people submit new tests to the test suite
> instead of solutions. That might prevent (or delay) the onset of
> hard-wired algorithm brittleness. There are obvious problems with
> simply submitting tests; clearly anyone with knowledge of the tests
> can use that information unfairly. At the same time, modifying the
> test suite would definitely exercise the entries in a new way. So
> I'm
> hoping that others will have some useful twists on this idea.
>
> -Ned Gulley.
> The Contest Team

Markus wrote:
>
>
> Mihi wrote:
>> The human readable version of my entries came in at place 20 <http://www.mathworks.com/contest/blockbuster.cgi/view_submission.html?id=31882>
>
> So the algorithm is still the same, you just saved some processing
> time, right?

Yes the general outline of the algorithm is still the same (yours),
but I'm using a slightly different clustering algorithm for the inner
loop which is more efficient if you only want to know the size of the
resulting clusters, and don't care where they are or what the board
would like when you bust them.

Yes, Tom makes a good point that I contributed lots of submissions
to the queue, but I also contributed a lot in terms of
pushing both the lowest result (until Mihi surpassed me)
and pushing the best score on many occasions.
I even recognize several of my own originally contribued lines
in the unobfuscated version of the winner.
(e.g., adjusting the board values after two-thirds of the moves,
"if count==flip, board = ...")

Anyway, I think it adds flavour to the contest to have a variety
of styles of participation, from prolific tweakers
to the occasional submission that makes a big leap.
If everyone waited until 15 minutes before the deadline
to submit their code, it wouldn't be as much fun.

I wonder if the Contest Team has considered farming out
the queue of submissions to a couple of (identical) compute servers
for execution (in parallel). This could speed up the process of
clearing the queue. ... although, part of the challenge
of actually WINNING the contest seems to be the abilty
to hold back your potentially winning entry while having
a very accurate estimate of the expected score on the
unknown suite of input data. (Well, ... or waiting to do
a quick tweak/update on a later submission while it is still
in the queue).

After the queue closed at 5:00PM, it still took until
roughly 1:00AM before the final winner was known for sure.
That's about 8 hours !! I think the queue is an inherent
part of the contest.

It's up to the Contest Team to decide the rules.
I'll do my best to be here next time.

-- David Jones

Tom wrote:
>
>
> The [man] with almost a quarter of the submissions attributed to
him
> doesn't think spamming the queue is harmful to the gaming public?
> whodathunkit.
>
> At a minimum it's disappointing to see a queue of 20 submissions
> from
> the same guy knowing that you won't be ranked for at least half an
> hour.
>
> On the unreadable code...
> The obfuscation is contrary to the sprint and the intent of the
> contest.
> Even if it can't be stopped by the server, it's rude.
> The very nature of the contest is communal reuse.
> If this idea frustrates you, then you're playing the wrong game.
>
> If it's that important to the spammers and the obfuscaters, then
> here's my testimony: "Yippee! you're all winners!! you are the l33t
> MATLAB h4x0r!!"
> so now that you're the big winner, please back off and let the
> contest be fun and accessible again.

Ned Gulley wrote:
>
> Someday I can imagine letting people set up their own contests
> whenever they want, but it might be a while before we get there.

As someone who teaches 800 engineering students (yikes!),
I would be interested in setting up a Matlab Contest
within my own course. Is there any chance you could release
some version of the contest software? I imagine the code
that accepts entries, stores them in the database,
and publishes the results on the web is actually quite compact,
since it is built on top of existing Matlab toolboxes.
Of course I realize there is a lot of brainpower that goes into
the actual design of the problem and selection of the test suite,
as well as commentary and analysis of the submissions afterwards.

Maybe you could release a "beta version" to a few universities
and even sell it as a Contest product/toolbox if it works out.

-- David Jones

> I want to float two ideas to see what people think.
>
> One is to introduce subjective voting of some kind. You know how
> some
> picture sites have a button that says "This picture has
> objectionable
> content"? We could have one of those, and if n people from n
> different IP addresses (choose appropriate value of n) clicked on
> "This entry has been maliciously obfuscated" we could yank it. That
> gets us (the contest team) out of the business of passing judgment
> on
> code.

I HATE the idea of some participants voting to yank another person's
entry. This would replace "malicious" obfuscating with "malicious"
voting. I would much prefer that the Contest Team either publish the
easy steps for unobfuscating and "smart indenting", ... or else even
automate this process, just like they have done with the "diff"
command.

In many cases, people consider code to be obfuscated when the
variable names do not explain what the code is doing.
Therefore, a rule against obfuscation is virtually equivalent
to a rule REQUIRING people to include meaningful variable names
(or even instructive comments!) that teach other participants
how to understand their algorithm. I think this is unreasonable.
In fact, ... what if someone submits code with variable names
that make perfect sense in a language other than English,
like German, French, Vietnamese, or Tagalog?

I think we should accept obfuscation as part of the contest.
Even Tim Vaughan attributes his winning tweak to his
function "a37" as a replacement for nthroot,
and then challenges us to understand it without any explanation.
(At least I think he said that.)
I think it both instructive and an important skill
to be able to understand what a short piece of code does
without having to rely on variable names to explain it to you.
In fact, ... when debugging faulty code (written by students),
this is what we do: we have to look past the variable names
that are what the author INTENDED the code to do,
and use our knowledge of MATLAB functions and syntax
to understand what the code ACTUALLY does.

-- David Jones

Ned Gulley wrote:

> Lots of good ideas here ...

Let me express one idea that may be the minority view, with respect
to twilight.

Personally, during EVERY phase of the contest, I get a feeling that
that particular phase is too short. During darkness, I wish there
was more time to develop a really solid algorithm. During twilight,
I wish the same thing. I think these sentiments are widely held.

However, I also wish that daylight were longer, because there are
generally lots of ways for the shared code to get further optimized,
even as the contest ends.

The only solution to this is to make the whole contest longer, but I
really REALLY do not want that! (Nor would my boss!) In the end, I
think the pressure of the deadlines makes things better, that the
relative lengths of the phases is quite good where it is, and that
daylight should extend right up until the end.

Ned Gulley wrote:

> Lots of good ideas here.

I think the contest team is getting a lot of good feedback from these
discussions. (Is "contest team member" a full-time position at
MathWorks, or do you all have to go back to your "day jobs" now, like
the contestants do?)

I'll add one more point, and I am not the first to say this. I think
that by far the most frustrating part of the contest is the CPU time
variability. Quantum mechanics dictates that there must be some
variability, but every effort to reduce it would mean more genuine
(but admittedly small) efficiency improvements would be recognized,
instead of being swamped by noise.

This would simultaneously help solve the queue flooding problem,
because a lot of "parameter tweaking" is not actually finding better
parameters, but rather just getting lucky on timing.

I would like to see the contest team focus all their efforts on this
problem, rather than fighting obfuscation, etc., which in my opinion
has only a minor impact on the contest.

Ned Gulley wrote:

> The other idea is less well thought out, but suppose one phase of
the contest involved letting people submit new tests to the test
suite instead of solutions.

A variant would merge the "tests" and "solutions", making the
programs into warriors competing face to face against each other.
This has worked well in Corewars and Gridwars, for example. It
encourages adaptivity and robustness, because you are not certain
exactly what you face, but leaves plenty of room for interesting and
complex code. Looking back at past Matlab competitions, it would be
easy to have competition between multiple trucking companies or
ants...

The mechanism in the ongoing corewars tournament is "king of the
hill": there are 20 warriors on the hill, and to get on you have to
fight them all and score higher on average than at least the worst.

The key problems I think would be to check that the contest idea
gives rise to a rich set of possible strategies, rather than
enforcing a uniform speed+greed approach, and to prevent co-operating
warriors from locking the hill.

David Jones wrote:
>
> score = k1 * result + k2 * exp(k3 * floor(runtime));
>
> Notice the runtime is truncated to the nearest second.

Neat idea.

A similar one, used by some record-recording bodies, is to ignore
improvements in a record if they improve by less than some amount D
on the existing record. That way the top 20 would consist of 20
different warriors each separated by at least D.

Its mainly a cosmetic change, but it should make the top 20 a more
interesting selection.

Robert

Yi Cao wrote:
>
> By the way, David, what made you use nthroot(board,2/(1+rot))
> instead of nthroot(board, 1/rot) to restore board values?

I first tried restoring the board values exactly
using "nthroot(board, 1/rot)", but then I played
around with various values, and 2/(1+rot) was a good compromise.
... and why it is best to do this about 2/3 of the way
through the moves was also empirically determined.

-- David Jones

Yi Cao wrote:
>
> Therefore, using nthroot, cluster 1 will be selected while using .^
> cluster 8 is selected.
That's what I meant with a different ordering (or sorting).