Thread Subject:

Fall 2008 MATLAB Contest, November 5th-12th

The Fall 2008 MATLAB Contest starts Wednesday at noon EST (17:00 GMT). It runs for a week, but includes several mid-contest awards. The programming challenge and full rules will be posted at the start of the contest on our website:

http://www.mathworks.com/contest/

We encourage you to reply on this thread to discuss any aspect of the contest, including rule clarifications, potential strategies, and commentary on the action. For administrative issues, e-mail us at contest@mathworks.com. For announcements, see our blog:

http://blogs.mathworks.com/contest/

Spoiler: We're introducing a form of limited head-to-head competition this time, bringing a new twist to our usual contest format.

Good luck, everyone. See you in the queue!

The concept of using the leading entry as the 'competing ants' is VERY COOL. This will introduce some very interesting dynamics to the competition.

Also, having a CAPTCHA on the entry page is a very welcome addition that's long overdue, unfortunately it doesn't seem to be working at all right now. It keeps saying the words I enter don't match the image. I even tried the audio version and that doesn't work either.

There seems to be some problem with some MATLAB version specific function. MATLAB Version 7.3.0.267 (R2006b). When I tried to run the runcontest got the following error

>> runcontest
??? Undefined function or method 'bsxfun' for input arguments of type 'function_handle'.

Error in ==> grade at 6
score = min(abs(bsxfun(@minus,(xf+1i*yf).',xm+1i*ym)),[],1)*f;

Error in ==> runcontest at 44
    scores(k) = grade(inputs{:},responses{k});

I checked.. There is no 'bsxfun' in this version? What to do? Any help...Looks like an interesting contest..

Best regards,
Abhisek

"Matthew Simoneau" <matthew@mathworks.com> wrote in message <ger09l$7v0$1@fred.mathworks.com>...
> The Fall 2008 MATLAB Contest starts Wednesday at noon EST (17:00 GMT). It runs for a week, but includes several mid-contest awards. The programming challenge and full rules will be posted at the start of the contest on our website:
>
> http://www.mathworks.com/contest/
>
> We encourage you to reply on this thread to discuss any aspect of the contest, including rule clarifications, potential strategies, and commentary on the action. For administrative issues, e-mail us at contest@mathworks.com. For announcements, see our blog:
>
> http://blogs.mathworks.com/contest/
>
> Spoiler: We're introducing a form of limited head-to-head competition this time, bringing a new twist to our usual contest format.
>
> Good luck, everyone. See you in the queue!

For other people having similar trouble (if so) with bsxfun with older MATLAB versions, I googled a bit and found this one. Seems to work:

http://www.mathworks.com/matlabcentral/fileexchange/18685

Follow the links to get a customized bsxfun function. I got the runcontest working, hope the values are same as with MATLAB's own bsxfun.

On the example testsuite, with the abovementioned bsxfun,

runcontest

ans =

results: 25128.03
time: 38.89

MATLAB team may verify..

Best,
Abhisek

> On the example testsuite, with the abovementioned bsxfun,
>
> runcontest
>
> ans =
>
> results: 25128.03
> time: 38.89
>

This is the same result I get with the latest version of MATLAB (R2008a), which has the bsxfun in it. Note this problem seems to consistently come up during the contests.

"Matthew Simoneau" <matthew@mathworks.com> wrote in message <ger09l$7v0$1@fred.mathworks.com>...
> The Fall 2008 MATLAB Contest starts Wednesday at noon EST (17:00 GMT).

Good luck to all!

One question: When the new opposing army is put into place, will all entries be rescored to determine the new leader? Or do future entries just have to compete against the new (and presumably better) army to get back on top?

This is indeed a very clever addition to the contest.

We've been trying to come up with a way to get a head-to-head aspect to the contest without changing our infrastructure much. Doug Hull came up with this "king of the hill" idea. We're happy to hear this initial positive feedback.

the cyclist, we won't be re-scoring previous days entries against the new leader. Each 24-hour period stands on its own. We're excited to see how the dynamics shake out. Will the best entry for each day be strictly "better" than the previous? Will there be daily oscillations leader's level of aggression? Anyone want to take a guess?

I'm excited by the new format, and look forward to seeing how it turns out. My guess is that agression will oscillate at first, and then approach some median.

A question: Is the opponent tactic on the testsuite the same as that in the zip file, or is it something else unknown?

Nicholas,

The currently running (Wed Nov 5) house ant is the 'dumb wanderer' (same as the solver.m that was sent as an example).

Enjoy the 'holiday' of having an easily defeated house ant. I have peeked at the current contenders, the house ant will not be so easy this time tomorrow.

-Doug


"Nicholas Howe" <NikHow@hotmail.com> wrote in message <gesuoe$dd5$1@fred.mathworks.com>...
> I'm excited by the new format, and look forward to seeing how it turns out. My guess is that agression will oscillate at first, and then approach some median.
>
> A question: Is the opponent tactic on the testsuite the same as that in the zip file, or is it something else unknown?

I just noticed another change you all made to the contest. The entry listing page is only updated every 20 minutes. Will this be the same for the 'queue' once we get into daylight? If so, that's a very clever way to eliminate the last minute resubmission rush that happens near the deadlines.

Bravo overall to the contest team for the various changes. Clearly you put a lot of thought into this over the summer!

Why does it say "[ym,xm] = find(main==-1);" in grade.m and not
main==1 as +1 are the users anthills and -1 are the opponents
anthils.... maybe i am missing something trivial but this seems to
grade the opponents score, not the user's score???????

l_combee@yahoo.no wrote in message <ba727466-d4f7-49de-8217-cb3547c63077@v13g2000pro.googlegroups.com>...
> Why does it say "[ym,xm] = find(main==-1);" in grade.m and not
> main==1 as +1 are the users anthills and -1 are the opponents
> anthils.... maybe i am missing something trivial but this seems to
> grade the opponents score, not the user's score???????

It looks like main=1 is for “home team” anthill.
We, as “guests” have -1 (see line # 60 armyantssimulator.m)

SY

> It looks like main=3D1 is for =93home team=94 anthill.
> We, as =93guests=94 have -1 (see line # 60 armyantssimulator.m)

Thanks, I see yes you are right. I also got divide by zero errors in
the line xx =3D r(u)+min(.8./w,1).*v+.1+(w=3D=3D0).*.4-.5; because w has
sometimes zero elements, fixed this line too. Annoying.

Looking at the entries that have run so far, it appears time is going to be a MAJOR problem this time around. The 'contest' machinery itself appears to take over 70 seconds just to run the default 'simple' job. This is nearly half of the max allowed time of 180 seconds. I don't recall ever having a situation like this in the past.

Because we have no control over the number of time intervals that get run or the number of boards, we'll never be able to get below that 70 second time frame. In fact, the situation is about to get worse, because once the house solver gets replaced by an user submitted code, it's going to add significantly to that overhead, perhaps as much as doubling it.

In developing my solver, I noticed that on some of the simple boards I was able to kill all the opposing ants and transfer all the sugar to the base in a very small amount of time (~200 time intervals). The remaining 800 intervals were just 'wasted time' at that point. Perhaps a change could be made to the run contest code that stops a board if all the ants are dead and all the sugar is at the base? Alternatively, maybe our solvers can provide an output that means stop running this board and score it as is? Either of these solutions would potentially allow us more control over the time it takes to run.

Wow.. the contest is off to an ominous start. It's nearly 5PM eastern and we still have entries from Darkness being processed. By my calculations entries are averaging about 2.5 mins each to run.. There are 25 entries still in the queue that were submitted prior to noon, meaning we won't be able to truly go into Twilight with a new 'house solver' until after 6PM.

Add to that the additional ~22 entries submitted after noon sitting in the queue, and someone submitting an entry right now won't be able to see how it reacts to the new house solver until after 7PM tonight (or to put it into perspective, approximately 30% of the time we have for twilight is lost. This makes the iterative and collaborative intent of the contest almost impossible to achieve.

Are there any intents from the contest team to try to make changes to the rules / execution in order to account for the issues we are seeing?

I agree. It will be MAJOR problem. I can see situation when best result for the day (next King of the Hill) will not be determined until the end of the next day. And this is even without intentional stalling the queue. One option is to cut number of steps to 200-500. Additional benefit of shorter run is preventing “kill everybody then collect” strategy.

"Alan Chalker" <alancNOSPAM@osc.edu> wrote in message <gevfju$q8d$1@fred.mathworks.com>...
> Looking at the entries that have run so far, it appears time is going to be a MAJOR problem this time around. The 'contest' machinery itself appears to take over 70 seconds just to run the default 'simple' job. This is nearly half of the max allowed time of 180 seconds. I don't recall ever having a situation like this in the past.
>
> Because we have no control over the number of time intervals that get run or the number of boards, we'll never be able to get below that 70 second time frame. In fact, the situation is about to get worse, because once the house solver gets replaced by an user submitted code, it's going to add significantly to that overhead, perhaps as much as doubling it.
>
> In developing my solver, I noticed that on some of the simple boards I was able to kill all the opposing ants and transfer all the sugar to the base in a very small amount of time (~200 time intervals). The remaining 800 intervals were just 'wasted time' at that point. Perhaps a change could be made to the run contest code that stops a board if all the ants are dead and all the sugar is at the base? Alternatively, maybe our solvers can provide an output that means stop running this board and score it as is? Either of these solutions would potentially allow us more control over the time it takes to run.

Hi all,

Jan Langers winning darkness entry will become the opposing ant colony but will be hidden. So he is the only contestant to know how the opposing ants act. This is quite a huge advantage for him for the twilight phase!! Did you consider this when introducing the "king of the hill" rule?

Markus

I love the way the challenge will change every day in this contest, and the addition of an opponent makes the challenge much more interesting. But I have some concerns to voice also:

- A lot of my entries in Twilight are timing out. I suspect this is because Jan's winning entry takes much longer to run than the original strategy, thus leaving essentially no time for the contender. Is this true? Can the contest machinery adjust the time computation by subtracting the time taken by the "house ant" software?

- Related to the above, I am concerned that it may be impossible to beat the score of the Darkness winner when faced with a more capable house ant. Of course, there are a lot of clever people working on this contest so I may be wrong about this one. Nevertheless, we may hit a point where the house ant gets so good that the scores move backwards rather than forwards.

> A lot of my entries in Twilight are timing out. I suspect this is because Jan's winning entry takes much longer to run than the original strategy, thus leaving essentially no time for the contender. Is this true? Can the contest machinery adjust the time computation by subtracting the time taken by the "house ant" software?

I have confirmed this by submitting the default solver again. The Matlab team's original entry took 75 seconds to run; my resubmission of an identical program running against Jan's code took 147 seconds.

"Matthew Simoneau" <matthew@mathworks.com> wrote in message <ger09l$7v0$1@fred.mathworks.com>...
> The Fall 2008 MATLAB Contest starts Wednesday at noon EST (17:00 GMT).

It seems there is a problem with the "Top 20 Ranked Entries" list on the "Queue and Top 20 Entries" page. Not sure what it is showing, but it is not the top 20 entries. 8-/

timing is odd, which said by Nicholas Howe above. The other interesting thing is the large regression in the score:)

Is it possible that move processing time (armyantsimulator.m) is included too?
(With different time to process different move.)

SY

Hi everybody. Sorry for the confusion. Here are a couple of quick clarifications.

Yes, the timing of both entries are included, so the runtime of the existing king of the hill will influence your total score. We anticipated this effect. Swapping in a new opponent introduces a discontinuity in the scoring, which is why the "top 20" now only shows entries that were scored in this phase. You're only being compared with other entries in this round.

The house's runtime doesn't leave much for you to work in before the exponential timing penalty kicks in, so Twilight entries will need to adjust accordingly. Fortunately, the winner of this round will necessarily have a shorter run time, leaving more room in the next round.

We're listening to your feedback. Twilight will go ahead as planned, but the contest team will meet tomorrow to see if we want to make any adjustments for later rounds. Because of the turn-based nature of the problem, simply subtracting run-time of the home team would be tricky. Please keep the suggestions coming.

"Nicholas Howe" <NikHow@hotmail.com> wrote in message <gf0701$k88$1@fred.mathworks.com>...
> > A lot of my entries in Twilight are timing out. I suspect this is because Jan's winning entry takes much longer to run than the original strategy, thus leaving essentially no time for the contender. Is this true? Can the contest machinery adjust the time computation by subtracting the time taken by the "house ant" software?
>
> I have confirmed this by submitting the default solver again. The Matlab team's original entry took 75 seconds to run; my resubmission of an identical program running against Jan's code took 147 seconds.

There is definitely something weird going on that I don't quite understand yet, but have a good idea on. The 'default entry was VERY SIMPLE, with only 2 calls to rand and round. It doesn't add anything to the cyc score or to the total execution time because it's so simple. As Nicholas points out, it take 147 seconds to run.

However, the current leading entry, submitted by Vishal, is 39 times as complicated according to the cyc and 10 times as long code wise. YET it runs in only 107 secs, shaving almost 30% off the run time.

Obviously this solver takes longer to actually execute since it's more complicated code, thus the time savings must be in the way the armyantsimulator code processes the outputs from the solver. I've profiled the armyantsimulator code and found that the 2 lines that take the majority of the time are the call to the solver and the update world one. Both of these are nested inside loops and get called repeatedly for the number of ants still alive (BOOOO for for loops. Couldn't the contest team vectorize this somehow?)

As an experiment I submitted the simplest of all possible solvers. It always returns no move, no action, no mark. And yet this code timed out. This leads me to a big conclusion: The only way to play this game is to go on the active hunt and attack and kill the opposing ants immediately. Unless you reduce the number of ants on the board very quickly, your code will time out. If this is indeed the case, it's kind of a bummer, since there might be some very cool searching / evade algorithms, as alluded to in the rules, but we'll never be able to use them due to the time limits.

Matthew: When you guys ran this contest internally did you have similar issues with a lot of entries timing out and having to 'go on the attack right away?

> As an experiment I submitted the simplest of all possible solvers. It always returns no move, no action, no mark. And yet this code timed out.

This sounds like a rather serious disadvantage for non-attacking ants OR the king of the hill is for such a scenario extremely inefficient and basically consuming your time?
Maybe using average time per ant or per round could be better.
Incorporating oponnents time into measuring is quite odd - as this will lead to nothing else than swapping time consuming strategies. 1st round wins processor heavy, became king of the hill, next round will simply have to win some very lite ant which will allow in 3rd round (day) to create processor heavy ant again. I dont think that this will settle somewhere around mid becasue number of entries seem to be quite big and especially when code will be revealed, people will be able tu push complexity to the limits of given round

"Petr " <cutallbeforedot.krejcip@gmail.com> wrote in message <gf10n5$act$1@fred.mathworks.com>...
> > As an experiment I submitted the simplest of all possible solvers. It always returns no move, no action, no mark. And yet this code timed out.
>
> This sounds like a rather serious disadvantage for non-attacking ants OR the king of the hill is for such a scenario extremely inefficient and basically consuming your time?

I could give a few hints about the attacking behavior of "ebi v1". Its rather simple :-)
But I'm not sure if the majority of participants likes to know. Either way, in 2.5 hours its open source.
Jan

There seems to be a substantial random factor in the timing: I tried two very subtle tweaks of an entry that finished with plenty of time and they both timed out. My hypothesis is that the entry that finished happened to generate chance encounters that killed many ants quickly, and the tweaked versions did not.

Well, at least the next day's house ant should be relatively quick-computing, giving our more elaborate ideas more time to work.

While we're all waiting for the queue to clear and get into daylight, I thought it'd be a good time to post my traditional analysis of the scoring formula. As in past contests, the formula is:

score = k1*result + k2*e(k3*runtime) + k4*max(cyc-10,0) + k5*nodes

The coefficients were only slightly tweaked compared to the spring contest, as the contest team typically does:

k1 = 1
k2 = 0.1
k3 = 2/30 (0.06666666…)
k4 = 1
k5 = 0.001

The current leading entry has a time of 161s, result of 22940, cyc of 54, and nodes of 1583. Here’s a breakdown of the current tradeoffs:

-cyc and score are a 1:1 ratio (i.e. each point shaved off cyc is a point shaved off the score)
-time and score are a 1:295 ratio
-result and score are a 1:1 ratio
-node and score are a 1:0.001 ratio

Note that the time component is at a very steep slope in the exponential curve. Adding just 5 seconds changes the ratio to 1:412, whereas saving 5 seconds changes the ratio to 1:226. It's not until we drop the time to about 75s that the ratio gets down to 1:1.

So what does this all mean?

1. Well given that we already know the shortest amount of time is never going to drop below about 70s due to the 'overhead' caused by the contest machinery, it will always be better to work on faster code than to work on a better result, unless there is some major algorithmic breakthrough.

2. Since it appears that all the codes have quite a bit of randomness in them as part of the 'searching phases', submitting the same code multiple times will likely yield very different scores. We already know from past contests that the contest system can have quite a variable workload, which results in small, but significant time differences. Just a small amount of change in the time can majorly alter the score one way or the other.

"Nicholas Howe" <NikHow@hotmail.com> wrote in message <gf1ghf$3ea$1@fred.mathworks.com>...
> There seems to be a substantial random factor in the timing: I tried two very subtle tweaks of an entry that finished with plenty of time and they both timed out. My hypothesis is that the entry that finished happened to generate chance encounters that killed many ants quickly, and the tweaked versions did not.

That happened for me as well. On two occasions _very_ small changes to my program made the difference between a runtime of 120-130s and timing out. However, this behavior has been reproducable for me both on my own computer and on the contest machinery.

 
> 1. Well given that we already know the shortest amount of time is never going to drop below about 70s due to the 'overhead' caused by the contest machinery, it will always be better to work on faster code than to work on a better result, unless there is some major algorithmic breakthrough.

Thanx Alan for your insight into the param. well, I also recognized the point of making faster code. I tried in that direction, but the result got worse. So kind of dilemma.
Nevertheless, it was a unique experience to fight against an unknown enemy. This goes towards promoting robust generalized codes.

To contest team, a potential suggestion to reduce the advantage of the darkness winner in twilight phase, one can choose the darkness winner entry and any other good entry which is not disclosed. Then, you can simply combine them as follows to create new enemy

if rand<0.5,
   run darkness winner
else
  run undisclosed good entry
end

Best,
Abhisek

Concerns that that the darkness winner had an unfair advantage seem unfounded, since it appears that someone else has won twilight.

A request for the contest masters, if possible: is there a way to view the top 20 entries on a particular house ant? I'd like to go back and look at the best entries from each phase, but it looks like the top 20 will only show the current day's activity.

"Nicholas Howe" <NikHow@hotmail.com> wrote in message <gf25o6$f62$1@fred.mathworks.com>...
> Concerns that that the darkness winner had an unfair advantage seem unfounded, since it appears that someone else has won twilight.
>
>

Well, I think I for one was a bit confused by the rules. I expected that the entry that won Darkness, "ebi v1", would win the Twilight phase because it continues to be the one with the best score. (And unless the scoring is changed, its score might never be beaten.)

Instead, it seems that the winner of each phase must have been submitted during that phase.

Hi contest team,

it seems the queue is stuck again. Maybe you can look after that.

Yours,
Markus

Hi,

the weekend is coming closer. Are there any mid-contest prizes planned? If yes, could you announce the deadlines now? If no, it is also worth to know. In this case I could spend the weekend doing something reasonable...

Will there be someone to look at stuck queues and new kings of the hill and such over the weekend?

Yours
Markus

The queue is still trucking away, but the web page isn't updating. We're trying to find someone on our web team who can help us debug this. Sorry for the delay-of-game.

P.S. Nick Howe is in first place:
http://www.mathworks.com/contest/armyants.cgi/view_submission.html?id=50312

1. too many submissions are random tests. And, the scale of testsuite cases is little.(8?)
2. why not add more Pcs to the contest to do parallel entry processing? we know, MATLAB has a Distributed Computing engine.
3. is it good to do 2 measurements? one is A vs B, the other is B vs A. using the average or the best as the final score?
4. time is not very enough for deeper algorithm

We can't get the Queue and Top 20 page to refresh. For now, please use this (ugly) page:
http://www.mathworks.com/contest/armyants.cgi/queue_top20_make.html

Looks like time stamp on some submissions is 1 hour off

the random bombing is rather meaningless. I sugguest to modifying the rule to mode of per round per day. In every round, the one makes the largest leap in score is the winner.
Now, the rule of sharing time makes the algorithm much "bloody". And, some unfair in the inter-killing make this condition intensified. So, if this rule(or the testsuite?) does not change, I guess that there are some dead-locks or ridiculous random scores in the next part of the contest.

Hi, I also see this odd problem, my "ttt08" should be the last entry before 12:00, but it showed the time 2008-11-08 10:59:54. I Add a "crazyroundover" entry ,which is inserted into 2008-11-08 11:04:50. Very tricky.

"Sergey" <ivssnn@yahoo.com> wrote in message <gf4gs0$i$1@fred.mathworks.com>...
> Looks like time stamp on some submissions is 1 hour off

Thank you for working on the queue page and maintaining the contest machinery. It is nice to see the contest beginning to heat up a little!

I was wondering if there were any plans for a statistics page this time? The one used in previous contests may not be completely appropriate given the discontinuity between phases, so it would be a bit of work to put together something new. But I for one would like to see a summary of how the entries have developed and improved over time. Thanks!

Hi everyone.
I have been looking over some of the high scoring entries and compare them to the rules I found a slight discrepancy. In the rules, it says that the scent an ant leaves is a number between 1 and 100, but several entries seem to thrive on using numbers much bigger than this. Is this an overinterpratation on my part, or a breech of the rules?

If it is not a breech of the rules, has someone used the escents to encode more complex measurements than the relative distance home? A double value is 64 bits long. Over the course of a match, you need to drop 10 bits of data to allow for the decrease in scent over time. That leaves (in theory) 54 pits per square to encode arbitrary data into. Using the neighbouring squares as well and ant has immediate read and rather quich write access to 9*54 bits, or over 60 bytes of persistent memory. This should be enough for the ants to encode the entire map into and thus begind working systematically from the start to the end.

Comments on this? I am not enough of a programmer to implement this, but considdering the level of some of you here, it should be a breeze for someone....

--DA

"Daniel Armyr" <firstname@lastname.se> wrote in message <gf4vq3$j2q$1@fred.mathworks.com>...
> Hi everyone.
> I have been looking over some of the high scoring entries and compare them to the rules I found a slight discrepancy. In the rules, it says that the scent an ant leaves is a number between 1 and 100, but several entries seem to thrive on using numbers much bigger than this. Is this an overinterpratation on my part, or a breech of the rules?

Daniel:

I think you missed the following part of the rules:
"The marking scent and death scent deposited by ants on a single square are both cumulative without bound. If five ants put a scent of 10 on a single square in the same time interval, that square will have a scent of 50 by the end of the turn."

While an individual ant can only leave a max scent of 100, a given square on the board can end up with a much higher scent due to this cumulative effect.

For those of you who missed this comment in the blog this morning, here's a copy of it (obviously the referenced entry isn't the current leading one, but the current King of the Hill is a minor derivative of it):

 For those of you interested in understanding how the leading solvers are working, the current top entry (Daylight Again 31) is a heavily commented version I created that should explain things pretty simply: http://www.mathworks.com/contest/armyants.cgi/view_submission.html?id=50436

It’s surprisingly short and simple to understand. Hopefully this will make it less intimidating for those of you wanted to jump in and compete over the weekend.

Credit goes to Nick Howe I believe who originally submitted this basic algorithm in his Cowardly Lion entry, which was subsequently improved upon by SY and David Jones before I took a stab at it.

Daniel,
Amount of scent deposited by one ant is limited to 0-100 by processing software. If ant attempts to deposit more than 100 it will be processed as 100.
About coding: Ants deposit scent “simultaneously” and they can not communicate with each other. For example, three ants in one cell can not reliably deposited together exactly five units of scent

SY

"Daniel Armyr" <firstname@lastname.se> wrote in message <gf4vq3$j2q$1@fred.mathworks.com>...
> Hi everyone.
> I have been looking over some of the high scoring entries and compare them to the rules I found a slight discrepancy. In the rules, it says that the scent an ant leaves is a number between 1 and 100, but several entries seem to thrive on using numbers much bigger than this. Is this an overinterpratation on my part, or a breech of the rules?
>
> If it is not a breech of the rules, has someone used the escents to encode more complex measurements than the relative distance home? A double value is 64 bits long. Over the course of a match, you need to drop 10 bits of data to allow for the decrease in scent over time. That leaves (in theory) 54 pits per square to encode arbitrary data into. Using the neighbouring squares as well and ant has immediate read and rather quich write access to 9*54 bits, or over 60 bytes of persistent memory. This should be enough for the ants to encode the entire map into and thus begind working systematically from the start to the end.
>
> Comments on this? I am not enough of a programmer to implement this, but considdering the level of some of you here, it should be a breeze for someone....
>
> --DA
>
>
>

I would like to second Markus:
"Are there any mid-contest prizes planned? If yes, could you announce the deadlines now? If no, it is also worth to know"

SY

"Markus Buehren" <mb_matlab.REMOVE@gmxTHIS.de> wrote in message <gf2dje$39m$1@fred.mathworks.com>...
> Hi,
>
> the weekend is coming closer. Are there any mid-contest prizes planned? If yes, could you announce the deadlines now? If no, it is also worth to know. In this case I could spend the weekend doing something reasonable...
>
> Will there be someone to look at stuck queues and new kings of the hill and such over the weekend?
>
> Yours
> Markus

> Amount of scent deposited by one ant is limited to 0-100 by processing software. If ant attempts to deposit more than 100 it will be processed as 100.

Further, the mark values are rounded to the nearest integer.

Markus

Umm...

I'm a little embarrassed to admit discovering this, but apparently the armyantsimulator dies at line 340 if all the ants on the board manage to simultaneously wipe each other out.

Since this is not really an error in the responsible entry, you might look into fixing it in the contest scoring routine. On the other hand, any algorithm that does this probably wasn't very good anyway. ;-)

"On the other hand, any algorithm that does this probably wasn't very good anyway. ;-)"
In a contrary, it may be very smart strategy.
Something like this: collect all food then create mutual destruction to cut program running time.

SY





"Nicholas Howe" <NikHow@hotmail.com> wrote in message <gf5nou$d76$1@fred.mathworks.com>...
> Umm...
>
> I'm a little embarrassed to admit discovering this, but apparently the armyantsimulator dies at line 340 if all the ants on the board manage to simultaneously wipe each other out.
>
> Since this is not really an error in the responsible entry, you might look into fixing it in the contest scoring routine. On the other hand, any algorithm that does this probably wasn't very good anyway. ;-)

Sergey has a good point. Anyway, I went ahead and dealt with this in my own copy of the program by adding the following at line 32 of armyantsimulator:

    if ~any(red(:))&&~any(black(:))
        break;
    end;

I haven't heard any response yet from the Matlab contest team about the potential scoring problem. (It's the weekend, and they probably have families to pay attention to and so forth.) But I think it may have bitten me during the last phase.

Here's the story: I spent quite a bit of time developing new algorithmic modifications on the last round. Some of my programs on the test suite were scoring in the 16000 range -- for comparison, I just tried out nearly immortal 8 and the best score it got over five runs was over 19000. I submitted a batch of entries toward the end of the phase hoping that one of them at least would be successful -- but all except one failed with the error "Cell contents indices must be greater than 0." My program doesn't use cell variables at all. That doesn't mean it's not a problem with my submission, but it seems like there is a very real possibility that the bug is actually in the simulation machinery, triggered when all the ants on a map are killed at the same time. If that is so, I'd like at least to know, and if possible find out what my entries would have scored against the house ant at the time.

I apologize if I sound bitter -- I'm not really. I love the Matlab programming contests, and look forward to them. The concept this time around has been great, and I appreciate all the effort by the Mathworkers to run things. I'm sure it's not easy to keep everyone happy. I'm just bummed because I probably won't have another large chunk of time to spend on the contest, and I'd like to understand why nearly all of my entries failed. If anyone can help me out, I'd be grateful. Thanks!

Nicholas Howe:

I pick up your some entry, and find you are right. I add a little tweak to avoid crazy killing, and this left more ants in the last to collect food, so everything works fine. I thought the killing unbalance was a intrinsic disadvantage (to see something in the rule, by the turn-based round ) before the above you said. But it is really a bug of the previous entries.
And so, there is definitely a bug in the contest armyantsimulator. Although, I don't think it is the best strategy to kil to nothing.

best regards,
Jin

> While an individual ant can only leave a max scent of 100.
I see. That is of course a limitation. However, as has been shown by several algorithms, the non-communication of the ants is a non-problem as all actions are doen step-wise and in a completely deterministic way.

"Matthew Simoneau" <matthew@mathworks.com> wrote in message <ger09l$7v0$1@fred.mathworks.com>...
> The Fall 2008 MATLAB Contest starts Wednesday at noon EST (17:00 GMT).

Is the "Grand Prize Winner" going to be the person at top of the "Top 20" on the last day, or the person at the top of the "Complete Rankings"?

I've discovered an interesting bug in the code I can't quite track down.

One of the things the contest machinery does that the runcontest code doesn't is seed the random number generator at the start of each run (which is why identical code submissions get identical results). In order to make it easier to track the affect tweaking changes have on the code, I added the following line at the start of runcontest.m: rand('twister',0) As a result, I get consistent run-to-run results.

However, the current King of the Hill (Nearly immortal 8, #50873) and leading entry (MutantCat 6, #51068) crash the test suite whenever I run them I can run each of the test boards individually just fine, however when I run the entire contest, it crashes with the following error:

??? Attempted to access r(0); index must be a positive integer or logical.

Error in ==> solver at 143
            dRow = sign(r(pick)-2); % convert to contest directions

If I remove the rand seed, the test suite doesn't crash.

Looking at the failed entries listed at http://www.mathworks.com/contest/armyants/rankings.html , this has now happened DOZENS of times on the actual contest machinery. Particularly with a lot of the entries just submitted in the past hour.

I'm not sure what the solution is to this, however it is an interesting side affect of the current code being so stochastic.

Hi,

I did not have time to compete this fall, but at least I managed to
spend some time studying different solvers and set up my own "ant-
death-matches" between various solvers. Really cool idea this king of
the hill! :) With currently about 900 passing solvers the possible
number of matches is more than enough..! Most matches are interesting,
especially in the beginning, but sometimes they stall (mostly when the
solvers are too equal), so I added a few lines in armyantsimulator.m
to get a "skip to Next map" functionality by pressing the letter 'n'.

At row 43 (original armyantsimulator.m version):

    if strcmp(get(gcf,'CurrentCharacter'),'n')
        set(gcf,'CurrentCharacter',char(0)); % reset figs
CurrentCharacter
        break;
    end


Another convenient fix when testing and comparing different solvers is
to call runcontest.m with function handle/s for the solver/s to use:

function [message,results,timeElapsed] =
runcontest(redteam,blackteam,drawboard,doBoards)
...
% Argument parsing.
if (nargin < 3)
    drawboard = 0;
end

load testsuite_sample testsuite

if (nargin < 4)
    doBoards = 1:numel(testsuite);
end
...
% Run the submission for each problem in the suite.
time0 = cputime;
for i = doBoards
   responses{i} =
armyantsimulator(redteam,blackteam,testsuite(i),drawboard);
end
...


To run and plot solverA.m and solverB.m against house_solver.m you now
call runcontest with:

runcontest(@solverA,@house_solver,1)

respectively

runcontest(@solverB,@house_solver,1)

This feature is quite handy, and I usually add it to my own
runcontest.m for each new contest. Maybe the Contest Team would like
to consider adding this as default functionality for future contests!?

Thanks for yet another great contest & have fun!
JohanH

"Alan Chalker" <alancNOSPAM@osc.edu> wrote in message <gf9qgi$nff$1@fred.mathworks.com>...
> I've discovered an interesting bug in the code I can't quite track down.

This is a bug in the current solver, but it only comes up for certain random seeds. Apparently there are configurations where the entire local map is masked out, so the search for moves returns an empty list. My guess is that when you run a board individually you happen to avoid this situation, but when you run it as a group the random seed is different (because of the random numbers used on preceding boards) and the problem situation arises.

We need to add:

if ~isempty(r)

before

pick = ceil(rand*numel(r));

SY

"Nicholas Howe" <NikHow@hotmail.com> wrote in message <gf9tum$cej$1@fred.mathworks.com>...
> "Alan Chalker" <alancNOSPAM@osc.edu> wrote in message <gf9qgi$nff$1@fred.mathworks.com>...
> > I've discovered an interesting bug in the code I can't quite track down.
>
> This is a bug in the current solver, but it only comes up for certain random seeds. Apparently there are configurations where the entire local map is masked out, so the search for moves returns an empty list. My guess is that when you run a board individually you happen to avoid this situation, but when you run it as a group the random seed is different (because of the random numbers used on preceding boards) and the problem situation arises.

"the cyclist" wrote
> Is the "Grand Prize Winner" going to be the person at top of
> the "Top 20" on the last day, or the person at the top of the
> "Complete Rankings"?

The Grand Prize Winner will be the person at the top of the Top 20 list on the last day, after the queue has emptied.

Ned Gulley
The Contest Team

What time Twilight starts?




"Ned Gulley" <gulley@mathworks.com> wrote in message <gfadn3$7or$1@fred.mathworks.com>...
> "the cyclist" wrote
> > Is the "Grand Prize Winner" going to be the person at top of
> > the "Top 20" on the last day, or the person at the top of the
> > "Complete Rankings"?
>
> The Grand Prize Winner will be the person at the top of the Top 20 list on the last day, after the queue has emptied.
>
> Ned Gulley
> The Contest Team

is there a late Twilight planned for this contest as well?

Abhisek


"Sergey" <ivssnn@yahoo.com> wrote in message <gfamrh$67q$1@fred.mathworks.com>...
> What time Twilight starts?
>

It looks like there is a similar bug in flippant8, the current house ant. Apparently it didn't get in the way of winning the last phase, but it probably means that some entries will fail randomly through no fault of their own. Russian roulette, anyone? :-)

One of my versions successfully crashes current house ant 90% of runs on test set

"Nicholas Howe" <NikHow@hotmail.com> wrote in message <gfc4lq$jbr$1@fred.mathworks.com>...
> It looks like there is a similar bug in flippant8, the current house ant. Apparently it didn't get in the way of winning the last phase, but it probably means that some entries will fail randomly through no fault of their own. Russian roulette, anyone? :-)

Nicholas,

My apologies, I wrote the bug that bit you here.

I thought it had been fixed it before the beginning of the contest, but I think I gave Matt the wrong version to post. The fixed version should have been pushed out with Monday's KOTH update.

-Sorry,
Doug

"Nicholas Howe" <NikHow@hotmail.com> wrote in message <gf85be$jsl$1@fred.mathworks.com>...
> I haven't heard any response yet from the Matlab contest team about the potential scoring problem. (It's the weekend, and they probably have families to pay attention to and so forth.) But I think it may have bitten me during the last phase.
>
> Here's the story: I spent quite a bit of time developing new algorithmic modifications on the last round. Some of my programs on the test suite were scoring in the 16000 range -- for comparison, I just tried out nearly immortal 8 and the best score it got over five runs was over 19000. I submitted a batch of entries toward the end of the phase hoping that one of them at least would be successful -- but all except one failed with the error "Cell contents indices must be greater than 0." My program doesn't use cell variables at all. That doesn't mean it's not a problem with my submission, but it seems like there is a very real possibility that the bug is actually in the simulation machinery, triggered when all the ants on a map are killed at the same time. If that is so, I'd like at least to know, and if possible find out what my entries would have scored against the house ant at the time.
>
> I apologize if I sound bitter -- I'm not really. I love the Matlab programming contests, and look forward to them. The concept this time around has been great, and I appreciate all the effort by the Mathworkers to run things. I'm sure it's not easy to keep everyone happy. I'm just bummed because I probably won't have another large chunk of time to spend on the contest, and I'd like to understand why nearly all of my entries failed. If anyone can help me out, I'd be grateful. Thanks!

No late twilight is planned.

"Abhisek Ukil" <abhiukil@gmail.com> wrote in message <gfaqst$esk$1@fred.mathworks.com>...
> is there a late Twilight planned for this contest as well?
>
> Abhisek
>
>
> "Sergey" <ivssnn@yahoo.com> wrote in message <gfamrh$67q$1@fred.mathworks.com>...
> > What time Twilight starts?
> >

Hi Contest team,

using the king of the hill will always bring the risk that it crashes du to some bug. Please put a try-catch-block around the call to the house solver AS FAST AS POSSIBLE to ensure a fair fight for the grand prize. Just replace the lines

[actions(curAnt).dRow, actions(curAnt).dCol, actions(curAnt).action, actions(curAnt).mark] ...
  = houseAntFH(AVmain, AVfood, AVblack, AVred, AVblackScent, AVredScent, AVblackDeath, AVredDeath);

by

try
  [actions(curAnt).dRow, actions(curAnt).dCol, actions(curAnt).action, actions(curAnt).mark] ...
  = houseAntFH(AVmain, AVfood, AVblack, AVred, AVblackScent, AVredScent, AVblackDeath, AVredDeath);
catch
  actions(curAnt).dRow = 0;
  actions(curAnt).dCol = 0;
  actions(curAnt).action = 0;
  actions(curAnt).mark = 0;
end

in armyantsimulator.m. This way the house solver will be punished for producing an error.

Yours,
Markus

sugguest to setting the last 1 hour or 2 hours as the late twilight to avoid crazy random tests.

"Doug Hull" <hull@mathworks.SPAMPROOFcom> wrote in message <gfc8jd$fh9$1@fred.mathworks.com>...
> No late twilight is planned.
>
> "Abhisek Ukil" <abhiukil@gmail.com> wrote in message <gfaqst$esk$1@fred.mathworks.com>...
> > is there a late Twilight planned for this contest as well?
> >
> > Abhisek
> >
> >
> > "Sergey" <ivssnn@yahoo.com> wrote in message <gfamrh$67q$1@fred.mathworks.com>...
> > > What time Twilight starts?
> > >

I second that. Please make the last few hours (or at least a half or quarter hour) twilight.

The 20 minute refresh timer on the queue doesn't help anything if people can browse all recently posted submissions on the view_submission.htm-page.

"Jin" <xx@xx.xx> wrote in message <gfdp22$js2$1@fred.mathworks.com>...
> sugguest to setting the last 1 hour or 2 hours as the late twilight to avoid crazy random tests.
>
> "Doug Hull" <hull@mathworks.SPAMPROOFcom> wrote in message <gfc8jd$fh9$1@fred.mathworks.com>...
> > No late twilight is planned.
> >
> > "Abhisek Ukil" <abhiukil@gmail.com> wrote in message <gfaqst$esk$1@fred.mathworks.com>...
> > > is there a late Twilight planned for this contest as well?
> > >
> > > Abhisek
> > >
> > >
> > > "Sergey" <ivssnn@yahoo.com> wrote in message <gfamrh$67q$1@fred.mathworks.com>...
> > > > What time Twilight starts?
> > > >

"Matthew Simoneau" <matthew@mathworks.com> wrote in message <ger09l$7v0$1@fred.mathworks.com>...
> The Fall 2008 MATLAB Contest starts Wednesday at noon EST (17:00 GMT).

Thanks to the Contest Team for running another exciting contest. The King of the Hill concept was a great new twist.

It's fun to be in the lead at the "closing bell", but I know how meaningless that is likely to be. Lots of interesting tweaks appeared in the waning moments, and probably some real algorithmic improvements as well.

It was particularly great that the comments tended to stay in the code, and that no one obfuscated! It was much more interesting that way.

so awsome queue... we will see the last top1 about two days later. round ridiculous...Good night, everyone^_^

Yet another interesting and fun contest is past us now. However, it might be quite a long time before we know the final winner. By my calculations, there are ~925 entries in the queue right now, which at about 2.5 mins per entry to run will take 38.5 hours to complete (i.e. early Friday morning). As many people predicted, this turned into tweaker war in the last 30 minutes of the contest.

Regardless, the new dynamics and mechanisms for the contest were a refreshing change from what we've seen in previous contests. The only thing similar I can recall is the golf one a while back that switched the testsuites everyday. Overall I think it's a very nice thing to have 'new' challenges each day. While there might be some small things to tweak about this type of contest format, I hope future contests will be of a similar construct.

I'd also be remiss if I didn't express my profound thanks to the contest team for running this event again. I really appreciate all the thought and effort that goes into it and hope you find it as equally enjoyable to run and watch from behind the curtains as I do from a competitors viewpoint. Thanks again!

>
> It was particularly great that the comments tended to stay in the code, and that no one obfuscated! It was much more interesting that way.

I hadn't even noticed that, but I agree that is a totally cool twist on this contest. I can't recall a contest in the past few years that didn't degenerate into obfuscated code at one point. The 'clean code' and the relative simplicity of it compared to previous contests definitely lowered the bar on new competitors being able to follow along and jump in.

Hi all,

the deadline is over, and now look at the queue! Short after deadline I counted 926 entries!!! The leading entries need 160 seconds to compute. Supposing that most of the entries in the queue are modifications of them, we can expect that the queue will take at least one and a half day to clear!! The one responsible for most of the delay is Alan Chalker. He submitted 397 entries during the last 23 minutes, obviously using some automatic upload program.

Alan, this is really a shame!!!

Please have a look at the "Hacking" section of the Contest Rules. It says "it's too hard for the few of us running the contest to keep ahead of the group intelligence" and "we ask that you not overwhelm the queue". Submitting nearly 400 entries may clearly be called "overwhelmed".

What do the other think about that? Shouldn't Alan's entires be disqualified due to violation of the rules?

Markus

Thanks once again, contest team! It's obvious that a lot of hard work and creative thinking went into this new format. The innovations made for a wonderful contest.

Good luck in the queue, everybody.

Nathan

"Markus Buehren" <mb_matlab.REMOVE@gmxTHIS.de> wrote
> What do the other think about that? Shouldn't Alan's entires be disqualified due to violation of the rules?


It will be interesting to see if Alan's strategy is successful. Given the large randomness effect in this contest, it's reasonable to expect that one of Alan's entries has a good chance of being best -- unless others have algorithms that are superior in some way, and also get lucky.

Personally I would be disturbed if this had happened during the middle of the contest, shutting down the queue while people were still working. But since it comes at the end it doesn't really prevent others from developing their own entries.

I want to add my own thanks to the Mathworks contest team for their work on another great contest concept!

I must agree with Markus - the queue is overwhelmed, by any reasonable definition. I don't want to see any disqualifications, but I hope we are not going to see this kind of mass submission technique in future contests.

Nathan

"Markus Buehren" <mb_matlab.REMOVE@gmxTHIS.de> wrote in message <gff3ou$7ms$1@fred.mathworks.com>...
> Hi all,
>
> the deadline is over, and now look at the queue! Short after deadline I counted 926 entries!!! The leading entries need 160 seconds to compute. Supposing that most of the entries in the queue are modifications of them, we can expect that the queue will take at least one and a half day to clear!! The one responsible for most of the delay is Alan Chalker. He submitted 397 entries during the last 23 minutes, obviously using some automatic upload program.
>
> Alan, this is really a shame!!!
>
> Please have a look at the "Hacking" section of the Contest Rules. It says "it's too hard for the few of us running the contest to keep ahead of the group intelligence" and "we ask that you not overwhelm the queue". Submitting nearly 400 entries may clearly be called "overwhelmed".
>
> What do the other think about that? Shouldn't Alan's entires be disqualified due to violation of the rules?
>
> Markus

Markus:

In response to your concerns, I'd like to point out a couple things:

1. According to the statistics page (http://www.mathworks.com/contest/armyants/statistics.html), I wasn't even close to being the person that submitted the most overall entries. David Jones submitted nearly 500, whereas I'm at around 250, while SY isn't far behind at 180. I'm not sure if that chart includes all the entries in the queue, but regardless, all 3 of us did submit a lot of entries, but no one person was way out of line compared to the others.

2. The first quote from the rules you gave is taken out of context. The full quote says: "Entries that compromise the contest machinery are no longer allowed. We've all learned some interesting MATLAB tricks in the past by contestants figuring out how to pass information from one entry to the next, or finding clever ways to execute disallowed functions, but it's too hard for the few of us running the contest to keep ahead of the group intelligence. In short, out of consideration for everyone participating in the contest, we ask that you not abuse the system. " Thus it's hacking the system that quote refers to, and I definitely did NOT do that.

3. You also didn't fully copy the second quote you gave, which is: "Tuning the entry to the contest test suite via tweak bombing or other techniques is still allowed, but we ask that you not overwhelm the queue. " Thus the rules EXPLICITLY allow tweak bombing. The overwhelm the queue part is there because when someone tweak bombs in the middle of the contest it prevents other contestants from getting feedback on their entries in a timely manner and thus reduces the 'feedback loop' and collaboration part of the contest. I didn't start submitting my entries until ~30 minutes prior to contest close, which due to the number of entries in the queue already was well beyond the time where anyone could submit a new entry and expect to see the result in time to make another entry based upon it. Thus my entries in no way prevented others from fully competing in the contest. The ONLY thing my tweak bombing affects is when we will know the final winner.

I'm sorry if you don't like the fact that I have a lot of entries clogging up the queue right now, but I respectfully submit to you that what I did is fully allowed by the rules and was well within the spirit of the competition. I carefully timed my actions such that they would NOT affect any other competitors abilities to fully partake in the contest. I believe what I did was a legitimate approach to the contest, and now it's just a matter of time to see whether it pays off.

"Nicholas Howe" <NikHow@hotmail.com> wrote in message
>
> It will be interesting to see if Alan's strategy is successful. Given the large randomness effect in this contest, it's reasonable to expect that one of Alan's entries has a good chance of being best -- unless others have algorithms that are superior in some way, and also get lucky.
>
> Personally I would be disturbed if this had happened during the middle of the contest, shutting down the queue while people were still working. But since it comes at the end it doesn't really prevent others from developing their own entries.
>

That's exactly why I waited until the very end to do this. I realized early on this would be a reasonable technique to try and win the contest, but wanted to be considerate of what impact it would have on other people.

Just to let everyone know, I did indeed develop an automated submission program that would 'cycle through' a given variable in a solver. I spent the morning going through all the previous day's entries and pulling out the top 1-2 leading entries, which I loaded into my program and setup to cycle through a given random threshold check. I probably ended up with something like 20 different solver variations that had all previously performed well. Each one I setup to cycle through a small range around the existing variable. The reason for the random numbers in the entry titles instead of a sequential list is I didn't want someone to see what was going on and mess up my program by submitting entries with names that would conflict with the next numbers my program would submit. Now it's just a waiting game to see if this pays off.

"Nathan" <nathoqXXX@yahooXXX.com> wrote in message <gff6kq$jbk$1@fred.mathworks.com>...
> I must agree with Markus - the queue is overwhelmed, by any reasonable definition. I don't want to see any disqualifications, but I hope we are not going to see this kind of mass submission technique in future contests.
>
> Nathan
>
Nathan:

Please see my previous response to Markus above about why I think what I did was a legitimate tactic.

Mass submission in one form or another has always been a part of the contest. We've been lucky enough to not have "Don Antonio Coimbra de la Coronilla y Azevedo" show up during recent contests. He had a nasty habit of submitting the SAME default entry hundreds of times during the middle of a contest and thereby truly overwhelming the queue for no reason whatsoever.

I'll be the first to welcome a CAPTCHA on the submission page, and in fact applauded the contest team at the start of this contest when I saw they had tried to put one up. If you look through previous contest newsgroup threads and blogs you'll also see I've made such a suggestion numerous times. Unfortunately it didn't work for some reason this time, and as a result I developed a strategy based upon that aspect of the contest.

Alan,

You've developed a robot to twiddle parameters with no rational programming input, and you maintain that this is "well within the spirit of the competition". This is a bizarre claim - surely the spirit of the contest is to reward ingenuity, algorithmic innovation and technical skills? I hope your tweakbot doesn't win.

As you say, mass submission has often been a part of the contest (but not always on this scale). That doesn't make it acceptable. It's unfortunate that the CAPTCHA didn't work this time, despite the contest team's efforts. That doesn't make automated mass submission acceptable. It's not the responsibility of the contest admins to police the vulnerabilities in the contest infrastructure - which is provided for our education and entertainment by the volunteer contest team and the Mathworks, free of charge. There will always be weaknesses in the structure, and as long as this remains a friendly, social, low-stakes competition, we competitors will have to trust each other.

To change the topic and tone completely - thanks for your ongoing campaign to comment the code. It's good to see that commenting now seems to be permeating the culture of the contest.

Nathan

"Alan Chalker" <alancNOSPAM@osc.edu> wrote in message <gff89a$f47$1@fred.mathworks.com>...

> Mass submission in one form or another has always been a part of the contest.
....

> 1. According to the statistics page (http://www.mathworks.com/contest/armyants/statistics.html), I wasn't even close to being the person that submitted the most overall entries.

The statistics page clearly does not contain entries that have not been evaluated yet. As I said, you submitted 397 entries during the last 23 minutes, making you the number one spammer. Which does not mean that I support David Jones and SY doing spamming as well!!

> 2. The first quote from the rules you gave is taken out of context. The full quote says: "Entries that compromise the contest machinery are no longer allowed. ...

You are right, I did not quote 100% correct. But the meaning of the sentence "it's too hard for the few of us running the contest to keep ahead of the group intelligence" can also be seen in context with the problems implementing a captcha.

> 3. You also didn't fully copy the second quote you gave, which is: "Tuning the entry to the contest test suite via tweak bombing or other techniques is still allowed, but we ask that you not overwhelm the queue." Thus the rules EXPLICITLY allow tweak bombing.

So the plea of the contest team does not mean anything to you???

> I'm sorry if you don't like the fact that I have a lot of entries clogging up the queue right now, but I respectfully submit to you that what I did is fully allowed by the rules and was well within the spirit of the competition.

If you think that it is in the spirit of the competition, please upload your submission script on Matlab Central. In the next contest we will all fill the queue so that we have to wait for weeks or month for the contest winner and the contest team will have to keep the contest machine running for that time. Great!!

By the way, winning the contest by luckily selecting the best parameter set is not a great achivement. Winning the darkness or twilight prize as well as contributing any code segment making the algorithm better/faster is much more honorable in my eyes.

Markus

the reason isn't relevant, spamming is spamming.

we all want a CAPTCHA, that's how we all know that the spamming isn't welcome or an intended use of the server.

I expect people are going to click like mad to get a few variations submitted in the last 30 minutes, but using a bot to spam it quicker puking tweak bombs in isn't clever enough to be cute past the first occurance.

I know other people did it more. I know if not someone, than someone else. These are terribly weak rationalizations for it. It's still just as noble as automated telemarketing.

I hope that rather than just delaying the contest outcome for a day or more that an automated spammer wins it. That should help motivate preventing it in the future.

"Nathan" <nathoqXXX@yahooXXX.com> wrote in message <gffa1i$hcl$1@fred.mathworks.com>...
> Alan,
>
> You've developed a robot to twiddle parameters with no rational programming input, and you maintain that this is "well within the spirit of the competition". This is a bizarre claim - surely the spirit of the contest is to reward ingenuity, algorithmic innovation and technical skills? I hope your tweakbot doesn't win.
>
.......
> To change the topic and tone completely - thanks for your ongoing campaign to comment the code. It's good to see that commenting now seems to be permeating the culture of the contest.
>


Nathan:

While I agree the items you list are part of the spirit of the contest, I don't think it's all inclusive. In fact, from the contest FAQ (http://matlabwiki.mathworks.com/Programming_Contest_FAQ): "Why are people allowed to tweak my code? .... One of the goals behind the MATLAB Contest is to learn more about MATLAB programming" I can assure you that I learned some things I didn't know before in developing my 'tweakbot' and who's to say that the ingenuity and technical skills have to apply only the the submitted solver code?

Of course reasonable people can disagree on this issue, and what really matters are the opinions of the contest team on this issue. I've pushed the boundaries of the contest rules before and as a result earned a permanent mention in the rules section that forbids extraction of the test suite. Perhaps the team will decide to somehow update the rules in the future as a result of my latest stunt.

Regardless, for me these types of things are what makes the contest so enjoyable. I can't even begin to compete with some of the more brilliant competitors who seem to consistently develop amazing algorithms. Thus I tend to focus on profiling, timing improvements, tweaking, and 'meta-contest' items like code commenting and scoring formula analysis. I believe the contest team wants to be as inclusive as possible, and thus sets up the contests in such a way to encourage and invite people other than 'code gurus'.

I do appreciate your kind words regarding my commenting campaign, and I was also pleased to see it permeate further during this contest with other people adding a 'change log' at the start of the leading solver. Regardless of any other currently controversial issues, this lone fact is the most telling of how the contest is really collaborative in nature and helps build up a small but passionate user community.

Markus:

I don't want to start a flamefest, but I think you are being a little bit hypocritical regarding this situation. I'd like to point out your commentary from the contest a year ago (at http://www.mathworks.com/contest/splicing/winners.html#winner1) that you were the grand prize winner of. Specifically:

"... So I started an incredible session of "copy & paste", randomly choosing submissions from the queue and pasting the repmat-tweak. Minutes before the deadline I clicked the "submit" button in about 40 open browser tabs. In the late evening (Central European time) I saw that "M07" took the lead and that I had luckily selected and tweaked that submission from the queue before."

So was that an 'honorable' way to win? Was it a 'great achievement'? You noticed something in that contest that could potentially improve the score and randomly 'rolled the dice' on submitting entries. While I submitted quite a few more entries this time, I essentially did the same thing in that I noticed something that could potentially improve the score, tried it against a lot of other entries, and am hoping to get lucky.

As I said in my other messages, this contest means different things to different people. I try playing a certain way and you try another way. My actions in no way impacted your ability to compete in the contest, so I'm not sure why you are so upset about them. If they have a impact on the contest organizers somehow, that's another issue. But nothing I've seen or heard indicates that is the case.

"Tom " <boundaryscan@gmail.com> wrote in message
> I expect people are going to click like mad to get a few variations submitted in the last 30 minutes, but using a bot to spam it quicker puking tweak bombs in isn't clever enough to be cute past the first occurance.
>

Tom:

So where do you draw the subjective line? If my ~400 entries aren't acceptable, are David Jones' ~100 entries (presumably all entered manually) in the last 30 minutes ok? If not, how about Fabio's ~70 entries, or the numerous other people who submitted around 30 entries each during the final push? As I said in my last message, Markus won the contest a year ago with ~40 randomly selected entries in the last 30 minutes.

> Tom:
>
> So where do you draw the subjective line?

That's a tough question that should be given some thought.
My gut reaction though, is to put the line well on the sweaty, clickety
side of computer generated code, computer submitted code, or similar.

Hopefully CAPTCHA will get fixed and will help throttle.

note to hosts: Make sure the CAPTCHA is time sensitive so that
transaction initiations can't be done well in advance.

What's next, hiring an online gold-farming company to make submissions?

> "... So I started an incredible session of "copy & paste", randomly choosing submissions from the queue and pasting the repmat-tweak. Minutes before the deadline I clicked the "submit" button in about 40 open browser tabs. In the late evening (Central European time) I saw that "M07" took the lead and that I had luckily selected and tweaked that submission from the queue before."

You are absolutely right! That contest win was also nothing more than luck with a little tweak. I am much more proud of my two Twilight Prize wins. I have written that view one paragraph below from where you took the references.

But I see one difference: My "repmat" tweak improved the score of some submissions about 0.01 or such. If someone else had submitted a new algorithm in the last seconds of the contest, I would not have won with that tweak. Your frequent submissions during the contest, however, lead to an extreme overfitting of the solver to the testsuite. This way an algorithmic improvement will have only small chances to make it to the top.

Markus

interesting discussion.
The submission by machine is not advocated at least to the contest. If the test samples are large enough, the result will walk through the solution space. The absence of CAPTCHA is definitely a "tragic" of the contest.
I guess there will be no respond from the official. In the later stage of this contest, the contest team seem inclined to keep silent. Many sugguestions have been proposed without any feedback. It's not a good smell^_^

"Markus Buehren" <mb_matlab.REMOVE@gmxTHIS.de> wrote in message <gffb57$80i$1@fred.mathworks.com>...
"Which does not mean that I support David Jones and SY doing spamming as well!!"

And I do not support Markus doing spamming as well.
Significant part on my submissions is testing of NEW code and modifications. Usually they require some adjustments which result in 10-20 submission per idea.
Even if I knew “the best” parameters I would never submit only one version. Because then somebody like Markus will pick it up and win by minor last minute tweak.
Just for sake of self-preservation one need to submit multiple decoys.

SY (Sergey)

Congratulations to all day-winners, especially to Jan and Nathan for winning Darkness and Twilight.

Also kudos to the people who initially contributed the two solvers we have been working on most of daylight. I think those are Nick Howe with his runner-up in darkness and especially Jack Snoeyink (thinkingslowly, #51687)? Sorry if I didn't track those back correctly. In my opinion this way more meaningful than winning Daylight and should also be worth a prize.

Thanks to the Contest Team for holding another very interesting and fun contest. See you in May!

I just drop in to congratulate all winners of this contests pay my respect to the mathworks team for another great contest.

Unfortunately, real life obligations kept me from participating this time, but they did not keep me from following the contest.

Hope to take part again next time.

"Andreas Bonelli" <andreas.bonelli@gmail.com> wrote in message <gfgkbb$8mb$1@fred.mathworks.com>...
...
> Also kudos to the people who initially contributed the two solvers we have been working on most of daylight. I think those are Nick Howe with his runner-up in darkness and especially Jack Snoeyink (thinkingslowly, #51687)? Sorry if I didn't track those back correctly.

I agree! Nick and Jack are the heroes of this contest. Jack coolly produced a leader inside the final 24 hours with brand new code, while most of us were scrambling for incremental modifications. Nice work!

I want to add some comment in the last:

One, the Jack's last day's new algorithm may be not really better. I guess it is just lucky. More thinks itself clever. NH's version is good enough. One of my last series "RandomKing 15", 18855.66 (cyc: 27, node: 1022) 146.5812 20627.4015, it beat the top1 in some smell, but I tweak one random parameter in the wrong way^_^ For the sample testsuite, I got 14000-20000/100s+ using RandomKing seriers; but got 18000-22000/120s+ using the top1's new algorithm. (RandomKing just fix 2-3 new bugs in the bug-fixed nh's entry, I think, and do some adjust parameters layout)
Two, the NH's entry is mainly similar to the older one(in dark and twlight stage) in the alogrithm (which has more or less relationship with the entry from the prevoius ants contest). The most important contribution of Nick is the killing bug fixature. The magic lies in,
mask(Y) = mask(Y)|(opAntMap(Y)>0); % make sure we don't blunder into an opposing ant!

I has a developing a new algorithm to enhance the NH's entry when the max scent==0 in the last several hours(the main weakness of NH's entry lies here). But I have not submit it. One thing is the improvement is case-sensitivity when the scale of cases is small. The another is that the nature of the contest do NOT advocate the new algorithm after the twlight stage. It's really depressed to see you lose the prize using your own new alogrithm. such as Ncik^_^ I really hope all of the bug of contest will be removed when the next contest.

Hi, all. see you next contest if time is allowed.


 
 

I appreciate the kind words, and am flattered that my code became the base everyone was working with for most of the contest. It was both humbling and exciting to see what others did with my original code. Thank you.

One of the frustrations of this contest for me was the way random variation could seem to swamp improvements to the algorithm. At the end I was working on an improvement that I hoped would give me an edge: looking at the opposing ant scent, and turning off the aggression if they happened to be moving sugar in the same direction. (I can't take credit for this idea; it was suggested by Doug Hull in the first contest video.) As far as I could tell from looking at the leading entries, nobody else was doing this. It seemed to work well on the test set, but in the end, it wasn't enoughto win the grand prize. (It did score what I believe was the best result using my old code base, 21801.3768 for Kumbayah 25.)

It looks like the winner was submitted by a bot. Nevertheless, congratulations to Fabio C. for his apparent first place finish!

Seems like the queue has been processed?

Congratulations to Fabio C. !

I take it back -- it looks like Jin had the best score using the older code base. Well done!

Hi all, thanks for congratulations!

I have no difficulties saying that I was lucky! In the wiring contest I lost the "1000 Node Challenge" for a copy/paste error and I made some stupid mistakes during the final rush. I was very disappointed. Now I was luckier.

This contest is quite different from the previous one. Here the solver function is called tens of thousands times and the computational complexity was very limited. So it was impossible for me to speed up the solutions of the other partecipants. The solutions was also very overtuned to the actual testsuite (and to the King of hill script) so it was necessary to add the following line:

xyz = rand(3,1);

at the begin of the function to get disastrous results and timeouts.
So I thought that
1) Overtuning of the best entries (i.e. "Vectorize b" di the cyclist) had to be preserved
2) It would be necessary to introduce a little variation the behaviour of such entries in the late stage of the execution

I introduced the following lines:

if opscent(12)>126 && mod(opscent(12),16) == 3
xyz=rand(2,1);
end

so when there is enough of opposing scent in a particular position (i.e. when the match isdeveloped executing the hyper-tuned algorithm) the random generator could be altered.

By this way, I get very different results.
16 out of 66 entries got timeout, the others had execution time between 132 and 179 seconds.
The winner entry got the best results by having a good execution time, but the entry "Useless 242pknonfhr" got the best score of the last day: 15467.41 (however with a dramatic execution time: 174s).

Finally, congratulations to the partecipants for the developed algorithms and to the contest staff for the interesting (and spectacular!) puzzle!

Congrats Fabio! That's definitely an interesting way to tweak the system only in certain cases.

Hi,

the contest team has quietly updated the hall of fame with the winners of the army ants contest. Thanks for all the organization again!

Yours
Markus

Hi contest team,

you will hate me for this, but anyway: How is the Captcha doing? Did you have time to finish the implementation that nearly worked last time?

Yours
Markus