TournamentCompare v0.004 - Player Pools

One slight deviation from the old Clarion application is that I have instituted player pools as a starting point for a tournament. All players for a tournament are entered into a sorted list (player pool) that the tournament template draws from. This allows the generation of the player pool to be independent from the tournament template and can therefore be used to test all sorts of varied starting positions and even seeding patterns.

For smaller tournament formats (8 players or less), every possible player pool combination can be generated and used in sequence to eliminate another possible bias. For 8 players this would be 40320 combinations (8 factorial). If a tournament is larger than 8, you could potentially take the top 8 combinations and interleave other random players to give 16, 32, 64, or 128 player pools. This would give a large control set (~40,000 samples), but at least give a somewhat even spread of higher candidates.Unfortunately it would also eliminate the random chance that the most skilled player meets the second most skilled player in round 1 of a 128 player tournament. Since generating a random player pool will be quite easy, I can also test the difference of these two formats and isolate a specific amount of bias due to potential seeding.      

TournamentCompare v0.003 - Validating a tournament template

Because the tournament template system is flexible enough to allow all sorts of formats, it is also flexible enough to bring in all sorts of loops, missing players, unlinked matches, etc, so I'll be making up a validator to vet each tournament template before running thousands of tests on an invalid tournament.

Here's the high level pseudocode:

1. Start with the final match and work back along the player entry locations until all matches have been processed into a list (vettedMatchList) and all the players are in another list (competitorList). This also sets the match layer (how deep into the tournament that this match represents)
2. Check to make sure that all TournamentTemplateMatches for this TournamentTemplate have been accounted for.
3. Check to make sure that all competitors have an entry into the tournament
4. Build a list (noExitMatchList) for each match that 2nd place continues in the tournament.This is combined with the match layer to evaluate final exit position for 2nd placegetters that do NOT continue.
5. Mark the tournament as valid and update the database with match layers & exit positions. 

TournamentCompare v0.002 - Enumerated Types in data first MVC

After tinkering with the controllers and views, I decided to look into a way to display the enumerated types used to set where the players were previously before this match. Ideally I wanted a radio control (as I had in Clarion), but the formatting eluded me for a super clean and simple implementation without losing data first design. In the end it turned out to be easier as a byte-stored enumerated type that can then be edited as a droplist.

Tournament Templates

One of the main parts of the program is to be able to put together any tournament format and then be able to run through thousands of tests to see the inherent bias of the tournament type. The tournament templates to be tested would then need to be anything from single elim, round robin, double elim, swiss, world cup rounds into single elim, etc. To achieve this, each tournament template would be a combination of TournamentTemplateMatches where each match accepts the 2 entering players from either the original player pool at the start of the tournament, another match previously held (Eg: single elim final match accepts the winners from the semifinals matches), a round robin subtournament, or an aggregator of previous scores to create a subtournament pool of ranked players.

Because each match could be pointing back to another match or another round (I'm assuming that subtournament aggregators can be handled by the round template with no rounds), I needed a concrete way of storing which type it was. Setting the PreviousType to "Match"would also mean that the PreviousMatchID is now valid, whereas if "Round"was the previous type then the PreviousRoundID would be valid.

Enumerated Types in data first MVC

Ideally I'd like to see in the database the type listed so that the data is a bit more human readable, but after investigation it seems that C# prefers integers or Bytes as the underlying enumerator value. I'd debated changing them to integers, but decided to use bytes instead (Tinyint in SQL) so that they appear as a different type in not to be confused with an ID. Changing the database wasn't too hard after manually changing the data to conform, then updating the field type in the designer.

Because I want to continually bring in database changes, I was keen to find a way that left the enumeration types already bound. I'd initially added the types in as another model, but found out later that you can add them directly into the model designer of the database with a right click > add >  enum  or by adding it to the enum types in the model browser. You can then change the fields to the new enumerated type. Doing this way it maintains the enum inside the database design (and scope) and also doesn't drop the assignment of the enum when the database is reloaded.

The display fields handled the enum perfectly, but the editing fields treated it like a string. There was some additional code to manually change the @html.EditorFor() to a @Html.EnumDropDownFor(), but I really wanted to be able to regenerate the controllers and views for any data first changes, so eventually settled on adding a html helper to overload the EditorFor() when an enum is detected.

TournamentCompare 0.001

I've been weighing up what programming project to work on for a while. I had Factor Friends as the front runner, but the move to a facebook friends variant didn't pan out as facebook now only allows access to friends of friends to only those who have installed / run your application. In some ways it is sad that the original concept cannot be done, but at least it wasn't made completely redundant if I had finished it before the changed the API.

On to TournamentCompare. I have been meaning to return to the Designing a Tournament series of articles for a while and it's really at the stage where I need some actual stats to pull together some comparisons. I had written a tournament analysis program back in 2006 to do some of the bias analysis, but since then I'd lost the Clarion development environment and couldn't continue modifying it for non-perfect play. So, a total rewrite is in order and hopefully make it web based and accessible for others to vet and build their own tournament types.

After a bit of digging it seems that another MVC application should be pretty easy, but use bootstrap to help with the presentation and graphs. Even though code first EF design seems to be more prevalent these days, I'm still a stickler for getting the data right so I've stuck with data first design.

Starting off the app in Visual Studio 2013 was pretty easy following this data first tutorial, although I'd created the MVC app first before creating the database. VS 2013 now has the ability to immediately add a SQL database right into the MVC project and create / modify tables and data right from within Visual Studio. I'm impressed! Within 1/2 hr I had the first TournamentTemplate tables done and looked like getting the others completed was just a matter of time. Went back to the tutorial to build up the entity framework for the starting tables, but unfortunately a VS crash took out the project connectors. Since it was so early I dumped and recreated the project with only the database making its way into the new project. EF done. Checked a quick update of the DB to make sure the EF would also update, and it looks pretty good so far.

Calculation of win percentages

This article is part of a series on designing a tournament:
Part 1 - Designing a Tournament
Part 2 - A grading system
Part 3 - Calculation of Win Percentages (this article)

One holdup in determining a grade for different tournament system is to settle on an algorithm to give consistent win percentages that scale with the difference in true skill of the competitors. Initially I had used a 100% win ratio to the higher ranked player for determining the inherent bias in a tournament system so that the result could be clear of as many other biases as possible, however this precludes tournaments that attempted to add more games to a match for bias reduction. Without some chance for the lower ranked player to overcome a higher ranked player (even a slim chance) there would be no need to play best of 3 matches, best of 5 matches and the like for the better player to prove their worth.

 Ideally I'd like to come up with a system that gives a win chance similar to a standard deviation bell curve or some sort of sigmoid function that granted ever higher win chances the greater the skill difference was between players. As it turns out, the bell curve presented as a cumulative chance for success also appears as a sigmoid. Also, the idea of having ranks of importance to also determine the relative skill differences appeals as the difference between 2nd & 5th should be far more significant than the difference between 32nd and 35th. Even though there may not be a bell curve of skill displayed at the tournament, it would be prudent to assume that we have the top end of the bell curve in attendance.

After a couple of attempts at mapping the ranks of importance onto units of standard deviation, another mathematical nicety presented itself: The inverse ratio of each player's true skill produces the same ratio between ranks: 2:1. Eg: 2nd in true skill play 4th in true skill (1 rank apart) gives 4:2 => 2:1. 8th plays 16th (1 rank apart) gives 16:8 => 2:1. 2nd plays against 8th (2 ranks apart) gives 8:2 => 4:1. 4th plays 16th (2 ranks apart) gives 16:4 => 4:1.

This inverse ratio means a difference of one rank has 66% chance to win over their opponent, a difference of 2 ranks gives 75% chance to win, 3 ranks gives 88.5% chance, etc. Not quite as steep as the standard deviation steps, but follows along the same shape as cumulative chance for success. This should fit quite nicely for now.

Criteria for selecting a competitive game

For a game to be selected for inclusion into Competitive Computer Games, it will need to be strong in these 4 areas:
1. Be Competitive – The game itself needs to have an aspect built toward competitive, multiplayer play. Any game could be used competitively, but certain games lend themselves toward fostering a competitive environment.
2. Be Internationally Competitive – The game should have some recognition by major esport organizations at an international level. This gives weight to the competitiveness of the game and ensures an existence of a competitive culture independent of Competitive Computer Games.
3. Have content appropriate for a school environment
4. Ease of installation – This would include the cost of the game, the cost of hardware / software peripherals to make the game work competitively, the effort to set up, etc.

Ranking vs Matchmaking

Sirlin wrote up an interesting post regarding the new Starcraft II ranking system with insights from his discussions with Rob Pardo about the merits of different systems such as Elo and TrueSkill. The focus is whether the ranking system used for matchmaking should be different to the displayed rank for player satisfaction. I like a lot of his conclusions, but found this quote a little odd:

Microsoft [through TrueSkill] makes another good point here that ONLY winning and losing can be allowed to affect these stats. You can't adjust the matchmaking stat by "experience points" or even by any skill-based stats such as headshots, number of kills, time to finish a lap in racing, etc. All those stats can be gamed, and you will end up trying to get more headshots or something instead of winning. Any formula that equates number of headshots (or any other stat besides wins/losses) with how likely you are to win or lose introduces a layer of imperfect simulation. If we want to know how likely you are to beat someone, we should only consider your wins and losses, and not any in-game stats.

When looking at a mathematical ranking system this makes sense, but it doesn't make sense if you are purely interested in efficient matchmaking. Here's an example:
Starcraft starts off and all players have no wins or losses and no stats. Legionnaire (pro player) gets drawn against a scrub in his first round and wins comfortably; the scrub had not even managed to get out of his starting area. Analysing the statistics after the game also indicate a cakewalk to even casual observers. Quick game time, aggressive economy expansion vs staggered economy graph, little waste vs high average waste, etc.
Another game is played between 2 average players which happen to be closely matched. The game includes a number of pushes, each player's expansion being demolished and culminates in a pitched battle at the 40 minute mark. The statistics show a variety of interesting observations for astute players, but a big dip in army count for the loser is probably the only giveaway as to who won.

In TrueSkill these 2 games would both have winners on 1-0 and be equally ranked. For their next match they would be as likely to be matched together as it would be for Legionnaire to be matched against another starting pro player. If there was a pool of 50,000 starting players (first day numbers), it would take ~10 matches before the top ten would even play each other, and many more before the win/losses settled into a nice 50/50 pattern indicating you're playing at your level.

If a human were in charge of looking at all the matches played and given the task of suggesting new evenly matched pairings for the next round, they would be easily able to break up the whole group into, say, 10 pools of varying skill level by looking solely at the stats. They may even pick up on the fact that the 2nd match documented looked to be pretty even, and would be worth a rematch immediately for added context to the players. Repeating this process only 4 times should have people within the top 10 getting matched up together to get right into those super-fun, closely matched games.

It would seem that additional statistics can be useful in terms of matchmaking, so why the intolerance from mathematical systems such as TrueSkill? To me it comes down to a couple of issues: differing goals of ranking vs matchmaking, ignorance of bias inherent in matches, and the need for objectivity.


Ranking vs Matchmaking
To me the main issue is that TrueSkill is both a ranking and a matchmaking system. If you replaced "matchmaking" with "ranking" in the above quote I'd agree wholeheartedly; you DO need a very rigorous way of ranking people. If you brought in anything else other than wins / losses you give advantage to those who play a certain way (finish quicker, headshot more, etc). As an aside this would also reduce the potential for innovation and metagames.

A good ranking system requires an independant, unbiased method of delineating between players of different true skill (the player's inherent skill that ideally would be on display in each match, but can be clouded by a number of factors) so that each player feels as though their attributed rank is in line with their perceived rank. There needs to be justification for your rank so that if someone questions why another player is higher or lower than them, there is evidence available to alleviate their concerns. Any biases in the ranking process detracts from the validity of the entire system.

Matchmaking, on the other hand, has a goal of maximizing entertainment to the competitors (and spectators). The premise is that closely ranked players are more likely to play epic battles with high engagement and player satisfaction. I have long supported this notion and have built it into a number of leagues and tournaments, however it's not the ONLY thing. Sometimes players enjoy playing above their rank for that underdog win feeling. Players may get more out of playing with/against friends. Players may enjoy certain matchups, or certain maps.

In one of the grand finals for Australian Warcraft 3 I had an interesting conversation with another tournament organiser. 2 of the favourites for the title who were long-standing rivals had been drawn in seperate pools, but were to meet in the semi-finals of single elimination rather than the finals. The pool allocation was seeded as much as possible through prior matches in the tournament, but as these players were from different states, there was no justification in placing them over and above other state winners. The other organiser was just trying to see the best match in the finals and wanted the pools reorganised. This is a classic case of conflict between ranking (through merit) and matchmaking (for entertainment).

Matchmaking for entertainment becomes a bit wishy washy. How do you know that someone is going to enjoy the match you just created? For a start you could give opportunity for the player to select certain traits before the game starts (initial team etc) that the player KNOWS they want, but you could also give opportunity for players to rate their enjoyment of a game after the game has finished (or after watching a replay for spectators). This needn't be an exhaustive analysis of the game, a score out of 10 or even a thumbs up / thumbs down would suffice as this would give rise to a new method of optimisation: to matchmake games that maximize player/spectator satisfaction. Could it be gamed? Sure, but the eventual loser would be the players themselves. The more accurate information you can give as to why it was fun, the quicker a system could deliver games that increase your entertainment.


Objective vs Subjective matchmaking
Putting the idea of matchmaking for entertainment to the side, let's revisit the intial premise; do you only need to wins/losses to adaquately matchmake? Yes, if you want to maintain objectivity. The example of using a human to view the game and statistics to more efficiently separate people requires having faith in a 'gut feeling'. A human can look at a short overruned game and see that the opponent was outclassed, but it's not just because of the game's length. A human can see that a better player can outmaneuver troops to maintain a winning advantage, but that's not a hard and fast rule for winning either. There is, in fact, many opportunities through the game that pro players can demonstrate their skill and these build up to give a general feeling of confidence that one player is superior over another.

When building back propagated AI networks in my uni days, we would continually use problems like these. You can't quite put your finger on what are the hard-and-fast rules to follow that led you to a decision on who is better, but you're pretty confident of giving a judgement either way. With the volume of statistics available after Starcraft matches, it should be achievable to devise an AI that delivered, say, 7 levels of superiority (totally pwnd, much better, better, same, worse, much worse, totally outclassed). After many games I'd hope that the system would be delivering 'same' players all the time, but the ability to recognise a large discrepancy in skill can help the initial setup as well as tracking large changes in behaviour (rapid improvement in play, returning from a long absence, etc). Would it deliver better matchmaking? yes. It would also be objective, but not justifiable. Without justification you'd have a hard time convincing others that the system is rigorous enough to provide a rank, but it might fly if rank and matchmaking were seperate beasts.


Controlling Bias
Another issue that a pure win / loss ratio overlooks are the biases inherent in the games played. If, for example, Terrans are heavily favoured in a TvZ matchup, then this should be reflected in the prediction of who is more likely to win. There are 3 different biases common to tournaments and ranked games, the player's playstyle (race choice and preference for certain strategies), the map bias (some maps are more suited to certains race / playstyles, or more well known than others) and the individual head-to-head bias (where someone just seems to have the wood over someone). In tournaments you also have a tournament bias where some players perform better or worse depending on the importance they place in a specific tournament (homeground advantage for international events). These biases are once again somewhat subjective, but could be deduced from large samples of games played. It is also a moving target as the biases could change through the metagame, direct patching, or sustained effort from players to eliminate weak points in their game.

Biases do not have to mean that the game needs fixing; biases can add strategic elements to play (zerg rush will always be faster, but weaker). It does mean that matchmaking needs to consider any biases if the goal is to ultimately produce epic, engaging games that are desirable for players and spectators. Placing a higher ranked player in a weaker position could still achieve this.


Summary
Wow, this went a lot longer than I meant and it feels like it needs a wrap-up. To me, matchmaking is about providing entertainment to the player and spectators. Players of close rank can LEAD to entertaining games, but I don't believe a rank based solely on wins and losses provides the most efficient method of matchmaking.

lol season 1: interview

PC Games - E3 2011 - League of Legends: Clash of Fates

Lotta hype, but the league structures (especially draft mode) are shaping up to be awesome. I really hope it catches on for large scale leagues and tournaments (like the upcoming WCG demonstration event), but I also hope mini tournaments are able to be pumped out to whet the appetite. Simple ones like 8 team beginner tourneys with a new skin as a prize (or simply riot points). Getting very VERY interesting.

A grading system for comparing tournament stuctures (part 2)

This article is the second in a series on designing a tournament:

Part 1 - Designing a tournament

Part 2 - A Grading System (this article)
Part 3 - Calculation of Win Percentages

A Grading System

Although I set out what I think are the boundaries of tournament systems (that of single elimination and league) at the end of the previous article, in this section I want to come up with a meaningful method of comparing the disparate tournament structures in terms of its ability to rank players, the desirability from a player's and spectator's perspective, the resources that it uses, the inherent tournament bias, and the individual matchup bias. Some of these (like resources) are a known quantity and can be mathematically compared, but others may stray into subjective territory. If there is no hard number accessible, a rating system will rank the specific tournament on a scale of A to E against the other tournament systems, where A is the best for that particular section and E is the worst. Lets start with some easy ones first:

Resources:

Even though location impacts on time eventually as a resource, there are still some benifits in having the 2 separate resources listed so that situations with ample amounts of one resource can choose appropriately. The first value is minimum time to complete the tournament. As I would like to compare the difference between one match and best-of-3 matches at a later date, the match length is used as the discrete unit of measure. An example will be an 8 man single elimination tournament has a minimum time of 3 match lengths, whereas an 8 man league has a minimum time of 7 match lengths (calculations formally defined later on).

Although it doesn't neatly cover the dilemma of limited locations, the number of matches in a tournament gives an indication of the size of the problem. An 8 man single elimination tournament is completed in 7 matches, but an 8 man league takes 28 matches.

Another way of demonstrating the time increases imposed by limiting locations is by recalculating the total time taken if only 1/2 the locations are available for a tournament of that size. This would give single elimination an increase from 3 to 4 match lengths for an 8 man tournament in contrast to an 8 man league going from 7 to 14 match lengths. [Maybe represent as a proportional increase?]

Tournament ranking ability:

A simple count of each unique rank given to participants should suffice. A 16 man single elimination tournament provides only 5 ranks(1st, 2nd, =3rd, =5th, =9th), whereas a 16 man league will rank all 16.

Inherent Tournament Bias:

Now it starts to get a bit tricky. A couple of years a go I developed an application to measure the inherent bias of a tournament structure against a competitor being ranked at his true skill. An extreme example of this is if the best player took on the 2nd best player in the first round of a 64 man single elimination tournament. The person who should have come 2nd will now come =33rd. The program takes a template of a tournament system and runs thousands of trials using random seeding and with competitors playing at their true skill to gain an average bias per player. An example would be a 64 player single elimination tournament having an average bias of ~0.59 levels after 100,000 trials.

[updated 15/7/09]

The unit chosen is a level of a single elimination tournament. This means that being misranked as 2nd to =3rd is the same importance as being misranked =17th to =33rd. This feels about right as the importance for accurately placing the top players has more perceived bearing on the bias of the tournament.

Using a Base 2 logarithm we can formalize the meaning of a 'level'. Log2(16) = 4, Log2(32) = 5, log2(64) = 6 etc. So someone who has a true skill that ranks them 32nd would receive an expected value of 5, but if they finish in 16th they receive an actual value of 4. The difference (1) is the same as finishing up a 'level'. So the formula for player bias is:

Player finishing bias = abs(log2(expectedPosition) - log2(finalPosition))

With a formula not reliant on the actual levels of a tournament, it allows the average tournament bias to compare totally different tournament structures as long as you know where players were supposed to finish compared to where they actually did.

Individual Matchup Bias

Although the bias should be able to be estimated once the situation is known, these factors by and large impact on all systems equally. I might leave this section as a special notes category to highlight specific independant matchup issues (such as robustness for home/away bias) until there's a standard way of presenting these types of bias across all structures.

Player satisfaction

This grade will need to be tempered with some subjectivity, but there are 2 areas that can be measured: average number of games played and the closeness of the competitor's skill levels. Average number of games played helps give an indication of how many rounds a player can expect to stay in the tournament and is a ratio to the minimum number of rounds of the tournament to normalize the result. Leagues would have 1.00 as players participate in every round (barring finals, leagues with finals will be analyzed seperately) whereas an 8 man single elimination tournament gives ~0.37.

The closeness in competitor skill should indicate both a greater potential for a close game and a greater potential for learning in a competitive environment. I'd like to collect some solid evidence (or even lots of circumstantial evidence) that this is the case, but it feels right from a what I've observed. Maybe it's a cutoff thing instead of proportionally based? I'll need to adjust the program to output this result anyway, so I'm open to suggestions.

Spectator satisfaction

Not sure what I can do in terms of measuring entertaining play, but close matches can glean off the closeness in competitor skill grade, with possible emphasis on the final games [logarithmic?]. The high degree of skill would emphasize tournaments that gave maximum opportunity to 1st and 2nd to play each other, and a logarithmic dropoff after that. [Very close to tournament bias?]

In the next couple of articles I'll look at the more common tournament structures and see how they stack up. I'm sure I'll be back onto this page at some point to tweak the grading process to more aptly fit the criteria. [should wikify it?]

Part 3 - Calculation of Win Percentages

Designing a tournament (Part 1)

This article is part of a series on designing a tournament:

Part 1 - Designing a Tournament (this article)

Part 2 - A grading system
Part 3 - Calculation of Win Percentages

Spent a bit of last week involved in a few discussions about ranking systems used in competitive computer gaming. This reminded me of an application I had virtually completed to calculate the bias of different types of tournament systems, but I hadn't written up the results. Hopefully this series of articles will address that.

Background:
Ever since the second season of QGL back in 1998 I've been designing and analysing tournament systems for use with computer games. Back then there was a great focus on player satisfaction as a goal for the type of tournament to run as we wanted to maximize the enjoyment and participation of players throughout the season. We developed a matching system akin to swiss to be run across several LANs that enabled players to compete against people of their rank and be resilient to players/teams dropping or joining the tournament.

As AusGamers grew to a national body for organising tournaments, we dabbled with finals formats consisting of mainly double elimination. At that time the format was relatively new on the scene, but it held up remarkably well to a number of concerns we had for finding a winner from disparate states in an efficient manner.

Through involvement running the Australian leg of CPL and WCG, a number of other systems were reviewed and the shift to a more spectator oriented position could be seen emerging through the formats chosen. As a keen advocate of double elimination at the time, I was a little disappointed in WCG's insistence on pools of players competing against each other with the top 2 advancing to a single elimination tournament. I could see the reasons why that type of structure was chosen, but believed that it was an inefficient method. I spent another 5 years involved with WCG (partly as Australian tournament director designing my own tournaments, and later at international level as head referee) and had continued to seek ways of eliminating bias.

There's got to be others in the world that have gone through a similar upbringing and who share a passion for tournament design. Hopefully these articles can stir up some debate so that we can develop a body of knowledge about how best to design a tournament for future competitions.

What makes a Tournament?

Wikipedia currently does an Ok job of defining what a tournament is, but not why it exists. What are the goals of a tournament? For me, the primary goal of a tournament is to provide an objective method for finding the competitor with the highest true skill. Who's the best gathered here today? As an adjunct to that, a tournament should also do its best to rank all competitors by their true skill. This becomes especially important when there are rewards given to lesser places.

There are also 2 other goals of varying degrees of desirability that depend on the context of the tournament: player satisfaction and spectator satisfaction. Players wish to derive satisfaction by playing as many games as possible, by being treated fairly, by being exposed to learning opportunities, and by demonstrating their skill to others, whereas spectators want to see entertaining play, close matches and a high degree of skill being displayed. Ideally these goals should be maximized and can shape the type of tournament selected, but not at the expense of its primary goal otherwize it's not a tournement (EG: WWF as a spectacle, handicapping for social play). These will hopefully be expanded upon in future sections.

True skill in a competitive sense is defined as the competitor's own abilities in the game being tested. Ideally everyone should be able to demonstrate their true skill each time they play, however a number of factors can cloud a player's true skill to produce a bias in the final result of each match in the tournament. Another topic for later, but it's mentioned here as acknowledgement that they exist. Initially when analysing tournament structures we will assume that players are able to play flawlessly to their true skill so that they will always win a matchup with someone of lower true skill.

The perfect tournament for me would be one where every competitor is able to play every other competitor simultaneously (but individually) and complete the match instantaneously, while playing at their true skill. Obviously this could never happen, but that type of tournament would be able to state with certainty who was the best competitor at that specific time and place. In reality there are biases all over the place, having to wait for your rounds, having to play people coming off a losing streak or winning streak, not playing everyone, etc. All are biases inherent with the tournament system that you choose. Understanding these biases and choosing a specific type of tournament structure that minimizes bias is at the heart of tournament design.

The final part of the puzzle is resource management. You are almost always going to be constrained to complete the tournament in a specified amount of time. If you can complete the tournament in a shorter amount of time then there are many ways to enhance the tournament to address bias issues or cater to secondary goals. The other main resource is the amount of locations available for matches to be played simultaneously. Not having enough locations for each round will impact on the total time taken for the tournament, however increasing the locations may be impossible or too costly.

Decisions, Decisions:

So, we want to deliver an objective ranking of all competitors while maximizing player and spectator satisfaction, minimizing bias inherent to the tournament structure or to individual matchups, and maximizing location use and time. No worries! Where do we start?

First port of call is to look at the resources you have and the anticipated players to see what types of tournaments are possible. Try single elimination initially as there will be no tournament structure with less resource usage for a competitor vs competitor style tournament. If you still don't have enough resources you'll need to cull the amount of players eligible or beg for more resources and use single elimination.

Next try a league format. If you can fit in a complete league then your tournament biases will be largely minimized, however the resources and time needed make these a rarity in most LAN and computer game competitions. You are most likely going to come out somewhere in the middle; more than enough resources for single elimination, but less than a full league. In this space there are a bevvy of different tournament structures to choose from, but little to no information to gauge how efficient each type of system is.

So how do you choose which is the best system for you? In the coming articles I hope we can come up with a grading system to compare tournament structures against each other in terms of bias and desirability and then explore what each type of tournament structure brings to the table.

Part 2 - A grading system

CGS belly up

Wow, this is huge. Out of all the new startup competitions these guys seemed to have the public edge. The competition structure itself was a bit of a let-down, but they had it all over WCG, ESWC, etc with their on-screen presentation. Admittedly it was still B-grade stuff compared to NFL, baseball, et.al, but was on par with extreme sports coverage and poker.

For me the big indicator was the forums. 1/2 the commentary on the forums were from team members. The fan base per team is non-existent (compared to other sports on the same tier). I would have thought the TV viewing numbers would have supported it though.

This is going to be a big setback. I can't imagine another company attempting a TV-centric competition in the next 5 years after this failure, so it looks like we're stuck with the old stalwarts for the time being.

5mill to starcraft

Another starcrafter hits it big playing poker

DJWheat drops GGL and EG

I was having a hunt for a likely PS3 title and found out the news that Epileptic Gaming had shut up shop when looking for a GTA IV review. Looking back it always seemed funny that DJWheat would do coverage for CGS while still in withh GGL, but I didn't think it would come to this. It does add even more credibility to the CGS that they can drag in talent like Marcus. I've been impressed at the stage production of CGS and it's the biggest threat to WCG of all the worldwide competition formulas.

Marcus also has a B'day close to mine ...

Zenith for Coaches

St Luke's coaches for sport picked up a set of Azuma Zenith shoes as thanks for the summer season. Although not as useful in Competitive Computer Gaming as other sports, they are a comfy shoe that should do well in touch footy.

Ventrilo

Need ventrilomix for [QGL]

GameCreate

Haven't looked at GameCreate for a while, but happened to notice that the new verion is out of beta. Might be just the shot for CCG.

Map Concede vs Map Select ver.2

After last WCG I was fired up to redo the numbers for map concede and map select for a size 5 map pool. The results were a little intriguing.


As the size of the skill difference between the players increase, the starker the difference in the different map selection methods. 4 map concede continues to remain at the same level regardless of additional map bias as it continually selects the middle ground where there is minimal bias. 2 map concede BO3 performs the next best with an additional bonus of map variety. Map select BO3 with random 3rd is another step below with only .4% difference in methods over a 10% map spread at 53% win chance for player 1. This difference in methods extends to 4% difference when player 1 should win 70% of the time and map spread is up to 20%. Single map random selection gets hammered quite severely, but more to do with the lack of 3 matches as it is more or less in sync with the true skill if you look purely at the average. The spread blows it out of the water though, giving 10% to 20% change in outcome depending on the map. Comparing this to other differences, it seems vitally important to not use random method when using 1 map. Map concede would keep the spread to whatever bias the middle map provides.

Finally some stats and figures to get my own head around the problem and to show the powers that be.

Carmac

I'm not much of a fan of carmac, he's outspoken and almost up himself. Last year he slammed double elimination as a format in favour of the intensity of single elimination, but then whinges when single elimination doesn't deliver. The thing is agree with most of his points, but for different reasons. He just seems to gloss over critical issues that have much more impact than the surface issues.

FIFA finals @ WCG

Study material for FIFA this term at St Luke's.

WCG announces games for 2007

WCG posted up the official list of games for 2007. Looks like there's a fair few new games with 12(!) in total including 2 team games.

Seems a bit top-heavy with 4 RTS games, and no single person FPS ? I do like the inclusion of Gears of War as the 2nd team-based game as it has a lot to offer that is different to CS. Tony Hawk has also been a long-time hardcore game with little recognition, so it's another good pick. The competition format is still being decided for Tony Hawk & various other new entries, so I'm not sure how it's going to play in a one-on-one competitive format.

There's also the Promo video for Seattle up on the site too.