Frank Karsten
9/20/2005
Click here to see the Classroom!

In this article, I will talk about playtesting. It's one of my favorite hobbies. Because I tend to encourage players (including myself) to test a bit too enthusiastically, I have gotten the nickname “playtest monster.” Last summer I organized two testing “colonies,” in which many Dutch Magicians came to playtest Magic for two weeks. I tested a lot for Kamigawa Block Constructed in particular, which paid off very well I guess. I mean, I made Top 8 at GP Salt Lake City and GP Mexico City. I'm sure that I wouldn't have done so well at those tournaments if I hadn't playtested as much as I did. But because I tested so many games, I had the best deck and I knew how to play it. Playtesting is the most important part of good tournament preparation.

Today I will give you some tips that might help you to playtest more effectively for tournaments.

Playtesting can serve up to four major goals:
1. You learn how one certain deck performs against other decks. Based on that information, you can choose the best deck for a tournament.
2. You learn how certain cards in your deck perform. You can then cut the bad cards, try out new cards, and up the count of cards that perform amazingly well.
3. You can improve your game in general. Your play skill will improve dramatically if you have a larger sample of games to draw experience from.
4. You learn how you should play your deck in a given matchup. Are you supposed to play the aggro or control role?

So goals 1 and 2 are about deck development and goals 3 and 4 are about play skill development. To achieve these goals, you need reliable, accurate playtesting results and the right testing conditions. Your playtesting environment should mimic a tournament environment.

What issues should you keep in mind while playtesting?

In a tournament you will encounter different versions than your playtest decks

In a tournament, everyone will have made tweaks and changes to known archetypes. You don't know exactly what's in their deck. If you have playtested 20 games against a Mono Black deck that ran 4 Sickening Shoal and no Hero's Demise for removal, you will subconsciously assume that your opponent in an actual tournament will have the same removal configuration. You're so used to facing Sickening Shoal that you don't even take the existence of Hero's Demise into account. Then you get wrecked by Hero's Demise in that tournament.

A good solution to this is to playtest against an “average” deck. If you expect half of the Mono-Black players to be playing with Sickening Shoal, a quarter with Hero's Demise and another quarter with Horobi's Whisper, you should put 2 Sickening Shoal, 1 Hero's Demise and 1 Horobi's Whisper in the Mono Black deck you test against! I know no one will play such a strange deck and you shouldn't give that list to a friend who needs a deck for the tournament. But this way you get to practice against a wide variety of cards in a short time period.

Which card is better?

This comes up quite often. Take this example. While you were building your Mono-Blue Control deck, you were wondering whether Condescend or Rewind is a better inclusion. You thought about the pros and cons of each card and you still couldn't make a decision. Well, you should be able to figure out which one is better during playtesting. Just put 4 Condescend in your deck and then think about the following question every time you draw one: “Which card would be better now; Rewind or Condescend?” You should note down the answer every time this comes up and consequently do this over the course of a large number of games. Afterwards, you can analyze that data and you will see that one certain card is better more often than the other one.

Playtest after sideboarding

In a tournament, roughly 60% of the games you play will include sideboard cards. This simple fact should encourage you to playtest more post-sideboarded games than pre-sideboarded games. Nevertheless, most people I know neglect to do that. I can't stress the importance of testing with sideboards enough, as a lot of matchups change completely just by adding a couple sideboard cards.

Also, you should make a sideboard plan. While you are building your sideboard, you should write down what cards you intend to bring in and what cards to take out against the major archetypes. When you do this, you might notice that against a certain deck you want to bring in 10 cards, while you can only reasonably take out 4. Or the opposite could happen. You want to take out 10 dead cards, but you only have 4 good cards to bring in. In such a case, you should take a critical look at your sideboard again. It's obviously better to know about such problems in advance. To me, making a sideboard plan is just as important, if not more important, than building the sideboard itself.

Try to find the right plays

You want to learn how to play a deck from your test games. On top of that, badly played games won't provide accurate results. So you should ask other people what they think of judgment calls, mulligan decisions, and other choices you make. You should do this often.

You should also be weary of strange plays your opponent makes. Ask him why he made a certain play. Maybe he missed something or maybe he's a genius. Whichever the case, there is something to learn for someone. And you should allow take backs in case it is clear that someone made a very wrong play.

If you are testing a very difficult matchup, in which you constantly have to make tough decisions and knowledge of the opposing player's hand won't greatly influence your decision making process, then I suggest playing with open hands. Just give your playtest buddy the same information you have, so that he can think about your decisions as well. Two players are better than one and you might learn a lot by playing this way. This works best if you test with a player who is better than you. You can learn more from his insight.

A good playtest team

Around ten is the ideal size for a playtest team. Often you won't have much choice, as you will just test with your friends from the local store and/or people who live nearby. But you should try to have trustworthy and hard-working people who are on the same skill level in your team. You should agree to share all information together, up to the moment the tournament ends. You should also keep track of playing style preferences of players inside your team; who likes control and who likes beatdown?

Plan to test with and against a lot of decks

Before you pick a deck for a tournament, it is advisable to have tried out all known archetypes to see what deck you like best.

It won't help you much to test your new deck against your friend Billy's five-color Genju of the Realms/Sunforger deck. You'll probably win, but that doesn't mean that your deck will perform against serious decks. Before you start testing a format, you should scout the internet and make a list of the 3-4 most played decks. Those decks will constitute your testing gauntlet. You should try out new decks against the testing gauntlet. And you should test decks out of the testing gauntlet only against other decks out of the testing gauntlet.

Furthermore, you should try to plan things. You can probably estimate how much time you want to put into testing. Then write a list of the matchups you want to test in advance and make sure you can test all of that in the time period you have available.

Take mulligans

Sometimes, I see people who mulligan to 7 in playtest games, backed up with an explanation like, “Hey, it's just a playtest game! Starting with fewer cards is frustrating and playtesting should be fun. So if you have a bad opening hand, you can just shuffle it back and draw 7 new cards.”

The problem with this “mulligan rule” is that now you won't be able to pick out decks that mulligan too often. I remember playtesting a Blue/Green Madness deck, which could get amazing opening hands or awful ones, with little in between. That deck was forced to mulligan a lot, with an average of once per game. Other decks took a lot less. It's not hard to imagine that if you don't go down to 6 or 5 cards (like you would in a real tournament) but instead start with a good 7 carder every time, your playtest results will be skewed.

Go first alternately

In a tournament the losing player decides who goes first. But that doesn't mean that you should adhere to the same rule in playtesting. Because you always start off with a die roll in tournaments, you should expect to go first as often as not. That means that in order to mimic tournament conditions, you should go first 50% of the time as well. It's good practice to write down on the life score sheet who played first, otherwise you'll inevitably forget it.

Furthermore, you should try to play an even number of games, because then both sides go first 50% of the time.

Let someone else play your deck now and then

You can switch sides or have other players test the same matchup. If you don't do this, you will get results that favor a better player rather than a better deck.

Use clear proxies

It's common practice to play with fake, or proxy, cards if you don't own the real ones. Nothing wrong with that. I usually use a land or useless common and write something on the back of the card with a permanent marker. I use the back of the card because it is easier to read something written on there; no pictures or rules text to distract you. For cards that everyone knows (for example, Kodama's Reach), you can just write down the card name. No ambiguity. But for more unknown cards (for example, cards out of a new set that was released 2 days ago) you'd better also write down things like mana cost, power/toughness and rules text. Otherwise you might end up forgetting what the card exactly does.

If you end up having to put lots of proxies in your deck, you can try making proxies more recognizable. You can write down an L in the top left and right corner of a land proxy; this way you can quickly discern between lands and non-land cards. Or you can try to find proxy cards with the same mana cost as the card they're representing, for easy reminders.

Anyway, don't use normal Magic cards as a proxy card without any indication that it is a proxy. If you don't want to write on Magic cards, then put a piece of paper in your sleeve or, well, do something! I remember a situation in which I had tested a lot of games with a Gifts deck that a friend lent to me. I was surprised at how many copies of Meloku I was drawing, while I did never draw a Gifts Ungiven. Afterwards, my friend told me, “Oh yeah, I forgot to tell you! Those Melokus were proxies for Gifts!” Right… Nice. Proxy.

Keep track of results

If you playtest to see how a certain deck fares against another, you should write down the results. If you don't, you will probably forget. And you should keep all results in one place, or you should email all results to members of your playtest team. I use the following format to keep track of playtest results:

I keep track of life totals, who plays first (designated by an “S” for start), what matchup this is about and the game score. All on one piece of paper. If I want to, I can also write down comments on what cards are usually game-breaking and/or why a deck won or lost. I use this format in tournaments as well and I find it very efficient.



Using the results to calculate a match win percentage and to choose the best deck

There is a huge difference between a match win percentage and a game win percentage. Let's say that you playtest deck A versus deck B. Deck A wins this session 10-6 and for simplicity, let's assume that no deck wants to sideboards anything. Simple mathematics allows you to conclude that deck A should win 62.5% of the time. True, but that is the probability that deck A will win one game, not an entire match. If you'd calculate the match win percentage then you'd end up with 68.4% of the time. That's an important difference. The impact of the post-sideboarded stats is even more important, because you will play more post-sideboarded games than pre-sideboarded.

The match win percentage is calculated as follows:

P(Match win vs deck X)=P(pre-sb) * P(post-sb) + P(pre-sb) * {1-P(post-sb)} * P(pre-sb) + {1-P(pre-sb) * P(post-sb) * P(post-sb)

In which P(pre-sb) stands for the probability of winning a game before boarding and P(post-sb) stands for the probability of winning a game after boarding.