Like so many other Magic players, I’ve been thinking about why the whole game is so messed up right now and what could be done about it. That lead me into thinking about all the analytics one could get from Arena.
We know they follow certain metrics. They reference win rates in their B&R announcements, so they follow those. They also have the smiley-face feedback after some matches, so I assume someone looks at those as well (although, it wouldn’t be the first time someone implemented a feature no-one ever actually used) and they must keep their eye on the amount of money being spent. However, they could go so much deeper.
Now, I don’t actually know what they are doing, but I have some experience in this and people tend to look at things that are way too easy to find. They have their KPIs (key performance indicators) and they don’t care about anything outside those. And yes, the amount of money being made is probably the most important metric (at least in the short term, which they are certainly emphasizing right now).
Here’s some ideas on what to look at:
How often do people scoop way too early? I haven’t seen this that much lately, but admittedly, I’ve done it myself. There was a time when the opponent would immediately concede when they saw the first turn [scryfall]Cauldron Familiar[/scryfall], because they knew how miserable those games were against the decks that used it. Now, I wasn’t even actually playing that deck, but that didn’t matter, because they didn’t even see what would have been my next play. If this keeps happening a lot, there is something even worse going on than your usual bannable card. If players don’t want to play against certain cards at all, perhaps you shouldn’t print something like that.
How many cards get played in a match? If you have ever done the quests and followed how long it takes, you know that often you cast only around four spells in a game. Is this enough? This does mean that often you need the kill in your opening hand. I don’t think this is good for the game, so perhaps someone should be following this stat.
There’s also the other quest goals. Sometimes being able to kill enough of your opponents creatures is just incredibly difficult, because you might just get a bunch of opponents, who don’t play creatures or play very few of them. This is actually one of the things that makes me think that they don’t follow metrics very much, because they haven’t fixed this yet.
How often are the games won by the player on play? I feel this is way too common at the moment. I don’t have any real stats on this, but it does feel like being on the play does give a disproportionate advantage. It is often just so easy to keep your opponent off-balance and roll them over. With the continual power creep, this is getting much worse.
These might not be as important as the obvious things, but they could still be good to look for the health of the game. I bet you could find a multitude of weak signals as well, if someone was trying to find them.
Of course, the problem with many of these is that you don’t necessarily know what is good. What’s a good number of spells being played per match? I don’t know, but I do believe it’s more than currently. It’s easy to look at how much money is being made, because more is always better, but the things I’ve talked about here more difficult than that.
Or take a compeltely different example, if you go to EDHREC and see that many of the cards used are from the most recent sets, is that a good or a bad thing? If they are being played, Wizards is making money, so it’s good in that sense. However, is that good for the health of the format? I don’t think so. I know many former commander players who got fed up with being pushed these cards you need to play instead of being able to dig deep. It feels just so patronizing. Sure, you can try to approach it in another way, but you would also have to convince your playgroup of that.
Anyhow, again, I don’t really know what they are doing, but from what I see, they seem to be kind of clumsy about how they gather feedback, so I wouldn’t be surprised if they weren’t looking hard enough at all the things they have at their disposal.