Science of Chess - Should we study GMs to understand chess and the mind/brain?

My goal for the next couple articles in this series about the Science of Chess is to focus on some specific research papers and talk through their methods, their results, and related ideas regarding how to examine the cognitive processes that support playing chess. Before moving forward with a specific paper or two, though, I think there is an interesting assumption that a lot of researchers (not all!) make when choosing how to study chess from a cognitive science perspective. I'd like to talk a little bit about that here as a little bit of food for thought as I start working through some interesting results about chess and the mind/brain.

The issue I want to think through with you here came up when I was talking to a colleague the other day about the Chessable Research Awards. We were exchanging some ideas about neat directions for her student's research and discussing some features of online chess platforms we might be able to use to help us develop and test good research questions about learning. At some point in the conversation, I mentioned something that I've been thinking about a lot as I've read more of the literature about the cognitive science of chess: There is a heavy bias in the field to study GMs and other titled players rather than the much larger population of players who aren't capable of awe-inspiring high-level play. "YES," she agreed, "We've noticed that. Why don't people want to study ordinary players?" Her student chimed in, too, "It's almost all about extreme expertise. It feels like there's a lot of room to study chess using more typical players." On one hand, this is great for the research community insofar as it means there are probably many interesting open questions to ask about learning and performance at a wider range of ability levels. On the other hand, what's going on with this bias, anyway?

Are the best players really the best people to investigate so we can understand chess in the mind and the brain? Photo by Jay Bhadreshwara on Unsplash

What about the Intermediate player?

Like a lot of adult improvers, I've run into a version of this question as I've tried to identify good resources to get better at the game. In trying to develop an opening repertoire that I liked, for example, it's all too easy to fall into the trap of rejecting a variation because it's "known" that it leads to an edge for the other player. That edge might matter a ton if you're staring down Fabiano Caruana or another player with deep knowledge of openings, but it also probably means you can have a lot of fun and good results at the club level. Likewise, there are study strategies and advice for structuring one's learning and practice that might indeed be great choices for a titled player, but have to be adapted for players who are looking instead to make a run at an OTB rating of 1600 ELO. By way of example, I bought a copy of Reuben Fine's (excellent) book about chess openings in the first weeks of my return to chess a few years ago, and while there are many things I love about it, there are also bits of advice and asides in the text that are...well...not really going to be relevant to me unless there's a miracle. Thankfully, there are a number of great places these days to get guidance about improving your game that are geared towards players who don't need a reminder about the basic rules, but also aren't heading for an NM or IM title anytime soon. As an example, I'm working my way through Ben Johnson's Perpetual Chess Improvement book and finding it to be highly practical and informative. I'm also a big fan of Jerry Silman's The Amateur's Mind, which does an excellent job of documenting how players at different levels make different mistakes in a range of positions.

The author (and odds are most of my readers!) is definitely in this blue shaded area rather than the one up above.

The scientific literature on chess and cognitive/neural processing isn't nearly so hospitable to the intermediate to advanced player, however. It's not a totally barren landscape, but if you start collecting studies to read more about, you'll quickly find that there is a clear focus on characterizing the chess expert's abilities. And why not? The goal of these papers isn't to make a broad audience of chess players feel welcome or have useful resources to take advantage of as they keep playing, after all. Instead, the goal is to try and understand the mechanisms that support playing chess, including both skill acquisition and actively working through a game or a puzzle. If the question is "What do the mind and the brain do to support playing chess?" surely the answer will come from studying individuals who do it best, right?

What I want to offer in this essay is not so much a counter-argument, but more like a cautionary note from one of my own areas of expertise: Face recognition. This might seem like a strange way to comment on this issue of who to recruit for chess studies, but there are more connections between playing chess and recognizing faces than you might think! Indeed, besides the points I'm going to raise in a moment, there are even a few studies comparing players' visual processing of chess positions to their visual processing of face images. It's not these possible mechanistic similarities between these two visual domains that I want to expand upon, though. Instead, it's the way that expertise functions as a key point of inquiry in both domains.

How do you recognize faces?

In general, people are very good at recognizing familiar faces. By itself, that's interesting if you're a cognitive or vision scientist because it's also objectively a tough problem. Faces tend to look a lot like one another insofar as there is a canonical structure to human heads and faces that constrains where facial features appear and how they tend to be shaped. Then again, any one person's face can also change appearance dramatically from moment to moment depending on facial expression, lighting conditions, viewpoint, and all sorts of other factors that change the contents of an image of a face without changing the identity of the individual it belongs to. Besides our impressive abilities as familiar face recognizers, the other thing that's interesting to researchers like myself about face recognition is that it seems like it depends on unique processes that are distinct from the tools you use to recognize other objects in your environment like animals, food, furniture, or most anything else you can name. If you like using the brain to motivate assertions like that, I've got some brains for you: There is evidence for an extended face network in brain made up of multiple neural loci that exhibit face-sensitivity and/or face-selectivity for different aspects of facial appearance. To put that more plainly, there are parts of your brain that appear to be most active when you are looking at images with different types of face-like appearance, and don't seem to be nearly as active when you are looking at other things. The most well-known of these sites is the Fusiform Face Area, which is located in extrastriate visual cortex at the occipito-temporal junction - the place where your occipital lobe meets the temporal lobe.

The fusiform face area (highlighted in red) is one part of a larger network of cortical regions that primarily contribute to face recognition. Polygon data were generated by Database Center for Life Science(DBCLS)[2]., CC BY-SA 2.1 JP <https://creativecommons.org/licenses/by-sa/2.1/jp/deed.en>, via Wikimedia Commons

You don't necessarily need to rely on the brain to support the claim that face recognition stands apart from other kinds of recognition, however. There are also some neat pieces of evidence that help us defend this assertion arising solely from comparing the manner in which you process and recognize face images to the manner in which you process other objects. I'd like to introduce you to just two of those here, both of which have been used to try and make the case that face recognition is worth talking about separately from object recognition more generally.

The first of these is the face inversion effect, which refers to the disproportionate difficulty most people have recognizing upside-down faces compared to upright faces. This may not surprise you much: Why should you expect to be any good at recognizing upside-down faces? Fair point, I suppose, but the key result here isn't just that you're worse at it, but that you are gloriously bad at it in some ways and not nearly so impacted by inversion if you are trying to recognize other objects. To show off briefly just how awful your inverted face perception can be, I will use a classic visual demonstration that you may have seen before: The Thatcherized Face. Briefly, to "Thatcherize" a face, one takes the eyes and the mouth and turns these upside-down while leaving the rest of the face untouched and unrotated. The result is a grotesque picture that many people find uncomfortable to look at. Turn that face upside-down, however, and that profound ugliness is much reduced. Take a look at the images below and my guess is that the two upside-down images look a little different, and you can probably see what's going on in them now that I've told you. The two upright images, however, likely look strikingly different to you. This difference between your sensitivity to this manipulation in upright and inverted faces is just one way to demonstrate how sensitive your face recognition is to orientation and how this is less the case with other objects.

The differential impact of "Thatcherizing" this face (which is in fact, Margaret Thatcher's) in the top row compared to the bottom row is a quick way to see how inefficient are upside-down face processing abilities are. Photography: Rob BogaertsImage manipulation: Phonebox, CC0, via Wikimedia Commons

Another way in which face recognition appears to differ from other kinds of object recognition is that faces appear to be recognized "holistically." What we mean by this is that you seem to process images of faces as one large pattern and are not so great at singling out smaller scale features like the eyes, nose, and mouth without the context of the whole face. One of the more robust ways to demonstrate this in the lab is via something called the Composite Face Effect, which is an experimental paradigm where researchers typically combine the top-half of one face with the bottom-half of another and measure how the appearance of one half of the face affects how the other half is perceived. That is, if I tell you to only worry about how the top half of a face looks, does it matter what I do to the bottom half? In the lab, we look for discrimination or recognition deficits (wrong answers) or inefficiencies (slow response times) to demonstrate that there is an influence between the two halves of the face suggestive of holistic processing. Here, I've got another quick demo for you to show off this effect, and it's one you can even try out at home. Full disclosure: I first saw this in a wonderful TikTok video, so the idea is not mine! The trick is this, though: Print out a life-size portrait or photograph of a face and cut it in half. Hold the top half of the image over your own face so that your head and the image are well-aligned. Next, start making different expressions with your mouth - this will likely give anyone watching you the vivid impression that the eyes of the portrait are also moving, even though they are absolutely not! The motion at the bottom of the image appears to be combined with a holistic face representation and influences what you experience of the top half as well.

The composite face effect highlights the holistic nature of face processing. Is this part of what makes face recogntition not just unique, but also effective? Photo by the author.

What makes you so good at face recognition? Let's ask the experts!

With these effects in mind, myself and other researchers remain interested in the relationship between our overall face recognition capabilities and these specialized mechanisms for recognizing faces. One line of research that a number of people (including my friend and colleague Dr. Richard Russell) have pursued over the last 15 years concerns individuals known as "Super-Recognizers," As you probably guessed, these are people with remarkably good face recognition abilities (like 98th percentile and above) as assessed by standardized instruments like the Cambridge Face Memory Test or the Glasgow Face Matching Test. If you want to understand the nature of face recognition, maybe these are the folks to work with! After all, they do demonstrate some remarkable anecdotal skills at remembering many people after only brief interactions and succeeding at recognition under bad ecological conditions like large age differences between one exposure to a face and the next. To understand how the things that make face recognition unique contribute to making our face recognition work well, this population of experts might be just the ticket - how sensitive are they to inversion? How susceptible are they to the composite face effect? If we ask them (politely) to climb into an fMRI, do the loci in their extended face network differ from more ordinary individuals' brains in consistent ways? All of these are questions that different labs have examined over the years and I won't drown you in references just for the sake of it. Suffice it to say that I hope it makes sense to you that these experts in face recognition seem like a good lens through which to try and examine face recognition in terms of the way the mind and the brain make it work.

But here is where things get complicated, friends, because these are not the only face recognition experts you could decide to work with.

Let me tell you about another kind of face expert that has also garnered some interest from the face recognition community. These are not untrained observers who just happen to be especially good at recognizing faces due to some lucky convergence of genetics and experience. Instead, these are individuals with extensive training that focuses specifically on making accurate recognition and discrimination judgments about faces: Forensic face examiners. Briefly, these are people who have worked for a long time to develop and refine face recognition skills for the purposes of forensic identification. In the laboratory, they also out-do a typical population sample at face recognition and discrimination tasks at a level comparable to the Super-Recognizers I described before. Neat, right? The catch is that though these two groups of extreme experts both excel at face recognition, there is a high likelihood that they do it in very different ways. In particular, a forensic face examiner's training and skills depend on spending lots of time (hours, in fact) examining a face, not just looking at it, to draw a conclusion. That examination also includes digital and analog tools that are used to measure specific things in the face that are used as the basis for a recognition decision. Successful forensic face identification can depend highly on the appearance of local, small-scale parts of the face in a way that differs from what it seems like typical individuals do.

This is very cool - this is neat to understand more deeply. This is also why I worry about focusing so strongly on GMs in chess research.

As you hear these descriptions of face recognition experts, I think it's worth reflecting on who the chess GM sounds a little more like. Is a GM someone with a natural aptitude that was supported by experience and training like face super-recognizers? On the other hand, is it more fair to say that they are more like someone with intense and specific training in using particular tools (in this case, perceptual or cognitive ones) to reach high levels of success given sufficient time to use them? It doesn't have to be just one of these or the other, of course, but there is an issue here of what we want to learn about chess and the mind and brain and how to pursue that. For my part, I think that studying the neural and cognitive processes very high-level players use to achieve their results is interesting stuff, but I also wonder about the patzers like me. The answers we get from one group vs. the other could potentially be very different and making sure we don't just assume that a beginner, intermediate or advanced player is just a lower-accuracy titled player seems important.

I'll also add that in the face recognition literature there is also a long and complementary line of work examining face recognition abilities and mechanisms in individuals who have a specific deficit in face processing: Individuals with prosopagnosia or face-blindness. Understanding the nature of perception and cognition at the other end of the spectrum also helps us gain insight into the mechanisms supporting specific abilities, which might be another important perspective to bring to chess research. Are there players who struggle mightily with specific parts of the game? Does this help us understand how the human mind plays chess in a way that complements what we learn from studying people who exhibit extraordinary abilities? Overall, my feeling is that we'll learn the most about the mind and the brain from looking across the broad range of chess ability to see how chess works when it's going great, going alright and even when it isn't going well at all.

Look for more Science of Chess soon, most likely a look at some research using eye-tracking to investigate how players examine chess positions and what this may tell us about the way they interpret and understand them (though I may change my mind and do something else - stay tuned either way!)

Support Science of Chess posts!

Thanks for reading! If you're enjoying these Science of Chess posts and would like to send a small donation ($1-$5) my way, you can visit my Ko-fi page here: https://ko-fi.com/bjbalas - Never expected, but always appreciated!

References

Boggan, A. L., Bartlett, J. C., & Krawczyk, D. C. (2012). Chess masters show a hallmark of face processing with chess. Journal of experimental psychology. General, 141(1), 37–42.

Russell, R., Duchaine, B., & Nakayama, K. (2009). Super-recognizers: people with extraordinary face recognition ability. Psychonomic bulletin & review, 16(2), 252–257.

Phillips PJ, Yates AN, Hu Y, Hahn CA, Noyes E, Jackson K, Cavazos JG, Jeckeln G, Ranjan R, Sankaranarayanan S, Chen JC, Castillo CD, Chellappa R, White D, O'Toole AJ. Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms. Proc Natl Acad Sci U S A. 2018 Jun 12;115(24):6171-6176.