PicoBlog

Welcome to Say Chess! This newsletter goes out to 4,017 chess players. If you haven't joined yet, sign up now and get the ebook '100 Headachingly Hard Mate In Two Puzzles Composed By Sam Loyd' for free.

This week, I will share an interview with Marcus Buffett, a passionate chess enthusiast and innovative programmer. Marcus and his co-founder, Ollie Campbell, are in the process of developing a chess opening tool designed to streamline the learning process, potentially providing an alternative to Chessable's extensive variations.

More about this intriguing tool at chessbook.com. In the conversation, Marcus sheds light on his inspiration for Chessbook and the story behind its inception.

I have this pattern where I get into chess, start practicing, then I go off and create my own tools instead of getting better at chess. It's terrible for my chess improvement, but I get a kick out of it. This time, I had been taking opening prep very seriously, and had been going through some lifetime repertoires on Chessable. I've got 1.3 million points, which I think translates to something like 26,000 moves reviewed or something; I was really serious about it. Then one day I got frustrated by a line that I had gotten wrong again and again, and wondered if I could just skip it. What I found out was that it had actually never been played, across millions of OTB games and billions of online games. After that I just lost all interest in reviewing – I didn't trust that I wasn't just wasting my time on a line that would never be played on a chessboard.

So initially I just built a tool for myself. I'd load up a repertoire into this little command-line program, it would find the lines that were expected in less than 1% of games at my level, and then spit out a new repertoire with those removed. Fast-forward through a ton of late nights and weekends, and eventually I had a somewhat working site, which I posted on Reddit and got a hugely positive response. Since then, myself and Ollie, who found me from that Reddit post, have just been working on the site whenever we can.

The name was originally Chess Madra, which came from a fantasy series called Cradle that I had been tearing through. Essentially everyone's got a different type of magic, and in that world it's called madra. Ollie and I always disliked the name though, and while brainstorming we came up with Chessbook. The idea is that we want the site to be THE Chess Book, like the definitive place to go for any openings. We're not close to that yet, but that was sort of the vision. 

How does Chessbook’s approach to chess opening training differ from Chessable?

Our north star is you should only spend time on the positions that you're actually going to see in your games. Chessable sorta pretends to offer this, with quick-start chapters and the option to limit the depths of lines, but these are just the easy approaches. Quickstart chapters are too minimal to be a useful repertoire, and limiting reviews by depth means that you'll be learning 10 moves in the mainlines that you'll see all the time (good), then also 10 moves of the sidelines that you'll virtually never see (a waste of your time).

The way we've tackled this is by looking at stats from a *lot* of online games (actually every non-bullet Lichess game ever played), and using that to determine what moves you'll see. After someone adds their repertoire, we can say:

"Your biggest gap is in this line of the Ruy Lopez. At your level people play d6 20% of the time, and so you'll see this move in 5% of your games as white, here are the moves you could respond with".

We're also focused on moves, not lines, which seems like a trivial distinction, but treating repertoires as a collection of lines is actually very limiting; it doesn't work for transpositions, and if you want to disable a move you have to disable the line, etc.

What science-backed methods have you incorporated into Chessbook to assist chess players in improving their skills? 

Reviewing on Chessbook is based on spaced repetition, which is a method of review based on the forgetting curve, the idea is that after you learn something, you'll forget it in say 1 day, then if you review it in time then you'll remember it for 2 days, then 6 days, etc, and that the most effective time to review is just before you forget. This is what any good memorization software is based on - Duolingo, Anki, Supermemo, Chessable, etc. The implementations vary a bit, but they're pretty similar in the end. I think our system strikes a good balance between making sure you don't forget, but not reviewing too often. It also helps that our system is based on moves, not lines. So you're not going to be asked to re-review whole lines when you just get one move wrong.

One interesting finding from studies on practice is that the closer you can make the practice environment to the real thing, the better you'll recall. So we've added chess.com and Lichess themes, to try to get as close to the online games you play as we can.

Another factor with spaced repetition for chess openings is how you learn to recognize positions. If you recognize positions by the moves that lead up to it, then you're going to be worse at identifying transpositions in your games. So if there are many ways to arrive at a position, Chessbook will mix up the order across reviews. It makes it a bit more difficult, but means that you'll identify positions by the actual position, not by the exact series of moves that led to it.

Who are your target users and how have their needs shaped the development of Chessbook?

Our initial tagline was something like "the opening builder for the online chess player", and that's definitely the audience we're interested in. There's a huge majority of chess players that aren't 2000+ FIDE and playing serious tournaments, but are still very interested in improving their game. The openings that a 1200, a 1600, and a 2000 ELO player will encounter are hugely different – you can't make a single repertoire that fits all these players. So we're focused on making a site for anyone from 800 to 2400 Lichess.

Part of this is connecting to the sites that people play on. For example, if you play on Lichess, you can import your games and we'll generate a repertoire based on what you play in your games. Then also we'll monitor your online games and when you come back to Chessbook you can see where you've gone wrong in your recent games.

The other part is that people play chess on the go, and will often want to add a move after playing a game; if your repertoire is on Chessbase then you'll just need to wait until you're back on your laptop, or if you've made your own course on Chessable then good luck on adding a new line from your phone. We've put a lot of effort into making the site work just as well on mobile as it does on a laptop.

Can you share some success stories of players who have improved using Chessbook? 

For sure! We get people mentioning games all the time in the #chess channel on our Discord, attributing their wins to out-prepping their opponents. My favorite is a member of our discord who set his son up with an account on Chessbook to learn openings, who keeps me updated on his son's tournaments. I love to hear that he’s getting winning positions out of the opening, and recently he placed third in a tournament and won a bike, which was awesome to hear.

We don’t have any success stories yet like “I studied on Chessbook and gained 200 elo in a month”, and probably won’t have much of those, which is just to be frank about the value of openings. I think someone who has totally neglected openings could gain something like 100 or 200 points by getting a really solid repertoire, but those would be the edge cases. So for me, a success story would be more about someone spending just enough time on Chessbook to build a solid repertoire and remember it, thereby freeing themselves up to focus on other areas - tactics, endgame study, strategy, visualization, etc.

What features are you currently working on adding? Do you think ai will be implemented at some point? 

We just recently released an iOS app, and so now we're building an Android app to get the trifecta (web/iOS/Android). There are a few smaller features on the shortlist; notifications so you know when moves are due to be reviewed, finding bad moves in your repertoire, and then there's always general UI polish. 

The bigger feature we're looking to add is around publishing repertoires, which someone could then add into their own repertoire. We'll start with pre-built repertoires based on the games of a handful of well-known players, but would love for anyone to be able to publish their openings soon after.

It's funny that you mention AI, I was actually just playing with some machine-learning models that would help in building a cohesive repertoire. The idea is that you train a model on an existing repertoire, so it can then identify moves that will fit in with that repertoire, resulting in a repertoire that's easier to learn and remember. We've got it working, but scaling this to all our users is a challenge, so it's on the back-burner for now. We've also tried to use large language models like ChatGPT to write annotations for moves, but the quality just isn't high enough. It will write something that sounds totally reasonable, like "Protecting the pawn on c5, and pinning the knight", but then when you actually look at the board you'll see that the pawn is on c4, and the knight hasn't been developed yet, stuff like that. I wouldn't be surprised if this works in the near future, or maybe it even works currently, and I just don't know the magic words.

What are your short-term and long-term visions for Chessbook?

Short-term, we're looking to support multiple repertoires, and publishing courses so other people can import them into their own repertoires. I think this will make it way easier for people to hit the ground running with building their repertoire. We're moonlighting this project, and this is a big feature, so I imagine that will keep us occupied for a while.

Long-term, there's too much to list, but we want to lean into the original idea of being like the definitive source of information on any opening. We currently have annotations for about 3,000 moves, which can be very helpful especially for lower-rated players, but this is just scratching the surface. Ideally, we'd want annotations for virtually every move that someone could add to their repertoire, and for the big decision points, like whether you're going to play an Accelerated Dragon or the Najdorf, it would be great to have a longer-form explanation with everything you might need to know to make that decision. I think there's a lot to be done around guiding people into the middle-game more effectively too. We already do more than most courses, but there's so much more we could do; tactics based on your openings, reviews of your middle-game plans, descriptions of the plans for each side, what the pawn structure means for your position, what pieces you should look to trade off, instructive videos that feature that opening, etc.

There's also the idea of expanding the target audience to professional players. I think Chessbook is already a great option for the online chess player, and lower-rated tournament players. But when you start talking about serious, high-rated tournament players, their needs are hugely different. They want to see exactly who has played a line, they want to prepare for a game based on what their opponent plays, they want to keep a record of all their games they've played in a given opening, etc etc. It's almost a totally different product, but it would be great if we could be the site/app for openings, even for the IMs and GMs.

You mentioned the utilization of data from online games to enhance the Chessbook experience. How do you ensure the accuracy of this data, and do you have plans to incorporate over-the-board (OTB) game data as well?

The online games come directly from Lichess’ publicly available monthly game archives, which give you anything you’d need for a project like this: time control, game result, moves, player ratings, etc. There are two primary fields to look at, when trying to see what openings you’re likely to face. There’s rating, and time control. The openings that a 1500 plays are not significantly different from those a 1300 plays, and so we bucket the games by ELO bands of 200; 1100-1300, 1300-1500, 1500-1700, etc. 

As for time control, there’s only big differences between bullet and the rest. There doesn’t tend to be much difference between, say, Rapid and Blitz. So we filter out the bullet games. We didn’t do this initially, and we’d come across terrible moves that had a good win-rate, a common one is Bh6 when the opponent is about to fianchetto, and you hope that they’ve pre-moved it, in which case you win two pieces. Bullet tricks like this are interesting, but just pollute the data of people playing other time controls.

We actually do have an OTB database too. If you’re in a line that enough masters have played before, then we show what % of them pick each move. That data comes from a database of all about 10 million games between high-rated players, and we further filter that down to only games between players rated 2400 and above. These databases come with a lot more issues than the Lichess database. There are invalid moves, slightly incorrect notation, that sort of thing. This has caused us a lot of trouble, and cleaning up all this data was a project in its own right.

Could you tell a bit more about the number of users you have currently and how it has developed since you launched. Any advice to other chess players who have an idea to build a chess tool?

Ever since we originally shared the project on Reddit over a year ago, there were a small number of people using it, and we kind of stuck with those <100 users for a while. It’s great to have some users, even if it’s not many, so you can see how they’re using the site, ask them what they’d like to see, etc. It’s also great for motivation to not just be building things for yourself. So I guess that’s one piece of advice, to build in public even if it’s just for 10 people. 

About 6 months ago we did a revamp of the site. We added onboarding, polished the UI, rebranded to Chessbook, added a homepage, all that miscellaneous stuff. It started to look like a real site, not just like a hobby project. With that and a bit of marketing, we’re seeing a lot more people trying out the site, about 30% more users every month. We’re still not quitting our day jobs anytime soon, but it also doesn’t seem totally out of reach.

As far as advice on building chess tools goes, I’ve got a few bits. The number one resource is Lichess; games databases, piece sets, stockfish integration, puzzles, position evals, javascript chessboard, and so on. It’s all really high-quality, and I have huge respect for Lichess for being so open with all of these things. One of the biggest highlights of building this site was when the developer of Lichess told me “your site looks great btw”.  Of course, make sure to donate if you end up using their stuff! There’s a lot more technical bits of advice I could give, but I’m not sure if anyone will care, so I’ll be quick. Make sure to handle transpositions well from the beginning, because that’s been the biggest source of bugs. Store EPDs, not FENs. Use a language with good chess libraries. Web is going to be much easier to develop on than native.

As far as getting some people to try out what you make, the chess subreddit is excellent. As soon as you start charging for anything they’ll lose all interest, but otherwise, you’ll get a lot of enthusiastic people checking your site out and giving feedback, which is invaluable.

Finally, has your evaluation of the importance of opening study changed through building Chessbook? Or have you gotten other interesting insights about the way chess players study the opening from user data?

I’m not sure my opinion on the importance of opening study has changed much. It’s great to have a solid opening repertoire – you avoid some traps, don’t waste your clock in the opening, hopefully, get a slightly better position – but whereas something like tactics training is an endless pursuit, I think openings can be “done”. For example, someone that’s rated 1300 on Lichess could make a manageable repertoire of say 100 moves for each side, then for the most part not worry about openings for a while, besides keeping up with reviews so they don’t forget their lines.

I hope you enjoyed the interview and maybe discovered a new tool. Next week I will write about tilt and share the first tests of a tilt-o-meter that I have built.
The idea is to measure chess players' tilt levels, so you can monitor when it might be a good idea to take a break.

Share

/Martin

ncG1vNJzZmirka6wqbHSrGWsrZKowaKvymeaqKVfpXyiuoyipa2doqu2psOMsKCtoF2irrOv1Kxkm62Wm7K1wA%3D%3D