- Blind mode tutorial
lichess.org
Donate

Is there a way to export rating/rank for multiple players at once? (in bulk)

I am trying to research the effect of experience and breaks from play on performance [i.e., does spaced-training improve ELO more than massed training?]

I can see here that the user of this research did something similar; however, it seems like he went way beyond and generated his own offline database (the setup seems like a LOT of work and is beyond the scope of what I need): link: https://lichess.org/blog/YXgv5xMAAMxG4VeC/how-long-does-it-take-to-get-good

I am simply looking for lets say ~10K of player ELO ratings over time (or from game-to-game). I can see the API lets you export this per user but I would like to export it for multiple players ideally from when they start their time at Lichess.

There are two methods I envision:

  1. use the lichess API ["Get rating history of a user"--> outsputs max 1 rating value per day] and choose lets say randomly 10K players: i'm not sure if the API will let me run it though for 10K distinct users... this user appeared to experience issues with exporting rankings for 300 users

  2. download the massive lichess opendata base (https://database.lichess.org/) and filter and transform the data until you get the users you want [problem: the files are large up to 34 GB and because there's only one file per month, I will not capture performance changes larger than one month, unless I want to endeavor to compile all the hundreds of GB together]

I am trying to research the effect of experience and breaks from play on performance [i.e., does spaced-training improve ELO more than massed training?] I can see here that the user of this research did something similar; however, it seems like he went way beyond and generated his own offline database (the setup seems like a LOT of work and is beyond the scope of what I need): link: https://lichess.org/blog/YXgv5xMAAMxG4VeC/how-long-does-it-take-to-get-good I am simply looking for lets say ~10K of player ELO ratings over time (or from game-to-game). I can see the API lets you export this per user but I would like to export it for multiple players ideally from when they start their time at Lichess. There are two methods I envision: 1) use the lichess API ["Get rating history of a user"--> outsputs max 1 rating value per day] and choose lets say randomly 10K players: i'm not sure if the API will let me run it though for 10K distinct users... this user appeared to experience issues with exporting rankings for 300 users 2) download the massive lichess opendata base (https://database.lichess.org/) and filter and transform the data until you get the users you want [problem: the files are large up to 34 GB and because there's only one file per month, I will not capture performance changes larger than one month, unless I want to endeavor to compile all the hundreds of GB together]

You don't know if these players also play on other accounts/websites/OTB, how much they study chess when they're not playing etc.

You don't know if these players also play on other accounts/websites/OTB, how much they study chess when they're not playing etc.

@ShiningDrongo said in #2:

You don't know if these players also play on other accounts/websites/OTB, how much they study chess when they're not playing etc.

Good points, it might help op state the assumptions of the research more explicitely. There might be some portion of lower activity that might be accounted by such missing information. Perhaps, filtering for users that have periods of both types of behavior (sparse or frequent or whatever measure of training intensity and quantity), might reduced that effect. (the more fluctuations the more likely it would be not about web site waltzing).

perhaps lichess being overwhelmingly good might be an argument to neglect the cases mentioned.....

Also, I don't see why the op question should be restricted in rating range. The data can come from anywhere.. the more range in input the more information, about the "function or trend" being sought or assessed for existence. I may miss something by saying that of course.

@ShiningDrongo said in #2: > You don't know if these players also play on other accounts/websites/OTB, how much they study chess when they're not playing etc. Good points, it might help op state the assumptions of the research more explicitely. There might be some portion of lower activity that might be accounted by such missing information. Perhaps, filtering for users that have periods of both types of behavior (sparse or frequent or whatever measure of training intensity and quantity), might reduced that effect. (the more fluctuations the more likely it would be not about web site waltzing). perhaps lichess being overwhelmingly good might be an argument to neglect the cases mentioned..... Also, I don't see why the op question should be restricted in rating range. The data can come from anywhere.. the more range in input the more information, about the "function or trend" being sought or assessed for existence. I may miss something by saying that of course.

Thank you all for the feedback.

@ShiningDrongo well put. yes this is a confound but I think the effect will shine through regardless.

@Toscani thank you, this is useful but as @dboing pointed out, I don't think this question is isolated to particular ranges. if anything, I would suspect expert players with ratings >2K would be more likely than others to practice outside of Lichess, and thereby confound the dataset even more.

I discovered that Lichess has an API from which I can export game history. If I had a list of a 500-1000 players who have played each 100s of games then I could use the API and export their game history (which includes rating changes).

Does anyone know where I can find such a list of players (of ALL rating levels) each who have played minimum ~500 games?

Thanks again!!!

Thank you all for the feedback. @ShiningDrongo well put. yes this is a confound but I think the effect will shine through regardless. @Toscani thank you, this is useful but as @dboing pointed out, I don't think this question is isolated to particular ranges. if anything, I would suspect expert players with ratings >2K would be more likely than others to practice outside of Lichess, and thereby confound the dataset even more. I discovered that Lichess has an API from which I can export game history. If I had a list of a 500-1000 players who have played each 100s of games then I could use the API and export their game history (which includes rating changes). Does anyone know where I can find such a list of players (of ALL rating levels) each who have played minimum ~500 games? Thanks again!!!

This topic has been archived and can no longer be replied to.