Export games does not have clock information in PGN files

Anyway I realized that downloading with clocks was not possible with the app, so I added all the options listed in the API ( including clocks ):

lichess.org/api#operation/gamesExportIds



Before download you can select what should the PGNs include ( default is moves and tags ):

I added a retry logic. If a batch fails, it waits a minute and retries. You get a message about this, asking you to wait.

I managed to download 5000+ games that way, within reasonable time.

@sefulesefarka
Thank so much, appreciate your quick help. It worked - the retry logic worked.I owe you a treat.
I noticed something ( just an observation)
total games 5475 found 5474 searching 4/27/2018, 2:59:23 AM elapsed 237.186 games/sec 23.083
There was no filter that was applied but the Total-games were 5475 but found 5474 ( one less)? could this be a bug?
Question 2:
if I need to download all the games for my analysis- is this the right place to download?
database.lichess.org/
Question 3:
This is a dumb question but I am not aware of the apitokens etc, I happen to generate a personal token.
Now do I have to paste that in the link that you have shared? ( in the token section ?) before the search?
Thanks so much again.

1) There is always a rating filter. May be some weird rating escaped the 600 - 3500 range.

2) Yes, this is the right place, if you have bandwidth, then you can download the several GB files. For variants it is much easier. Those are much smaller files.

3) Yes generate a token ( general scope, no need to check any special scope ), then paste it into the token box ( of course clear the old one first ), I recommend that you create a link with Create link button and bookmark this link, so that next time your token will be remembered. Also if you perform a search with Create code and search, the link will be updated.

@sefulesefarka
Thank you very much!!!!!!

Nice work @sefulesefarka

We'll make it a part of Lichess, see github.com/ornicar/lila/issues/5904

Hello @sefulesefarka ,
Your app is really good :fbserv.herokuapp.com/
I am trying to run some analytics for my research on all the PGN files, for that - instead of querying by username , can I make it generic. for example : If I want to find out the users who are above ratings of 1800 , is there anyway I can query that information? or to achieve that the only way is to download the raw pgn files ( parse them and store them in database ) and run queries on them? or is there any smarter way?
Requirement 2: I want to group users based on the total time spent playing per week . is this possible using your app? or again the only way is to create my custom database?
Thanks in advance.

#17

I don't think this is possible in general using the official API ( lichess.org/api ).

You can query leaderboards, which means 200 high rated players per leaderboard. You can query titled players ( I have a page for this ):

fbserv.herokuapp.com/titled?titles=GM

You can query tourneys and tourney games. Followers of a certain player, users followed by a certain player. Things like that.

These queries are limited and specific, but I'm not aware of any general type of query. In the old API at least you could fetch a random game, I don't if this is possible any longer.

Thanks @sefulesefarka , Just for my understanding - Your app does it use the official apis ( provided by lichess) and then fetch the data? or do you copy the raw pgn files (database.lichess.org/) and then mine the database? ( do you have the code for your app in github or is it private?)

#19

My app uses the official API. I have neither intention nor capability to download standard database files. I did download all atomic games played on lichess and built a book of games where both players were rated above 2200. This is the book @AtomicChessBot plays from.

The code is public:

github.com/handywebprojects/fbserv