Probably because your connection is trash so when your opponent moves it delays for one second.
@AnUtterlyKarpov But I have all five bars.
The server also adjusts your times depending on server latency, so if they other play had poor connection they could have some time boosted
