At first I did not see how reducing the board size could create a checkmate.
But now I see that it could. The checking piece, by vacating a rank or file, could cause the board size change, wich could then (by that size reduction) eliminate flight squares that otherwise would still have been available (despite the check) IF the board had not downsized.
It could also create a stalemate.
An 8 x 8 board could be reduced to 6 x 6, which would leave space remaining for 36 pieces -- so if this happened in the opening (very unlikely) it would still leave SOME room for all 32 pieces. In the middle game (still pretty unlikely) it would be even easier to accomodate.
In the endgame (where the size reduction would be most likely, it seems to me) the board size change MIGHT make for some interesting and faster endgames.
The more I think about the OP's idea, the more I think it is an interesting one. Nice, creative thought,
@Andreyka7 ! Congratulations!