Thanks to @nisima's suggestion (thanks!) I came across mini waffles by Marioinex and a much larger version called waffle blocks by Little Tikes.

The former comes closest to the #constructionToys I am looking for, but I still don't think it's the exact same one.
Most notable the shape of the centre hole does not match my memory of being in the shape of a plus-sign.

#MiniWaffle #WaffleBlocks #Marioinex #LittleTikes #MiniWaffleBlocks

@nisima ah, yes, the mini waffle blocks look a lot closer in size. :) And they are flat all around as well (no raised edges).
That only leaves the difference in the shape of the center hole (four squares for the mini waffle blocks, and plus-shaped for mine).
It does look like they would be compatible with each other though.

(Of course it is also possible that my own memory is failing me, and that the #MiniWaffleBlocks are indeed it).