Strange looking squares appears in the title bar when make window narrow. Making windows wider remove it. Happens in Pluma 
Long shot idea but you could try the marco compositor in Mate Tweak window settings.
Window manger is set to Marco(Adaptive compositor) and this still happens
On which Ubuntu MATE version is that and do you use any graphic cards like nVidia, AMD or hybrid? If so, maybe a driver is causing the issue...
It happens on all Ambiant-Mate version so i think is not theme specific. This happens only on this specific machine when I switch to another computer with the same theme and settings the black squares are gone