One thing I can suggest is if you’ve turned on some compiz features or tweaked some settings, reset compiz with the button provided in Mate Tweak and see if that helps.
I’ve been running compiz, I think since Ubuntu 8.04. Always nVidia hardware and proprietary drivers, too. It’s never ever been flawlessly stable.
I’ll bet something like a particular app or event like a screen saver starts it off. For example, with Mate 1.14 on UM 16.04 I see a definite interaction with the Software Updater notify screen and compiz: Strange Mode Starts With Software Updater
HTH