Hi y_cho,
I understand your confusion about the difference between observed behaviour and expected behaviour but I can assure you it is not a bug.
It is designed this way, and with a good reason.
Let me explain:
The standard to move a window in Ubuntu-MATE is Alt + LeftMouseButton.
This is to make it possible to drag any window without the fear of activating something within that window.
Every other way is undefined behaviour and up to the application.
This (actually very old) design is now more than ever a necessity because the introduction of "client side decorations" where sometimes (especially in internet browsers) there is no titlebar but also no room left (think of 30 tabs open) to grab the window anywhere.
With alt+LeftMouseButton this problem does not exist.
So you see that caja completely adheres to the standard.
What the other programs add to the mix is often convenient but is undefined territory.
I assume you would consider it convenient to add a bare LeftMouseButton 'grab' to caja
The best thing you can do is a "feature request" for caja in de link below: