Revert "[1.7.x] Fixed #23938 -- Added migration support for m2m to concrete fields and vice versa"

This reverts commit 1702bc52.

This doesn't work on stable/1.7.x because #23844 wasn't backported and we're
not willing to do so because it's a large change.