The first step in making the mask is the same sort of thing that Freodin described (make a mask and blur it), but it's the adding noise step plus blur and then scaling and/or thresholding of the layer mask that makes it look less like a broad, smooth transition.

The first image I showed is literally two solid colors, yellow on top and green below, blending through a noisy gradient mask (the mottled green area is the effect of the mask modulating the yellow onto the green). It's when the same mask is used to transition between the yellowish and greenish noise textures below that things look much better.