It's reeeeaaallly stretchy, so it will stretch on the hems. If your double needling it, just try to have less pressure on the foot, whether you do it by hand or if your machine has adjustable tension.
Also, I find that I need to make shirts "skinnier" than my pattern (so, smaller in width, but not length) a bit because it drapes differently than interlock and jersey. It stretches out and things always seem too big. What I always do is just make the shirt and if it isn't fitted or small enough, I just serge an inch or so off each side until I get it right.
HTH~