My son will be 5 at the end of the summer too and he knows all his shapes and colors, abcs and recognizes them, can count to 100 (needs minimal help), can write his name (he knows how to but won't do it lol) Doesn't know our address.. but we did just move. He was on his way to learning our old address before we moved. He can also read basic words (cat, dog, mom, dad, etc..)
In all honesty, if you put him in school, they will pick up on the areas where your son is not up to par with the other kids and they will help them. He will be able to get speech therapy to help with the stuttering, OT to help with the writing skills if they find a physical reason for not writing. (My DS is in OT/PT for low upper body muscle tone and they help him with his writing because he cannot hold a pencil correctly)
Despite the skills my son has, he could probably stand to be held back this year instead of entering kindergarten as one of the youngest kids in the class. But really, if he falls behind or isn't liking it, I can simply pull him out and have him do kindergarten over again next year, so I'm not stressing it.
If you really want to work with him over the summer, get some Pre-K/K workbooks at the teachers store (or pretty much anywhere) and work on some things with him.
And follow your gut. If at the end of summer you don't feel he's ready, don't put him in kindergarten.