The best way to gauge if a shirt is the perfect fit is through collar size. When buttoned up, you should be able to slip your forefinger between the collar of the shirt and your neck without any difficulty. Any looser and your tie will tend to ruck the collar up.
If your shirt does not fit perfectly then it will wear out faster. Why? Well, if the sleeves are too short, then the sleeve wears at the elbow. If, however, the sleeves are too long then the wear is noticed at the cuffs.