The launch vehicles themselves may not be that fragile, but payloads often are. Multimillion-dollar satellites can be damaged or completely non-functional from effects as small as being dropped 1 meter (see http://www.spacetoday.net/Summary/2230 , for example).
If the bolts holding the payload in place shear away under the lateral force of supporting the payload, the payload may fall and hit the hard inside of the launch vehicle, making the two events quite similar. (Indeed, the original drop I linked to occurred when the satellite was not correctly bolted to a table on which is was going to be tipped to one side in order to verify that it could withstand the stress of being in that physical orientation.)
That said, ugh's comment suggests my reasoning (as an explanation for vertical assembly) is either incorrect, incomplete, or both.
Ariane rockets are also assembled vertically [1]. Since assembly and adding the payload are two discrete steps, done in two different buildings [2], both of the times vertically, that tells me that there is more to it than the payload.
The main reason is merely tradition (really). Most systems are designed as evolutions of older systems, if the first system used vertical assembly, then so will the latest.
Note that the Russians have been doing horizontal to vertical launch vehicle assembly for decades (with manned and unmanned launches). There's no engineering reason why you must prefer one method or the other, it's a choice.
This does indeed suggest that there may be other factors at work. I could try tossing out other ideas, but they would be little more than hypothesis. (Bending during the tipping process damaging the joints in multi-segment launch vehicles? This might explain the difference with SpaceX's Falcons, which I believe do not have joints with O-rings.)
