I somewhat disagree about the irrelevance of visual design. A good or exceptional visual designer can quite often have an enormous impact on the user experience, and also save the development team a lot of work.
I don't think it's shallow, it's actually taken for granted. Look at how vertical scrollbars work on both mobile and desktop, for example.
We don't give it a second thought, but imagine how many layouts would break on mobile, with a default 20px scrolling rail on mobile browsers. Or the way the URL bar hides when you start scrolling.
Those kind of ideas, from my experience, rarely come from product developers without some visual design experience.
> you're being told how something should look by a designer, [...] Being ordered around by a gasp artistic person
Neither are very relevant. UI interface design is about supporting workflow. Visual appearance is not unimportant but it's shallow.
> about having real, non technical people interact with your code
yeah, that's how you do it! Assuming your company actually does UI testing at all.