I think some of the problem is that people who use pandas don't necessarily know how to drop one level in programming. Like, if the file isn't a nicely formatted csv, they don't know how to read and parse the file directly. If they can't use basic filtering or a boolean mask easily with pandas, they don't know how to use lists, loops, and conditionals directly. It's great to use pandas rather than reinventing the wheel, pandas is an excellent library, but if you're going to deal with data at a very intricate level, you do need to know how and when to punt and just write the code yourself.

I think this comes up particularly in the context of pandas, because it's a common entry point into a programming language for people who don't think of themselves as programmers and may resist the notion that this is actually what they're doing.