Yes, absolutely. The total sideways velocity required is fixed, going on a steeper trajectory requires more fuel for the same target orbit.
The reason Falcon 9 still does that sometimes is that a more vertical trajectory makes the return flight back to the launchpad easier. So if the payload is light enough that there is enough free delta-v, they want to do that. If there isn't, they land on the barge or fly expendable.
It is almost, but not quite, entirely about horizontal velocity. You also need sufficient altitude to be outside the atmosphere, as orbits with altitudes of under 100km are very short lived.
Additionally, it's generally not great to try to spend a lot of time accelerating through the atmosphere. For this reason rockets tend to fly upwards and then once they are out of the bulk of the atmosphere they fly mostly sideways.
Did the rocket just turn sideways to accumulate the necessary horizontal velocity to actually maintain an orbit at a much later phase in its flight?