As per your design the "back" button looks weird and out of place being there, so I can see why somebody has thought to question it. However, I still believe if you are going to allow a user to navigate back, then there should be a clear way to do so.
I assume there is some sort of "Next"/"Continue" button on your form? (maybe at the bottom of each page). I would suggest that you work the "Back" button into that design. Have them next to each other, or at least on the same vertical alignment - "back" on the left side, "next" on the right.
With regards to highlighting the step markers, it won't hurt to make it more obvious that they are clickable, but I don't think it's overly important as an average user is just going to use "back"/"next" for navigation.
In addition, if the argument for not having a back button is that the user can click on the steps, then present the counter argument for not having a "next" button for the same reason. That will hopefully highlight how silly it is to suggest a dedicated "back" button isn't required.
In short, keep the "back" button but move it inline with the "next" button.