I'm not an automation QA but my understanding is that the wait is added in the same class that instantiates the driver ie it is in place and active for all tests rather than being selectively added on a test by test basis.
TLDR: For all elements
If you are using your explicit waits in your framework in such a way that it gets applied behind the scenes of elements without much additional effort, there is no reason not to use it with everything. If done appropriately, performance impact will not be significantly noticeable in most cases.
If your architecture isn't designed this way however, I don't find much advantage in putting it everywhere. Most of your elements will likely be loaded before selenium recognizes the page as being ready to interact with. An additional explicit wait in these cases doesn't do any good. To be clear, Selenium already waits for the Document.readyState to be complete. It's only the case where elements load afterward where explicit waits need to be utilized.
I prefer an architecture that waits without you thinking about it, or only putting them in as you say, when the test fails.
One additional thing that can help, if you setup a way for the
PageObject to provide a method to Validate the page is loaded on instantiation, you can provide one or two elements to explicitly wait for, which you know are loaded last, that will eliminate the need to wait in other methods for elements.