Short of very complicated maths, here is what I would do, and may very well help you:
You need to know the specifications of your motor - how much power do they consume for a given amount of thrust? You can find this out either by looking at the spec sheet, or I would just add masses in 25g increments and take a look at the black box recording.
Once you know this, you can know how much power it will take to lift the extra mass of the battery, you can use P=IV to figure out the extra energy stored in the battery itself and see if it is enough to counteract the extra mass.
This is obviously very rough - you would need a large sample size and to operate in lots of different conditions, but it would give you a good idea of if it is worth it before you even buy the larger battery.
Disclaimer: don’t overload your quad and risk burning out your gear - do this at your own risk.
I've heard that you want to keep your battery weight no heavier than 1/3 your drone weight.
If you're like me and don't want to calculate: stay under 50% of your drones whole weight.
I don’t think there are diminishing returns, if the propeller+motor+controller efficiency stays the same regardless of overall weight.
Imagine a drone where all the weight is in the battery and the motors have just enough power to hover the drone. Double the battery and you’ll double the capacity but you’ll also double the weight (and required power output of the motors). So your flight time will always stay the same.
However, a real drone’s weight isn’t in batteries alone, though ideally it would be. Everything except the batteries is just dead weight when it comes to energy storage. So the more batteries you can add the less effect the other components should have and your overall flight time should increase.
All of this of course assumes that your drone has the same efficiency when running at higher power/rpm.
This question did not tell us if you are investigating only one change, the battery capacity, to an existing commercial quad, or whether you are designing a new custom quad that is not yet built. Trial and error methods such as adding weight to the current quad may be easy as suggested by @Drones and Whatnot. And I would refine this test method by looking at the available battery packs that you might consider and simply test the flight characteristics with the additional weight that this alternative pack would represent. There are only a limited number of batteries that would be feasible, so there would only be a limited number of tests that you would need to try. If the drone is still in the design phase, I would enter the parameters into eCalc. That would quickly tell you the calculated (estimated) flight duration with an alternative battery pack.