There are multiple factors which affect flight time.
1. Total weight of the drone
Weight is the huge factor in how long your drone will be able to fly, even small difference as 20g can change flight time on a minute on micro drones and for even smaller drone it becomes more and more important. So being able to put your drone on a most extreme diet is will greatly help to have longer flight times.
2. Well balanced props and good PID settings
The more vibrations drone experiences, because of either not very well balanced propellers or suboptimal PID settings the harder FC will make drone motors to work and it will suck juice out the batteries much faster, which will reduce flight times.
3. Capacity of the battery
Battery with higher capacity generally will allow you to get longer flight time, but there is a catch with it, and it will work as expected only when you drone is not overloaded, because of heavy weight.
Example
I was able to get 4mins from 2s 450mah battery with my Mini Owl which is around 80g total. From what I hear average well setup Mini Owl can fly around 4min. Somebody was running 850mah and getting longer flight times but then it runs heavy and more loud.I think that 650mah won't give dramatic increase in flight times, maybe another 30 sec or so.It is a bit vague but it is because experience varies a lot even between my builds of different drones, just too many factors are at play.