Not really a formula, but there are details that can help. Generally speaking, a 1″ increase in club length equals a 1 mph increase in head speed. 1 MPH in head speed yields 2.3 yards of distance. So, just based on the length alone, a 1.5″ increase should increase the distance 4.6 yards. IF, he fairway wood has a lighter shaft, that will add to the difference. Assume the Fairway wood shaft is 60 grams and the hybrid shaft is 80 grams. 20 grams difference in shaft weight could add another 2.27 yards. 20 gram lighter head weight could add approximately 2 additional yards. That gives you approximately 8.87 yards, if my math is correct. Different head shape and face construction means there is likely a difference in COR (ball speed off the face) and cg location, which will effect launch and spin, all of which will effect distance to some degree (but will vary greatly depending on the swing speed of the player). Higher the swing speed, the more these factors will effect the distance. With regards to COR, on clubs over 15 degrees loft, any change has a statistically insignificant effect on distance. So, beyond the length and weight differences mentioned above, any differences would most likely come from the launch and spin differences created by the two different head designs and their cg locations. I can give you my personal point of reference. My clubhead speed with a driver is 105 – 110. I had an 18 degree 5 wood with a 65 gram shaft that was 42″. I have a H2 Hybrid that is 17 degrees, has a 80 gram shaft and is 40″. Hit them both 225 yards, but the trajectory with the H2 hybrid is lower. Not “low”, but lower than the fairway wood, which I felt was too high and why I went to the lower lofted and shorter Hybrid. I know the differences in these two clubs of mine and the math I described above does not exactly equate, but that is my personal experience and is an example of what I always tell players, you have to build and compare clubs yourself, because everyone is different and you can’t know for sure how clubs will compare until YOU hit them.
Hope this helps.