How do you hit the 3w, hybrid and 4 iron? Not uncommon to have slightly different flexes in different clubs. I have worked with many players, good players, that will have one flex in their driver and another in fairway woods, hybrids, etc. It’s whatever combination of head(loft) and flex that gives them the feel and ball flight they want. That being said, if you like the feel of the driver and the irons, makes sense that you might want the X stiff in the others. As far problems having the stiff flex in those clubs, I really don’t think you would notice much. Might feel a little softer, might hit them very slightly higher. Some stiff flex shafts are stiffer than X stiff flex shafts, depending on the brand, model, etc. There is no standard in the industry for flex. There can be overlap in the designations.
My advice would be to evaluate the performance in them as they are and if they are OK, leave them the way they are. If your concerned or unsure, change them to X stiff.