Wisteria need a lot of water - particularly after flowering. We discovered that if you water consistently and constantly through summer and autumn, next year's growth is stunning.
All the pea family suffer if watering is erratic, if the flowers are 'puny' next year consider repalcing it as some varieties have much better racemes of flowers and deeper colur tones too