Don't put it outside unless you have a heated greenhouse. Keep it warm at a temperature in the mid twenties celsius. At this time of year you will have few insect pollinators so transfer the yellow pollen between flowers with a paintbrush or rolled tissue paper. I have had good success with either.
Yellowing could be due to cold shock or a lack of trace or major elements in the soil/compost. Use a tomato feed diluted as per tomato instructions and now it is flowering feed constantly throughout the fruiting period.
Good luck!