I don't see this as being too much of a problem, and as I work in HR, my team see this situation often.
My advise would be to be up front and honest to any future employer that offers you a position.
Whilst it is ilegal to give you a bad reference on things that cannot be substantiated, like your personality for example, if your previous employer saw you as unreliable due to your college hours, and that you left so suddenly he may quote that he "would not to re employ". (petty!) However if you performed well whilst you were working there, took little or no time sick, and was punctual, I doubt he'd even bother to mention it.
I would put a positive spin on the situation and explain that you were so commitment to college that you panicked and threw yourself into your work.
Also I would offer a professional reference (Ie someone that as known you for years and whom works in a professional capacity, a solicitor, college professors, policeman, J,P etc)
When I worked as a student I was sacked from Pizza hut, just because my boss "didnt like me" and that hasn't stopped me from working!
Well done for being so commited to college!