When I was at college, many moons ago, I always spoke less to the one(s) I really fancied - too afraid of making a prat of myself so I kept my mouth shut. Consequently the one I really liked didn't think I liked him coz I hardly ever spoke to him. Whereas I found speaking to the ones I didn't fancy easier.
Looking back it was all so awkward, but it sounds to be you are making progress; albeit slow! Stick with it, try & be as natural as possible - he's probably that nervous about making a prat of himself, he's not going to notice if you do.