I use Sugarflair Red Extra in my red velvet cake. It's a paste and you can mix it right into the sponge mix. You don't have to pre-mix it with anything. To make one red velvet cake I use about 1/2 tsp and my red velvet cake is always a vibrant red. If your cake looks more brown than red, then you haven't used enough.
I too use sugarflair red food colouring, the colour is strong and very concentrated, half a teaspoon should be fine for Red Velvet cake....maybe you used too much!........