I would not for one moment condone this man's behaviour insofar as you find it intimidating, but you need to be aware of cultural differences.
I am no expert on Japanese culture, but this habit of interupting you may be perfectly acceptbale in a manager / subordinate situation in japan, and maybe your manager has not been briefed about how this is perceived in Western culture.
Yes by all means have a one-to-one chat, but be prepared to be open, you may find your manager is surprised when he finds out your reaction to what he sees as standard ettiquette. It's always worth being nice first - then you can be nasty after, if necessary.
If your meeting does not resolve the situation, get your Union involved, they'll sort it out for you.