Yes.
They share great-great grandparents.
An easy way to remember how this works is that first cousins share grandparents (their parents are brothers or sisters). Second cousins share great-grandapents. (Their grandparents are brothers or sisters). Third cousins share great-great-grandparents (their great-grandparents are brothers or sisters).
A cousing "once removed" is the parent or offspring of a cousin.
In short, second, third (and so on) cousins are wider apart in the family tree. Cousins "removed" are higher or lower (in generation) in the family tree.