I would suggest re-reading the book, would rye bread make an acceptable substitution? If you feel it's too harsh perhaps you could have one small potato or slice of bread a day. My friend did a similar high protein diet and the only thing I ever heard from her was how much she missed her potatoes, she decided to go to WeightWatchers in the end.
Unfortunately rye bread is also carbohydrate. Surely the whole point of the Atkin's diet is that it is almost carbohydrate free so there is no point in looking for substitutes.