It is very difficult to confirm or refute what was in the mind of someone who painted a picture in the 16th / 17th c. Drawing on historical facts can only explain what the subject matter was, whether it was significant at the time and why it may have been painted in this or that particular style. To give it (the book) credence as a historical reference point is to make some very large leaps of faith and assume some fundamental parts of the book are accurate rather than twisted to fit.
An example of this would be for me to write a book about the universe, saying that it is like looking at the inside of a rolling ball and is expanding all the time so it is as big as we can imagine it. Then backing this up by taking statements and facts, mixing them to fit my theory and producing a believable end result. No-one can state absolutely and finally that I am wrong, because at the moment we don't know how big the universe is or if it is truly expanding or contracting, or if my inside of a ball theory is a reasonable assumption. Some very clever quantum physicists could hold a good arguement agianst my theory but proving it would still be down to a point of view.
There will always be the gullible ones that get carried away and believe it all, just as there will be those that decry Brown and call him a heretic and charlatan. That said, it was a cracking good read.