I'm a librarian who loves anime, manga, and reading a wide variety of genres.
If I didn't know better, I might guess that the anime came first, just because of the way some things are phrased. For example, when Lawrence tells Holo about peaches preserved with honey, she becomes consumed with a desire to have them. She is described like so:
"Her eyes were moist and blurred with overflowing anticipation." (17)
This sounded bizarre to me in the text, although I could bring to mind the kind of anime/manga reaction being described, and the illustration on the next page confirmed it.
It'll be interesting when I get to the books that haven't been adapted into anime. I like this series well enough so far, but I can't help but wonder if some of my reaction is due to already having been exposed to these stories in a visual format.