So now that I've actually finished the book and allowed myself to watch the trailers and makings of, here's what I worry about:
1) A lot of the book's appeal was in discovering the world's mythology and history. Their old sagas about dragons and the First People, etc. were very detailed and interesting, not to mention influential on current events. This is even more true for the history of the land, how the king got to be the king, his relationship with Ned, etc. These things are very text-heavy and I don't see how they can be tackled in the TV show in the required extent - there is definitely not enough time to show it in flashbacks or interludes (think movie LotR's prologue) and no way can characters just mention it in discussions.
2) While I appreciate how much money HBO is putting into the TV series, it still looks cheap. What's worse it looks small and insignificant. I recognized some of the scenes from the trailer and their counterparts in the book always were grander - had more people in it, the buildings (castles, cities) were bigger, etc. A lot of the shots also look very clean and "civilized" - the book was very much "realistic" fantasy, often taking great pride in depicting the foulness and stench of medieval-like life. The TV show seems to lack that.
3) On a positive note however, most of the dialog from the trailer was taken word-for-word from the book so my hope is they will be very faithful. That in turn should mean we are in for a hell of a ride
"We're on the side of the demons, Chief. We're evil men in the gardens of paradise, sent by the forces of death to spread devastation and destruction wherever we go."
-- Colonel Tigh