I have four games with a "Games for Windows LIVE" logo on the box:
007 Quantum of Solace
Bioshock 2
Fallout 3 (GOTY edition)
Star Wars - The Clone Wars: Republic Heroes
All of these are German releases. Other releases might behave differently.
My copy of Fallout 3 GOTY doesn't use GFWL anymore. I don't know why the box still has the GFWL logo on it.
The other three do use GFWL, but I could get all of them to work to some extend. I think I had to install the most recent patches, otherwise some of them wouldn't even start due to them requiring some sort of online activation. I think Bioshock 2 required a patch, maybe 007 QoS as well. I don't remember if Republic Heroes also needed to be patched. The biggest problem with these games is that they won't allow you to save your progress if you don't link the game to a GFWL account. I played through 007 QoS without saving and I think I was able to beat the whole game. The game locked up with a black screen at the end and I didn't get to see any ending credits, but based on what I've seen in Let's Play videos online, this must have happened after the final level. The other two games I've only played for a couple of minutes, since I knew I wouldn't have enough time for a full playthrough anyway.
I guess there's no definitive answer here. If you come across a game that requires online activation and there's no patch for it that removes that, the physical copy is pretty much useless.
Maybe there are cracks or GFWL wrappers out there that allow you to save even if you don't have a GFWL account. I don't know.