The only thing he had was a stockpile, a couple walls since I didn't even let him finish building his walls, and the main keep when I finally decided to push my lord up to help the others that were there with the final attack on the enemy lord. I could see his stockpile even though you can't demolish it and it was about empty and he's still building and maintianing peasants even though he had no granary to feed anyone. I guess you missed that I had taken out all the buildings including the granary and had enough troops around all of his land to prevent him from building again though he would occasionally sneak a wood cutter in as I was moving and attacking.