It's more of the 'Red Count is there so we the main body wouldn't meddle with him too much, after all, he has the bloodline of the emperor'.
Just like I said, it was more of a vacation village, does that mean no tax? No, but it meant that the military presence is somewhat lack, compared to...