Yes, of course, it's all about morale. In 7 rounds, your morale will be with 35 less. And if he starts with 85 morale, with some archers on field (200 are enough if attacker doesn't have any archer), he will finish the battle with about 45 morale, so there are chances that he will lose. And you don't need to fill the fort with archers. Sometimes they are useless.