If a fly wouldn't annoy you, would you still scare away the fly after it landed?

After seeing how quickly they can lay eggs, yes. I will always swat them away from uncovered food.