Even Good People Make Teams Go Slower
I have a simple model of why adding more people to your team – even good people who are working hard, etc – can make your team go slower.
First, let's take the case of a one-person organization. The person only has 100 "points" of energy to spend, but 100% of it goes on work, with 0% on coordinating with other people.

Now the person gets a colleague. This requires spending 20% of their effort/energy on co-ordinating with another person, but the upshot is that the total work done increases to 160 points.

When a third person joins, the co-ordination time goes up to 40% of everyone's efforts (20% per colleague per person), such that each person can only spend 60 points on the actual work. The third person still increases total output, but only from 160 to 180 – that is, they marginally increase output, but much less per-person than the two previous team members did, despite working hard.

Once you add a fourth person, the time spent coordinating gets to 60% per person, such that each person is only spending 40 points of their effort on actual work. As a result, the total work done by the team goes down to 160 – the fourth person has had a negative impact on output, even without considering their salary and other costs. And this is despite them being an equally-productive person as the rest of the team.

Now, you very reasonably might have complaints about this model. Does it really take 20% of your energy coordinating for every additional person you have on your team? Don't coordination costs have a large fixed component, such that additional team members after the second take up a smaller marginal share of your energy?
I think this is fine, and surely varies by team and activity, but the point of this model is just to show how this kind of dynamic could occur; if you change the numbers you can still get the same result at a larger team size.
One consequence of this model is a partial defense of managers:

If a manager takes on all coordination for the team, such that the other people coordinate only with her and don't have to spend bandwidth coordinating with anyone else, a manager can increase productivity even under the assumption that she does no work of her own.