Drivatar in private lobbies not equaling set amount

Is the amount of derivatives you can host per player in a lobby based on connection or is 5 per person the limit set so you’d need 4 players to have a full drivatar lobby?