According to the Mishna (18b), when the preparations of the Kohen Gadol were done, he was transferred by the Sages to the priestly elders who had him take an oath that his performance of the service would be done according to the teachings of the Sages. The Mishna concludes that following the oath, both the Kohen Gadol and the elders who executed it turned away and cried.
The Gemara explains: He turned aside and cried due to the indignity that they suspected him of being a Sadducee; and they turned aside and cried, as Rabbi Yehoshua ben Levi said: One who suspects the innocent of indiscretion is afflicted in his body. The High Priest might in fact be beyond reproach and they may have suspected him falsely.
The Sadducees (Tzedukim) were one of a number of sects that lived during Second Temple times, who had different interpretations of the passages that described the avodat Yom ha-Kippurim (Day of Atonement service), and they wanted to ensure that he would carry out the service properly. It was a particular concern because some of the essential parts of the service took place in the Holy of Holies where no one could see what was being done aside from the High Priest himself.
According to our Gemara, the elders cried because they were forced into a situation where they had to actively suspect someone of bad intentions, and Rabbi Yehoshua ben Levi taught that someone who suspects another without cause will suffer for having done so. According to the Jerusalem Talmud the elders cried because of the deterioration of the Second Temple period, when even the High Priest could not be trusted to carry out the Temple service properly.
The main argument between the Sages and the Tzedukim revolved around the definition of the passage (Vayikra 16:2) “for I appear in the cloud upon the ark-cover.” The Tzedukim interpreted this to mean that the incense cloud of the ketoret had to be lit by the kohen before entering the Holy of Holies (the Sages understood that the ketoret was lit only after the Kohen Gadol was inside). According to the Me’iri, lighting the ketoret outside appears to be a form of avodah zara, as it looks like the kohen is lighting the incense to honor a power outside the kodesh kodashim as well as the One inside the Holy of Holies.