There’s been a fair amount of discussion of this issue on the forum including some feature requests, so I’d suggest checking out some of the existing threads. A quick sampling:
https://forum.inaturalist.org/t/improve-id-function-and-name-of-agree-buttons-a-modest-proposal-not/6589
https://forum.inaturalist.org/t/identification-etiquette-on-inaturalist-wiki/1503
https://forum.inaturalist.org/t/overzealous-identification/5975
Some requests are open for comment, though some threads have closed. If you want to add to one of those discussions, you can tag a moderator and they can open it for you.
Some solutions for you as an IDer are to withdraw your initial ID or ID to a higher level so the observation doesn’t go to RG, to leave a comment explaining that users should only agree to their own level of expertise (many beginning users don’t understand this - see some of those linked threads), or to use the “Can still be improved” option in the DQA (though if you do this, please keep up to date with the observation, and remove the check when appropriate so it doesn’t remain in the Needs ID pool forever…)