My understanding is that it’s not harmful to the fungus itself, which is a bunch of mycelial threads in substrate - as you said, the mushroom is just a fruiting body of the fungus and will eventually decompose. And by picking the mushroom, mycologists have told me that’s releasing more spores but I can’t confirm that. I think as long as you put the mushroom back in the same place it should OK.
Whether or not it’s legal to do this in whatever area you’re in is another issue, and there are ethical questions surrounding whether or not you are having a negative impact on other people’s enjoyment of nature. For example, people probably find hiking less enjoyable when there’s broken bits of fungus littering a trail.
See this previous discussion: https://forum.inaturalist.org/t/reasons-not-to-pick-mushrooms/9024