i’ve been thinking about how this could even be represented, and i found this Activation Atlas for different layers of an image classification model: https://distill.pub/2019/activation-atlas. i think it provides an interesting visualization of how such a model might “see” different concepts. looking at the last layer will give the most human-friendly view of the concepts represented in the model.
i don’t know how difficult it would be to produce a similar activation atlas using iNat’s computer vision model, but i think this would be one way to give humans both an idea of what features the model is using to represent different taxa and also an idea of what other taxa might be “seen” as having similar features.
can you all even make sense of this activation atlas example and see how it could be used for any purpose other than just satisfying curiosity? is it interesting and useful enough to even contemplate asking staff to try to make one for the iNat CV?