Adaption of computer vision model to recognise backgrounds and aid certain traditional projects

Would it be possible to adapt the computer vision model to recognise and sort or tag pictures of animals with a particular background? As a specific example I am thinking of roads; adding observations to roadkill projects relies on individuals manually sifting through the mass of observations, and looking for ones that may be suitable for the project. This is obviously very time consuming and also prone to errors.
It seems it should be straightforward for the computer vision model to recognise tarmac, gravel, and perhaps dirt roads, and save the IDer much time.
This could also have a side benefit of allowing people that don’t want to see gore-y picture to be able to filter out potential pictures of roadkill.

Researchers have done similar things (e.g., use the iNat computer vision model to identify insects while using a different model to identify any flowers they are visiting). The way to do this is to use multiple models. iNat’s development team is likely focusing all their resources on the primary features of the site, such as increasing the number of organisms included in the computer vision model.

2 Likes

You could say the same thing about literally all of inaturalist. Why focus on roadkill specifically?

1 Like

More than that, it would be really helpful if more observers would describe the location/habitat of their finds in a few words.

2 Likes

Yellow flowers are a problem, for example. Almost every small insect sitting on a yellow flower of Asteraceae for iNat CV is a Listrus , no matter how different it is from a true Listrus.

1 Like

I don’t really have an answer to that, other than its something I’m familiar with and it’s seemingly low hanging fruit - it’s presumably easier for the computer vision model to recognise an animal on a tarmac background, than say a prey item in the mouth of a snake or bird, which are other traditional projects that are also time consuming.

Of course it would be easier still if all relevant observations had annotations and were appropriately tagged, but that is not the situation we are in, or likely to be in in the future.

Yeah, for some taxa the problem is exactly the opposite: the CV is actually sometimes frustratingly too good at learning the background rather than the features of the organism it is supposed to be identifying.

In cases where a species is closely associated with a host/food plant, I guess one could argue that this means the CV is accurately recognizing typical co-occurrences – but this would be more interesting and useful if the CV weren’t also overapplying this correlation to all sorts of cases where it isn’t warranted.

6 Likes