Psst - New Vision Model Released!

Suggestions welcome

Figure out a way to train with the location and date

People keep bringing up modifying the input images, but you can also just provide these as secondary inputs to the model architecture. For example if the architecture is currently:

  • [image] → [convolutional layers] → [output layer]

You could make it:

  • [image]-->[convolutional layers]-->[ concatenated
  • [location]------------------------> ........layer ]-->[ output layer ]

And location could simply be represented by the real value of the latitude and something like the cosine of the longitude after scaling from -pi to pi. (This enables it to wrap around.) You could do similar things to enable dates to wrap, giving a seasonality input value.

edit to add: The wrapping method I gave makes it so that 90W is exactly as far from 0 as 90E, but doesn’t capture that those are opposite from one another. If you use both the sine and the cosine you get both pieces of information. Intuitively, the sine and cosine denote a specific angle that points to a specific location on a circle (where the circle is a latitude line). Likewise, the sine of a date is about 0 at solstice and the cosine is about 0 at equinox, and the pair of them together tells you exactly where in the annual cycle you are.

I don’t know how useful this is / don’t have citations for its use, but it should be easy enough to try and test on a small dataset to see what happens.

4 Likes