Firstly buy second hand (let the original owner pay the depreciation for you). Just a note that it may be a good idea to include your definition of inexpensive as it varies greatly from person to person.
I’ve listed the best affordable macro lenses for Canon EF/EF-S mount below (roughly in order of the better lenses first):
- Sigma 105mm EX DG OS HSM Macro.
- Tamron 90mm SP Di VC USD Macro (there are two versions - both are good).
- Sigma 150mm EX DG OS HSM Macro (if you need a bit more working distance).
- Canon 100mm USM Macro (non L series).
- Canon 60mm USM Macro (shorter working distance than the others, EF-S mount only).
The first three lenses listed have image stabilizers which can make a world of difference at macro range. The ~100mm lenses are good as general purpose macro lenses but the 150mm is great if you need a bit more working distance (for skittish subjects for example).
The biggest thing in macro is probably the depth of field is generally quite narrow so focusing has to be accurate, the other big thing is light - it is 100% possible to do macro with natural light but most people use flash with a diffuser as it lets you use narrower apertures (more depth of field). Also as a last note autofocus generally doesn’t work very well at macro range - my Sigma 150mm macro does a decent job but I still have it set to back button (using the af-on button as opposed to half press of the shutter button) and use entirely manual focus at the maximum magnification end.