Is a 4-coiled #2 square jaw big enough for otter?

Set for otter using a drowning rig or coni. I have held otter with traps as small as a #1 longspring, but in that case, the trap was attatched to a plywood mink box with double 14 wire. The whole mess got tangled up in an alder thicker, giving the trap a spring when the otter pulled. There weren't many alder sprigs left when I came along, and the otter was totally exhausted.
In short, your trap would work fine. But I wouldn't just wire it to a tree. Otter roll and spin like crazy when caught. Use a drowner.

This right here. That trap will work fine. But like AV mention they are powerful critter. Anchor them good.

OK, thanks guys! I'll use it in a submersion set