I recently got an ad showing18650 size lithium batteries with many different mAmp ratings. Will a lithium battery with a higher amp rating burn brighter or longer, given the same voltage and the same LED bulb(s)?

there's an 'h' on the end of that rating, right? that would be milli-amp-hours. Its a capacity rating. It will affect run time. It will not affect brightness unless you've got some wimpy cells that aren't capable of supplying enouch current.

For example if its 2500 mah it can do 2.5 amps for 1 hour, or 1.25 amps for 2 hours, 1/2 amp for 5 hours, etc.

Sketchy brands usually inflate their ratings, so keep that in mind.

800 CCA is a real good battry