I have a 36 and love it but I recommend getting the minimum legal caliber for deer in your state.
I would agree if its your first muzzleloader. My first was a 32 and I sold it in 2 years to finance a 50 caliber to deer hunt with (and make the switch to a flinter). You can hunt squirrels with a 50, just need to take head shots only.
I have a 50 and a 54, so the 36 fills the small game slot nicely.
If you were strictly plinking with the rifle, then a small bore would be fine. But if you think you might want to deer hunt, get a 40 at minimum (depending on reg's).