HR Gurus, what is going on with my HR???

So I like to use heart-rate training when I am working on lactate threshold. I have been taking the same test (recommended by this website and multiple other sources) for years now. I do a warmup, then an all-out 30 min time trial and take my average HR over the last 20 minutes as my lactate-threshold heart rate. I even do the exact same route on my bike! I have used this test for running and cycling since I know your HR can differ between these two sports.
Here are some examples of recent test results:
May 2013 - my LTHR was 168
June 2013 - my LTHR was 174
Nov 2013 - this time it has jumped up to 178! I was riding my mountain bike for this one in full winter gear and I did it early morning on my
way to work.
Jan 2014 - this was my most recent one and it was 181! I also did this one on my mountain bike in full winter gear in early morning.
Now I know that your HR statistics can vary depending on weather, level of fatigue, time of day, etc. but it seems like my LTHR has seriously increased a lot! I am 29 years old by the way, male, in good shape. These results just seem a bit high to me.
Today I went out in the afternoon (on mountain bike in full winter gear) to do some sub-lactate threshold work. Heres the thing...during my intervals my HR was in the 160s (which should be in my zone 3 with a LTHR of 181), but I felt like I was riding at a lot higher effort! I felt like I was riding in zone 4 as in a time trial!
So what I am wondering is, can this difference be just because I rode in the afternoon? Or maybe I am just getting really good at my 30 min TT test and my results are higher than they should be. Like, maybe I am just good at bearing through the pain of an all-out time trial and so this 30 min test isn't working for me.
I realize that I could just be getting in better and better shape, but isnt a HR of 181 pretty high? Should I try a different test? Is it just because of the morning vs afternoon time difference? What is going on here? Thanks a ton for the replies!!