One thing that voters will have to consider when they head to the polls in November is the historic trend of second terms for Presidents not going nearly as well as their first terms. This has been particularly true in the modern era of American politics.
George W Bush had the housing bubble burst, the financial crisis and the start of our current recession. Bill Clinton had a recession (albeit much milder than the one we are in right now) and, of course, the whole Monica Lewinski/impeachment thing. Ronald Reagan had the Iran-Contra scandal, Richard Nixon had Watergate and resignation in shame, Lyndon Johnson had the draft and race riots and the US basically coming apart at the seams, Dwight Eisenhower had a recession and the embarrassment of Sputnik, Harry Truman had the Korean War stalemate and lost the first primary to Estes Kefauver in 1952 before pulling out of the race.
It's difficult to judge if Franklin Roosevelt's second term was worse than his first. He had spent billions of dollars on New Deal programs and projects (none of which had enrivonmental impact studies done for them or that used sustainable construction processes) but the Great Depression had eased little. That term might actually be considered a "push".
Why do second terms fail to live up to the standards set by a first term? The lame duck effect may have something to do with it. Congress is less likely to work with you knowing there is a countdown clock running on your time in the White House--while they can hold their seat forever if they want (an unfortunate result of the Republican over-reaction to FDR's four-term run with the 22nd Amendment). It could also be that economics are cyclical--with periods of bust and boom. Take away the Reagan era of growth and it's just hard to go eight years without some kind of recession. And then there is always the fatigue factor. Being President is 24/7/365 and after four years of that pressure, there is inevitably going to be a drop in enthusiasm and focus. Why do you think Presidential cabinets change so much from one term to the next?
So voters have to think "If this term is going to be the better of the two, what does that mean for the next four years?" Of course, with the backwards logic consistantly employed by this administration, maybe their selling point will be "Another four years couldn't possibly be worse than the first four years!"