All of the above.
Hornbeak is not lacking in athleticism, it's just that Buddy is better equipped to guard Smart due to his size and strength, to go along with his quick feet and better than average hops. Both players have really good hands, plus they're super aggressive in their on-the-ball coverage.
Don't get me wrong, Hornbeak is a good defender. I said after watching him play on televison two years ago that he is as sound fundamentally as any high school kid I had seen in sometime. There are no glaring weaknesses in his game that a little more experience won't fix. If I had to choose the best freshman guard of the three, I would pick Je'lon.
That said, I believe that Buddy or Cam are better suited to guard a unique talent like Keith Smart. IMO, no one on our roster is capable of shutting him down. I do think we have three, and maybe four counting Cousins, who can make him work for his points.