Since I was old enough to really start paying attention to college hoops, I'd say the best team to not win was that '91 UNLV team that entered the tournament undefeated. They had won the title the year before and had a win streak of something like 45 games before they lost to Duke in the semi-finals.
'88 OU would probably be #2 on my list. They were awesome.
Earlier teams would probably be the Phi-Slamm-Jamma team that lost to NC State and the Patrick Ewing Georgetown team that was upset in the finals.