When planning for Siem Reap, I thought that all there really was to see was Angkor Wat.  But, boy was I wrong!  We were only in Siem Reap for three days, splitting up the third day to take in different sites, but if I could do it again, I would want to see everything that our group saw collectively.  That's why I would say spend three full days in Siem Reap.  Most of the time because of flights, you'll end up spending two half days and two full days, so here's what I would consider the quickest itinerary where you can see it all (well, still almost, there's always more that you can find).