Question Making the Most of a Layover at KEF?

I was looking at a potential itinerary where I would have a layover, arriving at KEF at 3:35 pm one day and departing the next morning. I'm wondering if it's worth leaving the airport to explore Reykjavik or nearby areas, or if I should just stay at the airport. Has anyone done this? How easy is it to get into the city, and are there any must-see spots that are doable in such a short time? Also, any tips on where to stay overnight would be appreciated.