Is it worth it to go Lysefjord for 2 days from Bergen, or better just to visit the nearby Hardangerfjord?

My wife and I are going to Bergen in May, and want to see some of the fjords (obviously). We will we go up to Geirangerfjorden, but wonder if it's worth it to go down to Lyseford for 2 days (I know Stavanger is about 5.5 hours by bus. Yes, I want to climb Preikestolen), or just spend more time around Bergen. If we did that, we would have only about 2.5 days in Bergen. Thanks!