With more time: Visit the Scottish Parliament, climb Arthur's Seat, tour Holyrood Palace, or take a ghost walk in the evening.
With less time: Skip The Witchery by the Castle and eat at a cheaper pub instead. Also, visit Edinburgh Castle earlier in the day to give more time for other attractions or vice versa.