See everyone on your close friends list and remove any of them, so they no longer see your close-friends-only stories and posts. They aren't notified.
