At this point, if there are a few tiny hairs you missed, just go ahead and run over those. It’ll erase the background because this tool is apparently magic. You can also erase certain parts as well.
Here’s what I’m left with before hitting the OK button on the Refine Edge dialog box.