Brandy will impart the brandy flavor to the wine, now dessert port. If you use Everclear, it has no flavor, and will not change the flavor of your port.
So another question, if you are able to get 20% out of your yeast, then I assume it’s fermented bone dry, or is it just a tad back sweetened...