I don't think I'll ever fully understand the vehement hatred that some folks have to potassium sorbate. I don't add it to my wines that ferment dry and I don't backsweeten, but only because I figure, if it doesn't start back up again in the 6-12 months of bulk aging, odds are really quite low it will ever start up again. But If I backsweeten, I certainly add it (along with k-mta), at the proper dosage for the alcohol content of the wine and the ph and don't think a second longer about it.
I'm just at the point that I only put what's absolutely necessary in my wine. Kmeta, sure. Oak and tannin, yes. But, chitosan usually goes on a shelf (just in case I need it) and sorbate goes in the trash. Like you, I don't use it unless there's residual sugar in the wine.