This problem was spotted by Daniel Van Noord, from Optec. If a client application is set to run elevated ("As Asdministrator") and it loads a LocalServer driver, the server will also run with elevated permissions. That's bad from a security perspective, but perhaps even worse is that if an unelevated instance was already running, then you get a second elevated instance instead of the client attaching to the already running instance. This means the two instances will be in contention for resources (such as the serial port) and one of them will most likely fail. Very bad.
Daniel also researched a fix which is being considered for inclusion in the LocalServer template. While it is not a showstopper, it can lead to confusing failures for users, so Developers of existing LocalServer drivers might want to add this fix. I'm adding it to all of mine.
Here's a video demonstrating the problem and the fix on a production driver.