Serializing options for non-breaking spaces

I have an XML document, that I want to load manipulate and then save back to disk. One of the elements has text that contains a non-breaking space   in it. When I serialize the document back it comes out as a non-breaking space but not the code point like it was in the original. This problem also effects other code points like & and ", but not < and > since the later would not be legal in elements.

Is there any way to get it to serialize back as the code point?

XmlDocument doc = new XmlDocument {PreserveWhitespace = true};
doc.Load(fileStream.BaseStream);
var node = doc.SelectSingleNode("Some xpath");
if (node != null)
{
    node.ParentNode.RemoveChild(node);
    doc.Save(path);
}