Microsoft.Office.Core.CustomXMLParts events not firing due to instance change

I have a class:

class Bar {
    private readonly Document _document;
    Bar(Document document) {
        document.CustomXMLParts.PartAfterAdd += CustomXMLParts_PartAfterAdd;
        _document = document;
    }
    void CustomXMLParts_PartAfterAdd(CustomXMLPart NewPart) {
        MessageBox.Show("Test");
    }
}

Somewhere else, I add custom xml parts using

_document.CustomXMLParts.Add("<e></e>");

However, the PartAfterAdd event is not fired. So, I investigated this and it turns out the CustomXMLParts instance at the time of part addition is different. I modified my class:

class Bar {
    private readonly Document _document;
    private readonly CustomXMLParts _parts;
    Bar(Document document) {
        document.CustomXMLParts.PartAfterAdd += CustomXMLParts_PartAfterAdd;
        _document = document;
        _parts = document.CustomXMLParts;
    }
    void CustomXMLParts_PartAfterAdd(CustomXMLPart newPart) {
        MessageBox.Show("Test");
    }
}

and at my part addition code, I added a check

if (_document.CustomXMLParts != _parts) {
    MessageBox.Show("changed!");
}
_document.CustomXMLParts.Add("<e></e>");

My breakpoint at the messagebox is hit. I then proceeded to use the immediate window to check for hashcode & equals, which confirmed indeed the CustomXMLParts instance is different.

I figure this must be the source of the problem my events are not firing. I checked my code to see if I call into the interop API which may cause CustomXMLParts to change. I did not find any possible sources.

What can be the root cause of the CustomXMLParts instance changing?