JAXB won't unmarshal my previously marshalled interface impls

Here is my working marshalling code:

For starters I have an Entity class with something like the following:

@XmlRootElement
@XmlAccessorType(XmlAccessType.NONE)
public final class Entity implements {
    private Map<String, Component> components = new HashMap<String, Component>();

    @XmlElementWrapper(name="components")
    @XmlAnyElement
    public List<Component> getComponentList() {
        return new ArrayList<Component>(components.values());
    }

    public void setComponentList(List<Component> comps) {
        for(Component c : comps) {
            components.put(c.getComponentType(), c);
        }
    }

    ...
}

Component is a very varied interface but all implementations are nice and round beans carefully annotated.

Now, marshalling produces some good looking XML

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<entity>
    <components>
       <containerComponent>
           <children>
               <entity>
                   <components>
                       <containerComponent>
                           <children>
                        <entity>
                            <components>
                               <directionComponent>
                                  <direction x="0.0" y="1.0"/>
                               </directionComponent>
                               <sizeComponent>
                                  <size x="52.0" y="12.0"/>
                               </sizeComponent>
                               ...

As you can see, each component type has its very own tag name but that doesn't seem to be enough for JAXB because

Exception in thread "main" java.lang.ClassCastException: com.sun.org.apache.xerces.internal.dom.ElementNSImpl cannot be cast to tobacco.core.components.Component
    at tobacco.core.components.Entity.setComponentList(Entity.java:121)
    at tobacco.core.components.Entity$JaxbAccessorM_getComponentList_setComponentList_java_util_List.set(Unknown Source)
    at com.sun.xml.internal.bind.v2.runtime.reflect.Lister$CollectionLister.endPacking(Unknown Source)
    at com.sun.xml.internal.bind.v2.runtime.reflect.Lister$CollectionLister.endPacking(Unknown Source)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.Scope.finish(Unknown Source)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext.endScope(Unknown Source)
    at com.sun.xml.internal.bind.v2.runtime.property.ArrayERProperty$ItemsLoader.leaveElement(Unknown Source)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext.endElement(Unknown Source)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.SAXConnector.endElement(Unknown Source)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(Unknown Source)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(Unknown Source)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown Source)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(Unknown Source)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source)
    at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(Unknown Source)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(Unknown Source)
    at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(Unknown Source)
    at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(Unknown Source)
    at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(Unknown Source)
    at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(Unknown Source)
    at tobacco.core.serialization.XmlEntityConverter.toEntity(XmlEntityConverter.java:32)
    at tobacco.core.loader.XmlLoader.loadEntityTree(XmlLoader.java:43)
    at tobacco.game.test.main.Main.main(Main.java:75)

At the other end of the process I receive ElementNSImpl objects.

I though about using an ObjectFactory but after reading and testing I'm almost convinced it's no use: I couldn't find a way to access tag name on the creator method, and even if I could I'm not so sure JAXB would just accept my object and fill its properties happily or if I would have to fill them myself, which absolutely impractical.

I think this guy had the same problem. Does it work out of the box with MOXy? Is there a way to do this without switching implementations?