Passing arguments to command pattern

Okay, normally when I use the command pattern I pass in arguments through the constructor. However, my current design doesn't really allow that. The reason being is that I am loading data from an xml file on startup, so I might have something like this:

<Actor>
    <InputComponent>
        <Mapping Type="Drag" Key=1 Command="MoveCommand"/>
        <Mapping Type="Press" Key=16 Command="FillerCommand"/>
        <Mapping Type="Click" Key=2 Command="FillerCommand"/>
    </InputComponent>
</Actor>

Because they are read in as strings they are passed to a command factory object that then creates the correct command objects and hands em back to the input component. This means I don't have access to them as they are constructed and thus cannot specify special arguments via constructor. I could pass this extra info in through a void pointer to the Execute member function but that is terribly insecure. I could also pass the XML Element to the command factory and have they initialize themselves from further xml attributes (current method) but it still feels like a hack solution. Any suggestions would be welcome.