XML to Dictionary to Instances of Custom Class

I am wondering if there is a more direct/elegant way than what I show below, to take data from an XML file and make instances of a custom class from it. Below is how I current do it (works, but might be a bit clumsy).

My XML:

<Skills>
  <Fire>
    <Cast>0.00</Cast>
    <ReCast>90.00</ReCast>
    <MPCost>0</MPCost>
    <Button>8</Button>
  </Fire>
  <Ice>
    <Cast>5.98</Cast>
    <ReCast>2.49</ReCast>
    <MPCost>0</MPCost>
    <Button>9</Button>
  </Ice>
</Skills>

1) Loading elements from an XML into a Dictionary:

//Load Skill list
var skillXElement = XDocument.Load(path + @"\Skills.xml").Root;
if (skillXElement != null)
    SkillDictionary =
        skillXElement.Elements()
            .ToDictionary(e => e.Name.LocalName,
                e =>
                    new Skill(e.Name.LocalName, (double) e.Element("Cast"), (double) e.Element("ReCast"),
                        (int) e.Element("MPCost"), e.Element("Button").Value[0]));

2) Creating references to the Skill classes based on Dictionary:

class SkillInfo
{
    public Skill Fire { get; private set; }
    public Skill Ice { get; private set; }

    public SkillInfo()
    {
        Fire = Globals.Instance.SkillDictionary["Fire"];
        Ice= Globals.Instance.SkillDictionary["Ice"];
    }
}

3) Finally, I access the skills through a public property as such:

class Player : Character
{
    public Player()
    {
        SkillInfo = new SkillInfo();
    }

    public SkillInfo SkillInfo { get; private set; }

    private ExampleMethod()
    {
        UseSkill(SkillInfo.Fire);
    }
}

You might wonder why I do not just access each skill from the dictionary directly as such: UseSkill(Globals.Instance.SkillDictionary["Fire"]);. The reason is that each lookup in a Dictionary is fairly slow (about 500ms), despite being O(1). So, to avoid that delay each time I use a skill, I created the SkillInfo class.

Any tips or ideas on how to go from the XML to the class instances more elegantly would be highly appreciated.

Thanks in advance!