We have a large XML file we need to validate. Large is about ~260MB, which is not really that large. My old code, which was super-fast, is as follows:
XmlReaderSettings settings = new XmlReaderSettings();
settings.Schemas.Add(null, _xsdFilePath);
settings.ValidationType = ValidationType.Schema;
settings.ValidationEventHandler += new ValidationEventHandler(ValidationErrorHandler);
StreamReader streamReader = new StreamReader(_xmlFilePath);
string text = streamReader.ReadToEnd();
streamReader.Close();
XmlReader rdr = XmlReader.Create(new StringReader(text), settings);
_errorLog = "";
try
{
while (rdr.Read()) {}
}
catch (System.Xml.XmlException e)
{
_errorLog += "\n\nFatal Error: XML is not correctly formed!!!\nError = " + e.Message;
}
But this code also crashed pretty quickly with a System.OutOfMemoryException. Probably logical, trying to load the entire file into memory, and then handling the entire xml-validation in memory too.
I changed is as follows, reading directly from the file, and now it became incredible slow, and it crashes after a few hours, with the same error (out of memory exception).
XmlReaderSettings settings = new XmlReaderSettings();
settings.Schemas.Add(null, _xsdFilePath);
settings.ValidationType = ValidationType.Schema;
settings.ValidationEventHandler += new ValidationEventHandler(ValidationErrorHandler);
XmlReader rdr = XmlReader.Create(_xmlFilePath, settings);
_errorLog = "";
try
{
while (rdr.Read()) {}
}
catch (System.Xml.XmlException e)
{
_errorLog += "\n\nFatal Error: XML is not correctly formed!!!\nError = " + e.Message;
}
Any ideas?
No comments:
Post a Comment