How to efficiently read XML column from database and save to XML file and CSV file?



I have some logic already implemented that allows me to query for a set of records which contain an XML column. For each record I am to save the XML column data as a separate XML file as well as produce a CSV file of the same data. The problem is I can only do this for a limit number of records as I start to run out of memeory if too many records are being processed.


Currently the approach is:


To write to CSV, I am taking the InputStream for each XML column returned (using SQLXML objects for this) from query and parsing the inputstream using a DOM Parser along with some XPath in order to identify just the elements which contain any sort of text (child nodes, I don't really care about parents). Then using the element name as the header and text as the value in the csv file. The data is then being written to a file using a BufferedWriter along with StringBuilders to hold text (one for header, one for values).


Saving the data to XML file is being accomplished by taking the same InputStream mentioned above and converting it to a String and then finally writing it out to file using BufferedWriter. This really isn't the nicest looking at the end as the entire XML data is placed on a single line when it's written to the file, but it works for now.


I am quite new when it comes to working with XML in Java, so just wanting to get any advice or input on possible alternatives or more effecient practices to do this same process. Would like to be able to process at least 70 records at once.


Thanks in advance.


No comments:

Post a Comment