Working with multiple http request-response



I have a program which loops through an apps list.



Apps
--------
App1
App2
App3


Now, for each of them, i do a http request to get a list of builds for each app as an Xml.


So a request like



http://ift.tt/1tWIRp6


gives me a response like



<appid name="App1">
<buildid BldName="Bld3" Status="Not Ready"></buildid>
<buildid BldName="Bld2" Status="Ready"></buildid>
<buildid BldName="Bld1" Status="Ready"></buildid>
</appid>


Now I get the Highest build number with Status "Ready" and then do another web api call like



http://ift.tt/1vwBDv9


This gives me a response like



<buildinfo appid="App1" buildid="Bld2" value="someinfo"></build>


I feed these into internal data tables. But now, this program takes a painfully long time to complete (3hrs!!), since I have close to 2000 appids and there are 2 Web requests for each id. I tried sorting this issue using a BackgroundWorker as specified here . i thought of collating all info from http responses into a single XML and then using that XML for further processing. This throws the error "file being used by another process"


So my code looks like



if (!backgroundWorker1.IsBusy)
{
for(int i = 0; i < appList.Count; i++)
{
BackgroundWorker bgw = new BackgroundWorker();

bgw.WorkerReportsProgress = true;

bgw.WorkerSupportsCancellation = true;

bgw.DoWork += new DoWorkEventHandler(bgw_DoWork);

bgw.ProgressChanged += new ProgressChangedEventHandler(bgw_ProgressChanged);

bgw.RunWorkerCompleted += new RunWorkerCompletedEventHandler(bgw_RunWorkerCompleted);
//Start The Worker
bgw.RunWorkerAsync();
}


And the DoWork function picks the tag values and puts it into an XML.


What is the best way I can get the app- buildinfo details into a common file from all the http responses from all the background workers.I use .net 4.0. If there is an easier way and I'm doing it all wrong. Please do let me know :)


No comments:

Post a Comment