Rails Nokogiri XML parsing working locally, not on Heroku



I have the following script running in my rails app. When it runs locally, it parses all of the courses correctly (9673 results) but when I push it to Heroku, it returns exactly 2x this number (19346 results). Any ideas? Are there possible Heroku performance limitations that could be causing the cycle to repeat itself?



# Pulls all course data for specified year, based on subject list
subjectdoc= Nokogiri.XML(open("http://ift.tt/1xM0doJ"))
# Reads each subject and stores it in local variable prefix
subjectdoc.xpath("//subject/@subject").each do |prefix|
# Link to course pages, substituting in prefix in URL
classdoc= Nokogiri.XML(open("http://ift.tt/1BT5uBz"))
# Reads each course and stores listed vars
classdoc.xpath("/courses/course").each do |course|
num = course["catalog_nbr"] || "Not provided"
subj = course["subject"] || "Not provided"
title = (course.at("course_title/text()") || "Not provided").to_s
cid = (course.at("sections/section/@class_number") || "Not provided").to_s
inst = (course.at("sections/section/meeting/instructors/instructor/text()") || "Not provided").to_s
# Creates a cornell class in Cornellclasses table
Cornellclass.create(:prefix => subj, :coursenumber => num, :instructor => inst, :title => title, :courseid => cid)
end
end

No comments:

Post a Comment