Friday, 3 April 2015

Automatically import XML data from webpage into postgres/posgits DB



I am a newbie in this area and need a help from you to get started. My end goal is to get xml data (http://ift.tt/19V5SDJ) every 3 minutes automatically imported into a postgres/postgis DB which is running on Amazon's RDS cloud. The database is not installed on a virtual server.


Before importing the data from the XML file into the table I need to strip off unnecessary data. The only data I want to import is the car park name like PARNELL and the amount of car park spaces like 389.


I don't know what is the best approach to realize something like this and hope to get some ideas how I can achieve this.


My first thought was to get a cron job running on the postgres database which would establish every 3 minutes a connection to the xml file and read the file in. -> Would postgres support any cron job like this?


Also what would be the best approach to "clean" the XML file and just import the necessary data (carpark name + spaces)?


Would it be a better approach to use a Java to download the XML file, "clean" the XML file and imported the data through another Java script?


As last point, I don't have to use the postgres/postgis database from Amazon RDS, I could install a virtual machine on my system at home and install postgres/postgis.


Thank you very much for your help.


No comments:

Post a Comment