How to pass large data between systems



I have pricing data that is stored in XML format that is being generated on an hourly basis. It is roughly 100MB in size, if stored as XML. I need to send this data to my main system in order to process this. In the future, it is also possible that this data size is sent ever 1m.


What would be the best way to send this data? My thinking thus far was:



- It would be too large to send as JSON to a POST endpoint
- Possible to send it as XML and store it on my server


Is there a more optimal way to do this?


No comments:

Post a Comment