General advice for dealing with long running processes
Hi there. I'm getting ready to create a website for work that allows a user to start, monitor, and ultimately get results from a long running calculation.
The calculation will run long enough that a page would normally time out waiting for the server to respond.
The calculation code will be in .NET and most likely we will run the calculation on a Linux machine, possibly the web server itself (not sure about that yet).
What I need is some good ideas on handling things like:
1. How to get progress updates to the user, while the calc is running. The calculation code will be able to emit progress messages (such as 25% done), but I'm not sure how to get these to the user in the browser. I could probably require the user to press F5 every so often, to get updated results from the server, but this doesn't seem very friendly
2. How to inform the user the calculation is done and get results to him. I guess this is related to the first item; if I can figure out how to get progress updates to the user, then getting him a "job done" message, and the results themselves, may use the same solution.
3. What's a good architecture to handle the i/o between the web server app and the calculation that's running? The web server app will be Python/Pylons. In addition to starting a job, and getting updates and results, the user may cancel a job that has been started.
Any ideas on any of these topics would be greatly appreciated. I'm pretty sure I could create my own solutions from scratch, but if there are best-practices to follow, I'd rather do that!
Thanks for any ideas!