Results 1 to 3 of 3

Thread: General advice for dealing with long running processes

Hybrid View

  1. #1
    Join Date
    Jan 2014

    General advice for dealing with long running processes

    Hi there. I'm getting ready to create a website for work that allows a user to start, monitor, and ultimately get results from a long running calculation.

    The calculation will run long enough that a page would normally time out waiting for the server to respond.

    The calculation code will be in .NET and most likely we will run the calculation on a Linux machine, possibly the web server itself (not sure about that yet).

    What I need is some good ideas on handling things like:

    1. How to get progress updates to the user, while the calc is running. The calculation code will be able to emit progress messages (such as 25% done), but I'm not sure how to get these to the user in the browser. I could probably require the user to press F5 every so often, to get updated results from the server, but this doesn't seem very friendly

    2. How to inform the user the calculation is done and get results to him. I guess this is related to the first item; if I can figure out how to get progress updates to the user, then getting him a "job done" message, and the results themselves, may use the same solution.

    3. What's a good architecture to handle the i/o between the web server app and the calculation that's running? The web server app will be Python/Pylons. In addition to starting a job, and getting updates and results, the user may cancel a job that has been started.

    Any ideas on any of these topics would be greatly appreciated. I'm pretty sure I could create my own solutions from scratch, but if there are best-practices to follow, I'd rather do that!

    Thanks for any ideas!

  2. #2
    Join Date
    Aug 2004
    First thought that came to my mind (so it's not necessarily the best one)

    1. Make use of JavaScript AJAX calls to launch the process (so you never have to actually leave the web page), then to also periodically send requests to check on the status and get the results when it's done.

    2. The actual calculation process could be kicked off as a background process on the server, using whatever functions .NET provides to do so. I'm a PHP guy, so I'd probably do something like:
    PHP Code:
    $args getArrayOfArgsFromGetOrPost(); // or whatever
    $statusFile uniqid(''true).".txt";
    exec('nohup /path/to/script.sh '.implode(' '$args).' > '.$statusFile.' &'); 
    3. I'd stick $statusFile in the session data, or pass it back to the AJAX caller so that it can pass it back when making its status checks.

    4. Those status calls would call a server-side script that would check the contents of that file for a completion indicator and results data (or an indicator where to get the results in the database, perhaps?), or error conditions if it fails, and perhaps status messages?
    "Well done....Consciousness to sarcasm in five seconds!" ~ Terry Pratchett, Night Watch

    How to Ask Questions the Smart Way (not affiliated with this site, but well worth reading)

    My Blog
    cwrBlog: simple, no-database PHP blogging framework

  3. #3
    Join Date
    Jan 2014
    I agree ''The actual calculation process could be kicked off as a background process on the server, using whatever functions''.....

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
HTML5 Development Center



X vBulletin 4.2.2 Debug Information

  • Page Generation 0.09114 seconds
  • Memory Usage 2,943KB
  • Queries Executed 13 (?)
More Information
Template Usage (30):
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_global_above_footer
  • (1)ad_global_below_navbar
  • (1)ad_global_header1
  • (1)ad_global_header2
  • (1)ad_navbar_below
  • (1)bbcode_php
  • (1)footer
  • (1)forumjump
  • (1)forumrules
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (1)headinclude_bottom
  • (3)memberaction_dropdown
  • (1)navbar
  • (4)navbar_link
  • (1)navbar_moderation
  • (1)navbar_noticebit
  • (1)navbar_tabs
  • (2)option
  • (3)postbit
  • (3)postbit_onlinestatus
  • (3)postbit_wrapper
  • (1)showthread_list
  • (1)spacer_close
  • (1)spacer_open
  • (1)tagbit_wrapper 

Phrase Groups Available (6):
  • global
  • inlinemod
  • postbit
  • posting
  • reputationlevel
  • showthread
Included Files (27):
  • ./showthread.php
  • ./global.php
  • ./includes/class_bootstrap.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/functions_navigation.php
  • ./includes/class_friendly_url.php
  • ./includes/class_hook.php
  • ./includes/class_bootstrap_framework.php
  • ./vb/vb.php
  • ./vb/phrase.php
  • ./includes/functions_facebook.php
  • ./includes/functions_calendar.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_notice.php
  • ./includes/functions_threadedmode.php
  • ./packages/vbattach/attach.php
  • ./vb/types.php
  • ./vb/cache.php
  • ./vb/cache/db.php
  • ./vb/cache/observer/db.php
  • ./vb/cache/observer.php 

Hooks Called (71):
  • init_startup
  • friendlyurl_resolve_class
  • init_startup_session_setup_start
  • database_pre_fetch_array
  • database_post_fetch_array
  • init_startup_session_setup_complete
  • global_bootstrap_init_start
  • global_bootstrap_init_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • load_show_variables
  • load_forum_show_variables
  • global_state_check
  • global_bootstrap_complete
  • global_start
  • style_fetch
  • global_setup_complete
  • showthread_start
  • showthread_getinfo
  • strip_bbcode
  • friendlyurl_clean_fragment
  • friendlyurl_geturl
  • forumjump
  • cache_templates
  • cache_templates_process
  • template_register_var
  • template_render_output
  • fetch_template_start
  • fetch_template_complete
  • parse_templates
  • fetch_musername
  • notices_check_start
  • notices_noticebit
  • process_templates_complete
  • friendlyurl_redirect_canonical
  • showthread_post_start
  • showthread_query_postids_threaded
  • showthread_threaded_construct_link
  • showthread_query
  • bbcode_fetch_tags
  • bbcode_create
  • showthread_postbit_create
  • postbit_factory
  • postbit_display_start
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • memberaction_dropdown
  • tag_fetchbit_complete
  • forumrules
  • navbits
  • navbits_complete
  • build_navigation_data
  • build_navigation_array
  • check_navigation_permission
  • process_navigation_links_start
  • process_navigation_links_complete
  • set_navigation_menu_element
  • build_navigation_menudata
  • build_navigation_listdata
  • build_navigation_list
  • set_navigation_tab_main
  • set_navigation_tab_fallback
  • navigation_tab_complete
  • fb_like_button
  • showthread_complete
  • page_templates