How to make a script run for several hours...

[cusimar9]

I have a search script that I want to run, the problem is that it takes several hours to complete

I can 'save' its progress, but I have to restart the script every time it times out.

I've added this at the top of the page:


Server.ScriptTimeout = 14400 // 4 hours


But the page times out much before this.

Is there a way I can force the page not to time out? Or re-submit the search when it does time out?

[lmf232s]

Depending on what you have in the script i would look at converting the code to a .vbs file and then scheduling a windows task to run that bad boy.

Can i ask what it is that your searching where you need to have a web page run for 4 hours?

[cusimar9]

How do you mean 'converting to .vbs'? Can I then run that from a local machine rather than a webserver?

The script recursively searches websites to find trends in their content

[lmf232s]

a .vbs file is nothing more than renaming a text file w/ the extension .vbs.
its a VBScript File.

You can do all kinds of stuff in a .vbs file. You can communicate w/ the database and write and retrieve values, send emails, etc. Pretty much anything you do in a .asp file you can do in a .vbs file.

Now im not sure how your scrapping your pages for you content so this might not be the solution for you. If you have access to .net you could build the same thing in .net have run it as a desktop appliction but thats a beast of its own.

Again im not sure if this is the solution for you. Im not sure how your scrapping the pages for the content, how your traversing the web sites, what your doing w/ the data once you get it, etc..

Here is a sample of a sample .vbs file that i created a while ago. It looks like it was the first draft but the idea of the script was that when people in the company would open say solitar or windows media player, etc, it would call this script and then it would log it to a database and then send an email to the IT Dept. Agin this is partial working but the point is you can see how this same code could of been done in a web page.

When using a .vbs file you dont use <% %>. Not needed

'DECLARE VARIABLES
Dim sExecutable
Dim o
Dim colSystemEnvars
Dim colUserEnvVars
Dim objNetwork
Dim strConnection
Dim objConn
Dim objRS
Dim SQL
Dim body
Dim sApp

On Error Resume Next

'CREATE A BUNCH OF OBJECTS
set o = wscript.CreateObject("WScript.Shell")
Set colSystemEnvVars = objShell.Environment("System")
Set colUserEnvVars = objShell.Environment("User")
Set objNetwork = Wscript.CreateObject("Wscript.Network")


'GET THE COMPUTER NAME AND THE USERS NAME
samUser = objNetwork.UserName
computerName = objNetwork.ComputerName


'LETS FIRE OFF THE APPLICATION IN QUESTION
o.Exec sExecutable

'SAVE TO THE DATABASE
strConnection = "Provider=sqloledb;Data Source=?;Initial Catalog=?;USER ID=?;PASSWORD=?;"
set objConn = CreateObject("ADODB.Connection")
objConn.Open strConnection
Set objRS = CreateObject("ADODB.Recordset")

SQL = "INSERT INTO BUSTED (CUR_DATE, COMPUTER_NAME, USER_NAME, APPLICATION, APP_NAME) " & _
"VALUES ('" & NOW() & "', '" & computerName & "', '" & samUser & "', '" & sExecutable & "', '" & sAPP & "')"
set objRS = objConn.execute(SQL)


'CREATE BODY OF EMAIL
body = Now() & "<BR>"
body = body & samUser & "<BR>"
body = body & computerName & "<BR>"
body = body & sAPP & "<BR>"
body = body & sExecutable


'EMAIL START
Set objMessage = CreateObject("CDO.Message")
objMessage.Subject = "BUSTED WITH" & sAPP
objMessage.Sender = "AUTOALERT@IDINET.COM"
objMessage.To = "larryg@idinet.com"
objMessage.TextBody = body

'==This section provides the configuration information for the remote SMTP server.
objMessage.Configuration.Fields.Item ("http://schemas.microsoft.com/cdo/configuration/sendusing") = 2
'Name or IP of Remote SMTP Server
objMessage.Configuration.Fields.Item ("http://schemas.microsoft.com/cdo/configuration/smtpserver") = "IP ADDRESS"
'Server port (typically 25)
objMessage.Configuration.Fields.Item ("http://schemas.microsoft.com/cdo/configuration/smtpserverport") = 25
objMessage.Configuration.Fields.Update
'==End remote SMTP server configuration section==
objMessage.Send


'CLEAN UP ON ISIAL 10
set o = Nothing
set colSYstemEnvVars = Nothing
set colUserEnvVars = Nothing
set objNetwork = Nothing

[cusimar9]

I presume you can't have any html in it though?

How would you take input?

[lmf232s]

no you can have html but its created as a string.
Example

sHtml = "<Table><tr><td>TestMe</td></tr></table>"

How would you take Input?
What kind of input are we talking about. The html from the scrapping of the web page?

Depends on how your getting it and what you doing w/ it once you have it i guess.

[lmf232s]

cusimar,
I dont want to get you barking up the wrong tree. Im not sure what your code is doing and it appears that all is working just fine except that the web page is timing out. Based on that, im not sure if you can keep a web page running for 4 hours. I have never had the need to run a web page for more than a min or 2.

So again i just had a suggestion and not necessarly a solution. I have done something similar w/ a .net desttop application were i scrapped a single web site for stats but thats about as deep as that went.

So dont let me get you off track. Lets see if its possible for a web page to run for 4 hours.

[cusimar9]

I'm getting there... but how do you 'display' the contents of sHtml?

Though this isn't the solution I was looking for I can see the advantages of running it on a desktop so I'd like to get this working as well

[cusimar9]

I think I've figured it out

That's really cool, thanks! :)

[lmf232s]

cusimar9,
Please do tell. Ill take it you created a .vbs file w/ your script?

From here you then set up a windows scheduled task to run it?

Just curious as to see what you ened up doing?

[cusimar9]

Well hold onto your horses I've only just figured out how to print the output to a browser window lol

I'll let you know if it works ;)

[Ubik]

ok, I have a couple of similar problems, and a couple of solutions:

First, you are going to need to set the page URL as the home page for the browser of your choice, on the machine you are going to run this from. Set up a scheduled task for this machine to run at whatever intervals to open the browser. The browser will then open the home page, your script, and begin processing. So now, you have basically a windows CRON job, (as good as it gets with Windows, I am sorry to say).

Next, have your massive script process 'chunks' of your overall goal. Once you break it up into some kind of manageable chunks, which may be hard in itself, you can process the page by doing something like this:



dttmProcessStarted=now()

while dateadd("s", dttmProcessStarted, now()) <=10
'process a chunk that takes less than 10 seconds
if chunkprocessing=complete then
exit while
end if
wend

if chunkprocessing<>complete then
response.redirect ("backtothispage")
end if


I hope that helps.