www.webdeveloper.com
Page 2 of 4 FirstFirst 1234 LastLast
Results 16 to 30 of 55

Thread: Save an HTML table as CSV

  1. #16
    Join Date
    Mar 2009
    Posts
    467
    http://192.168.69.154

    I think your problem is here. The 192.168 indicates a local private IP address. From Wikipedia: (private networks)

    These addresses are characterized as private because they are not globally delegated, meaning that they are not allocated to any specific organization, and IP packets addressed with them cannot be transmitted through the public Internet. Anyone may use these addresses without approval from a regional Internet registry (RIR). If such a private network needs to connect to the Internet, it must use either a network address translator (NAT) gateway, or a proxy server.

  2. #17
    Join Date
    Mar 2005
    Location
    Behind you...
    Posts
    862
    My first question is about the get_web_page() function. I don't think that's a standard PHP function so I have to ask, do you have the actual function declared somewhere? Every reference to that I found is always a custom function written that simply runs CURL on a given URL. I only ask because if you don't actually have the function delcared somewhere you will get a 500 error (having nothing to do with the URL).

    I'm actually not familiar enough with CURL (or file_get_contents() for that matter) to know for sure, but generally if you try to connect to a page that isn't there you shouldn't be getting a 500 error. A 500 error indicates the script on the server encountered an error. The 400 range of errors are related to pages not being found (non-existent, moved or forbidden access). So I almost wonder if the issue isn't connecting to the correct address at all.

    One thing I like to do when I have absolutely no clue where an issue in my code is, would be to take out everything except for say... 1 line. Run the script and see if any errors occur. Then I slowly add back in lines (not always 1 line at a time as some code is dependent on other things) and rerun the script. This more or less allows you to see what line (or set of lines) is causing the error. So you might pull everything after $ch = curl_init(); and see if things work. Then add in some of the curl_setopt() lines to see if anything is wrong with those. Then try with the curl_exec() line added in, and so on.
    "Given billions of tries, could a spilled bottle of ink ever fall into the words of Shakespeare?"

  3. #18
    Join Date
    Mar 2014
    Location
    Corpus Christi, TX
    Posts
    82
    I dumbed this down a bit, and used a simple INCLUDE command.

    PHP Code:
    <?php 
    error_reporting
    (E_ALL); 
    ini_set'display_errors','1');
    include(
    'http://192.168.69.154/stalist.cgi');

    ?>
    Gives me the error:

    Code:
    Warning: include(http://192.168.69.154/stalist.cgi): failed to open stream: HTTP request failed! HTTP/1.0 404 Not Found in C:\Users\Michelle\Desktop\mowes_portable\www\www\getpagetest.php on line 4 Warning: include(): Failed opening 'http://192.168.69.154/stalist.cgi' for inclusion (include_path='.;C:\Users\Michelle\Desktop\mowes_portable\php5') in C:\Users\Michelle\Desktop\mowes_portable\www\www\getpagetest.php on line 4
    However, changing the code to:

    PHP Code:
    <?php 
    error_reporting
    (E_ALL); 
    ini_set'display_errors','1');
    include(
    'http://google.com');

    ?>
    Successfully loads google.com. This tells me I might have some error in my config.ini files. I confirmed fopen is enabled, Curl is installed and enabled, and PHP is the server.

    What else might be the problem?

  4. #19
    Join Date
    Mar 2014
    Location
    Corpus Christi, TX
    Posts
    82
    Quote Originally Posted by Tcobb View Post
    http://192.168.69.154

    I think your problem is here. The 192.168 indicates a local private IP address.
    I agree with you completely, and confirms my findings. I am searching various resources on how to connect to a local IP. Is this even possible? Do you have a recommendation where I can find some help on this?

    As a last resort, I can write a VBS to log in and see the table I need, but how do I extract it from that point?

  5. #20
    Join Date
    Mar 2005
    Location
    Behind you...
    Posts
    862
    Hmm, I mentioned a potential issue with the internal IP on your original 'Auto-Login' topic, but I had assumed you found it wasn't an issue since the topic moved on.

    If it is your problem then you will have to use the external IP, and the router would need to be forwarding the proper ports to the internal IP to which you are trying to connect. I don't know how well that plays into your case since this seems to include a number of devices/laptops, but if the target address is always going to be the same then setting up port forwarding should be possible. Essentially in the settings of your router (for the network on which this server is located) you would be forwarding port 80 to your internal IP address (192.168.69.154). Then you would just need to get the external IP (googling "what's my IP" while on that same network will give you your answer) and you could use this in your CURL.

    As for extracting your information, assuming you aren't going to use any javascript conversion methods, you can use PHP's DOMDocument to extract the information once you have the HTML of the page you need. If you end up using VBS, I'm not really sure how to bridge this with either the javascript converters provided earlier or DOMDocument.


    [EDIT]
    I don't know why, but all this time I neglected a seemingly obvious potential solution that I'd certainly be more familiar with, AJAX. You could, in theory, use AJAX in the same manor that you are using CURL in PHP. However I guess the first thing that needs to be addressed is the location of your data (which is written to a table after logging in) and the location of the device on which you are attempting to log in from. If these two things are on the same network (same router, VPN, etc.) then AJAX can certainly solve your problems. If they are not on the same network then we venture back to my comments about port forwarding, as this will be the only way to access a network resource remotely. Though I have to assume since you seem to be able to manually go to the page and log in, there has to be some sort of network-related connection.
    Last edited by Sup3rkirby; 04-08-2014 at 01:14 PM.
    "Given billions of tries, could a spilled bottle of ink ever fall into the words of Shakespeare?"

  6. #21
    Join Date
    Mar 2014
    Location
    Corpus Christi, TX
    Posts
    82
    I set up port forwarding in the router. 192.168.69.154 forwards to port 80. The script I am using is below. The script seems to hang while "Connecting to Localhost." Are there some other settings I am missing? I am getting the following errors:
    Code:
    Warning: include(http://71.40.200.85) [function.include]: failed to open stream: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. in C:\xampplite\htdocs\index.php on line 23
    
    Warning: include() [function.include]: Failed opening 'http://71.40.200.85' for inclusion (include_path='.;\xampplite\php\PEAR') in C:\xampplite\htdocs\index.php on line 23
    
    Fatal error: Maximum execution time of 60 seconds exceeded in C:\xampplite\htdocs\index.php on line 24
    PHP Code:
    <?php  
    error_reporting
    (E_ALL);  
    ini_set'display_errors','1'); 
    $url='http://71.40.200.85'
     
    $ch curl_init(); 
        
    $user_agent='Mozilla/5.0 (Windows NT 6.1; rv:8.0) Gecko/20100101 Firefox/8.0';
        
    curl_setopt ($chCURLOPT_URL$url); 
        
    curl_setopt ($chCURLOPT_USERAGENT$user_agent); 
        
    curl_setopt ($chCURLOPT_HEADER0); 
        
    curl_setopt ($chCURLOPT_FOLLOWLOCATION1); 
        
    curl_setopt ($chCURLOPT_RETURNTRANSFER1); 
        
    curl_setopt ($ch,CURLOPT_CONNECTTIMEOUT,120); 
        
    curl_setopt ($ch,CURLOPT_TIMEOUT,120); 
        
    curl_setopt ($ch,CURLOPT_MAXREDIRS,10); 
        
    curl_setopt ($ch,CURLOPT_COOKIEFILE,"cookie.txt"); 
        
    curl_setopt ($ch,CURLOPT_COOKIEJAR,"cookie.txt"); 
        echo 
    curl_exec ($ch); 
    function 
    get_web_page$url ){

    curl_close($ch);  
    echo 
    $ch;  
         }
         include(
    "http://71.40.200.85");
        
    ?>

  7. #22
    Join Date
    Mar 2005
    Location
    Behind you...
    Posts
    862
    Can you verify that the port forwarding works outside of the script (eg. simply navigating to the IP via browser). I don't know all of the specifics as to how you have things set up but I cannot navigate to that URL which would imply the script isn't failing due to errors really, it's just that you really cannot connect to that address.

    [EDIT]
    I feel I should also ask, if you take away all of the automatic scripting, how were you able to view this table with information manually? Supposedly you went to a page, logged in and then once logged in were given access to view the table printed in HTML? That being said, and if true, what URL did you use in your browser, and were you on the same network as this server (which contains the necessary data)? Or were you at a remote location on a separate network? I'm just trying to understand if this issue is simpler than it seems to be.
    Last edited by Sup3rkirby; 04-08-2014 at 02:08 PM.
    "Given billions of tries, could a spilled bottle of ink ever fall into the words of Shakespeare?"

  8. #23
    Join Date
    Mar 2014
    Location
    Corpus Christi, TX
    Posts
    82
    No, I cannot connect to the URL outside the connected network. Here is how I have set up the port forward.
    Code:
    Forwards
    Application	  Protocol	Source Net	Port from	IP Address	     Port to	  Enable
    M5                Both                              80          192.168.69.154    80        yes

    Using XAMPP on the local machine, are there any settings I need to specify?

    If I type in the IP address of the device with the extension of the page containing the data: (192.168.69.154/stalist.cgi) I am taken to a login page. Once logged in, the only information on the page is a table, including a header row, and a submit button.

  9. #24
    Join Date
    Mar 2014
    Location
    Corpus Christi, TX
    Posts
    82
    The other issue we are going to have is that the other IPs are going to be different for each laptop. The IP of 192.168.69.154 will be the same, but the latptop, and therefore the server, will be different for each machine.

  10. #25
    Join Date
    Mar 2014
    Location
    Corpus Christi, TX
    Posts
    82
    Correction - the button on the page, once logged in, is a REFRESH button, not a SUBMIT button.

  11. #26
    Join Date
    Mar 2005
    Location
    Behind you...
    Posts
    862
    Let's take a couple of steps back and realign our paths if we hope to achieve any success here. You have data that requires login to view. You want to take this data and store in a 'master database' on a separate server.

    We've been trying to setup a script that connects directly to this data source (the local 192.168.69.154 address?). However you say this data source will be changing, so that even with port forwarding (in an earlier example you'd be forwarding 71.40.200.85 to 192.168.69.154), you would at best have a working system on one laptop and not all?


    If everything above is correct then this is really running into a lot of non-coding related issues. Essentially if you have a bunch of local/private data sources and you want to migrate the data elsewhere, the infrastructure of all of these data sources needs to be set up in a way so that you can achieve the proper connectivity first.

    But then of course I still don't think I follow the entire situation here. This PHP script you've been trying to run (that automatically logs in and pulls the data), where is it actually located? On the same (local) server as the data source? Or is it on a public/global server somewhere and it's trying to remotely connect to each of these laptops?

    Regardless I see a worst case scenario being you have to manually log in, copy the table, paste it into a javascript converter which will convert and push to PHP, which will save it in your database. Obviously the best case scenario here is we work out an automated way of doing this, saving you time and effort overall.
    "Given billions of tries, could a spilled bottle of ink ever fall into the words of Shakespeare?"

  12. #27
    Join Date
    Mar 2014
    Location
    Corpus Christi, TX
    Posts
    82
    I agree, this project has come a bit convoluted. I appreciate your help in sorting it all out.

    Our company leases communication equipment (telephones) to large work areas. Each telephone is connected to an access point. All access points (20 or so) are connected wirelessly to one central device. This device is connected to a router which is also connected to a small laptop.

    The central device reads all the data from the stations and is located at the 192.168.69.154 address. I need to log in to that device and put that data into a central MySQL database that I have set up at one central location. The 69.154 address will not change for any of those central devices. However, the IP address for the laptop will change quite frequently. I might be able to set up a VPN through LogMeIn to make those IPs dynamic.

    Automation is almost a necessity as we have 150 or so work areas, and I have been asked to automate this so an hourly report can be generated for any specifications outside a set limit.

  13. #28
    Join Date
    Mar 2014
    Location
    Corpus Christi, TX
    Posts
    82
    Correction - Use the VPN to make IPs STATIC.... MySQL database is sitting next to me in my office.

  14. #29
    Join Date
    Mar 2005
    Location
    Behind you...
    Posts
    862
    Quote Originally Posted by TW79 View Post
    ...
    Our company leases communication equipment (telephones) to large work areas. Each telephone is connected to an access point. All access points (20 or so) are connected wirelessly to one central device. This device is connected to a router which is also connected to a small laptop.
    ...
    I'm still not sure I'm on the same page so I just want to make sure I understand this structure.
    You have a bunch of individual devices(A). These devices all connect to one central device(B) (which is connected to a router and laptop).

    Question) Now, without any of this fancy scripting stuff, how would you manually connect to get the data for an individual device(A)?
    I ask in regard to the specifics. I know there is a page you can visit and then log in to. Then you are given a table with some data and a refresh button. But the pieces I'm missing are throwing me off. Is it that you connect a laptop to this individual device(A) and then go to that IP address/page and log in? Or is it that you hop on the laptop that is already connected to the central device(B) and log in using credentials per each individual device(A)?
    "Given billions of tries, could a spilled bottle of ink ever fall into the words of Shakespeare?"

  15. #30
    Join Date
    Mar 2009
    Posts
    467
    This is all getting rather convoluted. Perhaps you're looking at it from the wrong end. Rather than trying to access the individual units from a myriad of possible configurations, why don't you have the individual machines report at intervals to a central location? If each machine has the ability to transmit data through the internet this seems like it would be a much cleaner approach.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
HTML5 Development Center



Recent Articles