www.webdeveloper.com
Results 1 to 14 of 14

Thread: build a Java application to log into a website

  1. #1
    Join Date
    Feb 2006
    Posts
    5

    build a Java application to log into a website

    Hello All,

    Iím trying to programmatically log into a web site.
    In-other-words, I would like to build an application/applet/script (whatever) so that when itís executed it uses my username & password to log into a web server and then allow me to perform searches and display the results (currently this is possible by loggin through a web browser).

    I have experience with c++, Java, perl and a little javascript. I wouldn't mind trying C# if it will get me to where I want to go.

    I have tried the Java.net package (URL & sockets) with no success.

    ray326 suggested I try java.net.HttpURLConnection and place a post here for help. I now have some java code running using HttpURLConnection but I still can't login through my java application.

    I'm also playing with PHP right now to see if I can get what I need.

    Please understand I'm trying to not put any code on the server, at this point Iím trying to work through the ďnormalĒ webtop interface (not around it).


    Thank you for any help or suggestions.

    p.s. please feel free to move this thread if it's in the wrong area.

  2. #2
    Join Date
    Dec 2002
    Location
    Calgary, Canada
    Posts
    6,137
    well you are not providing enough info. What happens when you open a connection to the webpage and try to login?

    See essentially java does not know whats on that page and have no idea if you want to login. What you will need to do is open the connection. read the page and submit to the page that actually logs in with required field with thier name and values so that server can performa login and when it returns the logged in page then you will have parse it again and...etc...etc
    Cheers

    Khalid

    Message Posting Guidelines In These Forums, Please read these before posting any question.
    Web site: webapplikations.com
    Web Resources Page:Web Resources

  3. #3
    Join Date
    Sep 2005
    Location
    Chicago, Il
    Posts
    144
    There are ALOT of ways to do what you are asking for. Your description is kind of vague though, so it is hard to give concrete examples.

    From what you have said though, it really sounds like a web service would be the best choice. Do you have access to the website or is it something that is black box? I would look to see if the site offers some sort of SOAP interfaces for you to get your info from, or maybe they offer an XML page. Parsing HTML for info would be a nightmare, especially if the page changes frequently.
    Last edited by BigDog; 02-24-2006 at 10:15 AM.

  4. #4
    Join Date
    Feb 2006
    Posts
    5
    First, thanks for your time.

    Currently there are 2 main layers to this system, the top layer is a proprietary commercial web interface (the black box) that accesses a SQL database underneath. Iím trying to add a third layer to the top for anonymous users. I could write code to directly access the database. But I want to go through the proprietary commercial web interface if possible.

    Quote Originally Posted by Khalid Ali
    What happens when you open a connection to the webpage and try to login?
    If Iím using a web browser I type in the URL, then enter my username and password into the fields and press the submit button, it then logs me in (it installs a cookie so it knows that Iím logged in for this session) I can now perform searches for various documents and then open & read those documents. What Iím trying to do is provide a way for an anonymous user to get in and perform the same searches but instead of actually opening the documents, the anonymous user would only see a brief description of the document(s) and then would be required to contact the author of the document(s) for permission.

    I chose Java because I have some background with it, but Iíll try any language if it will help me solve my problem.

    I have several versions of my Java code:

    1) this version uses a socket I can open a connection but then I get no response back from the server. I used the below code:

    socket = new Socket("url.of.the.server", 80);// URL of the login page gives errors
    out = new PrintWriter(socket.getOutputStream(), true);
    in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
    out.write("various statements");
    BufferedReader stdIn = new BufferedReader(new nputStreamReader(System.in));
    String userInput;
    while ((userInput = in.readLine()) != null) {
    System.out.println(userInput);
    }
    I think the reason I get no response back is because there is no code listening to my socket at the other end.

    2) this version I use java URL:

    URL u = new URL("URL of the login page");
    URLConnection connection = u.openConnection();
    connection.setDoOutput(true);

    with this version I get the login page source code back from the server (which is good) But I donít know how to go beyond this to actually send my username & password to server so it can login.

    3) this version I use HttpURLConnection:

    HttpURLConnection huc = (HttpURLConnection)u.openConnection();
    huc.setDoOutput(true);
    huc.setRequestMethod("POST");

    with this version I can get response codes (e.g. 200 ok) and the login page source code as with version 2.

    Quote Originally Posted by Khalid Ali
    What you will need to do is open the connection. read the page and submit to the page that actually logs in with required field with thier name and values so that server can performa login and when it returns the logged in page then you will have parse it again and...etc...etc
    Yes, the problem is that I am not sure how to submit my login information to the "black box" web software through my java program.

    Quote Originally Posted by BigDog
    Parsing HTML for info would be a nightmare, especially if the page changes frequently.
    The page will change depending on the search queries.

    sorry for being so long winded....
    Last edited by ruba; 02-24-2006 at 10:23 PM.

  5. #5
    Join Date
    Dec 2002
    Location
    Calgary, Canada
    Posts
    6,137
    its not a good solution however to answer ur question your connection url don't have to be what a user sees as a login page, as far as the webserver is concern it requires and user name (whatever name u have for that field) and a password field.
    so you will have to append these to the url
    something line
    "ur.server.com?userName=khalid&password=opensesame"
    now in some cases depending upon the server they do not respond back unless the querying tool tells them its an http complinat browser, so to be safe alwasy set the USER-AGENT property for the connection to a valid user-agent string
    Cheers

    Khalid

    Message Posting Guidelines In These Forums, Please read these before posting any question.
    Web site: webapplikations.com
    Web Resources Page:Web Resources

  6. #6
    Join Date
    Feb 2006
    Posts
    5
    Quote Originally Posted by Khalid Ali
    "ur.server.com?userName=khalid&password=opensesame"
    I've tried that. I've even tried several different variable names(password, pwd, passwd, etc). I've tried it in various ways with all three versions (from my previous post). What I get back is the source code for the login page plus some javascript. When I paste the exact same url string into a web browser I get the same thing back as my java program and I notice that my username is properly entered into the field but the password field is empty. If I hit enter I again I get a message: "The password applet is not fully loaded yet, please try again in a few seconds."

    I've also looked at the packets between my machine and the server using Etherreal. I've compared the packets for logging in from a web browser versus my java program. In the packet from the java program I can see the password, but in the web browser the password is encrypted. Obviously, I thought about copying the packet from the web browser and sending that in my java program, but I've noticed the values for the password in the login packet changes each time I login through the web browser.

    Is there some place that explains what communication occurs between a web browser and a server when logging into a website with a password (i.e. yahoo, hotmail, etc...)?

    It seems my hitch is the password. Other than the method discussed above how can I enter the password into the password field of the form code and submit the document....in the java program, not the web browser (web browser is easy )? Or is there something I'm missing from the url method mentioned above.
    Last edited by ruba; 02-25-2006 at 08:35 PM.

  7. #7
    Join Date
    Dec 2002
    Location
    Calgary, Canada
    Posts
    6,137
    if you like, you can send me the login pages html and the url where u need to login..I'll see whats going on....and ofcourse if there is a temp user with password thats even better...let me know...
    Cheers

    Khalid

    Message Posting Guidelines In These Forums, Please read these before posting any question.
    Web site: webapplikations.com
    Web Resources Page:Web Resources

  8. #8
    Join Date
    Mar 2008
    Posts
    2
    I have the same problem .... i am wondering if you could just point me out what i am doing so wrong.

    All i want to do is login, get the data from the site and analyze them in the way that i want.

    But login in is just a pain.

    So login form is :
    Code:
    <form name="login" action="login.php" method="post" target="_top"><input type="hidden" name="PHPSESSID" value="166c592c10598cfc0e8d2be5226cf33f" />
              E-mail: <input type="text"     size="27" maxlength="65" class="custom" name="email" title="Your E-Mail"   value="Your E-Mail"   onfocus="javascript:handleFocus(this);"><br><br>
            Password: <input type="password" size="27" maxlength="20" class="custom" name="pass"  title="Your Password" value="Your Password" onfocus="javascript:handleFocus(this);"><br><br>
            <input type="image" src="img/login2.jpg?PHPSESSID=166c592c10598cfc0e8d2be5226cf33f" value="Submit" name="submit" style="margin-right:60px;">
            <input type="hidden" name="submit" value="Submit">
        </form>
    And i am trying the following
    Code:
     public void tryLog() {
            try {
                // setup url connection. use POST to send forms data
                //url = new URL(HOUSE);
               
                urlConn = (HttpURLConnection) url.openConnection();
    
                urlConn.setRequestMethod("POST");
                urlConn.setDoInput(true);
                urlConn.setDoOutput(true);
                urlConn.setUseCaches(false);
                urlConn.setAllowUserInteraction(true);
                urlConn.setFollowRedirects(true);
                urlConn.setInstanceFollowRedirects(true);
                urlConn.setRequestProperty("content-type-Type", "text/html; charset=iso-8859-1");
                out = new DataOutputStream(urlConn.getOutputStream());
              //  String content = "email=" + URLEncoder.encode(EMAIL_ID, "ISO-8859-1") + "&pass=" + URLEncoder.encode(PASS_WORD, "ISO-8859-1") + "&action=login";
                 String content = "email=" + EMAIL_ID + "&pass=" + PASS_WORD + "&action=login.php";
                System.out.println(content + "\n" + "sending form to HTTP server ...");
                out.writeBytes(content);
               System.out.println(urlConn.getPermission());
                setCookie(urlConn.getHeaderField("Set-Cookie"));
                //System.out.println("sessionid: " + session_id);
    
                out.flush();
                out.close();
                // get input connection
                in = new BufferedReader(new InputStreamReader(urlConn.getInputStream()));
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    I am trying to login with this and then get the website content
    Code:
     public String getString() {
            String all = "";
            try {
                BufferedReader in = new BufferedReader(
                        new InputStreamReader(url.openStream()));
                String inputLine;
    
    
                while ((inputLine = in.readLine()) != null) {
                    all += inputLine + "\n";
                // System.out.println(inputLine);
                }
    
                in.close();
    
            } catch (MalformedURLException e) {     // new URL() failed
                System.out.println(e);
            } catch (IOException e) {               // openConnection() failed
                System.out.println(e);
            }
            return all;
        }
    And to read the content of the site


    Anyway, what ever i try all i get is the login page.
    How can i by pass this and actually get the first page when i login.

  9. #9
    Join Date
    Aug 2005
    Location
    The Garden State
    Posts
    5,634
    Have you thought of using Apache's Commons Http Client? It makes all of this a lot easier. What about JMeter?

  10. #10
    Join Date
    Aug 2008
    Posts
    1

    N B

    How Ali,
    I have the solution of problem.If you want into website with Java Application,you can the HttpClient API of Java.
    You can obtain the tutorial at this link:http://hc.apache.org/httpclient-3.x/tutorial.html

    An example is at this link:http://svn.apache.org/viewvc/httpcom...o.java?view=co
    This example shows how pass parameter to formula and submit it allowing you to log into website.
    Please excuse me i don't speak very well english.

  11. #11
    Join Date
    Nov 2012
    Posts
    2
    What version of HttpClient is use for the example that you put?
    I used httpClient 3.0 and 3.0.1 but throw a exception:

    java.lang.NoClassDefFoundError
    java.lang.ClassNotFoundException

  12. #12
    Join Date
    Nov 2012
    Posts
    2
    Quote Originally Posted by LuisMR View Post
    What version of HttpClient is use for the example that you put?
    I used httpClient 3.0 and 3.0.1 but throw a exception:

    java.lang.NoClassDefFoundError
    java.lang.ClassNotFoundException
    Is this related to the version of HttpClient?

  13. #13
    Join Date
    Nov 2012
    Posts
    2
    I think maybe I have a newer version and java not found the necessary classes for this reason.Or just my CLASSPATH is wrong.

  14. #14
    Join Date
    Nov 2012
    Posts
    5
    Yeah, maybe. Can you tell us the exact reason when you figure it out. Looking forward to knowing.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
HTML5 Development Center



Recent Articles