www.webdeveloper.com
Results 1 to 14 of 14

Thread: Ajax/dom tables VERY SLOW with 2000 records?

Hybrid View

  1. #1
    Join Date
    Jul 2008
    Posts
    92

    Ajax/dom tables VERY SLOW with 2000 records?

    I'm trying to create an ajax tables containing 12 columns and 2000 rows. As I've never done this before, I'm borrowing scripts from around the net and altering them to fit my needs (as I learn from example). So far, every script I have tried have been much slower than reloading the page. In fact, Safari pops up several times telling me the script is slow and I should stop trying to process it.

    I'm curious if Ajax/dom tables are the best way to go with 2k+ records? If so, can someone suggest a method to speed up their usage? Right now, it'd be much faster to reload the page w/ php $_GET's.

    The data in the tables is coming from a csv, which are dumped into php arrays, which then populate the dynamic table.

    I need a functioning sort by column and search by column as well.

    Thanks.

  2. #2
    Join Date
    Feb 2003
    Location
    Michigan, USA
    Posts
    5,774
    After populating the data into PHP arrays, is PHP creating the HTML for the table, or is that information being sent back to the browser and then JavaScript is adjusting the TABLE using document object model methods?

  3. #3
    Join Date
    Aug 2007
    Posts
    3,767
    It depends on the efficiency of the JavaScript. It should be done in order, creating the table, and then making it sortable. JavaScript is slow, but it can be make faster.

    That's a lot of information to be downloading anyway, so it'll take as long for the page to download the table as it will for the page to reload. If you separated into two or more requests, you could probably speed it up too.

  4. #4
    Join Date
    Jul 2008
    Posts
    92
    toi:

    I believe the php is creating the html for the tables. Is this the wrong way to go?

    Here's how I populate the table:
    Code:
                        <tbody>
                            <?php
                                    $i=0;
                                    foreach ($o1 as $key => $val) {
                                    echo "<tr id=".$i++.">";
                                            echo "<td class='alignright'>".$i."</td>";
                                            echo "<td><small>".strtoupper($d[$key])."</td>";
                                            echo "<td><small>".strtoupper($o2[$key])."</td>";
                                            echo "<td><small>".strtoupper($o1[$key])."</td>";
                                            echo "<td><small><a href=../file.php?action=run&src=STR&scenario=HF&xsec_type=C&isource=database&osource=database&mode=test&station=".$id[$key].">".$id[$key]."</td";
                                            echo "<td><small>".$o9[$key]."</td>";
                                            echo "<td><small>".$o3[$key]."</td>";
                                            echo "<td><small>".$o4[$key]."</td>";
                                            echo "<td><small>".$o8[$key]."</td>";
                                            echo "<td><small>".$o5[$key]."</td>";
                                            echo "<td><small>".$lat[$key]."</td>";
                                            echo "<td><small>".$lon[$key]."</td>";
                                    } ?>
                                    </tr>
                             </tbody>
    I know that the js is what is causing the slow downs. It must not be efficient (but I'm too new to this whole field to know), since the vanilla table itself loads very fast in php.

    For reference, the js I'm attempting to use is here: http://www.phatfusion.net/sortabletable/

    Like I said, I'm using it as a learning tool to understand how asynchronous requests work.
    Last edited by mith36; 07-15-2008 at 01:07 PM.

  5. #5
    Join Date
    Aug 2007
    Posts
    3,767
    The first thing is that ids cannot begin with a number, so that code is invalid. That actually can be improved a bit (won't be noticed, but it is more efficient).
    Code:
    <tbody>
    <?php
    $i=0;
    $str = '';
    foreach ($o1 as $key => $val) {
        $str.='<tr id='.$i++.'>';
        $str.='<td class="alignright">'.$i.'</td>'
        $str.='<td><small>'.strtoupper($d[$key]).'</td>';
        $str.="<td><small>".strtoupper($o2[$key])."</td>";
        $str.="<td><small>".strtoupper($o1[$key])."</td>";
        $str.="<td><small><a href=../file.php?action=run&src=STR&scenario=HF&xsec_type=C&isource=database&osource=database&mode=test&station=".$id[$key].">".$id[$key]."</td";
        $str.="<td><small>".$o9[$key]."</td>";
        $str.="<td><small>".$o3[$key]."</td>";
        $str.="<td><small>".$o4[$key]."</td>";
        $str.="<td><small>".$o8[$key]."</td>";
        $str.="<td><small>".$o5[$key]."</td>";
        $str.="<td><small>".$lat[$key]."</td>";
        $str.="<td><small>".$lon[$key]."</td>";
    }
    echo $str;
    ?>
    </tr>
    </tbody>
    What's your AJAX code?
    Last edited by Declan1991; 07-15-2008 at 01:12 PM.

  6. #6
    Join Date
    Jul 2008
    Posts
    92
    As I said, I'm trying to learn how to do this myself, but I do so through examples. As such I borrowed this code to learn from:

    Code:
    
    /**************************************************************
    
    	Script		: Sortable Table
    	Version		: 1.4
    	Authors		: Samuel Birch
    	Desc			: Sorts and filters table elements
    	Licence		: Open Source MIT Licence
    
    **************************************************************/
    
    var sortableTable = new Class({
    							  
    	getOptions: function(){
    		return {
    			overCls: false,
    			onClick: false,
    			sortOn: 0,
    			sortBy: 'ASC',
    			filterHide: true,
    			filterHideCls: 'hide',
    			filterSelectedCls: 'selected'
    		};
    	},
    
    	initialize: function(table, options){
    		this.setOptions(this.getOptions(), options);
    		this.table = $(table);
    		this.tHead = this.table.getElement('thead');
    		this.tBody = this.table.getElement('tbody');
    		this.tFoot = this.table.getElement('tfoot');
    		this.elements = this.tBody.getElements('tr');
    		this.filtered = false;
    		
    		/*for(i=0;i<10;i++){
    			this.elements.clone().injectInside(this.tBody);
    		}
    		this.elements = this.tBody.getElements('tr');*/
    		
    		this.elements.each(function(el,i){
    			if(this.options.overCls){
    				el.addEvent('mouseover', function(){
    					el.addClass(options.overCls);
    				}, this);
    				el.addEvent('mouseout', function(){
    					el.removeClass(options.overCls);
    				});
    			}
    			if(this.options.onClick){
    				el.addEvent('click', options.onClick);
    			}
    		}, this);
    		
    		//setup header
    		this.tHead.getElements('th').each(function(el,i){
    			if(el.axis){
    				el.addEvent('click', this.sort.bind(this,i));
    				el.addEvent('mouseover', function(){
    					el.addClass('tableHeaderOver');
    				});
    				el.addEvent('mouseout', function(){
    					el.removeClass('tableHeaderOver');
    				});
    				el.getdate = function(str){
    					// inner util function to convert 2-digit years to 4
    					function fixYear(yr) {
    						yr = +yr;
    						if (yr<50) { yr += 2000; }
    						else if (yr<100) { yr += 1900; }
    						return yr;
    					};
    					var ret;
    					//
    					if (str.length>12){
    						strtime = str.substring(str.lastIndexOf(' ')+1);
    						strtime = strtime.substring(0,2)+strtime.substr(-2)
    					}else{
    						strtime = '0000';
    					}
    					//
    					// YYYY-MM-DD
    					if (ret=str.match(/(\d{2,4})-(\d{1,2})-(\d{1,2})/)) {
    						return (fixYear(ret[1])*10000) + (ret[2]*100) + (+ret[3]) + strtime;
    					}
    					// DD/MM/YY[YY] or DD-MM-YY[YY]
    					if (ret=str.match(/(\d{1,2})[\/-](\d{1,2})[\/-](\d{2,4})/)) {
    						return (fixYear(ret[3])*10000) + (ret[2]*100) + (+ret[1]) + strtime;
    					}
    					return 999999990000; // So non-parsed dates will be last, not first
    				};
    				//
    				el.findData = function(elem){
    					var child = elem.getFirst();
    					if(child){
    						return el.findData(child);
    					}else{
    						return elem.innerHTML.trim();
    					}
    				};
    				//
    				el.compare = function(a,b){
    					var1 = el.findData(a.getChildren()[i]);
    					var2 = el.findData(b.getChildren()[i]);
    					//var1 = a.getChildren()[i].firstChild.data;
    					//var2 = b.getChildren()[i].firstChild.data;
    					
    					if(el.axis == 'number'){
    						var1 = parseFloat(var1);
    						var2 = parseFloat(var2);
    						
    						if(el.sortBy == 'ASC'){
    							return var1-var2;
    						}else{
    							return var2-var1;
    						}
    						
    					}else if(el.axis == 'string'){
    						var1 = var1.toUpperCase();
    						var2 = var2.toUpperCase();
    						
    						if(var1==var2){return 0};
    						if(el.sortBy == 'ASC'){
    							if(var1<var2){return -1};
    						}else{
    							if(var1>var2){return -1};
    						}
    						return 1;
    						
    					}else if(el.axis == 'date'){
    						var1 = parseFloat(el.getdate(var1));
    						var2 = parseFloat(el.getdate(var2));
    						
    						if(el.sortBy == 'ASC'){
    							return var1-var2;
    						}else{
    							return var2-var1;
    						}
    						
    					}else if(el.axis == 'currency'){
    						var1 = parseFloat(var1.substr(1).replace(',',''));
    						var2 = parseFloat(var2.substr(1).replace(',',''));
    						
    						if(el.sortBy == 'ASC'){
    							return var1-var2;
    						}else{
    							return var2-var1;
    						}
    						
    					}
    					
    				}
    				
    				if(i == this.options.sortOn){
    					el.fireEvent('click');
    				}
    			}
    		}, this);
    	},
    	
    	sort: function(index){
    		if(this.options.onStart){
    			this.fireEvent('onStart');
    		}
    		//
    		this.options.sortOn = index;
    		var header = this.tHead.getElements('th');
    		var el = header[index];
    		
    		header.each(function(e,i){
    			if(i != index){
    				e.removeClass('sortedASC');
    				e.removeClass('sortedDESC');
    			}
    		});
    		
    		if(el.hasClass('sortedASC')){
    			el.removeClass('sortedASC');
    			el.addClass('sortedDESC');
    			el.sortBy = 'DESC';
    		}else if(el.hasClass('sortedDESC')){
    			el.removeClass('sortedDESC');
    			el.addClass('sortedASC');
    			el.sortBy = 'ASC';
    		}else{
    			if(this.options.sortBy == 'ASC'){
    				el.addClass('sortedASC');
    				el.sortBy = 'ASC';
    			}else if(this.options.sortBy == 'DESC'){
    				el.addClass('sortedDESC');
    				el.sortBy = 'DESC';
    			}
    		}
    		//
    		this.elements.sort(el.compare);
    		this.elements.injectInside(this.tBody);
    		//
    		if(this.filtered){
    			this.filteredAltRow();
    		}else{
    			this.altRow();
    		}
    		
    		//
    		if(this.options.onComplete){
    			this.fireEvent('onComplete');
    		}
    	},
    	
    	altRow: function(){
    		this.elements.each(function(el,i){
    			if(i % 2){
    				el.removeClass('altRow');
    			}else{
    				el.addClass('altRow');
    			}
    		});
    	},
    	
    	filteredAltRow: function(){
    		this.table.getElements('.'+this.options.filterSelectedCls).each(function(el,i){
    			if(i % 2){
    				el.removeClass('altRow');
    			}else{
    				el.addClass('altRow');
    			}
    		});
    	},
    	
    	filter: function(form){
    		var form = $(form);
    		var col = 0;
    		var key = '';
    		
    		form.getChildren().each(function(el,i){
    			if(el.id == 'column'){
    				col = Number(el.value);
    			}
    			if(el.id == 'keyword'){
    				key = el.value.toLowerCase();
    			}
    			if(el.type == 'reset'){
    				el.addEvent('click',this.clearFilter.bind(this));
    			}
    		}, this);
    		
    		if(key){
    		this.elements.each(function(el,i){
    			if(this.options.filterHide){
    				el.removeClass('altRow');
    			}
    			if(el.getChildren()[col].firstChild.data.toLowerCase().indexOf(key) > -1){
    				el.addClass(this.options.filterSelectedCls);
    				if(this.options.filterHide){
    					el.removeClass(this.options.filterHideCls);
    				}
    			}else{
    				el.removeClass(this.options.filterSelectedCls);
    				if(this.options.filterHide){
    					el.addClass(this.options.filterHideCls);
    				}
    			}
    		}, this);
    		if(this.options.filterHide){
    			this.filteredAltRow();
    			this.filtered = true;
    		}
    		}
    	},
    	
    	clearFilter: function(){
    		this.elements.each(function(el,i){
    			el.removeClass(this.options.filterSelectedCls);
    			if(this.options.filterHide){
    				el.removeClass(this.options.filterHideCls);
    			}
    		}, this);
    		if(this.options.filterHide){
    			this.altRow();
    			this.filtered = false;
    		}
    	}
    
    });
    sortableTable.implement(new Events);
    sortableTable.implement(new Options);
    
    /*************************************************************/

  7. #7
    Join Date
    Jul 2008
    Location
    urbana, il
    Posts
    2,787
    javascript isnt that slow, the DOM is very very slow.

    there is more than one way to make a table.

    using the dom ala createElement("td"), appendThis(toThat) etc is painfully slow, likely unusable for your project. especiially if you are using the html to store all your data, then sorting the table cells themselves (ouch).


    building the table first in javascript (especally using arrays and not appending a string over and over) will prove much faster.

    i have found that building the array of arrays, and the .joining the rows and throwing into the tbody with .innerHTML to be the fastest by far with anything bigger than ~500 cells.

    to re-sort, it is faster to re-sort the array of arrays and simply throw away the old table, and .innerHTML the new one. don't try to sort html whatever you do.

    .innerHTML is not standard, but i dont know of any browsers than don't support it.



    EDIT - i wrote this before post #6 was posted.
    that post shows code that will be basically unusably slow and memory hungry for the size of table mentioned.

  8. #8
    Join Date
    Jul 2008
    Posts
    92
    How were you able to identify that it wouldn't be usable?

    Anything that helps me learn about this process is appreciated

    Btw, here's what it looks like atm. http://img381.imageshack.us/img381/7363/tableop2.png
    Last edited by mith36; 07-15-2008 at 01:36 PM.

  9. #9
    Join Date
    Jul 2008
    Location
    urbana, il
    Posts
    2,787
    Quote Originally Posted by mith36
    How were you able to identify that it wouldn't be usable?

    Anything that helps me learn about this process is appreciated.
    based upon the shear volume of cells, i would expect it to be quite slow.

    my directory listing page for instance is about 400 rows with 3 columns.

    using dom methods to build the table resulted in firefox not responding for about 2-4 seconds every resort. i expect on a table much larger, as the one you describe, it would push firefox into the dreaded "Script is taking to long.." message.

    after i moved over to an array of objects, resorting and drawing the table takes less than a second.

    this is for several reasons. consider sorting numbers for instance.
    to do this with the dom, you have to drill into the dom tree to find each cell.
    then you have to turn a dom property (.innerHTML or whatever) into a js string, then into a js number.
    if this were in an array of objects, it would already be the js Number you need.
    see how much work that saves?
    Last edited by rnd me; 07-15-2008 at 03:28 PM.

  10. #10
    Join Date
    Jul 2008
    Posts
    92
    Yes, that makes sense. I'm not sure how I would go about fixing the script though. Is there a particular js function set I should read up on to become more familiar with approaching the tables as an array of objects instead of the dom method?

    I'm familiar with sorting by arrays (as I actually programmed this entire thing to work in php very quickly--but the dynamic stuff was missing, hence the js), but I'm not sure if doing so in js is the same as with php.

  11. #11
    Join Date
    Aug 2007
    Posts
    3,767
    Are you using innerHTML to add the table to the page (that's the table sorting code you've posted)?

  12. #12
    Join Date
    Jul 2008
    Posts
    92
    Declan:

    I think so. Is that a bad thing? I'm borrowing the .js script as I said. Hopefully in a couple days I can write my own, but not until I can understand this one a little better.

  13. #13
    Join Date
    Feb 2003
    Location
    Michigan, USA
    Posts
    5,774
    You really wouldn't approach the table as an array of objects. You would actually create JavaScript arrays that hold the same data as the TABLE. The TABLE would merely be the display of that data. Any work on the data the TABLE displays would be done with the JavaScript arrays, then the table would update accordingly.

    Also, read up on the TABLE DOM:

    http://www.w3schools.com/htmldom/dom_obj_table.asp

    http://www.w3schools.com/htmldom/dom_obj_tablerow.asp

    http://www.w3schools.com/htmldom/dom_obj_tabledata.asp

    The browser automatically creates node references to all rows and cells within a table. This could be useful when traversing a table to display data.

  14. #14
    Join Date
    Jul 2008
    Location
    urbana, il
    Posts
    2,787
    Quote Originally Posted by toicontien
    You really wouldn't approach the table as an array of objects. You would actually create JavaScript arrays that hold the same data as the TABLE. The TABLE would merely be the display of that data. Any work on the data the TABLE displays would be done with the JavaScript arrays, then the table would update accordingly.
    arrays of objects and arrays of arrays are practically identical.
    the object would use the col head as a key, whereas an array would use the col# as an index.

    for instance, i parse in the incoming webDAV response into an array of object, each file getting an object like:
    Code:
    {name: "text1.txt", size: 432, date: new Date("123456789012"), path: "\/pub" }
    a prototype method can easily convert the object into an array, i call mine vals;

    Code:
    ob = {name: "text1.txt", size: 432, date: new Date("123456789012"), path: "\/pub" }
    ray = ob.vals() //===["text1.txt", 432,  new Date("123456789012"), "\/pub" ]
    i use prototype functions a lot. i would use Array.toiTable to create a HTML table from an array. it maps Array.toRow on the master's elements, and toTable from all the TRs.

    its a simple matter of preference. i find object easier to code because of the meaningful names attached.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
HTML5 Development Center



Recent Articles