Large html pages and slow load times/high CPU usage

[Madison Kelly]

Stewart C. Russell wrote:
> Madison Kelly wrote:
> 
>>  I am stress-testing my backup program now and I've noticed that with a
>>sample partition with ~26,000 directories when they are all set to be
>>displayed it creates a ~1.5MB html file.
> 
> 
> I have to ask -- is it ever really useful to present the user with this
> much data? It does sound like you're testing a slightly unusual option
> on your software. Even so, what user could make sense of even a tenth of
> this output?
> 
> It might be better to use some kind of data pager (there are Perl
> modules that will handle paging through query results with little extra
> programming effort). You could represent the data some other way (a
> tree?) to let the user drill down to see what they're interested in.
> 
> I don't think we can expect HTML to implement proper text tabulation any
> time soon. The time for that was HTML 2.0, or earlier. What it has is
> meant for small display tables, not thousands of lines of results.
> 
> cheers,
>  Stewart

It is a stress-test case... The situation arises what the user has many, 
many directories all opened in the file browser. In my test case, all 
~26,000 directories from my sample partition have been selected for 
display. Currently I am experiementing with a javascipt app that would, 
at the very least, require only a single load of the directory tree.

Currently Each time the directory branch is expanded or contracted the 
data is re-read from the database and redrawn. Waiting ~one minute on my 
(modest) test machine to load 26k directories once I can live with. 
Taking that long every time a single branch is expanded or contracted 
though is less tolerable. I am hoping I can intregrate this tree app I 
have to serve my purpose and if I can I will live with the current 
initial load time on one minute.

Madison

-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Madison Kelly (Digimer)
TLE-BU, The Linux Experience; Back Up
http://tle-bu.thelinuxexperience.com
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
--
The Toronto Linux Users Group.      Meetings: http://tlug.ss.org
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://tlug.ss.org/subscribe.shtml