How to write a code (in any programming language, preferably in java), which calculates the average loading time of any website (including all embedded elements such as images, Javascript, CSS, etc.) ?
I'd just use YSlow
Google just released Page Speed (a plug-in for firebug, just like YSlow) and it's got some pretty cool features.
I think websiteoptimization.com has all you need :)
I've always enjoyed using this tool:
http://www.websiteoptimization.com/services/analyze/
Depends what you mean by 'average loading time'.
If you sit at your PC with your site running locally and just hit F5 over and over again you'll get considerably different results to someone looking at the same code which is deployed in a data centre on the other side of the world.
Fiddler is also another option. See the Fiddler home page
A nice screenshot of the transfer timeline is here.
I had used souptag framework to parse html page and then found individual src attribute of all the tags, Then individually found size of each page mentioned in src attribute and then according to my internet speed found out average loading time.
Perceived loading time is important too. External scripts will delay rendering of everything below them until they've loaded and executed, in all browsers, so it's best to push them to the very bottom of the page when possible, as opposed to the very top which is the habit of many developers. Another source of delay in some browsers is img tags where the width and height are unspecified. The browser will give the image some time to load so that it can get the layout right the first try. Fixing both those issues will make your site appear a lot more responsive, even if total loading time is unchanged.
This has a nice JavaScript way to check load time of a web page:
http://javascript.internet.com/text-effects/loading-time-indicator.html
Related
I am currently in the process of developing an application that will request some information from Websites. What I'm looking to do is parse the HTML files through a connection online. I was just wondering, by parsing the Website will it put any strain on the server, will it have to download any excess information or will it simply connect to the site as I would do through my browser and then scan the source?
If this is putting extra strain on the Website then I'm going to have to make a special request to some of the companies I'm scanning. However if not then I have the permission to do this.
I hope this made some sort of sense.
Kind regards,
Jamie.
No extra strain on other people servers. The server will get your simple HTML GET request, it won't even be aware that you're then parsing the page/html.
Have you checked this: JSoup?
Consider doing the parsing and the crawling/scraping in separate steps. If you do that, you can probably use an existing open-source crawler such as crawler4j that already has support for politeness delays, robots.txt, etc. If you just blindly go grabbing content from somebody's site with a bot, the odds are good that you're going to get banned (or worse, if the admin is feeling particularly vindictive or creative that day).
Depends on the website. If you do this to Google then most likely you will be on a hold for a day. If you parse Wikipedia, (which I have done myself) it won't be a problem because its already a huge, huge website.
If you want to do it the right way, first respect robots.txt, then try to scatter your requests. Also try to do it when the traffic is low. Like around midnight and not at 8AM or 6PM when people get to computers.
Besides Hank Gay's recommendation, I can only suggest that you can also re-use some open-source HTML parser, such as Jsoup, for parsing/processing the downloaded HTML files.
You could use htmlunit. It gives you virtual gui less browser.
Your Java program hitting other people's server to download the content of a URL won't put any more strain on the server than a web browser doing so-- essentially they're precisely the same operation. In fact, you probably put less strain on them, because your program probably won't be bothered about downloading images, scripts etc that a web browser would.
BUT:
if you start bombarding a server of a company with moderate resources with downloads or start exhibiting obvious "robot" patterns (e.g. downloading precisely every second), they'll probably block you; so put some sensible constraints on what you do (e.g. every consecutive download to the same server happens at random intervals of between 10 and 20 seconds);
when you make your request, you probably want to set the "referer" request header either to mimic an actual browser, or to be open about what it is (invent a name for your "robot", create a page explaining what it does and include a URL to that page in the referer header)-- many server owners will let through legitimate, well-behaved robots, but block "suspicious" ones where it's not clear what they're doing;
on a similar note, if you're doing things "legally", don't fetch pages that the site's "robot.txt" files prohibits you from fetching.
Of course, within some bounds of "non-malicious activity", in general it's perfectly legal for you to make whatever request you want whenever you want to whatever server. But equally, that server has a right to serve or deny you that page. So to prevent yourself from being blocked, one way or another, you need to either get approval from the server owners, or "keep a low profile" in your requests.
I am using GWT 2.0.3 with ext version.When I run the application its take some time to load.As much as I know It take time to load some JS file (Not sure about it).For slow internet connection it wiil take more time.
I want to know what exactly GWT application do while loading.If it is loading some JS file the is there ant way to reudce loading time by dividing JS file or by nay other way?
Thanks in advance
When a GWT application loads, it loads all js files contained in your html host page, what means everything client side related is loaded.
To optimize this GWT introduced code splitting some time ago. You can check it here. The basic idea is to divide your application in logical parts, when a user wants to access to another part, its loaded on demand.
To speed up web app loading time, indeed split points are a first step.
But check as well this tool:
http://pagespeed.googlelabs.com
Slowness can also come from uncompressed pictures for instance.
Also, when you compile your GWT app, ensure you use "OBFUSCATED" mode for your compiled javascript, which makes it significantly smaller.
http://code.google.com/webtoolkit/doc/1.6/FAQ_DebuggingAndCompiling.html#Why_is_my_GWT-generated_JavaScript_gibberish?
Now if you are using Apache HTTPD as web server, Google has released a wonderful Apache module that implements web app best practices out of the box:
http://code.google.com/p/modpagespeed/
I am writing a web application that has a page that returns a list of results of objects that each have a profile photo.
I am unsure of how large I expect this page to get, but I want to account for it getting to a larger level. Right now, I am only returning 5-10 objects each with an associated photo, and I am seeing a performance decrease.
I am aware there are many resources on the internet regarding image optimization, and I have done some research, but am still lost as to what I should do to optimize this page.
I know that using css-sprites is an option, but these photos are added by members, so I'm not sure If I could keep up with updating the CSS Sprite.
I am also aware that some image types might be better for performance, but sometimes with a loss of compatibility(although I have read articles that argue against this, and say it is something of the past).
I am fairly new to the concept of needing to optimize images on my page, so if there is a more advanced technique that somebody believes might be over a more novice developers ability, then it might be the wrong thing for me.
Any advice would help. If I try one and am unable to figure out the answer, I can always try another.
I realize this is something that can go on StackOverFlow's sister website regarding web design, but I would like to get the server side of it as well because there may be some fixes I need to make when I am uploading the file, so please no comments on how this is not fit for StackOverflow, Thanks!
a good way of optimisng images is using cache, and plus thumbnail them!
if you dont know about caching:
this is good tutorial to start for
webdesigners/ devlopers:Cache Tutorial for web authors`
this is good tutorial on creating images into thumbnails using php and Gd(which comes default with php)
just check incase
Create thumbnails using PHP and GD`
Thumbnail them server-side to the EXACT dimensions needed, then convert them to a jpeg. Also, if possible, just use one "default" photo for all of the users then load the individual profile photos via AJAX once the page has loaded.
I'm working on a web application. There is one place where the user can upload files with the HTTP protocol. There is a choice between the classic HTML file upload control and a Java applet to upload the files.
The classic HTML file upload isn't great because you can only select one file at a time, and it's quite hard to get any progress indication during the actual upload (I finally got it using a timer refreshing a progress indicator with data fetched from the server via an AJAX call). The advantage: it's always working.
With the Java applet I can do more things: select multiple files at once (even a folder), compress the files, get a real progress bar, drag'n'drop files on the applet, etc...
BUT there are a few drawbacks:
it's a nightmare to get it to work properly on Mac Safari and Mac Firefox (Thanks Liveconnect)
the UI isn't exactly the native UI and some people notice that
the applet isn't as responsive as it should (could be my fault, but everything looks ok to me)
there are bugs in the Java UrlConnection class with HTTPS, so I use the Apache common HTTP client to do the actual HTTP upload. It's quite big a package and slows down the download of the .jar file
the Apache common HTTP client has sometimes trouble going through proxies
the Java runtime is quite big
I've been maintaining this Java applet for a while but now I'm fed up with all the drawbacks, and considering writing/buying a completely new component to upload theses files.
Question
If you had the following requirements:
upload multiple files easily from a browser, through HTTP or HTTPS
compress the files to reduce the upload time
upload should work on any platform, with native UI
must be able to upload huge files, up to 2gb at least
you have carte blanche on the technology
What technology/compontent would you use?
Edit :
Drag'n'Drop of files on the component would be a great plus.
It looks like there are a lot of issues related to bugs with the Flash Player (swfupload known issues). Proper Mac support and upload through proxies with authentication are options I can not do without. This would probably rule out all Flash-based options :-( .
I rule out all HTML/Javascript-only options because you can't select more than one file at a time with the classic HTML control. It's a pain to click n-times the "browse" button when you want to select multiple files in a folder.
I implemented something very recently in Silverlight.
Basically uses HttpWebRequest to send a chunk of data to a GenericHandler.
On the first post, 4KB of data is sent. On the 2nd chunk, I send another 4K chunk.
When the 2nd chunk is received, I calculate the round trip it took between first and 2nd chunk and so now
the 3rd chunk when sent will know to increase speed.
Using this method I can upload files of ANY size and I can resume.
Each post I send along this info:
[PARAMETERS]
[FILEDATA]
Here, parameters contain the following:
[Chunk #]
[Filename]
[Session ID]
After each chunk is received, I send a response back to my Silverlight saying how fast it took so that it can now send a larger
chunk.
Hard to put my explaination without code but that's basically how I did it.
At some point I will put together a quick writeup on how I did this.
I've never used it with files of 2GB in size, but the YUI File Uploader worked pretty well on a previous project. You may also be interested in this jQuery Plugin.
That said, I still think the Java Applet is the way to go. I think you'll end up with less portability and UI issues than you expect and Drag/Drop works great. For the record, Box.net uses a Java Applet for their multi-file quick uploads.
OK this is my take on this
I did some testing with swfupload, and I have my previous experience with Java, and my conclusion is that whatever technology is used there is no perfect solution to do uploads on the browser : you'll always end up with bugs when uploading huge files, going through proxies, with ssl, etc...
BUT :
a flash uploader (a la swfupload) is really lightweight, doesn't need authorization from the user and has a native interface which is REALLY cool, me thinks
a java uploader needs authorization but you can do whatever you want with the files selected by the user (aka compression if needed), and drag and drop works well. Be prepared for some epic bugs debuggin' though.
I didn't get a change to play with Silverlight as long as I'd like maybe that's the real answer, though the technology is still quite young so ... I'll edit this post if I get a chance to fiddle a bit with Silverlight
Thanks for all the answers !!
There are a number of free flash components that exist with nice multiple file upload capability. They make use of ActionScripts FileReference class with a PHP (or whatever) receiver on the server side. Some have recently broken with the launch of FP10 but I know for certain that swfupload will work :)
Hope this helps!
What about these two
Jupload
http://jupload.sourceforge.net/
and
jumploader
http://jumploader.com/
Both are java applets but they are also both really easy to use and implement.
what about google gears?
There are HTTP/HTTPS upload controls that allow multi-file upload. Here is one from Telerik, which I have found to be solid and reliable. The latest version looks to have most if not all of your feature requirements.
You can upload multiple files with HTTP forms as well, as Dave already pointed out, but if you're set on using something beyond what HTTP and Javascript offers I would heavily consider Flash. There are even some pre-existing solutions for it such as MultiPowUpload and it offers many of the features you're looking for. It's also easier to obtain progress information using a Flash client than with AJAX calls from Javascript since you have a little more flexibility.
You may check the Apache Commons FileUpload package. It allows you to upload multiple files, monitor the progress of the upload, and more. You can find more information here:
http://commons.apache.org/fileupload/
http://commons.apache.org/fileupload/using.html
Good luck
I have a Java Applet application ( achart) on my php Webpage ... Problem here is the Java Applet takes more time to load ... I am thinking of replacing these applets with some similar technology but fast ... I am counting on Ajax... what are my other options ... ?
Java applets load slowly. shrug Its the nature of the beast....
If you have multiple jars, you should check the order of the classpath you provide to your applet. Note that each jar is only downloaded "on demand" whenever a class needs to be loaded. It looks in the first jar, if it can't find the class it looks in the second and so on... You can reduce your startup time by ensuring that all of your classes required for starting up the app are in the first jar(s) listed.
Also, if you are attempting to load a class or resource which is not in your classpath, it must search through all the jars before returning AND hit the server codebase to look there. It can potentially greatly reduce your startup time.
Turn on applet tracing in the java control panel and you should get a better idea of how classes are being loaded out of the jars.
If the size of the applet's JAR library takes to long to download, you can shrink the size with the ProGuard tool. Here is a comparison of the compression ratio for some Java libraries.
Have you packed your jars.....
Have you tried Java deployment toolkit (http://blogs.oracle.com/vaibhav/entry/java_deployment_toolkit_6u10)
with jdk 1.6.10+.....
If its simple charts you want, have a look at:
http://code.google.com/apis/chart/
Yahoo has some nice charting components for actionscript 3 (flash) http://developer.yahoo.com/flash/astra-flash/charts/
Google has a service that will generate charts as images as Tim already pointed out.
Alternatively you could try to speed up the delivery of the applets that you are using - check if they have an Expires header so they only get downloaded once in a while instead of for every page (this won't help on the first view, but will speed up those after that)
Edit: if you only target specific browsers you could try and create the charts using javascript and the Canvas element, but that is definitely not supported by internet explorer. https://developer.mozilla.org/en/Canvas_tutorial
I know this answer is a bit late but, it could be that you are just being very inefficent with java or your machine is slow, I have a java program that has quite alot of code as well as accessing an SQLite DataBase and it runs quite quickly, loading within five seconds. If you have anything you don't use in your program then remove it, also try jaring and signing it(not sure if last two things would help.)