Thursday, December 28, 2006

Real Web 2.0: Bookmarks? Tagging? Delicious!

Learn how real-world developers and users gain value from a classic Web 2.0

In this article, you'll learn how to work with del.icio.us, one of the classic Web 2.0 sites, using Web XML feeds and JSON, in Python and ECMAScript. When you think of Web 2.0 technology, you might think of the latest Ajax tricks, but that is just a small part of the picture. More fundamental concerns are open data, simple APIs, and features that encourage users to form social networks. These are also what make Web 2.0 a compelling problem for Web architects. This column will look more than skin deep at important real-world Web 2.0 sites and demonstrate how Web architects can incorporate the best from the Web into their own Web sites.

When people list classic Web 2.0 sites, one that never fails to come up is del.icio.us (see Resources for a link). This site, despite its relative youth, is the best established social bookmarking site. It allows people to post links and add tags to those links. It provides Web feeds for link collections, including automatically aggregated tags such as the special popular tag, for links that have been posted by many users. It provides a simple and open API. del.icio.us also serves as a demonstration of how fuzzy a term Web 2.0 is. You might be expecting a slick, Ajax-driven Web page with Web boxes winking seductively at you. At one point, del.icio.us was nothing at all like that; it was very simple and functional. The front page has recently been dressed up quite a bit, but throughout most of its history it has had a very plain interface.

The idea of sharing browser bookmarks is not new. During the dot-com era, services such as Backflip emerged with which people could host and categorize their bookmarks online. Backflip was free of charge, but the breakthrough of sites such as del.icio.us is that the information is even more free than that: freely available for use in innumerable ways, and for sharing in innumerable patterns. Open APIs, Web feeds (such as those built on RSS and Atom), and tagging with automatic aggregation are the foundation of the more exciting developments on the Web, and these too often get lumped in with user interface tricks such as Ajax. These collaborative features are sometimes hampered by the typically limited interface of the Web, so sometimes a few fancy touches make it possible and attractive for more people to get involved with Web 2.0, and the value of such sites is often proportional to the number of participants.

The substance of an effective Web 2.0 site, and the points of interest for Web architects (as opposed to, say, Web designers), lie in how readily real developers and users can take advantage of open data features. From widgets that users can use to customize their bits of territory on a social site to mashups that developers can use to create offspring from Web 2.0 parents, there are ways to understand what leads to success for such sites, and how you can emulate such success in your own work. This column, Real Web 2.0, will cut through the hype to focus on the most valuable features of actual sites from the perspective of the Web architect. In this first installment, I'll begin with one of the ancestors of the genre, del.icio.us.

The front door

In this column, I'll refer to the main page of a Web 2.0 site as its façade. This is because, like the façade of a building, it's only the front face, and much of the actual business goes on in the less visible areas. I use del.icio.us almost every day, but I almost never bother with its façade. I use a Firefox plug-in to post interesting sites to del.icio.us. I follow items on the popular tag within my Web feed reader, and I use a simple Python script to automatically post my del.icio.us postings to my Weblog each day. This is what I mean when I say that much of the real business in Web 2.0 goes on behind the façade. Nevertheless, the front door is where you enter, so that's where I'll start.

The main page of del.icio.us tries to distill the major trends in tagging by users. It lists hot now Web pages that many people have recently bookmarked. It also lists some tags the editors deem particularly interesting. There is the mandatory Weblog by the site owners, and a recent entry celebrates the fact that the site now has 1 million registered users after just three years in business. del.icio.us is definitely an illustration of how having good architectural features for open data supports a community of contributors (developers of tools, plug-ins, mashups, and more), which brings in more users, which leads to more contributors, and so on in a virtuous circle.

Home sweet home

Registered users have a home page that is accessible using a very simple URL scheme: http://del.icio.us/. A very sensible URL scheme is one of the site's enduring strengths, and leads to value beyond the façade, as I'll discuss later. Your home page includes your recently posted links and your tags. Figure 1 illustrates a portion of my own del.icio.us page.

Friday, December 8, 2006

Top 20 Websites in US

Web metrics firm Compete has an interesting post, outlining the top 20 websites (for US traffic). According to Compete, all 20 of them got over 20 million unique visitors in October 2006. Here is the chart:



A couple of people noted in the comments that if you add Microsoft's 4 top 20 properties together (msn.com, live.com,microsoft.com and passport.net), then they would probably be number 1. However a counter to that is that a lot of passport.net domains currently re-direct to live.com. I think there may be some crossover between live.com and MSN too. So it may well be that Yahoo remains number 1, even accounting for Microsoft's multiple brands. Plus of course Yahoo and Google both have separately branded properties too - e.g. Flickr, YouTube. If I was to estimate, I'd put Microsoft at number 2 overall - but interested to hear what others think.


Compete notes that Adobe.com, Live.com, Wikipedia.org and YouTube.com are new to the top 20 over the past year,while Expedia.com, Monster.com, Paypal.com and Weather.com have all dropped out.


Looking at the Alexa data for US traffic, the top 20 is quite different:


1. yahoo.com
2. google.com
3. myspace.com
4. msn.com
5. ebay.com
6. amazon.com
7. youtube.com
8. craigslist.org
9. wikipedia.org
10. cnn.com
11. facebook.com
12. go.com
13. live.com
14. blogger.com
15. aol.com
16. microsoft.com
17. comcast.net
18. imdb.com
19. digg.com
20. flickr.com

The Alexa list is (I think) only counting US traffic, but it is quite different from Compete's stats. The presence of non-mainstream web 2.0 sites in Alexa's top 20 (blogger.com, digg, flickr) suggests that the traffic is heavily skewed towards technical users - which makes sense, given Alexa relies on toolbar downloads to get their stats.


Also interesting to note there is just one Microsoft property in the top 10 in Alexa, compared to 3 for Compete.

Thursday, December 7, 2006

Various web browsers

Browser

Release date

Platform

Rendering engine

Standards support

Microsoft Internet Explorer 6

2001

Windows, Linux, Unix

Trident
IV

CSS 1, some CSS 2, some CSS 3, ECMAScript, DOM (with proprietary implementations and quirks)

Microsoft Internet Explorer 5 and 5.5 (Windows)

1999 (5), 2001 (5.5)

Windows, Linux, Unix

Trident II (5) and III (5.5)

Most CSS 1, some CSS 2 (with bugs), partial support of ECMAScript/DOM

Microsoft Internet Explorer 5 (Macintosh)

2000

Macintosh

Tasman

CSS 1, some CSS 2, some CSS 3, ECMAScript, DOM (buggy and not complete)

Netscape Navigator 7

2002

Windows, Macintosh, Linux, Unix

Gecko

CSS 1, most CSS 2, ECMAScript, DOM

Netscape Navigator 4

1997

Windows, Linux, Macintosh, Unix

N/A

Some basic CSS 1, JavaScript, no DOM because it was written after NN4's release

Firefox 1.0

2005 (pre 1.0 release versions available in 2004)

Windows, Linux, Macintosh, Unix

Gecko

CSS 1, most CSS 2, some CSS 3, ECMAScript, DOM

Opera 8.5

2005

Windows, Linux, Macintosh, Unix

Presto

CSS 1, most CSS 2, ECMAScript, DOM (Opera 7 was the first version with DOM support)

Safari

2002

Macintosh OS X

KHTML

most CSS 1, some CSS 2, some CSS 3, ECMAScript, DOM (with bugs)

America Online

Various

Windows (there is a Mac version, but it isn't as well supported)

Trident

CSS 1, some CSS 2, ECMAScript, DOM (same as Internet Explorer, but expect additional buggy behavior)

Lynx

1993

Unix, Windows, Macintosh

N/A

N/A


Rendering Engines

A rendering engine, also known as a layout engine, is the code that tells the browser how to display web content and available style information in the browser window. The rendering engine is responsible for the size of an unstyled h1 heading or how a horizontal rule looks on the page. It's also the key to the correct implementation of CSS and other web standards.

The first separate and reusable rendering engine was Gecko , released by the Mozilla developers in 1998. It was notable for its small size and excellent support for standards. Now web developers pay attention to underlying rendering engines as a key to understanding a browser's performance.

The Wikipedia, an online collaborative encyclopedia, has a detailed comparison of rendering engines, where they are used, and what they support at http://en.wikipedia.org/wiki/Comparison_of_layout_engines.

How the Web Works?

The anatomy of a single browser selection

If you have browsed the World Wide Web even once, you are familiar with the basic method for getting from one place to another. A Web page presents you with a number of items you may select. Once you have selected something, you leave the current Web page and enter the one that you just selected. If you have a text-based browser, such as Lynx, you made this selection by choosing a number from a menu of items. If you have a graphical browser, such as Mosaic or Netscape, you made this selection by clicking on an underlined item, or perhaps an icon or other graphic.

Suppose you are happily browsing when you come upon an item labeled Sunsite, A Vast Compendium of Information. This certainly sounds intriguing, so you click on Sunsite (hereafter I will assume that you have a graphical browser; the principles are the same for text-based systems). After a few seconds, or several hours, depending on network traffic, how you are connected to the Internet, and a number of other factors, the Web page appears.

How did this actually happen? How did your index finger get such power? In this article we will examine what happens as a result of this single mouse click, and look at all the steps which contribute to making navigation on the World-Wide Web possible.

Anatomy of the URL

While you were sitting there with your finger poised over Sunsite, your browser probably displayed something similar to http://sunsite.unc.edu/. Mosaic, for example, displays this in its status bar at the bottom of the window. This is the infamous Uniform Resource Locator, or URL, which describes the location of the page. URLEvery link to another Web page has a URL. URLs themselves are worthy of an entire article, so I won't attempt to describe all of their nuances and quirks here. For our purposes we can think of a URL as being composed of four parts: the protocol, the hostname, the port, and the path (or document name), as shown in the figure.

The hostname says which machine to contact; the port number and the protocol say how to connect to that machine, and the path says what to get once connected. The protocol is located just before the first colon, and is http, the Hypertext Transfer Protocol, in this case. The hostname is located between the double slash (//) and the next separator, which will be another colon or slash. In this case the hostname is sunsite.unc.edu. If a port were given, the hostname would have been followed by a colon, a number, and then a slash. gopher://news.csie.nctu.edu.tw:4870/7gonnrp%20-g%20 is a gopher URL with a port number, namely 4870 (you might want to brush up on your Chinese before trying this URL). The final part of the URL, the path, is given immediately after the hostname or port. In our URL the path is simply /, known as the root. The format of the path part of a URL depends on the protocol being used, among other things, and can become quite complicated.

Finding a Host

The browser uses the various parts of the URL to find the host, connect to it, retrieve the path portion of the URL, and display the page. The first step is to convert the hostname into an Internet address. The hostname may be descriptive (that is its purpose), but it is also worthless in terms of making an actual connection. Converting a hostname to an address is often referred to as the "address resolution phase". The status bar of your browser may display something like "looking up hostname" during this process.

Addresses such as sunsite.unc.edu are part of a hierarchical addressing scheme known as the Domain Naming System, or DNS. Each of the components in the address is one level of the hierarchy. The rightmost part, "edu", is the top-level domain, and consists of all educational institutions on the Internet. The next level is "unc", a designation for the University of North Carolina, which has registered itself as a member of the edu domain. The final level is "sunsite", which is the name of an actual machine at UNC.

In order to resolve the address, your browser must construct a DNS query and submit it to a DNS server. This process also happens hierarchically. Somewhere very close to you (on your Internet Service Provider, or at your corporate headquarters, for example) there is almost certainly a local DNS Server. This DNS server probably won't know the address of sunsite.unc.edu, but it will know the address of a more authoritative DNS server. Eventually, some DNS server will actually known the answer, which will then make its way back to your browser. From my site, for example, it took two hops to discover that sunsite.unc.edu is actually at 198.86.40.81. My local DNS server contacted the unc.edu server, ns.unc.edu, which provided the numerical Internet address.

The Internet address is only half the equation, however. A fully qualified Internet address consists of a host address and a port number. If the port number is provided as part of the URL, then we have a fully qualified address already. If it is not, then the browser will use the well-known port associated with the protocol. All of the common protocols which the Web supports, including http, ftp, gopher, news (NNTP), WAIS, and Telnet, have well-known port numbers. For http this port number is 80. This information is usually stored in a file that is part of the local machine's network configuration. On Unix machines, for example, this information is stored in a file known as /etc/services.

The browser is now ready to connect to the host. It does this by building a connection request using TCP (some browsers use UDP, but that's another story). If you were to look at the browser code itself, you would see that once it has the address and port number, it makes the following system calls: socket(), to open a raw network endpoint; bind(), to bind that endpoint to the local address (the address of the machine running the browser), and finally connect(), to specify the remote address and try to talk to it. Inside the operating system the guts of connect() builds a TCP packet, and then hands it to the lower-level IP layer, which wraps that packet with IP protocol. This TCP/IP packet is then given to the lowest layer (usually a device driver), which often wraps it in yet another layer of protocol. On Ethernet this would be the Ethernet frame. After all this happens, the connection packet is unleashed on the Internet and goes off in search of the sunsite host.

However, this may not happen quite as directly as you might expect. Many machines are not connected directly to the Internet. Instead, their connection is mediated by one or more intermediaries, known as proxies, gateways, or firewalls, depending on their function. Firewalls, in particular, are becoming more and more common as concerns about security increase. Outgoing and incoming packets thus might be going through one or more stages of intermediate software, often designed to permit a limited set of outgoing connections and an even more limited set of incoming ones.

The first stop for the connection request packet might be an application such as SOCKS, which will peek inside the packet, look at the port number and protocol, and approve or disapprove it based on their values. This often explains why URLs based on specific protocols will fail while all others succeed. If you are behind a firewall, and your security administrator has decreed that Telnet connections may not cross that firewall, URLs that use Telnet will always fail during the connection phase. Firewalls can even cause DNS queries to fail, although few organizations are that paranoid.

Once the connection packet actually leaves your local site, it still won't go directly to sunsite.unc.edu. Unless you are extremely lucky and are connected directly to UNC's campus-wide network, it will instead pass through a number of other sites, each of which will try to route your packet to a site that is closer to the destination. Routes are born, expand, contract, and die all within the scope of milliseconds, so that it is quite possible that two identical packets sent by the same browser as quickly as possible will travel two vastly different routes to the same destination. If you have the traceroute utility, you can have hours of fun (well, maybe minutes of fun) tracing the route between your machine and another one across the globe, over and over again.

Going All the Way

Let's be optimistic and assume that your packet actually reaches sunsite.unc.edu. If that host is up, then its operating system will intercept and decode the packet. It will ask itself many penetrating questions, such as "Do I want to accept any connections right now?"; "Am I willing to accept a connection from this particular host?"; and, most important, "Is there a user application that has declared itself willing to deal with packets on this particular port?"

Such applications are typically called servers. If the http server, httpd, is running on sunsite, then it will have registered itself as a potential recipient for packets on http's well-known port. If httpd has available connections, it will accept our browser's connection request, and the operating system on sunsite will generate an appropriate connection response packet. If httpd is overloaded or otherwise unavailable, we will get a negative response, resulting in the dreaded "connection refused" message on most browsers.

In each case, the reply will work its way back through the network to the originating machine. An IP packet always includes the sender's address as well as the destination address, so sunsite will not have to do a DNS lookup in order to generate its reply. Some modern implementations even record the route taken by a packet from one machine to another, so that a response packet need not discover the return route for itself. It is not guaranteed that the machines over which the response traverses on its way back to us is the same as the outgoing route the connection request used, however. It is often quite unlikely.

Once the browser receives a successful connection reply, it is ready to actually request the path portion of the URL. In our example, that is the top-level path /. Since it is using the http protocol, it must ask for this path using that protocol. The http protocol is a text-based conversational protocol, in which the requesting client (our browser) sends out a series of strings in 8-bit extended ASCII (Iso-Latin-1 to be precise). Our browser will send out a GET request, indicating the path it wishes to receive, and a series of Accept qualifiers, indicating the possible formats it wishes to receive it in. This request is converted into a TCP packet containing the http protocol, wrapped in IP and more, and wends it way across the Internet again, to sunsite. Many browsers will display a progress message, such as "Sent HTTP request; waiting for response."

Sunsite's OS hands the path request to httpd, which then verifies that it is indeed a valid http request, and attempts to satisfy it. Our path is a special case, since we have asked for the root, rather than asking for a specific HTML document, such as exhibit.html (a multimedia link on sunsite). The traditional response to such a root request is to look for a document named index.html and send that.

It is possible, but very unlikely, that we would get back a listing of the top-level server directory, instead. The httpd server does not simply send the contents of this HTML document, however. It wraps the response in http, sending back a status line, header information, and the actual response. The status line indicates its success or failure in getting the requested path. If the request fails, the server tries to be helpful and tell you why, such as "that URL has moved to . . ."

On success, the status line is followed by one or more lines of header information describing what is to come. One header line that is always sent is the Content-Type line, which will take the form Content-Type: text/html in our case. This tells the browser the actual format of the reply data, which could have been in any number of formats.

Even though this conversation between our browser and sunsite's httpd server is in the http protocol, several of these headers are derived from the MIME (Multipurpose Internet Mail Extensions) system. MIME was created independently of http as a uniform method for moving non-ASCII data through e-mail. The http protocol adopted it as a way of doing the same thing for Web content.

Whew!

The final portion of the server's reply is the actual HTML document, index.html, a portion of which is shown below. The httpd server gives its http packet to Sunsite's Unix, which flavors it with TCP, IP, and an Ethernet frame, and sends it back to our browser. It also closes its end of the connection. This is because http was designed to be a stateless protocol, in which each transaction would be a complete exchange.

This design decision has profound implications on how the Web works. Since TCP connections are relatively precious (most systems allow only a few dozen to be simultaneously active in any one application), this helps to conserve system resources. However, it also means that if the browser needs more information, it must once again open a connection, send a request, and then wait for the reply. In our example this will happen repeatedly. When our local browser receives the contents of index.html and begins to parse it, in order to build the actual graphics for the page, it will discover these dangling references, and go back through the connect-request-reply steps for each piece of missing information. The status bar of the browser will flash a series of messages while this is happening.

When the browser finally has resolved all references and fully parsed the contents of index.html, it uses the various HTML directives, image data, and other content to construct the page. In the case of HTML, this is relatively easy, since the HTML language was designed to be nicely viewable. If we had requested a URL based on another protocol, such as FTP (File Transfer Protocol), the browser would have had to do more work. Older protocols such as these were not designed with graphics in mind, so that the browser must construe the textual replies from an ftp server and build a reasonable Web page from them.

As you can see, a very large number of software components, representing many diverse protocols that were not originally intended to work together, had to collaborate to execute the one mouse click that brought up the Sunsite Web page.

Wednesday, December 6, 2006

Subnet Mask

A mask used to determine what subnet an IP address belongs to. An IP address has two components, the network address and the host address. For example, consider the IP address 150.215.017.009. Assuming this is part of a Class B network, the first two numbers (150.215) represent the Class B network address, and the second two numbers (017.009) identify a particular host on this network.

Subnetting enables the network administrator to further divide the host part of the address into two or more subnets. In this case, a part of the host address is reserved to identify the particular subnet. This is easier to see if we show the IP address in binary format. The full address is:

10010110.11010111.00010001.00001001

The Class B network part is:

10010110.11010111

and the host address is

00010001.00001001

If this network is divided into 14 subnets, however, then the first 4 bits of the host address (0001) are reserved for identifying the subnet.

The subnet mask is the network address plus the bits reserved for identifying the subnetwork. (By convention, the bits for the network address are all set to 1, though it would also work if the bits were set exactly as in the network address.) In this case, therefore, the subnet mask would be 11111111.11111111.11110000.00000000. It's called a mask because it can be used to identify the subnet to which an IP address belongs by performing a bitwise AND operation on the mask and the IP address. The result is the subnetwork address:

Subnet Mask255.255.240.000 11111111.11111111.11110000.00000000
IP Address150.215.017.009 10010110.11010111.00010001.00001001
Subnet Address150.215.016.000 10010110.11010111.00010000.00000000

The subnet address, therefore, is 150.215.016.000.

IP address



DEFINITION - This definition is based on Internet Protocol Version 4. See Internet Protocol Version 6 (IPv6) for a description of the newer 128-bit IP address. Note that the system of IP address classes described here, while forming the basis for IP address assignment, is generally bypassed today by use of Classless Inter-Domain Routing (CIDR) addressing.

In the most widely installed level of the Internet Protocol (IP) today, an IP address is a 32-bit number that identifies each sender or receiver of information that is sent in packets across the Internet. When you request an HTML page or send e-mail, the Internet Protocol part of TCP/IP includes your IP address in the message (actually, in each of the packets if more than one is required) and sends it to the IP address that is obtained by looking up the domain name in the Uniform Resource Locator you requested or in the e-mail address you're sending a note to. At the other end, the recipient can see the IP address of the Web page requestor or the e-mail sender and can respond by sending another message using the IP address it received.

An IP address has two parts: the identifier of a particular network on the Internet and an identifier of the particular device (which can be a server or a workstation) within that network. On the Internet itself - that is, between the router that move packets from one point to another along the route - only the network part of the address is looked at.

The Network Part of the IP Address

The Internet is really the interconnection of many individual networks (it's sometimes referred to as an internetwork). So the Internet Protocol (IP) is basically the set of rules for one network communicating with any other (or occasionally, for broadcast messages, all other networks). Each network must know its own address on the Internet and that of any other networks with which it communicates. To be part of the Internet, an organization needs an Internet network number, which it can request from the Network Information Center (NIC). This unique network number is included in any packet sent out of the network onto the Internet.

The Local or Host Part of the IP Address

In addition to the network address or number, information is needed about which specific machine or host in a network is sending or receiving a message. So the IP address needs both the unique network number and a host number (which is unique within the network). (The host number is sometimes called a local or machine address.)

Part of the local address can identify a subnetwork or subnet address, which makes it easier for a network that is divided into several physical subnetworks (for examples, several different local area networks or ) to handle many devices.

IP Address Classes and Their Formats

Since networks vary in size, there are four different address formats or classes to consider when applying to NIC for a network number:
  • Class A addresses are for large networks with many devices.
  • Class B addresses are for medium-sized networks.
  • Class C addresses are for small networks (fewer than 256 devices).
  • Class D addresses are multicast addresses.

The first few bits of each IP address indicate which of the address class formats it is using. The address structures look like this:

Class A

0Network (7 bits)Local address (24 bits)

Class B

10Network (14 bits)Local address (16 bits)

Class C

110Network (21 bits)Local address (8 bits)

Class D

1110Multicast address (28 bits)

The IP address is usually expressed as four decimal numbers, each representing eight bits, separated by periods. This is sometimes known as the dot address and, more technically, as dotted quad notation. For Class A IP addresses, the numbers would represent "network.local.local.local"; for a Class C IP address, they would represent "network.network.network.local". The number version of the IP address can (and usually is) represented by a name or series of names called the domain name.

The Internet's explosive growth makes it likely that, without some new architecture, the number of possible network addresses using the scheme above would soon be used up (at least, for Class C network addresses). However, a new IP version, IPv6, expands the size of the IP address to 128 bits, which will accommodate a large growth in the number of network addresses. For hosts still using IPv4, the use of subnets in the host or local part of the IP address will help reduce new applications for network numbers. In addition, most sites on today's mostly IPv4 Internet have gotten around the Class C network address limitation by using the Classless Inter-Domain Routing (CIDR) scheme for address notation.

Relationship of the IP Address to the Physical Address

The machine or physical address used within an organization's local area networks may be different than the Internet's IP address. The most typical example is the 48-bit Ethernet address. TCP/IP includes a facility called the Address Resolution Protocol (ARP) that lets the administrator create a table that maps IP addresses to physical addresses. The table is known as the ARP cache.

Static versus Dynamic IP Addresses

The discussion above assumes that IP addresses are assigned on a static basis. In fact, many IP addresses are assigned dynamically from a pool. Many corporate networks and online services economize on the number of IP addresses they use by sharing a pool of IP addresses among a large number of users. If you're an America Online user, for example, your IP address will vary from one logon session to the next because AOL is assigning it to you from a pool that is much smaller than AOL's base of subscribers.

Static IP Address v/s Dynamic IP Address




DEFINITION

A static IP address is a number (in the form of a dotted quad) that is assigned to a computer by an Internet service provider (ISP) to be its permanent address on the Internet. Computers use IP addresses to locate and talk to each other on the Internet, much the same way people use phone numbers to locate and talk to one another on the telephone. When you want to visit whatis.com, your computer asks a domain name system (DNS) server (think telephone information operator) for the correct dotted quad number (think phone number) for whatis.com and your computer uses the answer it receives to connect to the whatis.com server.

It would be simple if every computer that connects to the Internet could have its own static IP number, but when the Internet was first conceived, the architects didn't foresee the need for an unlimited number of IP addresses. Consequently, there are not enough IP numbers to go around. To get around that problem, many Internet service providers limit the number of static IP addresses they allocate, and economize on the remaining number of IP addresses they possess by temporarily assigning an IP address to a requesting Dynamic Host Configuration Protocol (DHCP) computer from a pool of IP addresses. The temporary IP address is called a dynamic IP address.

Requesting DHCP computers receive a dynamic IP address (think temporary phone number) for the duration of that Internet session or for some other specified amount of time. Once the user disconnects from the Internet, their dynamic IP address goes back into the IP address pool so it can be assigned to another user. Even if the user reconnects immediately, odds are they will not be assigned the same IP address from the pool. To keep our telephone telephone analogy going, using a dynamic IP address is similar to using a pay phone. Unless there is a reason to receive a call, the user does not care what number he or she is calling from.

There are times, however, when users who connect to the Internet using dynamic IP wish to allow other computers to locate them. Perhaps they want to use CU-SeeMe or use a VoIP application to make long distance phone calls using their IP connection. In that case, they would need a static IP address. The user has two choices; they can contact their ISP and request a static IP address, or they can use a dynamic DNS service. Either choice will probably involve an additional monthly fee.

Using a dynamic DNS service works as if there was an old-fashioned telephone message service at your computer's disposal. When a user registers with a DNS service and connects to the Internet with a dynamic IP address, the user's computer contacts the DNS service and lets them know what IP address it has been assigned from the pool; the service works with the DNS server to forward the correct address to the requesting DHCP computer. (Think of calling the message service and saying "Hi. I can be reached at 435.44.32.111 right now. Please tell anyone who tries to reach me to call that number.) Using a dynamic DNS service to arrange for computers to find you even though you are using a dynamic IP address is the next-best thing to having a static IP.

IPv6, which has also been called "IPng" (IP Next Generation), lengthens IP addresses from 32 bits to 128 bits and increases the number of available IP addressess significantly, making static IP addresses easier and less expensive to obtain and maintain.

HTTP


Http Request Message Format

The client initiates an HTTP session by opening a TCP connection to the HTTP server with which it wishes to communicate. It then sends request messages to the server, each of which specifies a particular type of action that the user of the HTTP client would like the server to take. Requests can be generated either by specific user action (such as clicking a hyperlink in a Web browser) or indirectly as a result of a prior action (such as a reference to an inline image in an HTML document leading to a request for that image.)

HTTP requests use a message format that is based on the generic message format described in the preceding topic, but specific to the needs of requests. The structure of this format is as follows (see Figure 317):

Figure 317: HTTP Request Message Format

This diagram shows the structural elements of an HTTP request and an example of the sorts of headers a request might contain. Like most HTTP requests, this one carries no entity, so there are no entity headers and the message body is empty. See Figure 318 for the HTTP response format.


Http Response Message Format


Up and down; east and west; black and white; yin and yang. Well, you get the idea. Each request message sent by an HTTP client to a server prompts the server to send back a response message. Actually, in certain cases the server may in fact send two responses, a preliminary response followed by the real one. Usually though, one request yields one response, which indicates the results of the server's processing of the request, and often also carries an entity (file or resource) in the message body.

Like requests, responses use their own specific message format that is based on the HTTP generic message format. The format, shown in Figure 318, is:

Figure 318: HTTP Response Message Format

This figure illustrates the construction of an HTTP response, and includes an example of both message headers and body. The status code “200” indicates that this is a successful response to a request; it contains a brief text HTML entity in the message body. See Figure 317 for the HTTP request format.


Http Methods

A request sent by a client to a server obviously… requests that the server do something. All client/server protocols provide a way for the client to prompt the server to take action, generally by having the client give the server a series of commands. HTTP, in contrast, does not have commands but rather methods. Each client request message begins with the specification of the method that is the subject of the request.

What is the difference between a method and a command? In practical terms, nothing; they are the same. So why does HTTP use the term “method” instead of “command”? Good question. J The answer can be found in the abstract of the standard defining HTTP/1.0, RFC 1945. It states, in part, that HTTP is:

“a generic, stateless, object-oriented protocol which can be used for many tasks…”

In highly simplified terms, object-oriented programming is a technique in which software modules are described not as sets of procedures but as objects that possess attributes. These modules send messages to each other to communicate, and to cause actions to be performed, where the action taken depends on the nature of the object. In object-oriented programming, the procedures each object can perform are called methods.

HTTP is considered to be object-oriented because in many cases, the action taken by a server depends on the object that is the subject of the request. For example, if you ask a server to retrieve a text document, it will send that document, but if you ask for a directory, the server may instead return a default document for that directory. In contrast, a request that specifies the name of a program will result in the program being executed and its output returned (as opposed to the program’s source code being returned.)

URL v/s URI v/s URN

> This is a short list of URL's
>
> http://www.google.com
> http://www.msn.com
> http://www.irc2k.com
>
> This is a short list of URI's
>
> http://www.google.com/index.html
> http://www.irc2k.com/irc2k/index3.php

Both are lists of URIs and URLs, because by definition a URL
is a URI. The term 'URI', as it is widely employed in
Internet specifications today, comprises the two terms 'URL'
and 'URN' (Locator and Name); that is, 'URI' is the term given
to the superset of URLs and URNs. All three are discussed
formally in RFC3986: 1.1.3 <http://www.ietf.org/rfc/rfc3986>
and in greater detail in <http://www.ietf.org/rfc/rfc3305>.

The presence or not of an explicit path doesn't determine
whether a URI is a URL. The determining factor is whether the
URI specifies the 'location' of the resource. (Some URIs
don't, and aren't URLs; e.g., *.) By
that token, all HTTP URIs are URLs, since they specify the
'primary access mechanism', namely HTTP.

* This URN names /The Cambridge Encyclopedia of the English
Language/ by David Crystal. The URN namespace ISBN is
documented in RFC3187 <http://www.ietf.org/rfc/rfc3187>.

Transitional v/s Strict Markup (HTML v/s XHTML)

When promoting web standards, standardistas often talk about XHTML as being more strict than HTML. In a sense it is, since it requires that all elements are properly closed and that attribute values are quoted. But there are two flavours of XHTML 1.0 (three if you count the Frameset DOCTYPE, which is outside the scope of this article), defined by the Transitional and Strict DOCTYPEs. And HTML 4.01 also comes in those flavours.

The names reveal what they are about: Transitional DOCTYPEs are meant for those making the transition from older markup to modern ways. Strict DOCTYPEs are actually the default – the way HTML 4.01 and XHTML 1.0 were constructed to be used.

A Transitional DOCTYPE may be used when you have a lot of legacy markup that cannot easily be converted to comply with a Strict DOCTYPE. But Strict is what you should be aiming for. It encourages, and in some cases enforces, the separation of structure and presentation, moving the presentational aspects from markup to CSS. From the HTML 4 Document Type Definition:

This is HTML 4.01 Strict DTD, which excludes the presentation attributes and elements that W3C expects to phase out as support for style sheets matures. Authors should use the Strict DTD when possible, but may use the Transitional DTD when support for presentation attribute and elements is required.

An additional benefit of using a Strict DOCTYPE is that doing so will ensure that browsers use their strictest, most standards compliant rendering modes.

Tommy Olsson provides a good summary of the benefits of using Strict over Transitional in Ten questions for Tommy Olsson at Web Standards Group:

In my opinion, using a Strict DTD, either HTML 4.01 Strict or XHTML 1.0 Strict, is far more important for the quality of the future web than whether or not there is an X in front of the name. The Strict DTD promotes a separation of structure and presentation, which makes a site so much easier to maintain.

For those looking to start using web standards and valid, semantic markup, it is important to understand the difference between Transitional and Strict DOCTYPEs. For complete listings of the differences between Transitional and Strict DOCTYPEs, see XHTML: Differences between Strict & Transitional, Comparison of Strict and Transitional XHTML, and XHTML1.0 Element Attributes by DTD.

Some of the differences are more likely than others to cause problems for developers moving from a Transitional DOCTYPE to a Strict one, and I’d like to mention a few of those.

Elements that are not allowed in Strict DOCTYPEs

  • center
  • font
  • iframe
  • strike
  • u

Attributes not allowed in Strict DOCTYPEs

  • align (allowed on elements related to tables: col, colgroup, tbody, td, tfoot, th, thead, and tr)
  • language
  • background
  • bgcolor
  • border (allowed on table)
  • height (allowed on img and object)
  • hspace
  • name (allowed in HTML 4.01 Strict, not allowed on form and img in XHTML 1.0 Strict)
  • noshade
  • nowrap
  • target
  • text, link, vlink, and alink
  • vspace
  • width (allowed on img, object, table, col, and colgroup)

Content model differences

An element type’s content model describes what may be contained by an instance of the element type. The most important difference in content models between Transitional and Strict is that blockquote, body, and form elements may only contain block level elements. A few examples:

  • text and images are not allowed immediately inside the body element, and need to be contained in a block level element like p or div
  • input elements must not be direct descendants of a form element
  • text in blockquote elements must be wrapped in a block level element like p or div

Go Strict and move all presentation to CSS

Something that can be helpful when doing the transition from Transitional to Strict DOCTYPEs is to focus on what each element of the page you are working on is instead of how you want it to look.

Worry about looks later and get the structure and semantics right first.

Tuesday, December 5, 2006

What is Web 2.0

The bursting of the dot-com bubble in the fall of 2001 marked a turning point for the web. Many people concluded that the web was overhyped, when in fact bubbles and consequent shakeouts appear to be a common feature of all technological revolutions. Shakeouts typically mark the point at which an ascendant technology is ready to take its place at center stage. The pretenders are given the bum's rush, the real success stories show their strength, and there begins to be an understanding of what separates one from the other.

The concept of "Web 2.0" began with a conference brainstorming session between O'Reilly and MediaLive International. Dale Dougherty, web pioneer and O'Reilly VP, noted that far from having "crashed", the web was more important than ever, with exciting new applications and sites popping up with surprising regularity. What's more, the companies that had survived the collapse seemed to have some things in common. Could it be that the dot-com collapse marked some kind of turning point for the web, such that a call to action such as "Web 2.0" might make sense? We agreed that it did, and so the Web 2.0 Conference was born.

In the year and a half since, the term "Web 2.0" has clearly taken hold, with more than 9.5 million citations in Google. But there's still a huge amount of disagreement about just what Web 2.0 means, with some people decrying it as a meaningless marketing buzzword, and others accepting it as the new conventional wisdom.

This article is an attempt to clarify just what we mean by Web 2.0.

In our initial brainstorming, we formulated our sense of Web 2.0 by example:

Web 1.0
Web 2.0
DoubleClick --> Google AdSense
Ofoto --> Flickr
Akamai --> BitTorrent
mp3.com --> Napster
Britannica Online --> Wikipedia
personal websites --> blogging
evite --> upcoming.org and EVDB
domain name speculation --> search engine optimization
page views --> cost per click
screen scraping --> web services
publishing --> participation
content management systems --> wikis
directories (taxonomy) --> tagging ("folksonomy")
stickiness --> syndication

The list went on and on. But what was it that made us identify one application or approach as "Web 1.0" and another as "Web 2.0"? (The question is particularly urgent because the Web 2.0 meme has become so widespread that companies are now pasting it on as a marketing buzzword, with no real understanding of just what it means. The question is particularly difficult because many of those buzzword-addicted startups are definitely not Web 2.0, while some of the applications we identified as Web 2.0, like Napster and BitTorrent, are not even properly web applications!).

Monday, December 4, 2006

You Might Be A Programmer If

This summary is not available. Please click here to view the post.

Status Code Definitions

Each Status-Code is described below, including a description of which method(s) it can follow and any metainformation required in the response.

1 Informational 1xx

This class of status code indicates a provisional response, consisting only of the Status-Line and optional headers, and is terminated by an empty line. There are no required headers for this class of status code. Since HTTP/1.0 did not define any 1xx status codes, servers MUST NOT send a 1xx response to an HTTP/1.0 client except under experimental conditions.

A client MUST be prepared to accept one or more 1xx status responses prior to a regular response, even if the client does not expect a 100 (Continue) status message. Unexpected 1xx status responses MAY be ignored by a user agent.

Proxies MUST forward 1xx responses, unless the connection between the proxy and its client has been closed, or unless the proxy itself requested the generation of the 1xx response. (For example, if a

proxy adds a "Expect: 100-continue" field when it forwards a request, then it need not forward the corresponding 100 (Continue) response(s).)
1.1 100 Continue

The client SHOULD continue with its request. This interim response is used to inform the client that the initial part of the request has been received and has not yet been rejected by the server. The client SHOULD continue by sending the remainder of the request or, if the request has already been completed, ignore this response. The server MUST send a final response after the request has been completed. See section 8.2.3 for detailed discussion of the use and handling of this status code.
1.2 101 Switching Protocols

The server understands and is willing to comply with the client's request, via the Upgrade message header field (section 14.42), for a change in the application protocol being used on this connection. The server will switch protocols to those defined by the response's Upgrade header field immediately after the empty line which terminates the 101 response.

The protocol SHOULD be switched only when it is advantageous to do so. For example, switching to a newer version of HTTP is advantageous over older versions, and switching to a real-time, synchronous protocol might be advantageous when delivering resources that use such features.
2 Successful 2xx

This class of status code indicates that the client's request was successfully received, understood, and accepted.

2.1 200 OK

The request has succeeded. The information returned with the response is dependent on the method used in the request, for example:

GET an entity corresponding to the requested resource is sent in the response;

HEAD the entity-header fields corresponding to the requested resource are sent in the response without any message-body;

POST an entity describing or containing the result of the action;

TRACE an entity containing the request message as received by the end server.
2.2 201 Created

The request has been fulfilled and resulted in a new resource being created. The newly created resource can be referenced by the URI(s) returned in the entity of the response, with the most specific URI for the resource given by a Location header field. The response SHOULD include an entity containing a list of resource characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content-Type header field. The origin server MUST create the resource before returning the 201 status code. If the action cannot be carried out immediately, the server SHOULD respond with 202 (Accepted) response instead.

A 201 response MAY contain an ETag response header field indicating the current value of the entity tag for the requested variant just created, see section 14.19.
2.3 202 Accepted

The request has been accepted for processing, but the processing has not been completed. The request might or might not eventually be acted upon, as it might be disallowed when processing actually takes place. There is no facility for re-sending a status code from an asynchronous operation such as this.

The 202 response is intentionally non-committal. Its purpose is to allow a server to accept a request for some other process (perhaps a batch-oriented process that is only run once per day) without requiring that the user agent's connection to the server persist until the process is completed. The entity returned with this response SHOULD include an indication of the request's current status and either a pointer to a status monitor or some estimate of when the user can expect the request to be fulfilled.
2.4 203 Non-Authoritative Information

The returned metainformation in the entity-header is not the definitive set as available from the origin server, but is gathered from a local or a third-party copy. The set presented MAY be a subset or superset of the original version. For example, including local annotation information about the resource might result in a superset of the metainformation known by the origin server. Use of this response code is not required and is only appropriate when the response would otherwise be 200 (OK).
2.5 204 No Content

The server has fulfilled the request but does not need to return an entity-body, and might want to return updated metainformation. The response MAY include new or updated metainformation in the form of entity-headers, which if present SHOULD be associated with the requested variant.

If the client is a user agent, it SHOULD NOT change its document view from that which caused the request to be sent. This response is primarily intended to allow input for actions to take place without causing a change to the user agent's active document view, although any new or updated metainformation SHOULD be applied to the document currently in the user agent's active view.

The 204 response MUST NOT include a message-body, and thus is always terminated by the first empty line after the header fields.
2.6 205 Reset Content

The server has fulfilled the request and the user agent SHOULD reset the document view which caused the request to be sent. This response is primarily intended to allow input for actions to take place via user input, followed by a clearing of the form in which the input is given so that the user can easily initiate another input action. The response MUST NOT include an entity.
2.7 206 Partial Content

The server has fulfilled the partial GET request for the resource. The request MUST have included a Range header field (section 14.35) indicating the desired range, and MAY have included an If-Range header field (section 14.27) to make the request conditional.

The response MUST include the following header fields:

- Either a Content-Range header field (section 14.16) indicating
the range included with this response, or a multipart/byteranges
Content-Type including Content-Range fields for each part. If a
Content-Length header field is present in the response, its
value MUST match the actual number of OCTETs transmitted in the
message-body.

- Date

- ETag and/or Content-Location, if the header would have been sent
in a 200 response to the same request

- Expires, Cache-Control, and/or Vary, if the field-value might
differ from that sent in any previous response for the same
variant

If the 206 response is the result of an If-Range request that used a strong cache validator (see section 13.3.3), the response SHOULD NOT include other entity-headers. If the response is the result of an If-Range request that used a weak validator, the response MUST NOT include other entity-headers; this prevents inconsistencies between cached entity-bodies and updated headers. Otherwise, the response MUST include all of the entity-headers that would have been returned with a 200 (OK) response to the same request.

A cache MUST NOT combine a 206 response with other previously cached content if the ETag or Last-Modified headers do not match exactly, see 13.5.4.

A cache that does not support the Range and Content-Range headers MUST NOT cache 206 (Partial) responses.

3 Redirection 3xx

This class of status code indicates that further action needs to be taken by the user agent in order to fulfill the request. The action required MAY be carried out by the user agent without interaction with the user if and only if the method used in the second request is GET or HEAD. A client SHOULD detect infinite redirection loops, since such loops generate network traffic for each redirection.

Note: previous versions of this specification recommended a
maximum of five redirections. Content developers should be aware
that there might be clients that implement such a fixed
limitation.

3.1 300 Multiple Choices

The requested resource corresponds to any one of a set of representations, each with its own specific location, and agent- driven negotiation information (section 12) is being provided so that the user (or user agent) can select a preferred representation and redirect its request to that location.

Unless it was a HEAD request, the response SHOULD include an entity containing a list of resource characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content- Type header field. Depending upon the format and the capabilities of

the user agent, selection of the most appropriate choice MAY be performed automatically. However, this specification does not define any standard for such automatic selection.

If the server has a preferred choice of representation, it SHOULD include the specific URI for that representation in the Location field; user agents MAY use the Location field value for automatic redirection. This response is cacheable unless indicated otherwise.
3.2 301 Moved Permanently

The requested resource has been assigned a new permanent URI and any future references to this resource SHOULD use one of the returned URIs. Clients with link editing capabilities ought to automatically re-link references to the Request-URI to one or more of the new references returned by the server, where possible. This response is cacheable unless indicated otherwise.

The new permanent URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s).

If the 301 status code is received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.

Note: When automatically redirecting a POST request after
receiving a 301 status code, some existing HTTP/1.0 user agents
will erroneously change it into a GET request.

3.3 302 Found

The requested resource resides temporarily under a different URI. Since the redirection might be altered on occasion, the client SHOULD continue to use the Request-URI for future requests. This response is only cacheable if indicated by a Cache-Control or Expires header field.

The temporary URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s).

If the 302 status code is received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.

Note: RFC 1945 and RFC 2068 specify that the client is not allowed
to change the method on the redirected request. However, most
existing user agent implementations treat 302 as if it were a 303
response, performing a GET on the Location field-value regardless
of the original request method. The status codes 303 and 307 have
been added for servers that wish to make unambiguously clear which
kind of reaction is expected of the client.

3.4 303 See Other

The response to the request can be found under a different URI and SHOULD be retrieved using a GET method on that resource. This method exists primarily to allow the output of a POST-activated script to redirect the user agent to a selected resource. The new URI is not a substitute reference for the originally requested resource. The 303 response MUST NOT be cached, but the response to the second (redirected) request might be cacheable.

The different URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s).

Note: Many pre-HTTP/1.1 user agents do not understand the 303
status. When interoperability with such clients is a concern, the
302 status code may be used instead, since most user agents react
to a 302 response as described here for 303.

3.5 304 Not Modified

If the client has performed a conditional GET request and access is allowed, but the document has not been modified, the server SHOULD respond with this status code. The 304 response MUST NOT contain a message-body, and thus is always terminated by the first empty line after the header fields.

The response MUST include the following header fields:

- Date, unless its omission is required by section 14.18.1

If a clockless origin server obeys these rules, and proxies and clients add their own Date to any response received without one (as already specified by [RFC 2068], section 14.19), caches will operate correctly.

- ETag and/or Content-Location, if the header would have been sent
in a 200 response to the same request

- Expires, Cache-Control, and/or Vary, if the field-value might
differ from that sent in any previous response for the same
variant

If the conditional GET used a strong cache validator (see section 13.3.3), the response SHOULD NOT include other entity-headers. Otherwise (i.e., the conditional GET used a weak validator), the response MUST NOT include other entity-headers; this prevents inconsistencies between cached entity-bodies and updated headers.

If a 304 response indicates an entity not currently cached, then the cache MUST disregard the response and repeat the request without the conditional.

If a cache uses a received 304 response to update a cache entry, the cache MUST update the entry to reflect any new field values given in the response.
3.6 305 Use Proxy

The requested resource MUST be accessed through the proxy given by the Location field. The Location field gives the URI of the proxy. The recipient is expected to repeat this single request via the proxy. 305 responses MUST only be generated by origin servers.

Note: RFC 2068 was not clear that 305 was intended to redirect a
single request, and to be generated by origin servers only. Not
observing these limitations has significant security consequences.

3.7 306 (Unused)

The 306 status code was used in a previous version of the specification, is no longer used, and the code is reserved.
3.8 307 Temporary Redirect

The requested resource resides temporarily under a different URI. Since the redirection MAY be altered on occasion, the client SHOULD continue to use the Request-URI for future requests. This response is only cacheable if indicated by a Cache-Control or Expires header field.

The temporary URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s) , since many pre-HTTP/1.1 user agents do not understand the 307 status. Therefore, the note SHOULD contain the information necessary for a user to repeat the original request on the new URI.

If the 307 status code is received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.

4 Client Error 4xx

The 4xx class of status code is intended for cases in which the client seems to have erred. Except when responding to a HEAD request, the server SHOULD include an entity containing an explanation of the error situation, and whether it is a temporary or permanent condition. These status codes are applicable to any request method. User agents SHOULD display any included entity to the user.

If the client is sending data, a server implementation using TCP SHOULD be careful to ensure that the client acknowledges receipt of the packet(s) containing the response, before the server closes the input connection. If the client continues sending data to the server after the close, the server's TCP stack will send a reset packet to the client, which may erase the client's unacknowledged input buffers before they can be read and interpreted by the HTTP application.
4.1 400 Bad Request

The request could not be understood by the server due to malformed syntax. The client SHOULD NOT repeat the request without modifications.
4.2 401 Unauthorized

The request requires user authentication. The response MUST include a WWW-Authenticate header field (section 14.47) containing a challenge applicable to the requested resource. The client MAY repeat the request with a suitable Authorization header field (section 14.8). If the request already included Authorization credentials, then the 401 response indicates that authorization has been refused for those credentials. If the 401 response contains the same challenge as the prior response, and the user agent has already attempted authentication at least once, then the user SHOULD be presented the entity that was given in the response, since that entity might include relevant diagnostic information. HTTP access authentication is explained in "HTTP Authentication: Basic and Digest Access Authentication" [43].
4.3 402 Payment Required

This code is reserved for future use.
4.4 403 Forbidden

The server understood the request, but is refusing to fulfill it. Authorization will not help and the request SHOULD NOT be repeated. If the request method was not HEAD and the server wishes to make public why the request has not been fulfilled, it SHOULD describe the reason for the refusal in the entity. If the server does not wish to make this information available to the client, the status code 404 (Not Found) can be used instead.
4.5 404 Not Found

The server has not found anything matching the Request-URI. No indication is given of whether the condition is temporary or permanent. The 410 (Gone) status code SHOULD be used if the server knows, through some internally configurable mechanism, that an old resource is permanently unavailable and has no forwarding address. This status code is commonly used when the server does not wish to reveal exactly why the request has been refused, or when no other response is applicable.
4.6 405 Method Not Allowed

The method specified in the Request-Line is not allowed for the resource identified by the Request-URI. The response MUST include an Allow header containing a list of valid methods for the requested resource.
4.7 406 Not Acceptable

The resource identified by the request is only capable of generating response entities which have content characteristics not acceptable according to the accept headers sent in the request.

Unless it was a HEAD request, the response SHOULD include an entity containing a list of available entity characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content-Type header field. Depending upon the format and the capabilities of the user agent, selection of the most appropriate choice MAY be performed automatically. However, this specification does not define any standard for such automatic selection.

Note: HTTP/1.1 servers are allowed to return responses which are
not acceptable according to the accept headers sent in the
request. In some cases, this may even be preferable to sending a
406 response. User agents are encouraged to inspect the headers of
an incoming response to determine if it is acceptable.

If the response could be unacceptable, a user agent SHOULD temporarily stop receipt of more data and query the user for a decision on further actions.
4.8 407 Proxy Authentication Required

This code is similar to 401 (Unauthorized), but indicates that the client must first authenticate itself with the proxy. The proxy MUST return a Proxy-Authenticate header field (section 14.33) containing a challenge applicable to the proxy for the requested resource. The client MAY repeat the request with a suitable Proxy-Authorization header field (section 14.34). HTTP access authentication is explained in "HTTP Authentication: Basic and Digest Access Authentication" [43].
10.4.9 408 Request Timeout

The client did not produce a request within the time that the server was prepared to wait. The client MAY repeat the request without modifications at any later time.
4.10 409 Conflict

The request could not be completed due to a conflict with the current state of the resource. This code is only allowed in situations where it is expected that the user might be able to resolve the conflict and resubmit the request. The response body SHOULD include enough

information for the user to recognize the source of the conflict. Ideally, the response entity would include enough information for the user or user agent to fix the problem; however, that might not be possible and is not required.

Conflicts are most likely to occur in response to a PUT request. For example, if versioning were being used and the entity being PUT included changes to a resource which conflict with those made by an earlier (third-party) request, the server might use the 409 response to indicate that it can't complete the request. In this case, the response entity would likely contain a list of the differences between the two versions in a format defined by the response Content-Type.
4.11 410 Gone

The requested resource is no longer available at the server and no forwarding address is known. This condition is expected to be considered permanent. Clients with link editing capabilities SHOULD delete references to the Request-URI after user approval. If the server does not know, or has no facility to determine, whether or not the condition is permanent, the status code 404 (Not Found) SHOULD be used instead. This response is cacheable unless indicated otherwise.

The 410 response is primarily intended to assist the task of web maintenance by notifying the recipient that the resource is intentionally unavailable and that the server owners desire that remote links to that resource be removed. Such an event is common for limited-time, promotional services and for resources belonging to individuals no longer working at the server's site. It is not necessary to mark all permanently unavailable resources as "gone" or to keep the mark for any length of time -- that is left to the discretion of the server owner.
4.12 411 Length Required

The server refuses to accept the request without a defined Content- Length. The client MAY repeat the request if it adds a valid Content-Length header field containing the length of the message-body in the request message.
4.13 412 Precondition Failed

The precondition given in one or more of the request-header fields evaluated to false when it was tested on the server. This response code allows the client to place preconditions on the current resource metainformation (header field data) and thus prevent the requested method from being applied to a resource other than the one intended.
4.14 413 Request Entity Too Large

The server is refusing to process a request because the request entity is larger than the server is willing or able to process. The server MAY close the connection to prevent the client from continuing the request.

If the condition is temporary, the server SHOULD include a Retry- After header field to indicate that it is temporary and after what time the client MAY try again.
4.15 414 Request-URI Too Long

The server is refusing to service the request because the Request-URI is longer than the server is willing to interpret. This rare condition is only likely to occur when a client has improperly converted a POST request to a GET request with long query information, when the client has descended into a URI "black hole" of redirection (e.g., a redirected URI prefix that points to a suffix of itself), or when the server is under attack by a client attempting to exploit security holes present in some servers using fixed-length buffers for reading or manipulating the Request-URI.
4.16 415 Unsupported Media Type

The server is refusing to service the request because the entity of the request is in a format not supported by the requested resource for the requested method.
4.17 416 Requested Range Not Satisfiable

A server SHOULD return a response with this status code if a request included a Range request-header field (section 14.35), and none of the range-specifier values in this field overlap the current extent of the selected resource, and the request did not include an If-Range request-header field. (For byte-ranges, this means that the first- byte-pos of all of the byte-range-spec values were greater than the current length of the selected resource.)

When this status code is returned for a byte-range request, the response SHOULD include a Content-Range entity-header field specifying the current length of the selected resource (see section 14.16). This response MUST NOT use the multipart/byteranges content- type.
4.18 417 Expectation Failed

The expectation given in an Expect request-header field (see section 14.20) could not be met by this server, or, if the server is a proxy, the server has unambiguous evidence that the request could not be met by the next-hop server.

5 Server Error 5xx

Response status codes beginning with the digit "5" indicate cases in which the server is aware that it has erred or is incapable of performing the request. Except when responding to a HEAD request, the server SHOULD include an entity containing an explanation of the error situation, and whether it is a temporary or permanent condition. User agents SHOULD display any included entity to the user. These response codes are applicable to any request method.
5.1 500 Internal Server Error

The server encountered an unexpected condition which prevented it from fulfilling the request.
5.2 501 Not Implemented

The server does not support the functionality required to fulfill the request. This is the appropriate response when the server does not recognize the request method and is not capable of supporting it for any resource.
5.3 502 Bad Gateway

The server, while acting as a gateway or proxy, received an invalid response from the upstream server it accessed in attempting to fulfill the request.
5.4 503 Service Unavailable

The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. The implication is that this is a temporary condition which will be alleviated after some delay. If known, the length of the delay MAY be indicated in a Retry-After header. If no Retry-After is given, the client SHOULD handle the response as it would for a 500 response.

Note: The existence of the 503 status code does not imply that a
server must use it when becoming overloaded. Some servers may wish
to simply refuse the connection.

5.5 504 Gateway Timeout

The server, while acting as a gateway or proxy, did not receive a timely response from the upstream server specified by the URI (e.g. HTTP, FTP, LDAP) or some other auxiliary server (e.g. DNS) it needed to access in attempting to complete the request.

Note: Note to implementors: some deployed proxies are known to
return 400 or 500 when DNS lookups time out.

5.6 505 HTTP Version Not Supported

The server does not support, or refuses to support, the HTTP protocol version that was used in the request message. The server is indicating that it is unable or unwilling to complete the request using the same major version as the client, as described in section 3.1, other than with this error message. The response SHOULD contain an entity describing why that version is not supported and what other protocols are supported by that server.