Application-layer protocols
The application layer of the TCP/IP stack is where the protocols that people actually notice live: the web, email, file transfer and peer-to-peer sharing. The syllabus names six of them: HTTP, FTP, POP3, IMAP, SMTP and BitTorrent. Paper 3 questions ask you to state what each protocol is for, to choose the right one for a task, to describe how email moves from sender to recipient using SMTP and POP3/IMAP, and to explain how BitTorrent shares files between peers.
A quick map
| Protocol | Full name | Purpose | Typical port (transport) |
|---|---|---|---|
| HTTP | HyperText Transfer Protocol | Transfers web pages and other resources between a web server and a browser (client) | 80 (TCP) |
| HTTPS | HTTP Secure | HTTP encrypted using SSL/TLS | 443 (TCP) |
| FTP | File Transfer Protocol | Transfers files between a client and a server; upload, download, rename, delete, list directories | 21 control, 20 data (TCP) |
| SMTP | Simple Mail Transfer Protocol | Sends (pushes) email from a client to a mail server and between mail servers | 25 / 587 (TCP) |
| POP3 | Post Office Protocol version 3 | Retrieves (pulls) email from a mail server to a client, normally downloading and deleting it from the server | 110 (TCP) |
| IMAP | Internet Message Access Protocol | Retrieves email but keeps it on the server, synchronising folders across devices | 143 (TCP) |
| BitTorrent | Peer-to-peer file sharing: a file is downloaded in pieces from many peers at once | various (TCP/UDP) |
Port numbers are useful supporting detail; the purpose of each protocol is what earns marks.
HTTP and HTTPS
HTTP is the protocol of the World Wide Web. It is a client–server, request–response protocol: a client (usually a browser) sends a request, and the server sends back a response.
A request names a method and a resource:
- GET asks for a resource, such as
GET /index.html. - POST sends data to the server, such as the contents of a form.
- Others include PUT (upload/replace a resource) and DELETE.
The response starts with a status code: 200 OK, 301 Moved Permanently, 404 Not Found, 500 Internal Server Error. It then has headers (content type, length, cookies) and the body (the HTML, image or other data).
HTTP is stateless: each request is independent, and the server does not remember earlier requests. Websites that need memory (log-ins, shopping baskets) use cookies or session identifiers sent with each request.
A web page usually needs many HTTP requests: one for the HTML, then one for each image, stylesheet and script it refers to.
HTTPS is HTTP sent through an encrypted SSL/TLS connection. The messages are the same; the difference is that they cannot be read or altered in transit and the server's identity is verified with a digital certificate. See SSL and TLS.
FTP
FTP transfers files between a client and a server across a network. A user (or program) logs in to an FTP server with a user name and password (or as anonymous on a public server) and can then:
- upload and download files
- list, create and remove directories
- rename and delete files
FTP uses two connections: a control connection (port 21) that carries commands such as USER, PASS, LIST, RETR (retrieve a file) and STOR (store a file), and a separate data connection (port 20 or a negotiated port) that carries the file contents. Files can be sent in text (ASCII) mode, which converts line endings between systems, or binary mode, which copies the bytes exactly.
Plain FTP sends passwords and data unencrypted. Secure alternatives are FTPS (FTP over SSL/TLS) and SFTP (file transfer over SSH).
Typical use: uploading the files of a website to a web server; transferring large files between organisations.
Email: SMTP, POP3 and IMAP
Email uses different protocols for sending and receiving.
SMTP is a push protocol used to send email: from the sender's email client to the sender's mail server, and from the sender's mail server to the recipient's mail server.
POP3 and IMAP are pull protocols used by the recipient's email client to retrieve email from the recipient's mail server.
- The sender writes the email in an email client (or webmail).
- The client uses SMTP to send the email to the sender's mail server (outgoing SMTP server).
- The sender's mail server uses DNS to find the mail server responsible for the recipient's domain (the part after
@). - The sender's server uses SMTP to send the email to the recipient's mail server, possibly via other servers.
- The recipient's mail server stores the email in the recipient's mailbox.
- When the recipient opens their email client, it uses POP3 or IMAP to fetch the email from their mail server.
Webmail (using an email account in a browser) adds a layer: the browser talks to the webmail server with HTTP/HTTPS, and the webmail server itself uses SMTP and IMAP behind the scenes.
POP3 compared with IMAP
| POP3 | IMAP | |
|---|---|---|
| Where emails are kept | Downloaded to the device and (by default) deleted from the server | Kept on the server; the client shows a synchronised copy |
| Several devices | Poor: an email downloaded to the laptop is not on the phone | Good: every device sees the same mailbox and folders |
| Folders and flags | Stored locally only | Folders, read/unread and flags stored on the server and synchronised |
| Offline reading | Yes, everything is on the device | Only messages already cached |
| Server storage needed | Little | More (all mail stays on the server) |
| Download | Whole messages | Can fetch headers first, and parts of messages (e.g. skip attachments) |
| Backup | Only the user's device | Server (usually backed up by the provider) |
IMAP is the normal choice now that people read email on several devices. POP3 suits a single computer with limited server storage or a user who wants a local copy that does not depend on the server.
BitTorrent: peer-to-peer file sharing
In client–server file transfer (HTTP, FTP), one server sends the whole file to every client. If ten thousand people want the same large file at once, the server's bandwidth is the bottleneck. BitTorrent removes the bottleneck by making every downloader an uploader too.
BitTorrent is a peer-to-peer protocol for distributing files. The file is split into small pieces; each peer downloads pieces from many other peers simultaneously and, at the same time, uploads the pieces it already has to other peers.
The vocabulary you need:
| Term | Meaning |
|---|---|
| Peer | Any computer taking part in sharing the file |
| Swarm | All the peers sharing (uploading or downloading) a particular file |
| Seed (seeder) | A peer that has the complete file and is uploading it |
| Leech (leecher) | A peer that is downloading and does not yet have the complete file (sometimes used for peers that download without uploading) |
| Torrent file | A small file containing metadata about the file to be shared: its name and size, the size of each piece, a hash of every piece, and the address of the tracker |
| Tracker | A server that keeps a record of which peers are in the swarm; it does not hold the file itself |
| Pieces (chunks) | The fixed-size parts the file is split into |
- Someone who has the file creates a torrent file describing it (piece size, hash of each piece, tracker address) and publishes the torrent file, for example on a website. They become the first seed.
- A user who wants the file downloads the small torrent file and opens it in a BitTorrent client.
- The client contacts the tracker, which returns a list of peers in the swarm (seeds and other downloaders).
- The client connects to several peers and requests different pieces from different peers at the same time, often the rarest pieces first so that every piece stays available.
- Each piece received is checked against its hash in the torrent file. A corrupted or tampered piece fails the check and is requested again.
- As soon as the client holds a piece, it uploads that piece to other peers who need it. Peers that upload more tend to be rewarded with faster downloads (tit-for-tat).
- When every piece has been received and checked, the client reassembles the file. If it keeps running, it becomes a seed.
Benefits and drawbacks:
- Benefits: very fast downloads of popular files, because many peers upload at once; no single server needs huge bandwidth; the more popular a file is, the faster it spreads; no single point of failure once the swarm exists; hashes ensure integrity.
- Drawbacks: a file can become unavailable if there are no seeds and no peer has every piece; each user's upload bandwidth is used; peers' IP addresses are visible to each other (privacy); the protocol is often used to share copyrighted material illegally, and files from unknown sources may contain malware.
Legitimate uses include distributing Linux operating-system images, large game updates and open data sets.
Worked examples
State the most appropriate protocol for each task.
(a) A web designer uploads the new files of a website to the company's web server. (b) A browser requests a news page. (c) A user reads their email on a phone, a tablet and a laptop, and wants the same folders on each. (d) A software company wants 50 000 users to download a 6 GB file in the same hour without buying a very large server. (e) An email client sends a message to the user's mail server.
Solution
(a) FTP (or a secure version such as SFTP/FTPS). (b) HTTP (HTTPS for a secure site). (c) IMAP, because messages and folders stay on the server and are synchronised across devices. (d) BitTorrent, so users download pieces from each other and the company's server is not the bottleneck. (e) SMTP.
Ali, whose address is at mailA.com, sends an email to Bea, whose address is at mailB.org. Bea reads it on her laptop using an email client. Describe the protocols used and the role of each. [5]
Solution
- Ali's email client sends the message to Ali's mail server (
mailA.com) using SMTP. - Ali's mail server uses DNS to look up the mail server for
mailB.org. - Ali's mail server forwards the email to Bea's mail server using SMTP; SMTP is a push protocol used to send mail between servers.
- Bea's mail server stores the email in Bea's mailbox until it is collected.
- Bea's email client retrieves the email from her server using POP3 (downloads it, normally removing it from the server) or IMAP (synchronises a copy, leaving it on the server).
A travelling salesperson often has no internet connection, uses only one laptop, and has a very small mailbox allowance on the server. Recommend POP3 or IMAP, with reasons.
Solution
POP3. It downloads the emails to the laptop and deletes them from the server, so the small server allowance is not filled; the emails are then stored locally and can be read while offline. The usual disadvantage of POP3 (emails not available on other devices) does not matter because the salesperson uses only one laptop.
Explain how the BitTorrent protocol allows a large file to be distributed to many users quickly. [6]
Solution
- The file is split into many small pieces.
- A torrent file holds metadata including the size and hash of each piece and the address of a tracker.
- A user's BitTorrent client opens the torrent file and contacts the tracker, which supplies a list of peers in the swarm that have all or part of the file.
- The client downloads different pieces from many peers simultaneously, so the download is not limited by the bandwidth of a single server.
- As soon as it has a piece, the client uploads it to other peers, so every downloader also adds upload capacity; the more peers, the faster the distribution.
- Each piece is verified using its hash; when all pieces are received they are reassembled, and the user can continue as a seed.
A user types http://example.com/news.html into a browser. The page contains three images. Describe the HTTP messages exchanged.
Solution
The browser (client) opens a TCP connection to the server on port 80 and sends an HTTP request GET /news.html. The server sends an HTTP response with status 200 OK, headers (including the content type text/html) and the HTML of the page in the body. The browser parses the HTML, finds three image references and sends a GET request for each (often over the same connection), receiving a 200 OK response containing each image. If one image did not exist the response would be 404 Not Found. Because HTTP is stateless, each request is handled independently.
- SMTP sends email; POP3 and IMAP receive it. Saying "POP3 sends email to the server" is a common lost mark.
- The BitTorrent tracker does not store or send the file; it only tells peers about each other.
- HTTP is not encrypted; HTTPS is. Do not describe HTTP as secure.
- Peer-to-peer does not mean "no servers at all": BitTorrent normally uses a tracker server, though the file itself moves only between peers.
- Questions often give a scenario and ask "identify a protocol and state its purpose". Name the protocol and say what it does in the scenario: "IMAP retrieves the email from the server and keeps it there so that it can be read on several devices".
- For BitTorrent, use the vocabulary: swarm, seed, leech/peer, tracker, torrent file, pieces, hash. A description without the terms earns fewer marks.
- Email journey questions are marked on order: client to server (SMTP), server to server (SMTP), server to client (POP3/IMAP).
- HTTP: request–response transfer of web resources between browser and web server (GET, POST, status codes); stateless; HTTPS adds SSL/TLS.
- FTP: transfer and management of files between client and server; separate control and data connections; unencrypted by default.
- SMTP pushes email from client to server and between servers.
- POP3 pulls email to one device and usually deletes it from the server; IMAP keeps email on the server and synchronises across devices.
- BitTorrent is peer-to-peer: a torrent file describes the pieces and their hashes, a tracker lists peers, and each peer downloads pieces from many peers while uploading to others.
Practice questions
- State the purpose of each of these protocols: HTTP, FTP, SMTP.
- Explain why email needs a different protocol for sending and for receiving.
- Give two differences between POP3 and IMAP.
- Define the terms seed, swarm and tracker in the context of BitTorrent.
- Explain how BitTorrent ensures that the file a user receives has not been corrupted.
- Give two drawbacks of BitTorrent compared with downloading a file from a single server using HTTP.
- A user accesses their email through a web browser. Identify the protocols used (a) between the browser and the webmail server, (b) when the webmail server sends a message the user has written.
- A games company releases a 40 GB update. Compare distributing it with HTTP from the company's servers and with BitTorrent.
Answers
-
HTTP: transfers web pages and other resources from a web server to a client (browser) using requests and responses. FTP: transfers files between a client and a server, including upload, download and file management (rename, delete, list). SMTP: sends email from a client to a mail server and between mail servers.
-
Sending is a push operation: the sender's client and servers forward the message towards the recipient's server whenever it is sent (SMTP). Receiving is a pull operation: the recipient's mail is held on their server until their client connects and asks for it, which may be much later; this requires a protocol that can authenticate the user, list and fetch messages from a mailbox (POP3/IMAP). The recipient's device is not always online, so it cannot receive pushed mail directly.
-
Any two of: POP3 downloads emails and usually deletes them from the server, IMAP keeps them on the server; IMAP synchronises across several devices, POP3 does not; IMAP keeps folders and read status on the server; POP3 needs less server storage; POP3 allows full offline access to all downloaded mail; IMAP can download headers only.
-
Seed: a peer that has the complete file and uploads it to others. Swarm: all the peers sharing (uploading or downloading) a particular file. Tracker: a server that keeps a list of the peers in a swarm and gives it to clients so that they can connect to each other.
-
The torrent file contains a cryptographic hash of each piece. When a piece is downloaded, the client calculates its hash and compares it with the value in the torrent file. If they differ, the piece is corrupted (or tampered with) and is discarded and downloaded again, possibly from a different peer.
-
Any two of: availability depends on peers; if there are no seeds the file may not be complete; uses the user's upload bandwidth; IP addresses visible to other peers; risk of malware from untrusted sources; often blocked by networks or associated with illegal sharing; needs a client program.
-
(a) HTTP (in practice HTTPS). (b) SMTP.
-
With HTTP, every player downloads all 40 GB from the company's servers, so the company needs enormous bandwidth and server capacity; at release, demand peaks and downloads become slow or fail; but it is simple, works through any browser or launcher and the company controls availability. With BitTorrent, players download pieces from each other as well as from the company's seeds, so the company's bandwidth costs fall and downloads get faster as more players join; hashes guarantee integrity. However, players use their own upload bandwidth, some networks block peer-to-peer traffic, and players' IP addresses are visible. A hybrid (company servers as seeds plus peer sharing) is common.