UNP – http

Given below is a reference to slides from a US university (links obtained via Google search and so presumed to be freely accessible to anybody on the net) which are the key teaching slides for this topic:

www.cs.rpi.edu/courses/fall02/netprog/notes/web/web.ppt – till slide 39, you may ignore the later slides.

[The slides that I used when I taught the course could not be located in US university sites easily via Google Search. But I found them at the following two links:



Additional references:

HTTP Made Really Easy: http://www.jmarshall.com/easy/http/

RFC1945 – HTTP/1.0: http://www.faqs.org/rfcs/rfc1945.html

RFC2616 – HTTP/1.1: http://www.faqs.org/rfcs/rfc2616.html

RFC 2396 – Uniform Resource Identifiers (URI): Generic Syntax: http://www.faqs.org/rfcs/rfc2396.html

RFC822 – ARPA Internet Text Messages Format: http://www.faqs.org/rfcs/rfc822.html

RFC 1521 – MIME (Multipurpose Internet Mail Extensions) Part One: http://www.faqs.org/rfcs/rfc1521.html

RFC 1522 – MIME (Multipurpose Internet Mail Extensions) Part Two: http://www.faqs.org/rfcs/rfc1522.html

Tutorial – HTTP Proxy – http://www.ragestorm.net/tutorial?id=15

RFC 2617 – HTTP Authentication: Basic and Digest Access Authentication: http://www.faqs.org/rfcs/rfc2617.html

Base 64 Resources

Online encoder/decoder: http://www.paulschou.com/tools/xlate/

Source code for encode/decode: http://www.adp-gmbh.ch/cpp/common/base64.html


 Assignment 1

Write a browser “brow” implementing HTTP GET request:       brow –d GET ipaddress /………

The server response should be displayed on stdout. There should be a debug option –d to display (on/off) the reply messages.

UNP – Threads

Given below is a reference to slides from a US university (links obtained via Google search and so presumed to be freely accessible to anybody on the net) which are the key teaching slides for this topic:


[The slides that I used when I taught the course could not be located in US university sites easily via Google Search. But I found them at the following two links:

http://www.learningace.com/doc/1205385/e48a70888d6eecc154d027da7ecc4279/threads http://www.docstoc.com/docs/111876661/Threads-Programming]

An interesting additional reference is: http://www.cs.fsu.edu/~baker/realtime/restricted/notes/pthreads.html

Show/explain the following code:

nonblock/strclifork.c     threads/strclithread.c

threads/tcpserv01.c      threads/tcpserv02.c


Multi-threaded program which increments a global variable incorrectly (‘No Synchronization’)


Corrected version of above program using a mutex to protect the shared variable


Condition Variables

‘Web client and simultaneous connections’ using condition variables to specify which thread to wait for


Client/Server Design Alternatives

TCP Prethreaded Server, per Thread accept()

server/pthread07.h              server/serv07.c            server/pthread07.c

TCP Prethreaded Server, Main Thread accept()

server/pthread08.h             server/serv08.c           server/pthread08.c


Assignment 1

Modify the Saisays server and client (stresser) programs of Assignment 4 – IntroSockets by making them ‘multi-threaded’ instead of spawning multiple processes for client connections.

Repeat the stress test against this server and perform a comparison of these results with the results obtained in the previous assignment.

Write a report giving your observations and conclusions regarding the assignment.

Assignment 2

Understand and try out the ‘mutexes’ and ‘condition variables’ examples in Chapter 26 (Threads) of the course book.

Understand and try out the different examples of client/server design alternatives examined in Chapter 30 of the book.

Assignment 3

Consider a fictitious Simple File Transfer Protocol (SFTP) which is as follows:

  • It is a line oriented ASCII protocol i.e. requests and replies both are in ASCII (not binary) and terminated by \n
  • It allows clients to access files in just one directory (called as SFTP data directory) managed by the server. It can list the files in the directory and also get a file. To simplify the protocol we will limit the files in the directory to be only text files (and no subdirectories).
  • Requests consist of just request keywords and optional parameters. They are as follows:
    • List\n
    • Get filename\n
    • Quit\n
  • Responses consist of a mandatory header and optional body.
  • Response Header always has a status line followed by a content length line as follows:
    • Status nnn\n
    • Content-Length  nnnnnn\n
  • Status code values are as follows:
    • 200      – Means OK
    • 401      – Invalid Command
    • 402      – File not found
  • All responses will have status. If the request has been handled successfully then the Status response of 200 is returned otherwise an appropriate Status error response is returned.
  • Content-Length line gives the length of the contents that follow after the header. Examples are as follows:
    • Content-Length 0\n    – For the case when there is no data that follows. Typically for error status responses and if for List there are 0 (no) files in the SFTP data directory.
    • Content-Length 246\n   – For the case when 246 bytes of data follow the header. Typically for List if the string containing the filenames including the return characters is 246 bytes and for Get if the text file whose data follows the header has a file size of 246 bytes.
  • List command lists the files in the SFTP data directory. Each filename is on a separate line ended by a \n. If there are no files in the directory then no data is returned to client (but response header is returned).
  • Get command gets the file data that it requests. E.g. Get abc.txt\n. If the file does not exist a suitable error status is returned to the client.
  • Quit command causes the server to return status of OK and Content-Length 0, after which the server closes the connection.

Write an SFTP server program which implements the application level protocol (SFTP) Simple File Transfer Protocol. (Implement only the ‘GET’ command).

The implementation should be multi-threaded with the server using prethreading to create a pool of available threads when it starts.

Implement the server program first with the main thread calling accept() and then with each thread in the pool calling accept().

Compare the performance of the two approaches using statistics like the response time of the server.

Write a report giving your observations and conclusions regarding the assignments.

Reading Assignments

Chapter 26 Threads

Chapter 30 Sections 30.11 and 30.12

Additional Reading

Implementing a read-write mutex:




Need mutex for multiple read:

Do I need to lock a mutex for reading a variable:

Per-thread data example: http://www.cs.fsu.edu/~baker/realtime/restricted/examples/threads/perthread.c

POSIX Threads programming tutorial: Lawrence Livermore National Laboratory

Introduction to Parallel Computing: Lawrence Livermore National Laboratory
(Has a section on Parallel Programming Models where it notes that Threads Model and Message Passing Model are two of the many models for Parallel Programming)

UNP – Sockets – Miscellaneous

Line Oriented IO Issues

read functions which buffer data may not work properly with select

readline() function reads a buffer of data and from that buffer returns a line of input back to caller. Additional read data that is available will be kept in readline() function’s buffer and returned at subsequent call. Further readline() will block till at least one line of data is read.

Show/explain the following code:

test/readline1.c, lib/readline.c, lib/readn.c

Such a readline function is very useful for network programs dealing with line oriented protocols like HTTP, FTP etc. However there are many issues with such buffered IO functions:

  • If select is used to wait for input from multiple sources (say socket and stdin), select may block even though readline() has a line of data (read during earlier invocation of readline where read returned multiple lines in one call). Select uses the stdin fd and is unaware of the internal buffer of readline().
  • The second version of readline() above allows for such issues to be fixed by the programmer first checking readline’s exposed buffer for data before calling select. But this increases complexity of programming.
  • If readn and readline function calls are mixed unexpected behaviour may occur.
  • Even if the network programs expect data to be exchanged only in lines, due to bugs or malicious attempts some data may be sent which is not line terminated. Using a function like readline will make it difficult for the network program to detect such data and flag it as an error.
  • For the same reasons mentioned above stdio functions like fgets should not be used in socket programs.

Readn function suffers from some of the issues that readline has. So ideally one should avoid using readn type of functions as well.

So what should one do if lines (or fixed amt of data) have to be read (and written)

  • Always think in terms of buffers of data being read and written over sockets.
  • If a line is expected, read data into a buffer and check the buffer to see whether it contains a line
  • If a fixed amt of data is expected typically one has to continue reading till at least expected amt of data has arrived. But this should be done at a top level instead of in a readn kind of function. This way the partially read data is always available in a buffer for the top level code to do any error checking or similar kind of task, instead of the partially read data being hidden away in a readn() routine’s private buffer.

Reading Assignments

Section 3.9 (readn, writen, and readline functions)

Section 6.4 to Section 6.7

Socket Options

  • Various attributes are used to determine the behavior of sockets.
  • Setting options tells the OS/Protocol Stack the behavior we want.
  • Support is provided for generic options (apply to all sockets) and protocol specific options.

getsockopt() gets the current value of a socket option.

setsockopt() is used to set the value of a socket option.

int getsockopt( int sockfd,

int level,

int optname,

void *opval,

socklen_t *optlen);

level specifies whether the option is a general option or a protocol specific option (what level of code should interpret the option).

SO_RCVTIMEO         Option to be set for so that socket receiving functions timeout after specified time

But man page for socket (7) says that it cannot be set by user on Linux!!!

recv and send functions are also available instead of read and write for sockets.

Reading Assignment

Chapter 7         Socket Options


Name Address Conversions

DNS provides Name and address conversion.

/etc/hosts file can be used to create some name to address mapping entries so that name address conversion facility in limited form is made available without DNS.

gethostbyname() function returns ip address for name. Reentrent version is also available.


gethostbyaddr() function takes ip address and returns hostname

/etc/services file maps service names to ports

getservbyname()  gives port number for a service name, protocol name pair

e.g. sptr = getservbyname(“ftp”, “tcp”);


gethostbyname, gethostbyaddr support only IPv4. getaddrinfo() supports both IPv4 and IPv6 and handles both name-to-address and service-to-port translation.

Reading Assignments

Chapter 11 Name and Address Conversions


Daemons and inetd

Given below is a reference to slides from a US university (links obtained via Google search and so presumed to be freely accessible to anybody on the net) which are the key teaching slides for this topic:


Show/explain the following code:

lib/daemon_init.c  : Linux has daemon function which does the same.

Inetd run server

lib/daemon_inetd.c          inetd/daytimetcpsrv3.c

Reading Assignments

Chapter 13 Daemon Process and the inetd SuperServer



Assignment 1

Modify the Saisays server and client programs that you have written, so that both of them use name-to-address and service-to-port conversions. The server program should take the service name as an argument and the client should take the IP address/hostname of the server, and the service name as arguments.

Assignment 2

Modify the Saisays server program so that it runs as a daemon process. It should also report appropriate messages to the syslog daemon (the central logging facility of UNIX) using the syslog() call with appropriate priority levels for different types of messages.

The daemon should write to the log all significant events which include messages on the daemon starting up, on the server being brought up, on acceptance of a connection from a client, on the occurrence of an error and anything else you consider important.

Assignment 3

Improve the Saisays server program by making it possible for it to be run as a Superserver based server (i.e. based on xinetd or inetd). It should write appropriate messages to syslog.

UNP – Robust Sockets

Typical Server and Client

Show/explain the working of TCP Echo server and client (version 1) from the book source code:

  • tcpcliserv/tcpcli01.c
  • tcpcliserv/tcpserv01.c
  • lib/str_echo.c
  • lib/str_cli.c

Robustness and Responsiveness  Issues with above server and client


Server listener should clean up worker server processes by SIGCHLD handler and waitpid.

Show/explain the following code:

tcpcliserv/tcpcli04.c : Multiple connects to server. On exit results in zombies which are not effectively handled by wait in SIGCHLD handler (needs waitpid in a loop).

tcpcliserv/tcpserv04.c      tcpcliserv/sigchldwaitpid.c

(Flawed SIGCHLD handler) tcpcliserv/sigchldwait.c

Server Process Termination

If server is terminated prematurely tcpcli04.c will know of it only on attempting a socket library call. Typically it will be waiting for user input at fgets(). A more responsive client would detect that the server has terminated even while waiting for user input at fgets() and immediately inform the user of the situation. Solution is to use select() function to block on both stdin as well as socket.

Show/explain the following code:


Shutdown function

The shutdown function allows us to terminate only read or write directions of the socket. This is useful in scenarios where we want to initiate close from one end but don’t want to lose any data that the other end may have sent us but which is in transit. Using close() would simply throw away the data in transit. Shutdown of write end of socket followed by read on read end will ensure proper closure of connection without any loss of data which was in transit at the time of shutdown inititiation.

int shutdown (int sockfd, int howto)

howto: SHUT_RD – Only read half of connection is closed

SHUT_WR – Only write half of connection is closed

SHUT_RDWR – Both read and write halves of connection are closed.

The following version of str_cli function uses shutdown() instead of close(). It also operates on buffers instead of line centric code (e.g. Readline). Show/explain the code.


Reading Assignments

Chapter 4 Elementary TCP Sockets

Chapter 5 TCP Client/Server Example

Section 6.3 to Section 6.8 of Chapter 6 I/O Multiplexing: The select and poll Functions.


Assignment 1

You should try out all the examples mentioned above and write down your observations.

Assignment 2

Implement what you have learnt about robustness and responsiveness to improve Assignment 4 of Introduction to Sockets. Write down your observations of the same.

UNP – Introduction to Sockets

Given below are references to slides from a US university (links obtained via Google search and so presumed to be freely accessible to anybody on the net) which are the key teaching slides for this topic:

1) Basic Socket API 1: http://www.cs.rpi.edu/academics/courses/spring98/netprog/lectures/ppthtml/sockets/sockets.ppt

2) Basic Socket API 2: http://www.cs.rpi.edu/academics/courses/spring98/netprog/lectures/ppthtml/tcp_sockets/tcp_sockets.ppt

The references to chapters and examples below are of the course book (and not the above slides).

Show/explain the working of:

  • intro/daytimetcpcli.c
  • intro/daytimetcpsrv.c

Reading Assignments

Chapter 1 Introduction

Chapter 2 Sockets Introduction


Assignment 1

Study and try out the daytime tcp client and server example.

Assignment 2

Write Sayings client and server programs with the following behavior

  • As in the daytime server, whenever the server accepts a connection from a client, it sends a saying to the client and then closes the connection.
  • The client just reads the saying from the server and then displays it.
  • Make just one source file each for the client code and the server code. Avoid using the library functions such as readn, writen. The assignment folder containing the source files should be named Saisays.
  • The client program should take the ipaddress and port of the server program as an argument in the form of ipaddr:port. e.g. “./saicli”. This will enable us to test the client program with any Saisays server. The server program should also take the same argument. In the case of the client the argument refers to the server ip address and port. In the case of the server also it refers to the server’s ip address and port. This will allow us to copy and run the server program to any machine without program modification and recompilation.

Assignment 3

Write a program called stresser which will subject the server to a stress test by flooding it with requests from multiple clients. Simulate server doing time consuming tasks by adding a sleep for some milliseconds, if required.

Assignment 4

Then increase the ‘availability’ of the server by creating a separate process for each connection accepted by it. This new process handles the communication with the client.

Also, subject the modified server program to a stress test by connecting to it from multiple clients, initially from the same machine and later on from many different client machines.

Write a report giving your observations and conclusions regarding the assignments. For submission of assignments put solutions for each assignment in a separate folder named appropriately (e.g. assign1).

UNP – Introduction to Networking

Theory courses on networking would have covered the important concepts of networking, OSI model and TCP/IP protocol. However, if that has not been done so far or your knowledge of them is fuzzy then here are some references to slides from US universities (links obtained via Google search and so presumed to be freely accessible to anybody on the net) with suggestions on how to study them as well as the particular topics to be studied (within those slides).

1) http://www.cs.cmu.edu/afs/cs/academic/class/15441-f01/www/lectures/lecture01.ppt
Suggested speed: Quick run through

Suggested (slide) topics to study/read/refresh: Internet, Protocol, Network Edge, Network Core, Circuit Switching, Packet Switching

2) http://www.cs.cmu.edu/afs/cs/academic/class/15441-f01/www/lectures/lecture02.ppt
Suggested speed: Quick run through;

Suggested (slide) topics to study/read/refresh: Layering

3) http://www.cs.cmu.edu/afs/cs/academic/class/15441-f01/www/lectures/lecture03.ppt
Suggested speed: Medium

Suggested (slide) topics to study/read/refresh: Applications and Application Layer Protocols, Client-Server Paradigm, UDP, TCP, Port Numbers, Names and Addresses. You may exclude the Socket API detailed slides as that is covered later on in this course.

4) https://cs.nmt.edu/~liu/CSE389/Lect_01.ppt

Suggested speed: Quick run through

Suggested (slide) topics to study/read/refresh: Headers, Router, Byte Ordering, Network Byte Order, MultiPlexing, Modes of Service, Error Control, Flow Control, End-to-End v/s Hop-to-Hop, Buffering, Addresses, Broadcasts

5) https://cs.nmt.edu/~liu/CSE389/Lect_02.ppt

Suggested speed: Quick run through

Suggested (slide) topics to study/read/refresh: Ethernet, CSMA/CD, IP datagrams, IP Addresses, Class A, B.., IP Services, UDP, TCP, Modes of Service, Connection Oriented, TCP vs. UDP. You may exclude the TCP segment detailed slides, the three-way handshake slides, the IP datagram example detailed slides and the HTTP slides.

6) http://www.cs.utep.edu/cheon/cs3331/notes/network.ppt

Suggested speed: Quick run through

Suggested (slide) topics to study/read/refresh: Socket Programming, Server vs. Client Sockets, Server Sockets, Client Sockets, Echo Server, Echo Client. You may exclude Multi Echo Server and the whole set of RMI slides (last part of the slides).

Unix Network (socket) Programming including pthread Programming

This course may have been last taught by me in 2008 in a deemed university in Andhra Pradesh, India. I may have taught this course to/for three or four batches/years. The class size was typically around 14 to 18 students.

Last updated on 23rd March 2014

This is a Lab course I taught for I. M.Tech. (Comp. Sc.).  It teaches network client and server programming using sockets and threads.

On completion of the course, most students have a platform from which they can take off to areas like writing an Internet browser client (like Firefox) or even an HTTP server (like Apache). Of course, we get time only to cover the fundamentals and so the student will have to get deeper into these topics on his/her own. But the platform would have been laid. As a by-product the student would have learned pthread programming which enables him/her to do any multi threaded programming tasks (using pthread or other thread libraries). I have been given to understand that multi-threaded programming is useful for multi-core programming and so I have ensured that adequate coverage is done for this topic.

Prerequisites for the Course

C Programming including debugging skills, User level knowledge of Unix (Unix commands like ls, chmod, mkdir, kill etc.) knowledge of file i/o and signal Unix system calls.

Course Book
Unix Network Programming Volume 1, 3rd edition: The Sockets Networking API by W. Richard Stevens, Bill Fenner and Andrew M. Rudoff (http://www.informit.com/store/unix-network-programming-volume-1-the-sockets-networking-9780131411555). Here’s the book support site: http://www.unpbook.com/.

Ideal Coverage (subject to time limitations)

  1. Quick revision of TCP/IP fundamentals. Almost all of the topics of TCP/IP fundamentals are covered in a Computer Networks theory course done at P.G. level. So we just did a quick revision. Topics revised: Internet, protocols, client-server model, TCP, connection oriented service, UDP, connection less service, packet switching, sequencing, error control, flow control, full duplex, half duplex, protocol stack, protocol headers. IP address, IPv4, IPv6, Port numbers, well known ports, hostnames, dns, Byte ordering, Network byte order, Ethernet, MAC address.
  2. Introduction to sockets:
    1. Socket APIs: sockaddr structure, socket(), htonx, ntohx functions, bind(), inet_aton(), inet_ntoa(), inet_pton(), connect(), listen(), accept(), read(), write(), close()
    2. Simple daytime socket client and server program.
    3. Assignment: Writing a Sayings client and server
  3. Robust sockets:
    1. TCP echo client and server. Server uses worker processes for each connection
    2. Avoiding zombies by handling SIGCHLD signal
    3. Using select() to wait on multiple file descriptors for I/O. How select enables writing of more responsive client and server programs.
    4. Shutdown function enabling closure of only Read or Write halves of the connection. How shutdown() enables graceful closure of connection between client and server.
    5. Assignments: Bringing in all above mentioned features, step-by-step, into Sayings client and server.
  4. Posix threads (pthreads):
    1. Basic thread functions: Pthread_create(), pthread_exit(), pthread_join(), pthread_detach()
    2. Thread synchronization using thread mutexes and condition variables: pthread_mutex_lock(), pthread_mutex_unlock(), pthread_cond_wait(), pthread_cond_signal(); Dangers of multi-threaded program and how to avoid them.
    3. Assignments: Sayings server implemented as a worker pool using multithreading – two versions, last version uses condition variables for higher efficiency.
  5. HTTP:
    1. Quick introduction to HTTP protocol
    2. Optional HTTP Assignment: Program which makes an HTTP browser request and displays/stores response from server.

Index of Links to Chapter Wise Course Pages

Please note that, if I recall correctly, I used slides from usually US university links (obtained usually via Google search  and so presumed to be freely accessible to anybody on the net) for all of the topics – why reinvent the wheel? However the topics links below clearly specify the sections covered/to be read as reading assignments and the assignments, most, if not all, of which were created by me specially for this course. So I think they give a good framework for students to learn Network (socket) Programming and pthread programming in a semester in college/university environments. [In some years, due to lack of time the pthread programming followed by the http parts of the course got moved to a separate course done after the Network programming course. Two separate courses allows/gave time for the students to attempt/work on most assignments of the course(s) thereby strengthening the knowledge they gained of the topics covered by the course(s).]

Further, please note that, as a first step, I have focused on putting up the course content used by me to teach this course in the deemed university in Andhra Pradesh, India, suitably modified, on this blog. As part of the minor modifications for putting it up on the publicly accessible blog, some errors may have crept in. I have not checked all the modifications for accuracy (due to other demands on my time). If this course (on this blog) does get utilized by students then the errors in the modified part will come to light and will get fixed by me (or others). I think that is a better way of investing my (and others) time for fixing any errors in this course (on this blog).

Introduction to Networking

Introduction to sockets

Robust and Responsive socket programs

Sockets – Miscellaneous



Former Student Feedback

A former student who had been taught these courses by me, wrote me on 22nd March 2014:

These courses (Advanced Unix Programming and Unix Network Programming) went a long way in helping me land my job at Alcatel-Lucent. I had a one-on-one interview with my hiring manager that was entirely on Unix. After joining the company I learned that this person(manager) was a big time ‘Unix fan’. It was very satisfying to have done well in that interview. On the job, we completely relied on Solaris Unix based servers and the concepts of processes and threads gained from these course(s), went a long way in helping me grasp the software.
Thank you Ravi Sir.