Sockets

Sockets are an inter-process network communication implementation using a Internet Protocol (IP) stack on an Ethernet transport. Sockets are language and protocol independent and available to "C", Perl, Python, Ruby and Java (and more) programmers. The "C" language BSD API is used on Linux, all popular variants of Unix, Microsoft Windows (NT,2000,XP,... and later) and even embedded OSs like VxWorks. It is by far the most popular implementation of inter-process network communication.

Sockets allow one process to communicate with another whether it is local on the same computer system or remote over the network. Many other higher level protocols are built upon sockets technology.

The sockets API provides many configuration options so we will try and cover the socket API components and then give examples of a few implementations. It would be very difficult to cover all variations of its use.

Typically one configures a socket server to which a socket client may attach and communicate. The IP protocol layer will also require that the domain name or IP addresses of the communicating processes be made known as well. Within the IP protocol it is also important to provide the mechanism used: TCP or UDP.

The BSD is a "C" programming API. Examples shown are compiled using the GNU C++ compiler on Linux:

Basic steps in using a socket:

• Socket include files:

  Include File	Description
  sys/types.h	Types used in sys/socket.h and netinet/in.h
  netinet/in.h	Internet domain address structures and functions
  netinet/tcp.h	Socket option macro definitions, TCP headers, enums, etc
  sys/socket.h	Structures and functions used for socket API.i accept(), bind(), connect(), listen(), recv(), send(), setsockopt(), shutdown(), etc ...
  netdb.h	Used for domain/DNS hostname lookup
  sys/select.h	Used by the select(), pselect() functions and defines FD_CLR, FD_ISSET, FD_SET, FD_ZERO macros
  sys/time.h	select() uses argument of type struct timeval and pselect() uses struct timespec defined by this include file.
  arpa/inet.h	Definitions for internet operations. Prototypes functions such as htonl(), htons(), ntohl(), ntohs(), inet_addr(), inet_ntoa(), etc ...
  unistd.h	Defines constants and types
  errno.h	Defines sytem error numbers

• Create the socket instance:

  Open a socket using TCP: Basic declarations and call to "socket".

  #include <iostream>
  #include <sys/types.h>   // Types used in sys/socket.h and netinet/in.h
  #include <netinet/in.h>  // Internet domain address structures and functions
  #include <sys/socket.h>  // Structures and functions used for socket API
  #include <netdb.h>       // Used for domain/DNS hostname lookup
  #include <unistd.h>
  #include <stdlib.h>
  #include <errno.h>
  
  using namespace std;
  
  main()
  {
     int socketHandle;
  
     // create socket
  
     if((socketHandle = socket(AF_INET, SOCK_STREAM, IPPROTO_IP)) < 0)
     {
        close(socketHandle);
        exit(EXIT_FAILURE);
     }
  
     ...
     ...
  }
      

  Socket function prototype:
  int socketHandle = socket(int socket_family, int socket_type, int protocol);
  Choose socket communications family/domain:
  • Internet IPV4: AF_INET
  • Internet IPV6: AF_INET6
  • Unix path name (communicating processes are on the same system): AF_UNIX
  Choose socket type:
  • TCP: SOCK_STREAM
  • UDP: SOCK_DGRAM
  • Raw protocol at network layer: SOCK_RAW
  Choose socket protocol: (See /etc/protocols)
  • Internet Protocol (IP): 0 or IPPROTO_IP
  • ICMP: 1
  • ...
  Also see:
  • socket man page
  • protocols man page

• Configure the socket as a client or server:

  Comparison of sequence of BSD API calls:

  TCP		UDP
  Socket Server	Socket Client	Socket Server	Socket Client
  socket()	socket()	socket()	socket()
  bind()	gethostbyname()	bind()
  listen()
  accept()	connect()
  recv()/send()	recv()/send()	recvfrom()/sendto()	recvfrom()/sendto()
  close()	close()	close()	close()

  This is specific to whether the application is a socket client or a socket server.

  • Socket server: TCP
    • bind(): bind the socket to a local socket address. This assigns a name to the socket.
    • listen(): listen for connections on a socket created with "socket()" and "bind()" and accept incoming connections. This is used for TCP and not UDP. Zero is returned on success.
    • accept(): accept a connection on a socket. Accept the first connection request on the queue of pending connections, create a new connected socket with mostly the same properties as defined by the call to "socket()", and allocate a new file descriptor for the socket, which is returned. The newly created socket is no longer in the listening state. Note this call blocks until a client connects.

       ...
       ...
    #define MAXHOSTNAME 256
       ...
       ...
    
       struct sockaddr_in socketAddress;
       char sysHost[MAXHOSTNAME+1];  // Hostname of this computer we are running on
       struct hostNamePtr *hPtr;
       int portNumber = 8080;
    
       bzero(&socketInfo, sizeof(sockaddr_in));  // Clear structure memory
    
       // Get system information
    
       gethostname(sysHost, MAXHOSTNAME);  // Get the name of this computer we are running on
       if((hPtr = gethostbyname(sysHost)) == NULL)
       {
          cerr << "System hostname misconfigured." << endl;
          exit(EXIT_FAILURE);
       }
    
       // Load system information into socket data structures
    
       socketInfo.sin_family = AF_INET;
       // Use any address available to the system. This is a typical configuration for a server.
       // Note that this is where the socket client and socket server differ.
       // A socket client will specify the server address to connect to.
       socketInfo.sin_addr.s_addr = htonl(INADDR_ANY); // Translate long integer to network byte order.
       socketInfo.sin_port = htons(portNumber);        // Set port number
    
       // Bind the socket to a local socket address
    
       if( bind(socketHandle, (struct sockaddr *) &socketInfo, sizeof(struct sockaddr_in)) < 0)
       {
          close(socketHandle);
          perror("bind");
          exit(EXIT_FAILURE);
       }
    
       listen(socketHandle, 1);
    
       int socketConnection;
       if( (socketConnection = accept(socketHandle, NULL, NULL)) < 0)
       {
          close(socketHandle);
          exit(EXIT_FAILURE);
       }
    
       ...
       ...
    
       // read/write to socket here
    
    

    Socket functions:
    • bind():
      Function prototype:
      int bind(int sockfd, struct sockaddr *my_addr, socklen_t addrlen);
      Bind arguments:
      • int sockfd: Socket file descriptor. Returned by call to "socket".
      • struct sockaddr: Socket information structure
      • socklen_t addrlen: Size of structure
      • Returns 0: Sucess, -1: Failure and errno may be set.
      Also see the bind man page

    • listen():
      Function prototype:
      int listen(int s, int backlog);
      Listen arguments:
      • int s: Socket file descriptor. Returned by call to "socket". Identifies a bound but unconnected socket.
      • int backlog: Set maximum length of the queue of pending connections for the listening socket. A reasonable value is 10. Actual maximum permissible: SOMAXCONN
        Example: int iret = listen(socketHandle, SOMAXCONN);
        The include file sys/socket.h will include /usr/include/bits/socket.h which defines the default value for SOMAXCONN as 128.
        The actual value set for the operating system: cat /proc/sys/net/core/somaxconn
        In kernels before 2.4.25, this limit was a hard coded value and thus would require a kernel recompile with the SOMAXCONN value as defined in /usr/include/linux/socket.h
        For very heavy server use, modify the system limit in the proc file and set "backlog" to the same value (eg. 512).
      • Returns 0: Sucess, -1: Failure and errno may be set.
      Also see the listen man page

    • accept():
      Function prototype:
      int accept(int s, struct sockaddr *addr, socklen_t *addrlen);
      Accept arguments:
      • int s: Socket file descriptor. Returned by call to "socket".
      • struct sockaddr *addr: Pointer to a sockaddr structure. This structure is filled in with the address of the connecting entity.
      • socklen_t *addrlen: initially contains the size of the structure pointed to by addr; on return it will contain the actual length (in bytes) of the address returned. When addr is NULL nothing is filled in.
      • Returns:
        • Success: non-negative integer, which is a descriptor of the accepted socket. Argument "addrlen" will have a return value.
        • Fail: -1, errno may be set
      Also see the accept man page

    [Potential Pitfall]: If you get the following message:

    bind: Address already in use
    

    The solution is to choose a different port or kill the process which is using the port and creating the conflict. You may have to be root to see all processes with netstat.

    netstat -punta | grep 8080

    tcp 0 0 :::8080 :::* LISTEN

  • Socket client: TCP
    • connect(): initiate a connection with a remote entity on a socket. Zero is returned on success. Support both TCP (SOCK_STREAM) and UDP (SOCK_DGRAM). For SOCK_STREAM, an actual connection is made. For SOCK_DGRAM the address is the address to which datagrams are sent and received.

       ...
       ...
    
       struct sockaddr_in remoteSocketInfo;
       struct hostent *hPtr;
       int socketHandle;
       char *remoteHost="dev.megacorp.com";
       int portNumber = 8080;
    
       bzero(&remoteSocketInfo, sizeof(sockaddr_in));  // Clear structure memory
    
       // Get system information
    
       if((hPtr = gethostbyname(remoteHost)) == NULL)
       {
          cerr << "System DNS name resolution not configured properly." << endl;
          cerr << "Error number: " << ECONNREFUSED << endl;
          exit(EXIT_FAILURE);
       }
    
       // Load system information for remote socket server into socket data structures
    
       memcpy((char *)&remoteSocketInfo.sin_addr, hPtr->h_addr, hPtr->h_length);
       remoteSocketInfo.sin_family = AF_INET;
       remoteSocketInfo.sin_port = htons((u_short)portNumber);      // Set port number
    
       if( (connect(socketHandle, (struct sockaddr *)&remoteSocketInfo, sizeof(sockaddr_in)) < 0)
       {
          close(socketHandle);
          exit(EXIT_FAILURE);
       }
    
       ...
       ...
        

    Connect function prototype:
    int connect(int sockfd, const struct sockaddr *serv_addr, socklen_t addrlen);
    Connect arguments: (Same as server's bind() arguments)
    • int sockfd: Socket file descriptor. Returned by call to "socket".
    • struct sockaddr: Socket information structure
    • socklen_t addrlen: Size of structure
    • Returns 0: Sucess, -1: Failure and errno may be set.
    Zero is returned upon success and on error, -1 and errno is set appropriately.
    Also see the connect man page

    ---

    The sockaddr_in data structure: /usr/include/linux/in.h

    /* Internet address. */
    struct in_addr {
            __u32   s_addr;   /* Defined as 32 or 64 bit address (system dependent) */
    };
    
    /* Structure describing an Internet (IP) socket address. */
    #define __SOCK_SIZE__   16              /* sizeof(struct sockaddr)      */
    struct sockaddr_in {
      sa_family_t           sin_family;     /* Address family               */
      unsigned short int    sin_port;       /* Port number                  */
      struct in_addr        sin_addr;       /* Internet address             */
    
      /* Pad to size of `struct sockaddr'. */
      unsigned char         __pad[__SOCK_SIZE__ - sizeof(short int) -
                            sizeof(unsigned short int) - sizeof(struct in_addr)];
    };
    

    Note:
    • IP addresses: Note the specific IP address could be specified:

           #include <arpa/inet.h>     // IP from string conversion
      
           socketInfo.sin_addr.s_addr = inet_addr("127.0.0.1");
      
           cout << inet_ntoa(socketInfo.sin_addr) << endl;
      

      or bind to all network interfaces available:

           socketInfo.sin_addr.s_addr = htonl(INADDR_ANY);
      

    • Port numbers: Note the specific port can be specified:

           socketInfo.sin_port = htons(8080);
      
           cout << ntohs(socketInfo.sin_port) << endl;
      

      or let the system define one for you:

           socketInfo.sin_port = htons(INADDR_ANY);
      

• Read from or write to socket: TCP/UDP

  Use send() and recv(), or write() and read(), or sendto() and recvfrom() to read/write to/from a socket.

  • TCP recv() or UDP recvfrom(): receive a message from a socket

       char *pcIpAddress;
       unsigned short shPort;
    
       ...
       ...
    
       if (iSocketType == SOCK_STREAM)
       {
          rc = recv(socketHandle, (char*) _pcMessage, (int) _iMessageLength, 0);
          if ( rc == 0 )
          {
              cerr << "ERROR! Socket closed" << endl;
          }
          else if (rc == -1)
          {
              cerr << "ERROR! Socket error" << endl;
              closeSocket();
          }
       }
       else if (iSocketType == SOCK_DGRAM)
       {
          int iLength;
          struct sockaddr_in stReceiveAddr;
          iLength = (int) sizeof(struct sockaddr_in);
          memset((void*) &stReceiveAddr, 0, iLength);
    
          rc = recvfrom(socketHandle, (char*) _pcMessage, (int) _iMessageLength, 0,
                       (struct sockaddr *) &stReceiveAddr, (socklen_t*) &iLength))
          if{ rc == 0 )
          {
             cerr << "ERROR! Socket closed" << endl;
          }
          else if (rc == -1)
          {
             cerr << "ERROR! Socket error" << endl;
             closeSocket();
          }
       }
    
       pcIpAddress = inet_ntoa(stReceiveAddr.sin_addr);
       shPort = ntohs(stReceiveAddr.sin_port);
    
       cout << "Socket Received: " << _iNumRead << " bytes from "
            << pcIpAddress << ":" << shPort << endl; 
    
       ...
       ...
    
        

  • read(): read a specific number of bytes from a file descriptor

       int rc        = 0;  // Actual number of bytes read by function read()
       int count     = 0;  // Running total "count" of bytes read
       int numToRead = 32; // Number of bytes we want to read each pass
       char buf[512];
     
       ...
       ...
    
       while(bcount < numToRead)
       {
          // rc is the number of bytes returned.
          if( (rc = read(socketHandle, buf, numToRead - count)) > 0);
          {
             count += rc;
             buf += rc;      // Set buffer pointer for next read
          }
          else if(rc < 0)
          {
             close(socketHandle);
             exit(EXIT_FAILURE);
          }
    
          cout << "Number of bytes read: " << count << endl;
          cout << "Received: " << buf << endl;
       }
    
       ...
       ...
    
        

  • send(): send a message from a socket. Used only when in a connected state. The only difference between send and write is the presence of flags. With zero flags parameter, send is equivalent to write.

    #include <string.h>
    
       ...
       ...
    
       char buf[512];
     
       strcpy(buf,"Message to send");
    
       ...
       ...
    
       send(socketHandle, buf, strlen(buf)+1, 0);
     
       ...
       ...
    
        

  • TCP send() or UDP sendto():

       int iSocketType;
       int iBytesSent = 0;
       char *pMessage = "message to send";
       int  iMessageLength = 16;  // number of bytes (includes NULL termination)
       sockaddr pSendAddress;
    
       ...
       ...
       
       if (iSocketType == SOCK_STREAM)
       {
           if ((iBytesSent = send(socketHandle, (char*) pMessage, (int) iMessageLength, 0)) < 0 )
           {
               cerr << "Send failed with error " << errno << endl;
               close(socketHandle);
           }
       }
       else if (iSocketType == SOCK_DGRAM)
       {
           if ((iBytesSent = sendto(socketHandle, (char*) pMessage, 
                                    (int) iMessageLength, 
                                    0,
                                    (struct sockaddr*) pSendAddress,
                                    (int) sizeof(struct sockaddr_in))) < 0 )
           {
               cerr << "Sendto failed with error " << errno  << endl;
               close();
           }
       }
       else
       {
           // Failed - Socket type not defined
       }
    
       ...
       ...
        

• Close the socket when done: TCP/UDP

  #include <unistd.h>
  
     ...
  
     close(socketHandle);
  
     ...
      

  This is the "C" library function to close a file descriptor. Returns zero on success, or -1 if an error occurred.

Simple TCP Socket Server:

#include <iostream>
#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>
#include <stdlib.h>
#include <strings.h>
#include <stdio.h>
#define MAXHOSTNAME 256
using namespace std;

main()
{
   struct sockaddr_in socketInfo;
   char sysHost[MAXHOSTNAME+1];  // Hostname of this computer we are running on
   struct hostent *hPtr;
   int socketHandle;
   int portNumber = 8080;

   bzero(&socketInfo, sizeof(sockaddr_in));  // Clear structure memory

   // Get system information

   gethostname(sysHost, MAXHOSTNAME);  // Get the name of this computer we are running on
   if((hPtr = gethostbyname(sysHost)) == NULL)
   {
      cerr << "System hostname misconfigured." << endl;
      exit(EXIT_FAILURE);
   }

   // create socket

   if((socketHandle = socket(AF_INET, SOCK_STREAM, 0)) < 0)
   {
      close(socketHandle);
      exit(EXIT_FAILURE);
   }

   // Load system information into socket data structures

   socketInfo.sin_family = AF_INET;
   socketInfo.sin_addr.s_addr = htonl(INADDR_ANY); // Use any address available to the system
   socketInfo.sin_port = htons(portNumber);      // Set port number

   // Bind the socket to a local socket address

   if( bind(socketHandle, (struct sockaddr *) &socketInfo, sizeof(socketInfo)) < 0)
   {
      close(socketHandle);
      perror("bind");
      exit(EXIT_FAILURE);
   }

   listen(socketHandle, 1);

   int socketConnection;
   if( (socketConnection = accept(socketHandle, NULL, NULL)) < 0)
   {
      exit(EXIT_FAILURE);
   }
   close(socketHandle);

   int rc = 0;  // Actual number of bytes read
   char buf[512];

   // rc is the number of characters returned.
   // Note this is not typical. Typically one would only specify the number 
   // of bytes to read a fixed header which would include the number of bytes
   // to read. See "Tips and Best Practices" below.

   rc = recv(socketConnection, buf, 512, 0);
   buf[rc]= (char) NULL;        // Null terminate string

   cout << "Number of bytes read: " << rc << endl;
   cout << "Received: " << buf << endl;
}

Forking Socket Server:

In order to accept connections while processing previous connections, use fork() to handle each connection.

Use establish() and get_connection() to allow multiple connections.

#include <iostream>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/wait.h>
#include <netdb.h>
#include <stdlib.h>
#include <strings.h>
#include <stdio.h>
#include <errno.h>
#include <unistd.h>
#include <signal.h>
#include <netinet/in.h>
#define MAXHOSTNAME 256
using namespace std;

// Catch signals from child processes
void handleSig(int signum)
{
   while(waitpid(-1, NULL, WNOHANG) > 0);
}

main()
{
   struct sockaddr_in socketInfo;
   char sysHost[MAXHOSTNAME+1];  // Hostname of this computer we are running on
   struct hostent *hPtr;
   int socketHandle;
   int portNumber = 8080;

   signal(SIGCHLD, handleSig);

   bzero(&socketInfo, sizeof(sockaddr_in));  // Clear structure memory
  
   // Get system information 

   gethostname(sysHost, MAXHOSTNAME);  // Get the name of this computer we are running on
   if((hPtr = gethostbyname(sysHost)) == NULL)
   {
      cerr << "System hostname misconfigured." << endl;
      exit(EXIT_FAILURE);
   }

   // create socket

   if((socketHandle = socket(AF_INET, SOCK_STREAM, 0)) < 0)
   {
      close(socketHandle);
      exit(EXIT_FAILURE);
   }

   // Load system information into socket data structures

   socketInfo.sin_family = AF_INET;
   socketInfo.sin_addr.s_addr = htonl(INADDR_ANY); // Use any address available to the system
   socketInfo.sin_port = htons(portNumber);      // Set port number

   // Bind the socket to a local socket address

   if( bind(socketHandle, (struct sockaddr *) &socketInfo, sizeof(struct sockaddr_in)) < 0)
   {
      close(socketHandle);
      perror("bind");
      exit(EXIT_FAILURE);
   }

   listen(socketHandle, 1);

   int socketConnection;
   for(;;)  // infinite loop to handle remote connections. This should be limited.
   {
      if( (socketConnection = accept(socketHandle, NULL, NULL)) < 0)
      {
         close(socketHandle);
         if(errno == EINTR) continue;
         perror("accept");
         exit(EXIT_FAILURE);
      }
      switch(fork())
      {
         case -1:
            perror("fork");
            close(socketHandle);
            close(socketConnection);
            exit(EXIT_FAILURE);
         case 0:   // Child process - do stuff
            close(socketHandle);
            // Do your server stuff like read/write messages to the socket here!
            exit(0);
         default:    // Parent process, look for another connection
            close(socketConnection);
            continue;
      }
   }

}

Simple Socket TCP client:

#include <iostream>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>
#include <unistd.h>
#include <stdlib.h>
#include <errno.h>
#define MAXHOSTNAME 256
using namespace std;

main()
{
   struct sockaddr_in remoteSocketInfo;
   struct hostent *hPtr;
   int socketHandle;
   const char *remoteHost="localhost";
   int portNumber = 8080;

   bzero(&remoteSocketInfo, sizeof(sockaddr_in));  // Clear structure memory

   // Get system information

   if((hPtr = gethostbyname(remoteHost)) == NULL)
   {
      cerr << "System DNS name resolution not configured properly." << endl;
      cerr << "Error number: " << ECONNREFUSED << endl;
      exit(EXIT_FAILURE);
   }

   // create socket

   if((socketHandle = socket(AF_INET, SOCK_STREAM, 0)) < 0)
   {
      close(socketHandle);
      exit(EXIT_FAILURE);
   }

   // Load system information into socket data structures

   memcpy((char *)&remoteSocketInfo.sin_addr, hPtr->h_addr, hPtr->h_length);
   remoteSocketInfo.sin_family = AF_INET;
   remoteSocketInfo.sin_port = htons((u_short)portNumber);      // Set port number

   if(connect(socketHandle, (struct sockaddr *)&remoteSocketInfo, sizeof(sockaddr_in)) < 0)
   {
      close(socketHandle);
      exit(EXIT_FAILURE);
   }

   int rc = 0;  // Actual number of bytes read by function read()
   char buf[512];

   strcpy(buf,"Message to send");
   send(socketHandle, buf, strlen(buf)+1, 0);
}

Note that this runs on a single system using "localhost".

• g++ server.cpp -o server
• g++ client.cpp -o client

Start the client: ./client
This will connect to the server and write the message "Message to send". The server will receive the message and write it out.

Note that the above compile/link commands apply to Linux (and Cygwin which includes socket and network libraries in cygwinl.dll).

• g++ server.cpp -o server -lnsl -socket
• g++ client.cpp -o client -lnsl -socket

• DNS: Name resolution associates IP address to a hostname using DNS (Domain Name System) name servers. Name resolution invokes a series of library routines to query the name servers.
• TCP/IP ports and associated services
• Network Protocols
• Network name information

Data Structures returned:

   struct hostent {
       char  *h_name;		 /* official name of host */
       char **h_aliases;	 /* alias list */
       int    h_addrtype;	 /* host address type */
       int    h_length; 	 /* length of address */
       char **h_addr_list;	 /* list of addresses */
   }

   struct servent {
       char    *s_name;        /* official service name */
       char    **s_aliases;    /* alias list */
       int     s_port;         /* port number */
       char    *s_proto;       /* protocol to use */
   }
   

   struct protoent {
       char    *p_name;        /* official protocol name */
       char    **p_aliases;    /* alias list */
       int     p_proto;        /* protocol number */
   }
   

   struct netent {
       char    *n_name;          /* official network name */
       char    **n_aliases;      /* alias list */
       int     n_addrtype;       /* net address type */
       unsigned long int n_net;  /* network number */
   }
   

Socket options:

One can "get" (read) the current socket options or "set" them to new values. The default values are obtained from the OS:

For a full list of options see the TCP man page

For a full list of IP options see the IP(7) man page

• int sockfd: Socket file descriptor. Returned by call to "socket".
• int level: See table above
• int optname: See table above
• void *optval: Pointer to value or data structure
• optlen: Length of "optval"
• Returns 0: Sucess, -1: Failure and errno may be set.

Code to read socket options:

#include <sys/socket.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
#include <errno.h>
#include <stdio.h>

int main()
{
   int socketHandle;

   // create socket

   if((socketHandle = socket(AF_INET, SOCK_STREAM, IPPROTO_IP)) < 0)
   {
      close(socketHandle);
      perror("socket");
   }

    int iSocketOption = 0;
    int iSocketOptionLen = sizeof(int);;

    struct linger SocketOptionLinger;
    int iSocketOptionLingerLen = sizeof(struct linger);;

    getsockopt(socketHandle, IPPROTO_TCP, TCP_NODELAY, (char *)&iSocketOption, &iSocketOptionLen);
    printf("Socket TCP_NODELAY = %d\n", iSocketOption);

    getsockopt(socketHandle, IPPROTO_TCP, TCP_MAXSEG, (char *)&iSocketOption, &iSocketOptionLen);
    printf("Socket TCP_MAXSEG = %d\n", iSocketOption);

    getsockopt(socketHandle, IPPROTO_TCP, TCP_CORK, (char *)&iSocketOption, &iSocketOptionLen);
    printf("Socket TCP_CORK = %d\n", iSocketOption);

    getsockopt(socketHandle, IPPROTO_TCP, TCP_KEEPIDLE, (char *)&iSocketOption, &iSocketOptionLen);
    printf("Socket TCP_KEEPIDLE = %d\n", iSocketOption);

    getsockopt(socketHandle, IPPROTO_TCP, TCP_KEEPINTVL, (char *)&iSocketOption, &iSocketOptionLen);
    printf("Socket TCP_KEEPINTVL = %d\n", iSocketOption);

    getsockopt(socketHandle, IPPROTO_TCP, TCP_KEEPCNT, (char *)&iSocketOption, &iSocketOptionLen);
    printf("Socket TCP_KEEPCNT = %d\n", iSocketOption);

    getsockopt(socketHandle, IPPROTO_TCP, TCP_SYNCNT, (char *)&iSocketOption, &iSocketOptionLen);
    printf("Socket TCP_SYNCNT = %d\n", iSocketOption);

    getsockopt(socketHandle, IPPROTO_TCP, TCP_LINGER2, (char *)&iSocketOption, &iSocketOptionLen);
    printf("Socket TCP_LINGER2 = %d\n", iSocketOption);

    getsockopt(socketHandle, SOL_SOCKET, SO_REUSEADDR, (char *)&iSocketOption, &iSocketOptionLen);
    printf("Socket SO_REUSEADDR = %d\n", iSocketOption);

    getsockopt(socketHandle, SOL_SOCKET, SO_ERROR, (char *)&iSocketOption, &iSocketOptionLen);
    printf("Socket SO_ERROR = %d\n", iSocketOption);

    getsockopt(socketHandle, SOL_SOCKET, SO_BROADCAST, (char *)&iSocketOption, &iSocketOptionLen);
    printf("Socket SO_BROADCAST = %d\n", iSocketOption);

    getsockopt(socketHandle, SOL_SOCKET, SO_KEEPALIVE, (char *)&iSocketOption, &iSocketOptionLen);
    printf("Socket SO_KEEPALIVE = %d\n", iSocketOption);

    getsockopt(socketHandle, SOL_SOCKET, SO_SNDBUF, (char *)&iSocketOption, &iSocketOptionLen);
    printf("Socket SO_SNDBUF = %d\n", iSocketOption);

    getsockopt(socketHandle, SOL_SOCKET, SO_RCVBUF, (char *)&iSocketOption, &iSocketOptionLen);
    printf("Socket SO_RCVBUF = %d\n", iSocketOption);

    getsockopt(socketHandle, SOL_SOCKET, SO_LINGER, (char *)&SocketOptionLinger, &iSocketOptionLingerLen);
    printf("Socket SO_LINGER = %d  time = %d\n", SocketOptionLinger.l_onoff, SocketOptionLinger.l_linger);

    getsockopt(socketHandle, SOL_SOCKET, SO_RCVLOWAT, (char *)&iSocketOption, &iSocketOptionLen);
    printf("Socket SO_RCVLOWAT = %d\n", iSocketOption);
}
    

getsockopt man page: get a particular socket option for the specified socket.

Set socket options:

• Socket "keep-alive":

  int iOption = 1; // Turn on keep-alive, 0 = disables, 1 = enables
  
  if (setsockopt(socketHandle, SOL_SOCKET, SO_KEEPALIVE, (const char *) &iOption,  sizeof(int)) == SOCKET_ERROR)
  {
         cerr << "Set keepalive: Keepalive option failed" << endl;
  }
      

Set socket client options:

• Socket re-use:

  int iOption = 0;  // Reuse address option to set, 0 = disables, 1 = enables
  
  if (setsockopt(socketHandle, SOL_SOCKET, SO_REUSEADDR, (const char *) &iOption,  sizeof(int)) == SOCKET_ERROR)
  {
         cerr << "Set reuse address: Client set reuse address option failed" << endl;
  }
      

  When a socket connection is closed with a call to close(), shutdown() or exit(), both the client and server will send a FIN (final) packet and will then send an acknowledgment (ACK) that they received the packet. The side which initiates the closure will be in a TIME_WAIT state until the process has been completed. This time out period is generally 2-4 minutes in duration. It is hoped that all packets are received in a timely manner and the entire time out duration is not required. When an application is abnormally terminated, the TIME_WAIT period is entered for the full duration.

  Setting the SO_REUSEADDR option explicitly allows a process to bind a port in the TIME_WAIT state. This is to avoid the error "bind: Address Already in Use". One caviat is that the process can not be to the same address and port as the previous connection. If it is, the SO_REUSEADDR option will not help and the duration of the TIME_WAIT will be in effect.
  For more info see How to avoid the "Address Already in Use" error.

  Solution: Enable socket linger:

  linger Option;
  Option.l_onoff = 1;
  Option.l_linger = 0;
  
  if(setsockopt(socketHandle, SOL_SOCKET, SO_LINGER, (const  char *) &Option,  sizeof(linger)) == -1)
  {
      cerr << "Set SO_LINGER option failed" << endl;
  }
      

  This allows the socket to die quickly and allow the address to be reused again. Warning: This linger configuration specified may/will result in data loss upon socket termination, thus it would not have the robustness required for a banking transaction but would be ok for a recreational app.

• Broadcast:

  int iOption = 0;  // Broadcast option to set, 0 = disables, 1 = enables
  
  if (setsockopt(socketHandle, SOL_SOCKET, SO_BROADCAST, (const char *) &iOption,  sizeof(int)) == SOCKET_ERROR)
  {
         cerr << "Set reuse address: Client set reuse address option failed" << endl;
  }
      

  Struct: remoteSocketInfo.sin_addr.s_addr = htonl(INADDR_BROADCAST);

setsockopt man page: set a particular socket option for the specified socket.

#include <sys/select.h>
#include <sys/time.h>

...
...

bool isReadyToRead(int _socketHandle, const long &_lWaitTimeMicroseconds)
{
    int iSelectReturn = 0;  // Number of sockets meeting the criteria given to select()
    timeval timeToWait;
    int fd_max = -1;          // Max socket descriptor to limit search plus one.
    fd_set readSetOfSockets;  // Bitset representing the socket we want to read
                              // 32-bit mask representing 0-31 descriptors where each 
                              // bit reflects the socket descriptor based on its bit position.

    timeToWait.tv_sec  = 0;
    timeToWait.tv_usec = _lWaitTimeMicroseconds;

    FD_ZERO(&readSetOfSockets);
    FD_SET(_socketHandle, &readSetOfSockets);

    if(_socketHandle > fd_max)
    {
       fd_max = _socketHandle;
    }

    iSelectReturn = select(fd_max + 1, &readSetOfSockets, (fd_set*) 0, (fd_set*) 0, &timeToWait);

    // iSelectReturn -1: ERROR, 0: no data, >0: Number of descriptors found which pass test given to select()
    if ( iSelectReturn == 0 )  // Not ready to read. No valid descriptors
    {
        return false;
    }
    else if ( iSelectReturn < 0 )  // Handle error
    {
        cerr << "*** Failed with error "   << errno << " ***" << endl;

        close(_socketHandle);
        return false;
    }

    // Got here because iSelectReturn > 0 thus data available on at least one descriptor
    // Is our socket in the return list of readable sockets
    if ( FD_ISSET(_socketHandle, &readSetOfSockets) )
    {
        return true;
    }
    else
    {
        return false;
    }

    return false;
}

1. int fd_max: Highest socket descriptor number. When opening a socket with the call socket(), the function returns a socket descriptor number. The call to select() will loop through all of the socket descriptors from zero up to fd_max to perform the "test".
2. fd_set *readSetOfSockets: This is a pointer to the variable holding the set of bits representing the set of sockets to test for readability. (Read will not block)
   The default number of bytes detected for the socket to be considered ready to read is 1. To change this default use (in this example 8 bytes):
   int nBytes = 8;
   setsockopt(socketHandle, SOL_SOCKET, SO_RCVLOWAT,(const char *) &nBytes, sizeof(int))
3. fd_set *writeSetOfSockets: This is a pointer to the variable holding the set of bits representing the set of sockets to test for writeability. (Write will not block)
4. fd_set *exceptfds: This is a pointer to the variable holding the set of bits representing the set of sockets to test for exceptions.
5. struct timeval *timeout: This structure holds the upper bound on the amount of time elapsed before select returns. It may be zero, causing select to return immediately. If timeout is NULL (no timeout), select can block indefinitely.

       struct timeval {
           long    tv_sec;         /* seconds */
           long    tv_usec;        /* microseconds */
       };
           

Also see the select man page

The use of port numbers below 1024 are limited to the root process. For a list of port numbers with established uses see the file /etc/services.

Posts are 16 bit identifiers. Many are reserved and managed by the Internet Assigned Numbers Authority (IANA).
See RFC 1700.

Note that when transferring data between different platforms or with Java, the byte order endianess will have to be considered. The network (the neutral medium) byte order is Big Endian and the byte order to which data is usually marshalled.

Host processor byte order:

Also note: Java processes and stores data in big endian byte ordering on any platform.

Character data is not subject to this problem as each character is one byte in length but integer is.

Integer data can be converted from/to host or network byte order with the following routines:

• ntohl/htonl, ntohs/htons: convert values between host and network byte order
• htobe16()/be16toh(), also 32 and 64 bit: convert values between host and big-/little-endian byte order

Include file/code snipit to determine the processor endianess:

#ifndef __IS_BIGENDIAN__
#define __IS_BIGENDIAN__

const int bsti = 1;  // Byte swap test integer

#define is_bigendian() ( (*(char*)&bsti) == 0 )

#endif // __IS_BIGENDIAN__

#include <netdb.h>
#include "is_bigendian.h"

/**
  In-place swapping of bytes to match endianness of hardware

  @param[in/out] *object : memory to swap in-place
  @param[in]     _size   : length in bytes
*/
void swapbytes(void *_object, size_t _size)
{
   unsigned char *start, *end;

   if(!is_bigendian())
   {
       for ( start = (unsigned char *)_object, end = start + _size - 1; start < end; ++start, --end )
       {
          unsigned char swap = *start;
          *start = *end;
          *end = swap;
       }
   }
}

#include <netdb.h>
#include "is_bigendian.h"

void swapbytes(void *_object, size_t _size);

// Struct size of four bytes

struct ExampleA
{
#ifdef BIGENDIAN
   unsigned int a:1;
   unsigned int b:1;
   unsigned int c:1;
   unsigned int d:1;
   unsigned int e:4;
   unsigned int f:8;
   unsigned int g:8;
   unsigned int h:8;
#else
   unsigned int h:8;
   unsigned int g:8;
   unsigned int f:8;
   unsigned int e:4;
   unsigned int d:1;
   unsigned int c:1;
   unsigned int b:1;
   unsigned int a:1;
#endif
};

...
...

    // Bits:
    //   |B31....B25 |B24....B16 |B15....B8  |B7....B0   | Big endian
    //   |B7....B0   |B15....B8  |B24....B16 |B31....B25 | Little endian

    // Intel host to network:
    //   Reverse bit field structure and then byte swap.
    //   Just byteswap for int.

    // Code body 

    struct ExampleA exampleA;
    char tmpStore[4];

    ...

    // assign member variables:
    exampleA.a = 0;
    exampleA.b = 0;
    exampleA.c = 1;
    exampleA.d = 1;
    exampleA.e = 3;
    ...

    // Use memcpy() because we can't cast a bitfield
    memcpy(&tmpStore, &exampleA, sizeof(ExampleA));

    swapbytes((void *)&tmpStore, sizeof(ExampleA));

    ...

For complete coverage on big and little endian byte order and byte order conversion see the YoLinux Endian Byte Order tutorial.

• Re-connect: If a socket client attempts to connect to a socket server and fails, one should attempt to re-attach after a given waiting period, a fixed number of times.

  ...
  ...
  
  int iTry = 0;
  int mRetrymax = 10;
  int mRetrywait = 2;
  
  // If this client can not connect to the server, try again after period "Retrywait".
  
  while ( connect(socketHandle, (struct sockaddr *)&remoteSocketInfo, sizeof(sockaddr_in)) < 0 )
  {
      iTry++;
      if( iTry > mRetrymax )
      {
          cerr << "Failed to connect to server. Exceeded maximum allowable attempts: " << mRetrymax << endl;
          break;  // Done retrying!
      }
      sleep( mRetrywait );
  }
  
  ...
  

• Check return codes: Just because the socket was opened doesn't mean it will stay open. Dropped connections do happen. Check all read/write socket API function return codes:
  • return code > 0: Data received/sent
  • return code == 0: Socket closed
  • return code == -1: Check system errno and interpret (or call "perror()")

  For even more robust code for a socket client, close, then open a new socket connection for return codes 0 or -1.

• Read entire message: (Variable and fixed length messages) When sending messages using a socket, be aware that the other side of connection must know how much to read. This means that you must:
  • Send fixed message sizes of a known number of bytes where all messages sent are of the same size
    or
  • Send a message with a header of a known size where the message size is sent in the header. Thus read the fixed size header and determine the size of the whole message then read the remaining bytes.

  Note that with a UDP client/server communications, the socket read will read the whole frame sent. This is not necessarily true for a TCP stream. The function used to read the stream will return the number of bytes read. If the required number of bytes have not been read, repeat the read for the remaining bytes until the whole message has been read.

  In this example, our message header is a single integer used to store the message size in the first 32 bits of the message.

  #include <iostream>
  #include <arpa/inet.h>
  #include <sys/socket.h>
  #include <sys/types.h>
  #include <stdio.h>
  #include <error.h>
  
  using namespace std;
  
  ...
  ...
  
     // Data: Message Header and Size
  
     int rc = 0;
     int messageSize = 0;
     int remainingMessageSize = 0;
  
     int messageHeaderSize = sizeof(int);
     unsigned char *headerBuf= new unsigned char[messageHeaderSize ];
  
     // Read Message Header
  
     int socketConnection;
  
     // Prototype: ssize_t recv(int s, void *buf, size_t len, int flags);
     // rc is the number of bytes returned.
  
     rc = recv(socketConnection, headerBuf, messageHeaderSize , 0);
  
     if ( rc == 0 )
         cerr << "Socket closed" << endl;
     else if ( rc == -1 )
         cerr << "Socket error" << endl;
     else if ( rc != messageHeaderSize ) 
         cerr << "Problem reading header" << endl;
  
     // Read message
  
     int messageTotalSize;
     int tmp = htonl((uint32_t)headerBuf);
     memcpy((void *)&messageSize,(void *)&tmp, messageHeaderSize );
  
     // Create storage buffer for the message
     unsigned char *messageTotalBuf = new unsigned char[messageTotalSize ];
  
     // Copy header into message buffer
     memcpy(messageTotalBuf, &headerBuf, messageHeaderSize );
  
     // How much more to read
     remainingMessageSize = messageTotalSize - messageHeaderSize;
  
     // Character buffer pointer math: Put rest of message after header.
     messageTotalBuf += messageHeaderSize;
  
     while(1)
     {
         rc = recv(socketConnection, messageTotalBuf, remainingMessageSize, 0);
  
         if ( rc == 0 )
         {
             cerr << "Socket closed" << endl;
             break;
         }
         else if ( rc == -1 )
         {
             cerr << "Socket error" << endl;
             perror("recv");
             close(socketConnection);
             break;
         }
         else if( rc != remainingMessageSize)
         {
              // Still more to read but less by what we have just read.
              remainingMessageSize = remainingMessageSize - rc;
  
              // Character buffer pointer math: Put rest of message after header
              //  and what has already been put there by recv().
              messageTotalBuf += rc;
         }
         else break;
     }
  
     // Now have message in messageTotalBuf of size messageTotalSize
  
  ...
  ...
  

  This example applies to either client or server TCP sockets.
  Another method of ensuring a complete read of a fixed size message is to use the MSG_WAITALL flag:

  ...
  ...
  
  int flags = MSG_WAITALL;
  int rc = recv(socketConnection, buffer, length, flags); 
  
  ...
  ...
  

  On "SOCK_STREAM" (TCP) sockets, this flag requests that the recv() function block until the full amount of data specified (length) can be returned. The function may return the smaller amount of data if the socket is a message-based socket, if a signal is caught or if the connection is terminated.

• UDP message order: TCP will guarentee that the order of the message delivery is maintained. UDP will not guarentee the message delivery order and thus you will have to maintain a counter in the message if the order is important.

• Signals: The socket layer can issue signals, which if not handled, can terminate your application. When a socket is terminated at one end, the other may receive a SIGPIPE signal.

  Program received signal SIGPIPE, Broken pipe.
  0x000000395d00de45 in send () from /lib64/libpthread.so.0
  (gdb) where
  #0  0x000000395d00de45 in send () from /lib64/libpthread.so.0
  #1  0x00000000004969c5 in bla bla bla
  ...
  ...
      

  Note GDB will report a received signal even if it's being ignored by the application.
  In this example, we tried to send() over a broken socket link.
  Set up a signal handler at the beginning of your program to handle SIGPIPE:

  #include <stdlib.h>
  #include <signal.h>
  
  /// Ignore signal with this handler
  void handleSigpipe(int signum)
  {
     cout << "SIGPIPE ignored" << endl;
     return;
  }
  
  
  int main()
  {
    /// Ignore SIGPIPE "Broken pipe" signals when socket connections are broken.
    signal(SIGPIPE, handleSigpipe);
  
    ...
    ...
  }
      

  Also see:
  • C++ Signal class tutorial - YoLinux.com tutorial
  • signal: Signal handling
  • sigaction: examine and change a signal action

The beauty of the BSD API is that it is nativly supported by Microsoft Visual C++ but there are some slight differences which can be wrapped in an "ifdef".

#ifdef WIN32              // Windows
#include "StdAfx.h"
#include <windows.h>
#include <winsock2.h>
#include <winsock.h>
#else                     // Unix/Linux
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
#endif

  ...
  ...

#ifdef WIN32
    SOCKET socketHandle;
    WSADATA oWSAData;
    memset(&oWSAData, 0, sizeof(oWSAData));
#else
    int socketHandle;
#endif

  ...
  ...

#ifdef WIN32
        int iWSAError;

        // Load the WSA structure

        if ((iWSAError = WSAStartup(MAKEWORD(2, 2), &oWSAData)) != 0)
        {
            cerr << "Creating the WSAStartup failed with error " << iWSAError << endl;
        }
#endif

  ...
  ...
        //  Make sockets calls here
  ...
  ...

#ifdef WIN32
        closesocket(socketHandle);
#else
        close(socketHandle);
#endif

        // Manage errors here

        cerr << "Failed with error "
#ifdef WIN32
                << WSAGetLastError() << endl;

        WSACleanup();
#else
                << errno << endl;
#endif

Microsoft Visual C++ settings (VC5.0):

Error codes: The error codes returned by the MS/Windows BSD API socket calls use different defined macro representations and may have to be handled differently.

Config files: MS/Windows has an equivalent of /etc/hosts which can be found in %SYSTEMROOT%\SYSTEM32\DIRVERS\ETC\HOSTS where the format is identical to the Unix/Linux file format.

• socket: establish socket interface
• gethostname: obtain hostname of system
• gethostbyname: returns a structure of type hostent for the given host name
• bind: bind a name to a socket
• listen: listen for connections on a socket
• accept: accept a connection on a socket
• connect: initiate a connection on a socket
• setsockopt: set a particular socket option for the specified socket.
• close: close a file descriptor
• shutdown: shut down part of a full-duplex connection

• select: synchronous I/O multiplexing
• FD_ZERO(), FD_CLR(), FD_SET(), FD_ISSET(): Set socket bit masks
• poll: check on the state of a socket in a set of sockets. The set can be tested to see if any socket can be written to, read from or if an error occurred.
• getsockopt: retrieve the current value of a particular socket option for the specified socket.

• recv/recvfrom/recvmsg: Read socket
• send/sendto/sendmsg: Write socket

• ntohl/htonl, ntohs/htons: convert values between host and network byte order
• inet_pton: Create a network address structure
• inet_ntop: Parse network address structures

• exit: Terminate process
• perror: Output explanation of an error code
• protocols: Network Protocols (see /etc/protocols)

• Endian Byte Order tutorial - YoLinux
• Apache Portable Runtime (APR) - [documentation] - Portable socket wrapper functions.
• SCTP - Stream Control Transmission Protocol
• RPC - Remote Procedure Calls
• IBM: Secure programming with the OpenSSL API

Books:

	"UNIX Network Programming, Volume 1: Sockets Networking API" Third Edition by W. Richard Stevens, Bill Fenner, Andrew M. Rudoff, Richard W. Stevens ISBN # 0131411551, Addison-Wesley Pub Co; 3 edition (October 22, 2003) This book covers POSIX, IPv6, network APIs, sockets (elementary, advanced, routed, and raw), multicast, UDP, TCP, Threads, Streams, ioctl. In depth coverage of topics. I recommend this book for any sockets programming task.
	"UNIX Network Programming, Volume 1: Networking APIs - Sockets and XTI" Second Edition by W. Richard Stevens ISBN # 013490012X, Prentice Hall PTR This book covers network APIs, sockets + XTI, multicast, UDP, TCP, ICMP, raw sockets, SNMP, MBONE. In depth coverage of topics.
	"UNIX Network Programming Volume 2: Interprocess Communications" by W. Richard Stevens ISBN # 0130810819, Prentice Hall PTR This book covers semaphores, threads, record locking, memory mapped I/O, message queues, RPC's, etc.
	TCP/IP Sockets in C, Second Edition: Practical Guide for Programmers by Michael J. Donahoo, Kenneth L. Calvert ISBN #0123745403 IPv4 and IPv6
	"Advanced Linux Programming" by Mark Mitchell, Jeffrey Oldham, Alex Samuel, Jeffery Oldham ISBN # 0735710430, New Riders Good book for programmers who already know how to program and just need to know the Linux specifics. Covers a variety of Linux tools, libraries, API's and techniques. If you don't know how to program, start with a book on C.
	"Advanced UNIX Programming" Second Edition by Marc J. Rochkind ISBN # 0131411543, Addison-Wesley Professional Computing Series
	"Advanced Programming in the UNIX Environment" First Edition by W. Richard Stevens ISBN # 0201563177, Addison-Wesley Professional Computing Series It is the C programmers guide to programming on the UNIX platform. This book is a must for any serious UNIX/Linux programmer. It covers all of the essential UNIX/Linux API's and techniques. This book starts where the basic C programming book leaves off. Great example code. This book travels with me to every job I go to.
	"Advanced Unix Programming" by Warren W. Gay ISBN # 067231990X, Sams White Book Series This book covers all topics in general: files, directories, date/time, libraries, pipes, IPC, semaphores, shared memory, forked processes and I/O scheduling. The coverage is not as in depth as the previous two books (Stevens Vol 1 and 2)
	"Linux Programming Bible" by John Goerzen ISBN # 0764546570, Hungry Minds, Inc This covers the next step after "C" programming 101.
	C++ How to Program by Harvey M. Deitel, Paul J. Deitel ISBN #0131857576, Prentice Hall Fifth edition. The first edition of this book (and Professor Sheely at UTA) taught me to program C++. It is complete and covers all the nuances of the C++ language. It also has good code examples. Good for both learning and reference.

   

Advertisements