fsxNet Wiki

OK, so now we have a basic Telnet hello world example going - let's move forward.

The plan for the tutorial at this stage is to handle some feature negotiation in this article, and the remaining negotiations and input on the next one. Haven't decided what to do after that, but probably a state machine interaction post or two, and then you should know most of what it takes to build a basic old-school BBS system. So, on to feature negotiation.

I've been using SyncTerm as my terminal problem for these posts. When connecting to the prototype BBS we're building, I noticed the feature negotiation flow using the “toggle options” telnet client trick from the intro post. We'll focus on a subset of options, and not the whole spectrum of possible options. IANA has a list of all Telnet options. After implementing these, others should hopefully be easy to implement as well.

Our prototype will:

• show local echo
• suppress Go-Ahead
• ask for the user's terminal type
• set the connection to binary-mode (so we can get input immediately without Enter keypresses)

All of this will be handled at the start of every connection, so let's revisit our connection handler method:

private def handle_connection(socket)
  socket << "hello world\n"
  socket.close
end

Pretty bare at this moment. Let's start by telling the client we'll be handling feature negotiation. Also, to hold the connection info, let's introduce a new variable in the mix.

private def handle_connection(socket)
  socket << "detecting client features..."
 
  conn = ConnHash.new
 
  socket.close
end

We could write down the whole Crystal type, but since it's cumbersome we'll alias it to a shorter name (in this case, ConnHash). To keep our main file uncluttered, let's move this to a separate file.

module BBS
  alias ConnHash = Hash(Symbol, Bool | String | Int32)
end

Save this in the same folder as main.cr and name it aliases.cr since we'll be adding more aliases to this file as needed. Next, require it on main.cr:

require "./aliases"

The first option we want to negotiate is WILL ECHO to signal the client we want user input to be shown locally. That messag's IAC byte array will look like:

• 1st byte - 0xff (255) / IAC
• 2nd byte - 0xfb (251) / WILL
• 3rd byte - 0x01 ( 1) / ECHO

We could create the byte array “by hand” for each message, but let's make it a utility method:

private def iac(*args)
  Slice(UInt8).new(args.size + 1) do |i| 
    i == 0 ? 255.to_u8 : args[i - 1].to_u8
  end
end

Looks weird, but it's quite simple. We start off by splatting the arguments passed. This means we'll be able to handle a variable list of arguments, which serves us since some IAC messages have more arguments than others. We then create a byte array (Slice(UInt8) in Crystal terms) with a size equal to the size of the arguments passed plus one - since we don't want to pass the IAC byte in every method call, we just insert it in the array, thus the plus one being added to the size, making room for it.

In the block's main line, we use the i variable to loop through the array being constructed. When we're at the 0 index, we insert 255 (IAC) as an unsigned 8bit value, and if we're not at index 0, we just pass the arguments from the method into the byte array. Ingenious, and gets the job done.

However, the 255 here does not make for very clean code, and we hate that, so let's create a file to house all of our constants. Save this as constants.cr in the same folder as our main.cr file:

module BBS
  IAC  = 255
 
  DO   = 253
  DONT = 254
 
  WILL = 251
  WONT = 252
 
  SB   = 250
  SE   = 240
 
  BINARY        =  0
  ECHO          =  1
  SUP_GOAHEAD   =  3
  TERMINAL_TYPE = 24
end

As before, don't forget to require it in the main file:

require "./constants"

With this file in place, we can now change the IAC method's source to something a bit more readable:

private def iac(*args)
  Slice(UInt8).new(args.size + 1) do |i|
    i == 0 ? IAC.to_u8 : args[i - 1].to_u8
  end
end

Alright, with all this in place, let's abstract the negotiating to its own method on the main handler method:

private def handle_connection(socket)
  socket << "detecting client features..."
 
  conn = ConnHash.new
 
  negotiate(iac(WILL, ECHO), :echo, socket, conn)
 
  socket.close
end

The negotiate method will take in a byte array (provided by our new iac utility method), a key (:echo in this case), the socket and the conn (our hash to keep connection state). The key being passed is to be set on the conn hash as either true/false once the negotiation is done. The negotiate method looks like this:

private def negotiate(slice, key, socket, conn)
  socket.write slice
 
  reply = Slice(UInt8).new(3)
  socket.read_fully reply
 
  case reply[1]
  when DO, WILL
    conn[key] = true
  when DONT, WONT
    conn[key] = false
  else
    puts "error negotiating #{key}!/#{reply.inspect}"
  end
end

First off, we write the slice (byte array) passed onto the socket. This will signal the client we want to initiate the negotiation for the feature encoded on the array. Then, we read the reply. The reply is always three bytes long, we so create a reply slice and read the response onto it from the socket. Then we get a bit lazy and only check the middle byte in the reply! Since the first byte will be IAC, and the last one is a repetition of the value we sent, we only care if the middle byte is DO/WILL or DONT/WONT. Beware that in a proper system, we'd need some error handling here. Based on the yes/no character of the reply, we set the key passed on the conn hash as true or false.

Thus, after this method runs, the hash will hold if the client agreed to local echo or not! Let's add the remaining options, and print out a summary at the end:

private def handle_connection(socket)
  socket << "detecting client features..."
 
  conn = ConnHash.new
 
  negotiate(iac(WILL, ECHO), :echo, socket, conn)
  negotiate(iac(WILL, SUP_GOAHEAD), :suppress_goahead, socket, conn)
  negotiate(iac(DO, BINARY), :binary, socket, conn)
 
  puts "features negotiated: #{conn}"
 
  socket.close
end

If you run make and connect to your local server with SyncTerm now, you should see:

/bbs_release
now listening in port 2023
features negotiated: {:echo => true, :suppress_goahead => true, :binary => true}

Great success! The client has agreed to all of our feature requests. There's one more request, but it requires special treatment. We want to get the “terminal type” which is a string that the terminal sends to identify itself. Of course, the reply being a string, it won't be a standard IAC 3-byte reply. The format of these special requests is:

• 1st byte - 0xff / IAC (as usual)
• 2nd byte - 0xfa / SB (special begin)
• … (request itself)
• byte before last - 0xff / IAC
• last byte - 0xf0 / SE (special end)

So, to make the client tell us its name, we'll send TERMINAL_TYPE ECHO from our constants (making the meat of the request 0x18 0x01'). The response to this request will be:

• 1st byte - IAC
• 2nd byte - SB
• 3rd byte - TERMINAL_TYPE (0x18)
• 4th byte - IS (0x00)
• …. (ANSI coded string)
• byte before last - 0xff / IAC
• last byte - 0xf0 / SE (special end)

Let's have another utility method to craft the IAC extended byte arrays:

private def iac_sb(*args)
  Slice(UInt8).new(args.size + 4).tap do |slice|
    slice[0] = IAC.to_u8
    slice[1] = SB.to_u8
    args.each_with_index do |arg, idx|
      slice[idx + 2] = arg.to_u8
    end
    slice[-2] = IAC.to_u8
    slice[-1] = SE.to_u8
  end
end

This is pretty much the same logic as the iac method before, just making room for more predetermined bytes in the array being constructed.

Now, to get the terminal string from the client:

private def get_terminal_string(socket)
  socket.write iac_sb(TERMINAL_TYPE, ECHO)
 
  4.times { socket.read_byte }
 
  term_string = ""
  loop do
    this_byte = socket.read_byte
    if this_byte
      break if this_byte == IAC && socket.read_byte == SE
      term_string += this_byte.chr
    end
  end
 
  term_string
end

So we start off by sending the IAC extended request for the terminal type, using the iac_sb utility method. Then we skip the first four bytes (since we know they will be IAC / SB / TERMINAL_TYPE / IS) and start reading the string sent.

We break if the byte being read is IAC and the next one is SE - we could just break on IAC, but by reading the next byte from the socket, we “empty” the bytes present on the socket's buffer. If the byte is not IAC, we add its ANSI value (using .chr) to the terminal string, which is then returned.

Let's change the main handler method once more:

private def handle_connection(socket)
  socket << "detecting client features..."
 
  conn = ConnHash.new
 
  negotiate(iac(WILL, ECHO), :echo, socket, conn)
  negotiate(iac(WILL, SUP_GOAHEAD), :suppress_goahead, socket, conn)
  negotiate(iac(DO, BINARY), :binary, socket, conn)
 
  negotiate(iac(DO, TERMINAL_TYPE), :terminal_type, socket, conn)
  conn[:term_string] = get_terminal_string(socket)
 
  puts "features negotiated: #{conn}"
 
  socket.close
end

You will see now if you connect to the server using SyncTerm, the server's output will show “ANSI” and if you use the command line telnet command, it will be “XTERM-256COLOR”. Pretty exciting!

In the next part, we'll handle responding to features being requested by the client, and actual input. Stay tuned, see you next time!

Here's the full listing of our main.cr file so far:

require "yaml"
require "socket"
 
require "./aliases"
require "./constants"
 
module BBS
  class Config
    YAML.mapping(
      settings: Hash(String, Int32)
    )
  end
 
  class Main
    @config : Config
 
    def initialize
      @config = load_config
    end
 
    def go!
      Signal::INT.trap { puts "SIGINT, exiting gracefully!"; exit 0 }
 
      telnet_port = @config.settings["port"]
 
      server = TCPServer.new("localhost", telnet_port)
      puts "now listening in port #{telnet_port}"
      loop do
        if socket = server.accept?
          spawn handle_connection(socket)
        end
      end
    end
 
    private def iac(*args)
      Slice(UInt8).new(args.size + 1) do |i|
        i == 0 ? IAC.to_u8 : args[i - 1].to_u8
      end
    end
 
    private def iac_sb(*args)
      Slice(UInt8).new(args.size + 4).tap do |slice|
        slice[0] = IAC.to_u8
        slice[1] = SB.to_u8
        args.each_with_index do |arg, idx|
          slice[idx + 2] = arg.to_u8
        end
        slice[-2] = IAC.to_u8
        slice[-1] = SE.to_u8
      end
    end
 
    private def negotiate(slice, key, socket, conn)
      socket.write slice
 
      reply = Slice(UInt8).new(3)
      socket.read_fully reply
 
      case reply[1]
      when DO, WILL
        conn[key] = true
      when DONT, WONT
        conn[key] = false
      else
        puts "error negotiating #{key}!/#{reply.inspect}"
      end
    end
 
    private def get_terminal_string(socket)
      socket.write iac_sb(TERMINAL_TYPE, ECHO)
 
      4.times { socket.read_byte }
 
      term_string = ""
      loop do
        this_byte = socket.read_byte
        if this_byte
          break if this_byte == IAC && socket.read_byte == SE
          term_string += this_byte.chr
        end
      end
 
      term_string
    end
 
    private def get_terminal_size(socket, conn)
      socket.write iac(DO, NAWS)
 
      reply = Slice(UInt8).new(9)
      socket.read_fully reply
 
      puts reply.inspect
 
      cols = reply[4]
      lines = reply[6]
 
      puts "NAWS -> cols: #{cols} lines: #{lines}"
 
      conn[:term_width] = cols.to_i
      conn[:term_height] = lines.to_i
    end
 
    private def handle_connection(socket)
      socket << "detecting client features..."
 
      conn = ConnHash.new
 
      negotiate(iac(WILL, ECHO), :echo, socket, conn)
      negotiate(iac(WILL, SUP_GOAHEAD), :suppress_goahead, socket, conn)
      negotiate(iac(DO, BINARY), :binary, socket, conn)
 
      negotiate(iac(DO, TERMINAL_TYPE), :terminal_type, socket, conn)
      conn[:term_string] = get_terminal_string(socket)
 
      get_terminal_size(socket, conn)
 
      puts "features negotiated: #{conn}"
 
      socket.close
    end
 
    private def load_config
      Config.from_yaml(File.read("config.yml"))
    end
  end
end