What is Megaphone?

What is Megaphone?
The Megaphone project is about enhancing open source chat software. Specifically, the goal is to allow ejabberd to support 1,000,000 simultaneous users. See The Plan page for more details on how I plan to solve this problem. See the About this Blog page for more details on why I created this blog.

Friday, January 13, 2012

Making Connections

Previously...
  • I created a new distribution and the mod_adhoc problem went away.
  • I decided to keep using the ejabberd BOSH server.
  • I started messing around with nodejs TCP servers.
I modified ECM to use a straight TCP style server.  Everything seems to work with that and it looks like the HTTP headers make their way to the megaphone module that is living in ejabberd.  

The code to set up the server is pretty straight-forward:

function Exchange (socket, connectionID, ecm)
{
    var me = this;

    me.initialize = function (socket, ecm)
    {
        me.ecm = ecm;
        me.socket = socket;

        me.socket.on('data', me.receiveFromClient);
        me.socket.on('end', me.unregister);
    };

    me.receiveFromClient = function (data)
    {
        var packetLength = data.length + 32;
        var packet = "";

        packet += toPaddedString(packetLength, 10) + "|";
        packet += toPaddedString(me.connectionID, 20) + "|";
        packet += data;

        me.ecm.sendToServer(packet);
    };
...

Looking at the output from this I got

0000000408|00000000000000000001|POST /http-bind/ HTTP/1.1
Host: ubuntu2
User-Agent: Pidgin 2.6.6 (libpurple 2.6.6)
Content-Encoding: text/xml; charset=utf-8
Content-Length: 224

<body content='text/xml; charset=utf-8' secure='true' to='ubuntu2' xml:lang='en' xmpp:version='1.0' ver='1.6' xmlns:xmpp='urn:xmpp:xbosh' rid='328237668647899' wait='60' hold='1' xmlns='http://jabber.org/protocol/httpbind'/>

While looking at this, I was again hit with the question as to how to treat the header to each segment.  The approach I'm using is to zero pad the various fields.  For example:

0000000408|00000000000000000001|POST /http-bind/ HTTP/1.1

This indicates that the complete packet is 408 bytes long (including the header) and that this is for connection 1.  Another approach would be to use a variable sized header.  For example:

408|1|POST /http-bind/ HTTP/1.1 ...

The thing I like about the fixed sized header is that the receiving side always knows to read that number of bytes to get the header.  Furthermore, calculating the packet size is simpler.  If I were to use variable sized headers, I would probably change packet size field to be a content length field - it would contain just the number of bytes of data in the content rather than the number of bytes in the complete packet including the header.

A fixed size makes the packets easier to process, while a variable size makes them more extensible.  For example, if I were to decide to use UUIDs for the connection ID, I would have to increase the size of the header to accommodate a 32 character value rather than the current 20 character scheme.

I pride myself on being decisive.  I just may end up making the same decision over and over again.

Next steps...
  • Modify megaphone to use erlang:decode_packet 

No comments:

Post a Comment