What this Blog is About

This blog is about the megaphone project --- a software subsystem that is supposed to allow many more users to connect to the ejabberd chat system than was previously possible.

Ejabberd is a chat system like Yahoo! Instant Messenger, Microsoft Messenger or AOL Instant Messenger.  The difference is that ejabberd is not tied to any particular company and anybody can set up their own chat system using the server.

As of this writing, ye basic ejabberd requires about 100kB per user hosted.  This works just fine until you start getting hundreds of thousands of users on the system.  For 1,000,000 users, this means that you will need around 100GB of memory --- possible but not what I would call a "commodity system."

The goal of the megaphone project is to allow 1,000,000 connected users, and only using around 10GB of memory.  Of that 10GB, less than 4GB would be used by the TCP connections, and the rest would be used by ejabberd to deal with processes and the like.

The idea is to use a library called libevent to handle the TCP connections and then multiplex all the users over a single TCP connection to ejabberd.  This would require

• a libevent module
• an ejabberd module

The libevent module would handle up to about 1,000,000 connections and multiplex them onto a single TCP connection to the ejabberd server.  After a bit of looking around, it appears that modifying a system called node-xmpp-bosh would simplify the libevent portion and would get megaphone BOSH and websocket connections in the bargain so the plan is to use that as the connection server. 

The ejabberd server would demux the connection into individual sessions, and then treat each of them as a regular XMPP connection.  This part will likely be based on the http_bind module that currently comes with ejabberd, since that module has to deal with something other than a straight XMPP connection.

The reason for the blog is to:

• Provide a backup for my notes
• To help someone else who might want to continue the project
• To help others in their efforts to create modules for ejabberd

Will this project work?  What will the author discover about ejabberd?  What did Kosh mean by "We have always been here"  To (possibly) find out the answers to these and other questions, keep reading.