What is Megaphone?

What is Megaphone?
The Megaphone project is about enhancing open source chat software. Specifically, the goal is to allow ejabberd to support 1,000,000 simultaneous users. See The Plan page for more details on how I plan to solve this problem. See the About this Blog page for more details on why I created this blog.

Wednesday, February 29, 2012

Slow Going

Previously...
  • I made some progress in wiring up ejabberd to megaphone.
  • My primary computer's power supply died.
  • I salvaged one of the drives from my dead computer.

I finally managed to get back to doing some testing on megaphone.  It appears that data is getting to ECM OK, but that it gets stuck in megaphone.  I get the following in the log:

=ERROR REPORT==== 2012-02-29 07:18:38 ===
Error in process <0.367.0> on node 'ejabberd@localhost' with exit value: {{badmatch,{more,undefined}},[{megaphone,parse_packet,1},{megaphone,update_table,3},{megaphone,put_data,3},{megaphone,main_loop,1}]}

This indicates to me that the following line is failing:

parse_packet(Data) ->
    {ok, Request, Remainder} = erlang:decode_packet(http, Data, []),
    parse_headers([Request], Remainder).

Specifically, erlang:decode_packet is returning "{more undefined}" instead of "{ok, <something>, <something>}" which means that I need to go back and read the decode_packet documentation some more.

Next time: causes and solutions to the above problem.

Tuesday, February 28, 2012

Please Bear with Us

Previously...
  • I removed mod_megaphone for the list of ejabberd modules.I
  • I made some progress in wiring up ejabberd to megaphone.
  • My primary computer's power supply died.

The good news is that I managed to salvage one of the hard drives from my primary system and am therefore able to blog.  The bad news is that I don't have a whole lot to write about.

Some questions that I have for when I am able to run through some tests against megaphone/ejabberd are: 
  • Is the BOSH subsystem receiving data from megaphone.
  • Is the BOSH subsystem trying to send data to the megaphone subsystem.
Small steps because of the current situation.

Monday, February 27, 2012

Technical Difficulties

Previously...
  • I tried running ejabberd with megaphone: things did not go well.
  • I removed mod_megaphone for the list of ejabberd modules.
  • I made some progress in wiring up ejabberd to megaphone.

And then all hell broke loose.  First I needed to go on a short trip that prevented me from blogging for a few days.  But then the real bomb hit: my computer power supply seems to have reached the end of its short life.  

Fortunately, I have several different computers that I work on, and the ejabberd code lives on one of the machines other than the one whose power supply was fried.  Hopefully, this will only prove to be a short interruption.

But I know better.

Next time, hopefully back on the primary system.

Friday, February 24, 2012

ECM to Megaphone to ejabberd

I tried hooking up ECM to Megaphone.  It connected and this was good.  Then I tried connecting directly to ejabberd via Pidgin and that worked and that was good.  Then I tried connecting to ejabberd via Pidgin via ECM and that did not work.  While this was not good, it was not surprising.

Looking at what ECM is printing in the way of error messages I saw the following:


config = { server: { host: 'localhost', port: 6280 },
  client: { port: 8280 },
  misc: { prefixLength: 32 } }
server is now listening to port 8280
now connected to localhost:6280


0000000409|00000000000000000000|POST /http-bind/ HTTP/1.1
Host: ubuntu2
User-Agent: Pidgin 2.6.6 (libpurple 2.6.6)
Content-Encoding: text/xml; charset=utf-8
Content-Length: 225


<body content='text/xml; charset=utf-8' secure='true' to='ubuntu2' xml:lang='en' xmpp:version='1.0' ver='1.6' xmlns:xmpp='urn:xmpp:xbosh' rid='2535528048031658' wait='60' hold='1' xmlns='http://jabber.org/protocol/httpbind'/>


sendToServer

This gibberish means that I have to make up less stuff to fill out a blog posting, but it also means that ECM is able to connect to Megaphone and furthermore that I was able to send some fairly credible looking data to Megaphone.

Looking at the ejabberd log I get the following:

=INFO REPORT==== 2012-02-23 10:48:39 ===
D(<0.368.0>:megaphone:57) : megaphone is starting up with Socket = #Port<0.3805>

This means that at least megaphone was able to start up in response to ECM.  I don't see anything from Megaphone in response to the packet sent from Pidgin, which means it's back to the source code to put in more debugging statements.  

When taken together what this all means is that the damn thing isn't working, but I am making some progress.  One can infer a number of things from this, but I choose to think that I am making titanic, frog-like leaps of progress through a swamp of confusing messages.

Really.

Next time: more debugging statements from inside Megaphone.


Thursday, February 23, 2012

More Testing with ejabberd

Previously...
  • I created comments for the megaphone_sender module.
  • I fixed some stuff and ran out of excuses not to start testing with ejabberd.
  • I tried running ejabberd with megaphone: things did not go well.

Rooting around with ejabberd.cfg I found the part where it was starting up megaphone:

  {mod_version,  []},
  {mod_megaphone, []}
 ]}.

When you define a module this way, ejabberd invokes it at startup simply by calling 

<module name>:start(Host, Options)

Looking at the source for that part I found the start code:

start(_Host, _Opt) ->
    megaphone:start().

stop(_Host) -> ok.

As it turns out, megaphone does not have a start function that takes no arguments, hence the (current) problem.  I removed mod_megaphone from the ejabberd.cfg file and ejabberd was able to start up again.

One down, must more fun to go.

Wednesday, February 22, 2012

Yack! Gag! Integration with ejabberd

Previously...
  • I created comments for the megaphone_receiver module.
  • I created comments for the megaphone_sender module.
  • I fixed some stuff and ran out of excuses not to start testing with ejabberd
> Look
> You are in a room with an ejabberd module called "megaphone" and the ejabberd system
> Connect the damn thing 
> I see no "damn thing."
> Connect megaphone to ejabberd 
> Megaphone is now connected to ejabberd 
> Turn on ejabberd 
> ejabberd explodes with a flat "kaboom" which blows up the machine, the room, and the house.  By some miracle you are not killed.  It is now, however, completely dark and you are therefore eaten by a grue.  
You have died.  You have earned 2 out of 768 possible points.  This gives you the rank of rank amateur.

Well...I can't say that I'm surprised that it didn't work, but I am very disappointed.  VERY.  DISAPPOINTED.

Now granted, this is what always happens but certain coworkers, who were somewhat inebriated at the time, assured me that this time everything would work out just fine, given the exhaustive review that they had put my system through before allowing me to throw the switch.

In this particular situation there are two ways that you module can be kicked off: when the system starts up and when the web portion of the system gets a connection.  The part I am trying out is the one that starts up when the system starts up.

Then I need to figure out what API that megaphone needs to implement in order to be notified and how it should respond.

All fun topics that I will be wrestling with for quite some time.

Tuesday, February 21, 2012

Don't Start the Party Quite Yet

Previously...
  • I created comments for the megaphone module.
  • I created comments for the megaphone_receiver module.
  • I created comments for the megaphone_sender module.

First a shout out to Italy, which in addition to inventing Chianti, have fostered a number of people of very good taste, as demonstrated by their visiting this site.  I'm not saying that they taste good or endorsing cannibalism, just that they really know their sites; along with the Russians.

Anyhow, right when I was really looking forward to testing out the system while wired up to ejabberd, I ran into a few minor technical glitches.  These manifested themselves in compile time errors that I did not clean up yesterday when I was finishing up with my code clean-up.

The many people who read this blog, though they consist mostly of people I bribe along with the few who think that if they visit the site they have a chance to win a new laptop or something (you don't but if I ever sell out completely and start doing adds, then I might get one), my plans to begin testing using ejabberd will have to wait another day.  I am hoping that this will be the start of many angsty posts about how depressed I am over the delay and blah, blah, this and whine whine that.  But the changes were minor enough that tomorrow I shouldn't have any excuses to hide behind.  

Now is the time for people who actually follow this blog to mail in suggestions about how to put that wretched time off or, if you are of a more optimistic bent, you can send in ideas or exhortations about this will just make these more challenging.

If nobody bothers to write in I will be forced to create some made up feedback which I always feel momentarily bad about but not being tremendously proud gives me an advantage.  Then there's always the possibility that I will actually do something relevant to the project.

I tells ya, the excitement is getting palpable...


Monday, February 20, 2012

Comments: megaphone_sender

Previously...
  • I finished testing the send functionality (go me!).
  • I created comments for the megaphone module.
  • I created comments for the megaphone_receiver module.

Before I go any further I want to say that I've always felt that Russia has the best Vodka.  And this is not because I am either trying to pad this posting or pander to my readers.

Ahem.  Now then.  I finished commenting the megaphone_sender module, which means that I have finished commenting the system.  I also checked my changes in version control (git).  Once the subsystem has a vague chance of working, I plan on checking the project into github. 

During the whole checkin process, I managed to remove some unused files.  I find that some extra files can accumulate as time goes on.  The problem is that, after a while, I'm not sure if they belong in the project any more.  What's worse is that I can end up making changes to those files because I want to make sure everything is up to date.  

A nice thing about erlang is that it will tell you if a function is not being used by anyone, assuming that the function is not exported.  If the function is exported, then you have to go through the whole grep song and dance to figure out if anything references it and if not whether it's really a library function, etc.  

At this point I have run out of excuses to start testing the thing.  Except that I am sure it will work on the first try.   The good news is that I can put that off until tomorrow.  So...next time: testing the system with ejabberd...unless I can think of something else to put this off with.

Sunday, February 19, 2012

Comments: megaphone_receiver

Previously...
  • I started testing the send functionality.
  • I finished testing the send functionality (go me!).
  • I created comments for the megaphone module.

This time around I created some comments for the non-test methods of the megaphone_receiver module.  One thing of note that I discovered was that one of the functions to defined, to get the status of the module, would not work.  The reason for this goes back to the reason for creating the module in the first place.

At this point, I know of no way to listen at a socket and to listen for an erlang message at the same time.  Since the subsystem needs to listen for messages to send as well as new data, there is a need for this process.  The thing is, this process is busy listening for new data over the TCP port so it will not hear other messages telling it to respond to a status message or to shutdown.  

Therefore the status API function will never work with the module in its current form: because the status message would come in via the erlang message route instead of the TCP route, the process would never see it.  In the spirit of code cleanup, I therefore removed it.

This may seem like a small and trivial detail.  That's because it is.  But it does illustrate the larger point of cleaning up code.  Without cleanup, source code accumulates non-functional effluvia that makes the code harder to understand.  Performing cleanup is vital to keeping the forces of entropy at bay.

Of course, the laws of thermodynamics tell us that this is all a waste of time and that chaos will win in the end, but still...

Next time: commenting megaphone_send.

Friday, February 17, 2012

Comments: megaphone



Previously...
  • I puzzled through some ejabberd issues with send.
  • I started testing the send functionality.
  • I finished testing the send functionality (go me!).

Rather than wiring megaphone together with ejabberd, I decided to add some comments to the various functions that make up the megaphone module.

When trying to understand any source code, comments are incredibly important.  Unfortunately, these are often missing in open source projects so I find myself taking an inordinate amount of time just trying to understand what is going on, rather than contributing or whatever to the project.  I resolved therefore that my project would be different, hence my taking some time to comment what I have done.

Note that this is not due to something like cowardice with regards to seeing my system crash and burn when I try it out.  No sir.

Next time: comments for the sender and receiver modules.

Thursday, February 16, 2012

Testing megaphone_sender: Go Me!

Previously...
  • I completed testing for receiving data from the BOSH client.
  • I puzzled through some ejabberd issues with send.
  • I started testing the send functionality.

I managed to get the send functionality to the point where it seems to work.  In situations like this, I find it better to stop when things appear to be working rather than inquiring too closely as to whether they are actually working - at least this way I have the delusion of progress.

The problem with einval was actually a test to allow readers to learn more about erlang.  That's my story and I'm sticking to it!  This was caused by the dreaded active mode/passive mode aspect of gen_tcp and not at all because I made the same mistake twice.

Sigh.

Elsewhere in the news, have I mentioned how much I admire Denmark?  This is not at all because a recent surge in hits for this site appear to have come from there.  Nope.  Not at all.

After resolving the issue with active/passive TCP sockets, I realized that I had not actually written the code to send packets.  As it turned out this was not a huge problem because the code to send a packet consisted of the following:

send(ConnectionID, Data) ->
    megaphone ! { send_data, ConnectionID, Data }.

I'm not exactly thrilled by the fact that I have three processes to do this one job.  And that one of those processes sole duty is to pass data on to the other two processes.  Of course I can always rewrite it and then claim that was my plan all along.  I can even point to these posts as evidence...hmmm.....

At this point I suppose the next step is to try and wire the thing up to ejabberd and see what breaks.  I'm sure the fact that I found all these annoying bugs and problems at this stage instead of after I have it connected to ejabberd will save me a lot of time.  Of course the only way to figure out how much time I saved is to put all the bugs back in and then...oh never mind.

Next time: connecting to ejabberd.

Wednesday, February 15, 2012

Foo Luck

Previously...
  • I encountered some problems with states.
  • I completed testing for receiving data from the BOSH client.
  • I puzzled through some ejabberd issues with send.

Unfortunately, I did not make as much progress as I had hoped for.  I'd like to say this was someone else's fault, so I will go ahead and blame this on Clippy, the evil paper clip from Office <insert date here>.  That's right folks, I was all set to make some serious progress when he came storming in and messed up everything.

At any rate, I created some more test code:


test_sender_receiver(Port) ->
    io:format("trying to connect to server via port ~p~n", [Port]),
    {ok, Socket} = gen_tcp:connect("localhost", Port, [binary, {packet, 0}]),
    gen_tcp:controlling_process(Socket, self()),
    test_sender_loop(Socket).

test_sender_loop(Socket) ->
    io:format("socket ~p~n", [Socket]),
    {ok, Data} = gen_tcp:recv(Socket, 0),
    io:format("received ~p~n", [Data]),
    test_sender_loop(Socket).


test_send() ->
    start_test(),
    spawn(?MODULE, test_sender_receiver, [4567]),
    receive after 1000 -> ok end,
    send(24, "<body>some response</body>").

When running this I get the following error:


=ERROR REPORT==== 15-Feb-2012::19:24:30 ===
Error in process <0.35.0> with exit value: {{badmatch,{error,einval}},[{megaphone,test_sender_loop,1}]}

I think I could probably clear this problem up, but I'm too lazy to do that right now, so I'll just leave it at that.

Next time: hopefully completed sender testing.

Tuesday, February 14, 2012

ejabberd_http_bind Interface Issues


Previously...
  • I resolved some problems with dictionaries.
  • I encountered some problems with states.
  • I completed testing for receiving data from the BOSH client.

"Sending" data to the client of a BOSH connection is a little strange.  The thing is, data is always "sent" by responding to an HTTP POST that the client has issued.  The client issues a POST when it wants to send data to the server and the server responds immediately.  

The client also issues a POST when it wants to receive data from the server.  In that case the server delays responding to the POST until either a timeout occurs, at which point it responds with an empty packet, or the server has some data for the client, in which case it puts the data in the response to the POST. 

This poses something of a problem for megaphone: how does the system figure out which virtual connection the server is responding to?  


It turns out that the HTTP stuff really doesn't matter from the standpoint of megaphone: it just sends and receives TCP segments.  While megaphone is marginally aware of the nature of HTTP, in that it has use erlang:decode_packet on the data that it hands off to ejabberd when it sends data back to the client it has no idea if this is a response to an HTTP request - it's just another segment as far as megaphone is concerned.  


As to the issue of which connection the response is going to, when ejabberd_http_bind wants to send something, it includes the socket as one of the arguments.  In the case of megaphone, this includes the connection ID.


This resolves most of the issues associated with the interface except for one thing: how is ejabberd_http_bind notified of a new client connection or new client data.


Next time: just how does megaphone tell ejabberd about new connections?

Monday, February 13, 2012

Receive: Excelsior!


Previously...
  • I resolved some problems with lists and test data.
  • I resolved some problems with dictionaries.
  • I encountered some problems with states.

While I have a nagging belief that saying that something is working will immediately cause the code gremlins to break everything, I finally managed to get the receive portion of the Megaphone tests to at least look like they are working!

The output of the test_receive_read looks like this:


1> megaphone:test_receive_read().
will try to send packet to port 4567
megaphone is now listening on port 4567
megaphone now has a connection on port 4567, starting...
megaphone is now running
Sent test packet to 4567
{ok,{http_request,'POST',{abs_path,"/"},{1,1}}}
{ok,{http_header,24,'User-Agent',undefined,
                 "curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15"}}
{ok,{http_header,14,'Host',undefined,"localhost:4567"}}
{ok,{http_header,8,'Accept',undefined,"*/*"}}
ok
2> 

A bunch of gibberish I know, but the basics are that the first call to recv results in the line 

{ok,{http_request,'POST',{abs_path,"/"},{1,1}}}


Which denotes that the connection received an HTTP POST.  The next three lines contain some of the header lines from the POST.


{ok,{http_header,24,'User-Agent',undefined,
                 "curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15"}}
{ok,{http_header,14,'Host',undefined,"localhost:4567"}}
{ok,{http_header,8,'Accept',undefined,"*/*"}}


A small victory I know, but I may as well savor it, since so much of what I do seems to be fixing problems in my stuff.  It's nice when something works for a change.

Next time: start on send tests.

Sunday, February 12, 2012

Fun with States

Previously...
  • I encountered a problem with my test data.
  • I resolved some problems with lists and test data.
  • I resolved some problems with dictionaries.

The next problem that I resolved had embarrassingly little to do with the erlang libraries.  With "imperative" languages like Java I am used to doing something like this:

int foo = 1;
foo++; // foo now has the value 2

Whereas with a functional language like erlang, one instead uses something like this:

Foo = 1;
Foo2 = 1 + Foo;

This is because in languages like erlang, you cannot change a variable once it has been set.  Personally, I don't think one should use the term "variable" to describe such an object in erlang, but I digress.

The problem I was running into was that I was not updating the variable that I changed.  Thus I would use something like this:


put_data(State, ConnectionID, Data) ->
    NewState = update_table(State, ConnectionID, Data),
    { Result, NewerState } = remove_waiter(ConnectionID, NewState),
    case Result of
        undefined -> NewerState;
        Waiter -> notify_waiter(Waiter, ConnectionID, State)
    end.

Whereas what I really wanted was something like this:

put_data(State, ConnectionID, Data) ->
    NewState = update_table(State, ConnectionID, Data),
    { Result, NewerState } = remove_waiter(ConnectionID, NewState),
    case Result of
        undefined -> NewerState;
        Waiter -> notify_waiter(Waiter, ConnectionID, NewerState)
    end.

Nothing really Earth shattering here, just ye basic functional vs. imperative programming styles.  This will not stop me from complaining, but I'm just saying...

Next time: more testing goodness.

Saturday, February 11, 2012

The Foreign Legion

Previously...
  • I encountered an annoying problem and thereby vindicated my efforts.
  • I encountered a problem with my test data.
  • I resolved some problems with lists and test data.

I couldn't help noticing that in addition to rent-a-page-view and one other person (hi Mom!), I got a lot of hits the other day from Germany.  I must say that I think you guys make the best beer on Earth; something that has been very important to this project so far :-)

The problem I was having with dictionaries was pretty simple.  With the erlang "dict" module, it is an error to call dict:fetch with a key that is not in the table.  The first time I received data for a particular connection I was not checking first, hence the error.

Mind you, the function could have just returned undefined but nooooooooo!  They had to throw an exception.  Of course I wasn't checking for undefined either, but it's the principal of the thing.

The next point of interest that I found was that, if a client asks for some data that is not available, the system would simply forget about the client completely.  It should be recording the client in the list of waiters for that connection, so I ended up writing another function to take care of that:


add_waiter(ConnectionID, State, PID) ->
    Table = State#megaphone_main_state.waiters,
    Waiters = case dict:is_key(ConnectionID, Table) of
        false -> [];
        true -> dict:fetch(ConnectionID, Table)
    end,
    NewWaiters = Waiters ++ [ PID ],
    State#megaphone_main_state{waiters = NewWaiters}.

I noticed that this could also introduce a potential memory leak: what happens if no data ever comes in for that connection?  I created a new parking lot issue to address this.

Next time: more recv testing.

Friday, February 10, 2012

Progress Most Slow and Painful

Previously...
  • I resolved to test from the client side of the API.
  • I encountered an annoying problem and thereby vindicated my efforts.
  • I encountered a problem with my test data.

Using my trusty curl command line utility, I was able to figure out what the test data really ought to look like:

test_string() ->
    "0000000281|00000000000000000024|"
    "POST / HTTP/1.1\r\n"
    "User-Agent: curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15\r\n"
    "Host: localhost:4567\r\n"
    "Accept: */*\r\n"
    "Content-Length: 24\r\n"
    "Content-Type: application/x-www-form-urlencoded\r\n"
    "\r\n"
    "<body>hello world</body>".

The contents were obtained via

curl -X POST -d @tfile http://localhost:4567

I had to play around with the packet length parameter a bit to get the correct value.  

After that, everything just magically worked.  

Well, actually it didn't, but it would have been nice if it had.

The next problem I ran into was on account of my lack of familiarity with erlang.  I was using a statement like this:

SomeList ++ SomeElement

The problem with this is that the ++ operator really wants to concatenate two lists together, rather than a list and an element.  Switching things around a little resulted in erlang being happier:

SomeList ++ [ SomeElement ]

There's probably some other form of syntax that would accomplish this without having to construct a new list, but I don't know what that is at this point.

And that, everything just magically worked.

OK, so that phrase is getting old, but you have to admit, it was funny the first couple of times.  

The current problem has something to do with how I'm using dictionaries.  That will be the topic for next time.

Thursday, February 9, 2012

Obvious Foul

Previously...
  • I tested the ability of the megaphone module to receive data.
  • I resolved to test from the client side of the API.
  • I encountered an annoying problem and thereby vindicated my efforts.

Well, actually, I just found a problem last time, but nevertheless, I feel vindicated.  More or less.

At any rate, the problem that I was running into was due to the wrong message being sent to the megaphone process by the megaphone_receiver process.  Specifically, I needed to send a message that said "Hey!  There's some data that you should keep track of!"  Whereas I was sending a message that was more like "Here's some data.  I'm not going to tell you it's something you should keep track of, so there!"

Anyhow, that problem was quickly fixed.  And then I ran into another problem.  But it wasn't my fault.

The test data that I was sending had the form:

<body>whatever</body>

whereas the megaphone process was expecting HTTP.  And just because I was the person who wrote the data, does not mean that it was my fault.  

Actually I guess it does.

Oh bugger.

Next time: better test data.

Wednesday, February 8, 2012

Blue Wednesday

Previously...
  • I finished testing the megaphone_sender component.
  • I tested the ability of the megaphone module to receive data.
  • I resolved to test from the client side of the API.

After resolving to test the client side I tried out megaphone:recv when some data was already present.  That failed.

I suppose I should feel vindicated that the test will save time in the long run, but for right now I just wish the thing worked.  Ah well.

I created some new test code for this attempt:

test_receive_read() ->
    start_test(),
    test_receiver(),
    recv(24).

The test sets up the client and ECM connections, sends a packet of data, then attempts to read that packet via the recv method.

When I tried this out the first time and it just sat there like a bump on a log, I tried putting in some debug statements and discovered that megaphone did not even think that any data had been received for that connection.  

That was as far as I got, so next time I will try to figure out where the data is disappearing to.

Monday, February 6, 2012

Blue Monday

Previously...
  • I finished testing the megaphone_receiver component.
  • I finished testing the megaphone_sender component.
  • I tested the ability of the megaphone module to receive data.

After looking at what I had done for testing megaphone reception, I became convinced that it was a good start but not enough.  What I really needed was to be able to test the ability of ejabberd to receive data through megaphone. 

On the other hand, my real goal with testing is to save time that would be spent debugging megaphone in the far more difficult context of a running ejabberd system.  While creating a simulation of ejabberd is nice, if it ends up taking as much or more time than simply using megaphone in a running ejabberd system, then I would really be wasting time.

Decisions, decisions.

I think what I shall do is to test a relatively simple receive and then a send.  After that, I will try hooking the module into ejabberd and see what happens.

Next time: calling megaphone:recv




Sunday, February 5, 2012

Testing: megaphone


Previously...
  • I started testing the megaphone_receiver component.
  • I finished testing the megaphone_receiver component.
  • I finished testing the megaphone_sender component.
So...back to the megaphone module...

I added some more test code.  Rather than trying to figure out what changed here is what I used:


start_test() ->
    spawn(megaphone, test_start_server, [4567]).
    
test_start_server(Port) ->
    {ok, LSocket} = gen_tcp:listen(Port, [binary, {packet, 0}, {active, false}]),
    io:format("megaphone is now listening on port ~p~n", [Port]),
    {ok, Sock} = gen_tcp:accept(LSocket),
    io:format("megaphone now has a connection on port ~p, starting...~n", [Port]),
    ?MODULE:start({gen_tcp, Sock}, undefined),
    io:format("megaphone is now running~n").


test_receiver() ->
    Port = 4567,
    test_receiver(Port).
    
test_receiver(Port) ->
    io:format("will try to send packet to port ~p~n", [Port]),
    {ok, Socket} = gen_tcp:connect("localhost", Port, [binary, {packet, 0}]),
    receive after 1000 -> ok end,
    gen_tcp:send(Socket, "0000000053|00000000000000000024|<body>whatever</body>"),
    io:format("Sent test packet to ~p~n", [Port]).
   
After many trials and tribulations, the test code eventually worked (go me times three!).  The biggest gotcha I ran into was the following:

When using gen_tcp:listen it is important to use the correct option re: binary mode.  At first I used the following:

{ok, LSocket} = gen_tcp:listen(Port, [{packet, 0}, {active, false}])

Note the missing binary option in the options to gen_tcp:listen.  Because of this, the calls to things like byte_size were failing because gen_tcp:recv was returning strings (lists) rather than binaries.  

Next time: status and (hopefully) the sender.

Saturday, February 4, 2012

Testing megaphone_sender: Times Two!

Previously...
  • After some initial tests, I resolved to test the different components in turn.
  • I started testing the megaphone_receiver component.
  • I finished testing the megaphone_receiver component.

I finished testing the megaphone_sender module.  A proud person might point out that this module is rather, erm, modest.  Fortunately, I am not proud: Go Me!

At any rate, I had to create some test harness stuff for the sender module:

test() ->
    %% 
    %% Create a connection for the process to talk to.
    %%
    Port = 4567,
    test_listener(Port),

    %%
    %% give the listener a little time to start up
    %%
    receive after 100 -> ok end,
    test_sender(Port).

test_listener(Port) ->
    { ok, LSocket } = gen_tcp:listen(Port, [binary, {packet, 0}, {active, false}]),
    spawn(?MODULE, test_listener_start, [LSocket]),
    io:format("tester is now listening on port ~p~n", [Port]).

test_listener_start(LSocket) ->
    {ok, Socket} = gen_tcp:accept(LSocket),
    io:format("tester is now connected.~n"),
    listen_loop(Socket).

listen_loop(Socket) ->
    Data = gen_tcp:recv(Socket, 0),
    io:format("tester has received ~p~n", [Data]),
    listen_loop(Socket).

test_sender(Port) ->
    io:format("test_sender is trying to connect to port ~p~n", [Port]),
    {ok, Socket} = gen_tcp:connect("localhost", Port, [binary, {packet, 0}]),
    io:format("test_sender is now connected to port ~p~n", [Port]),
    spawn(?MODULE, start, [Socket, undefined]),

    %%
    %% give the system a bit of time to start up
    %%
    receive after 100 -> ok end,

    %%
    %% test sending a packet through the system
    %%
    io:format("trying to send message to sender~n"),
    megaphone_sender ! { 16, "<body>some response</body>" },
    io:format("sent messsage~n").

After a few typos, misformats and other assorted bugs, the sender code seems to be working correctly.  

Note that all during testing I have had to make various changes to the underlying code, hence the original stuff that I posted will not work correctly.  Rather than reposting the updated modules here, I think it would be more useful to just post the tested stuff up on github.  

Besides posting the revised version of the code would be a cheap out and I absolutely resolve not to do that.  Absolutely.  Will not.  At least until I feel lazy.

Next time: back to the megaphone module.