A while back a friend and started developing an online multiplayer Geo-Racing game where a group of people are able to compete against each other in a race from Point A to Point B anywhere in the world (almost). In all its simplicity it is basically like any other running gps tracker out there, with an additional built in competitive system. This post is simply just a small introduction and overview of some of the technologies that is included in the project and a brief introduction of the backend mechanics of the game.

My project partner has written a nice post with some more detail on the technology layers and some more details on the beneficial common layering :

http://www.squeed.com/blog/2013/12/mqtt-powered-event-bus-for-android-applications/

http://www.squeed.com/blog/2013/12/common-jpa-entities-for-android-and-backend/

Why MQTT?

Because we thought it was a fun idea exploring the possibilities of creating a fully message driven system on Android. Also with its messaging being tiny, power efficient, low cost keep alive and low latency push capabilities, it is a perfect fit for the system we are building.

I will not touch upon the subject of performance, there is already an excellent post from Stephen Nicholas taking up some of these aspects http://stephendnicholas.com/archives/219.

Technology choices

There are more and more brokers out there supporting the MQTT protocol. Some of the brokers we initially tried out was ActiveMQ, ActiveMQ Apollo and Mosquitto. The natural choice fell on Mosquitto given it is a native MQTT broker with good documentation, a hot mailing list and fast turnarounds for bugs and support (http://mosquitto.org/).

I’v come across two settled opensource MQTT client libraries that can be used for the server and client setup.

• Eclipse Paho (http://www.eclipse.org/paho/) "is the primary home of the reference MQTT clients that started at IBM", has good documentation and supports a variety of languages
• Fusesource MQTT - Client (https://github.com/fusesource/mqtt-client) (the one used in our project) in all its simplicity has some nice predefined ways of setting up your connecting, such as, BlockingConnection - FutureConnection and CallbackConnection

The topology

So the architecture of this whole setup is pretty straight forward. Basically what we have from start-up is a Server application connecting and subscribing to an (what we call it) Action Topic (AT) on the Mosquitto broker. Any android application (A) connecting to the broker will start subscribing to a unique topic for that user. If we have 100 users, we will have 101 topics in the system, that may or may not actively subscribe to their own topic. This means that any action that results in a message delivery to one to many topics will result in a very fast push of information to all the consuming devices.

The message handling

Has been abstracted as a common library included by both the server and the android applications. Since the MQTT message payload is of type byte, we made the decision to define any message sent between the client and the server (and vice versa) to be consisting of a byte[] serialized java class. This consists of a top level abstract class “Application Message“ which any new message type would then extend. This means that any payload sent between the client and server will be a deserializable byte payload known to both the client and server message handler.

So sending a message from the client to the server could go like this:

1. Create a new message and add all necessary information
2. Serialize the message to a byte[]
3. Add the data to the buffer payload and publish mqtt message on destination topic.

Now the server will retrieve this buffer payload, deserialize it, and by Visitor Pattern determine what kind of “Application Message” where sent and operate on the message accordingly. The beauty of this now is that the Android application will react in the exact same way. The three steps would be taken by the server for any message that might be sent to the client, and by convention the client subscribing to that topic would determine what message were sent by deserializing and dropping it through the message visitor and act on it accordingly. And for any additional processing necessary on either the server or android application we can simply override the basic behavior for the visit method for that message type and define our own logic.

The database back-end has also been included in the common packaging for both the server and the client. OrmLight (http://ormlite.com/) is a relatively lightweight ORM java package for persisting java objects in an SQL database. Here we define all our database entities by JPA annotations and define our DAO layer. The beauty of this is that we are simply able to make use of the same dao for both the server and android back-end. Since the servers main objective is “only” to mirror and persist the state of our complete network the way we define our CRUD can be commonly shared.

The servers main objective is to persist and store information. In some scenarios it also is responsible for forwarding messages between users when they lack information to send the message directly to the user themselves. It is also responsible for Registration and Login. Since MQTT by nature is asynchronous, we have implemented a custom synchronous messaging between the client and server using callbacks for handling logins and registration.

The android client will periodically push its information to one or more topics when active. The messages sent by the android app will be published to either client, client&server or server. When publishing the information, MQTT supports three different types of Quality Of Service (QoS). When choosing in what QoS the message should be sent, there will be an affect on the applications battery consumption etc since there are more or less messages being sent in the background.

Information below on QoS is taken from the paho documentation: http://www.eclipse.org/paho/files/mqttdoc/Cclient/qos.html

• QoS0, At most once: The message is delivered at most once, or it may not be delivered at all. Its delivery across the network is not acknowledged. The message is not stored. The message could be lost if the client is disconnected, or if the server fails. QoS0 is the fastest mode of transfer. It is sometimes called "fire and forget". The MQTT protocol does not require servers to forward publications at QoS0 to a client. If the client is disconnected at the time the server receives the publication, the publication might be discarded, depending on the server implementation.
• QoS1, At least once: The message is always delivered at least once. It might be delivered multiple times if there is a failure before an acknowledgment is received by the sender. The message must be stored locally at the sender, until the sender receives confirmation that the message has been published by the receiver. The message is stored in case the message must be sent again.
• QoS2, Exactly once: The message is always delivered exactly once. The message must be stored locally at the sender, until the sender receives confirmation that the message has been published by the receiver. The message is stored in case the message must be sent again. QoS2 is the safest, but slowest mode of transfer. A more sophisticated handshaking and acknowledgement sequence is used than for QoS1 to ensure no duplication of messages occurs.

Message aggregation

In all its simplicity we want to assure that any message sent will reach its destination. How we propagate the messages in order for all involved parties to receive their messages is a challenge. The following is an example of how the message propagation works when we do a race-invite to multiple users.

A1 sends a race invite to A2 and A3 containing the race information and a list of anyone that might need further information regarding this race (information so that A2 and A3 may communicate since they might not be friends)

A2 Accepts the invitation by creating the race in its local database, return his information to A1, A3 and the server so that the two get all information and the server can persist.

Finally A3 also accepts the race invite, creating the race in his local database and sending his information to both A1, A2 and the server for persistence.

Whats left out in these examples is the actual retry policy that is underlying for both the server and the android client. There is no support in MQTT (AFAIK) that takes care of persisting and automatically retrying to publish any failed publications. Lets say that the mosquitto broker were to be offline at the point of a message publication, the retry to send this message needs to be handled on client level, hence we have written our own retry policy to take care of these events.

To sum up…

So far having chosen MQTT as our primary communications protocol it has been  a fast and fun experience learning some new aspects diverging from the more common ways of communicating in a distributed environment. The project is still in working progress, so we are still eager for any interesting problems that might arise.