Introduction

A little while ago I performed some power profiling of MQTT on Android to try and put a figure on just how efficient this technology is on mobile devices. I think I got some pretty good results (enough to give an idea anyway), but the next question people always ask is: ‘So how does that compare to the alternatives?’.

Well, first of all, it is really Apples vs. Oranges :D

Of my three main choices for Android mobile push notifications: MQTT, HTTP or C2DM, each is designed for a different purpose; with different features, bad points and good points. I’m not going to do a full comparison here, but to summarise:

• MQTT – designed to provide low latency, assured messaging over fragile networks and efficient distribution to one or many receivers. Protocol focuses on minimising the amount of bytes flowing over the wire and low power usage. Maximum message size of 256MB, but not really designed for sending large amounts of data; better at a high volume of low size messages. Provides a two-way communication channel.
• HTTP – designed as a request-response protocol for client-server computing. Best known as the foundation of data communication for the World Wide Web (well, duh). It can be ‘abused’ to provide push like capabilities, for example by using Comet style approaches, but it really isn’t designed with this in mind. Provides two-way communication capabilities.
• C2DM – designed as a one-way ‘shoulder-tap’ notification system. On the plus side, Google takes care of getting the messages to the device and to your application; waking it if necessary. However there are a number of restrictions. The main one is, as the Google Docs themselves say, that ‘C2DM makes no guarantees about delivery or the order of messages’. Therefore, the recommendation is that a C2DM message should never contain the data itself but, rather, be used to provide a notification that there is data available; and applications should then contact their own server to get the data. Other restrictions include:
  • Only available on devices running > Android 2.2.
  • Requires a registered Google account on the device.
  • Message size is limited to 1024 bytes.
  • Google limits the number of messages a sender sends in aggregate.
  • And the number of messages a sender sends to a specific device.

However, fruit rivalry aside, it’s still an interesting question and my points above don’t tend to stop people asking, so I thought I’d try a comparison against the most equivalent & open approach, in my opinion, (and also the easiest one for me to test): HTTP. Also, what with security on mobile being such a big issue, I decided I’d test the secure versions: HTTPS and SSL enabled MQTT.

What was I testing?

On the MQTT side, it was very similar to what I tested previously: a simple application using a custom wrapper around the standard Java MQTT client offered by IBM; but this time using an SSL connection against a SSL enabled instance of Micro Broker and performing mutual certificate based authentication between the client and server.

For the HTTPS side, I decided to use Comet style long polling. If you’re not familiar with this, the client-server interaction looks something like this:

Basically, the client makes a HTTPS request to the server, which is kept open until the server has new data to send to the client. When this data arrives, the server sends it to the client and closes the request. The client then initiates a new long polling request in order to obtain subsequent events.

On the server side, I used a simple SSL enabled Comet style Pub/Sub server I wrote recently in Node.js. And for the client side, I wrote a simple Android app that connects to this as needed using a standard HttpsURLConnection. Again performing mutual certificate based authentication between the two.

How did I test them?

Again, it was very similar to what I did previously; using the Android applications mentioned above, along with simple desktop apps to drive them where required. The tests were kicked off manually, however I’ve improved the overall automation of them. The results from here should be reliably comparable with the previous ones.

Caveats / Specifics

• The power profiling was performed on my HTC Desire, now running Android 2.2.2 – Build 2.33.161.6 CL345089.
• ‘% Battery / Hour’ refers to the % of the fully charged capacity of my phone’s battery that is used per hour. My phone has a standard Li-Ion battery that is rated at: 1400mAh & 3.7V.
• The tool I used to capture the power usage data (PowerTutor) uses real data, combined with a power usage model for some aspects. This model has been tailored against the type of device I used for this testing and is reported to be accurate to within 0.8% on average, with a 2.5% margin of error.
• I’ve made a number of decisions on my implementation of the two approaches. I’ve done this to try and make MQTT and HTTPS more equal and make things fairer; not to bias things one way or the other. I’ve tried to do this sensibly and I’ll explain myself as I go along; however some people may have decided to do things slightly differently.
• I’ve tried my best to produce correct, consistent and usable results; however I am human and so there’s a chance I have made mistakes. As such, these figures shouldn’t be treated as gospel, but I would expect them to be representative of what you could expect to see and they should provide an accurate comparison between the two approaches.

Results

Establishing & Maintaining a Connection

I decided to start out by looking at the cost of establishing and then maintaining an open connection for each approach. And I pretty much ran straight into one of the main challenges of this testing: how to make it a fair fight? How do I structure things so I’m not being un-necessarily biased to one technology or the other?

Straight out of the box, MQTT is much more feature rich than HTTPS and the main feature involved here is the Keep Alive. Basically this is a way for the client to detect in a timely manner when the server connection has been lost (and vice-versa) without having to wait for the often long TCP/IP timeout. To do this, the MQTT client and server exchange keep-alive messages every so often. This also serves to maintain the raw TCP/IP connection; as in some circumstances (e.g. on some 3G networks) long-running connections that have no data flowing my be purged.

HTTPS doesn’t have this built in, but it seems like useful functionality for an application to have and so I decided to add this into my HTTPS client by using a read timeout of the same duration as the MQTT keep alive interval. This means that if the server doesn’t respond with any data within x seconds then an Exception is thrown and the connection torn down. The client will then establish a new one. If it can’t, then the client knows the server is unreachable.

Hopefully that sounds like a fair thing to do, but don’t worry, it also flows the other way. As, in my implementation, the HTTPS client also ‘subscribes’ as part of it’s connection (the topic of interest being part of the URL), I decided to consider the act of MQTT connecting to include both the connection and a subscription step.

Anyway, enough waffling, onto the results. First of all, the amount of power taken to establish the initial connection to the server:

% Battery Used
3G		Wifi
HTTPS	MQTT	HTTPS	MQTT
0.02972	0.04563	0.00228	0.00276

As you can see, HTTPS wins this one hands down; and by quite a significant amount (~30% for 3G). However this isn’t really that much of a surprise:

With the HTTPS approach, all we’re doing is opening a connection to the appropriate URL and exchanging certificates. Whereas, for MQTT, we establish the raw connection, perform the certificate exchange and then flow additional information; including the unique Client Id. We then wait to receive a confirmation from the server and then subsequently send additional messages to subscribe to the test topic (which shouldn’t be required for subsequent reconnections; as the server can remember our subscriptions for us).

Note: just to emphasise, this is an artificial limitation that I’ve placed on MQTT. If you don’t need the subscribe step, then the actual cost of connection is very comparable. However, this is something that I decided to do to make the comparison more fair.

Now onto the cost of ‘maintaining’ that connection (in % Battery / Hour):

	% Battery / Hour
	3G		Wifi
Keep Alive (Seconds)	HTTPS	MQTT	HTTPS	MQTT
60	1.11553	0.72465	0.15839	0.01055
120	0.48697	0.32041	0.08774	0.00478
240	0.33277	0.16027	0.02897	0.00230
480	0.08263	0.07991	0.00824	0.00112

As you can see, this is where MQTT gains back ground. In all cases it uses less power, and in most cases a fair bit less. So the longer the connection is established, the ‘cheaper’ MQTT is to use.

If we consider the 3G case (the most relevant to the mobile story) and a keep alive / read timeout interval of 240 seconds (my personal favourite) then by my calculations we make up for the difference in the cost of connecting after ~5 ½ minutes of being connected:

Note: You can click & drag on an area of the graph to zoom to it.
Single click anywhere to zoom back out.

3G – 240s Keep Alive – % Battery Used Creating and Maintaining a Connection

After that, everything else is gravy, and based on this I reckon you’d save ~4.1% battery per day just by using MQTT over HTTPS to maintain an open stable connection.

The reason for this is simple. While it costs MQTT more to create the initial connection, this is essentially a one off and the cost of the following keep alives is comparatively small. Whereas for HTTPS it needs to perform the ‘expensive’ connection stage every time it has to reconnect (up to once each keep alive interval, in my implementation).

Interestingly, this does seem to suggest that if the phone is constantly cycling / dropping connections and you’ve decided not to let the server remember your connections (so that MQTT has to perform the additional subscribe step on every reconnect – cleanstart=true for those in the know) then HTTPS may be a better choice for that particular scenario; and in that case MQTT would be better in slightly more stable situations. The obvious question then is: how often is the connection likely to drop in typical mobile usage?

I’ve tried searching, but not really found any sensible statistics so far. The mobile network should handle routing TCP/IP connections from tower to tower, so that shouldn’t be an issue; however you might go out of range or hit a tower with no spare capacity. Although I guess that’s not likely to happen very often (?). It’s something that I’d be interested in finding out, but it sounds like it would need a large field trial and that’s a little outside the scope of this blog post :)

Receiving

The next thing I looked at was receiving messages on the phone. To do this, I sent messages to the phone in two different ways:

1. Sporadically

To try and emulate a more realistic style of notification sending, I decided to send 6 messages to the phone, with an average of 1 message per 10 minute interval, but with the message being sent ‘randomly’ during that time. This would enable me to test the long-term performance of each approach, but also to add in some unpredictability. For MQTT, the messages were delivered at QoS 1. The results are shown below:

% Battery Used
3G		Wifi
HTTPS	MQTT	HTTPS	MQTT
0.34645	0.27239	0.04817	0.00411

Note: the timing of the messages was calculated beforehand, so both MQTT and HTTPS experienced exactly the same delays, etc.

And here’s some graphs:

3G – Receive 6 x 1 byte messages over 60 minutes – Total mW

Wifi – Receive 6 x 1 byte messages over 60 minutes – Total mW

As you can see, in both cases MQTT wins; by ~30% for the 3G case and an order of magnitude for Wifi.

Also, I actually could’ve made the MQTT implementation even more efficient: Currently I blindly send the keep alive message every x seconds, regardless of when the last message was exchanged between client and server. Whereas I’d actually only need to send the keep alive message x seconds after any message was last sent / received. Interestingly, the HTTPS implementation already exhibits this behaviour and so has a slight advantage.

One final thing of note is that the Wifi graph above nicely shows the difference in the keep alive cost between the two approaches. This is not as obvious in the 3G graph, as the CPU & data transmission costs are overwhelmed by the cost of having the 3G active.

2. As Fast As Possible

To be consistent with the results I produced previously, I also decided to test sending 1024 messages, of 1 byte a piece, to the phone, as quickly as possible. The results are shown below:

</td>

	3G		Wifi
	HTTPS	MQTT	HTTPS	MQTT
% Battery / Hour	18.43%	16.13%	3.45%	4.23%
Messages / Hour	1708	160278	3628	263314
% Battery / Message *	0.01709	0.00010	0.00095	0.00002
Messages Received	240 / 1024	1024 / 1024	524 / 1024	1024 / 1024

 
* – % Battery / Message is a bit of a silly metric. There is a fixed cost in having the Wifi or 3G active and so the actual cost of just sending / receiving a single message would be higher. However these figures do serve to indicate the difference between the two approaches and hopefully give you an indication of the battery usage involved.

And here’s some more graphs:

3G – Receive 1024 x 1 byte messages – Total mW

Wifi – Receive 1024 x 1 byte messages – Total mW

Note: the data in the graphs for each approach stops 5 seconds after all messages have been received.

Again, MQTT wins this one. Despite it being a bit of a silly metric, the % Battery / Message really is the interesting one to look at, and in the 3G case MQTT is over two orders of magnitude better than HTTPS. This highlights both the low power usage of MQTT and also the speed with which the messages were received (averaging 160278 messages per hour for MQTT versus only 1708 for HTTPS).

Another important thing to highlight is the reliability (or perhaps, more accurately, success rate) of delivery. For 3G, every MQTT message got through, whereas HTTPS only managed 240 of the 1024, ~24%. This is because a lot of messages were missed in the interval between when the connection closed with the previous message and was subsequently re-established to receive the next. This is somewhat mitigated on Wifi, as the connection can be re-established more quickly. I could’ve decided to implement a queueing mechanism on the server to help with this, but as MQTT didn’t need it, I thought it was fair not to.

Sending

Finally, I looked at sending messages from the phone. I found it somewhat difficult to get sensible & consistent figures for sending a single message and so I decided to scale up to sending 1024 messages, of 1 byte a piece, as quickly as possible. The results are shown below:

</td>

	3G		Wifi
	HTTPS	MQTT	HTTPS	MQTT
% Battery / Hour	18.79%	17.80%	5.44%	3.66%
Messages / Hour	1926	21685	5229	23184
% Battery / Message *	0.00975	0.00082	0.00104	0.00016

And here’s, yes you guessed it, yet more graphs:

3G – Send 1024 x 1 byte messages – Total mW

Wifi – Send 1024 x 1 byte messages – Total mW

And again, MQTT wins. It uses significantly less power (see % Battery / Message) and is a lot quicker. I don’t think there’s anything else I need to say about this one.

Conclusions

Well, I have to admit I’m somewhat relieved. I’ve been using MQTT on mobile for a while now and I’ve been confident that the power consumption was lower than that of HTTPS, but it’s good to see that the figures support that. I’m definitely an MQTT fan, but I’ve been keen to make this testing fair and unbiased and I would’ve reported the results whichever came out on top (although I know a few people who would be none too happy about that). So definitely relieved :)

As this testing has shown, MQTT uses less power to maintain an open connection, to receive messages and to send them. It also does these last two more quickly and reliably. The only place where it loses out is in establishing the initial connection (with cleanstart=true) and that’s mitigated after ~5 ½ minutes of being connected.

There’s a number of other benefits of MQTT over HTTPS as well, which I’ve not really included in this testing, which include:

• Assured delivery
• Retained messages
• Last will & testament
• Multiple subscriptions ‘multiplexed’ over one connection

These could theoretically be built into a HTTPS approach, but they’re not there by default.

So I think it’s definitely fair to say that MQTT wins overall and is my technology of choice for providing true push capabilities on Android.

What next?

I’ve addressed a few of the additional use cases I identified in my last bout of MQTT power profiling, but there’s still a couple of things I’d like to look in to:

• Testing the effect of message size on sending & receiving.
• Testing power usage when running on fragile networks that keep dropping & being re-established.
• Performing some longer term live field testing.

---

So, congratulations on reading this far. That’s a bit of an epic post, my longest to date, but again hopefully it was interesting and informative. And I welcome any comments.