Wear Mandelbrot – Smooth Colouring

Colouring the Mandelbrot Set is relatively easy, you choose a different colour for each iteration value. This gives the visual effect where the Mandelbrot Set has solid bands of colour, like so:

Mandelbrot Set coloured using iteration value

Now this looks okay, but it’s possible to produce a more aesthetically pleasing image by using an algorithm to produce a smooth gradient, like in this picture:

Same portion of the set with Smooth Colouring

Comparing the two, the later is definitely preferable! (NOTE: back when these fractals were first being rendered there weren’t computers with 16 million colour palettes, typically restricted to just 16 often garish options, so this kind of smoothing wasn’t feasible)

We achieve this by seeing “how far” the x-y pair is into the colour band, in essence calculating a fractional part for the iteration count. For our Android Wear app we’ve  used a algorithm called the “Normalized Iteration Count” which uses the number of iterations to calculate a value between 0 and 1. This value can then be used to interpolate the colours across the band, creating a colour gradient.

The equation to calculate this value is such: nu = iter -  log(log(sqrt(x*x + y*y)/ log(2))/log(2)

The quickest way to grab the fractional part of the value in java is just to cast the double to an int and subtract it from the original value, i.e. double nu_frac =  nu - (int)nu. Now you have a number between 0 and 1 which you can use to interpolate between the colour for the iteration value and the iteration value + 1, like so:

int r = (int)(Color.red(colours[iter]) * (1 - nu_frac) + Color.red(colours[(iter + 1)]) * nu_frac);
int g = (int)(Color.green(colours[iter]) * (1 - nu_frac) + Color.green(colours[(iter + 1)]) * nu_frac);
int b = (int)(Color.blue(colours[iter]) * (1 - nu_frac) + Color.blue(colours[(iter + 1)]) * nu_frac);

That’s all you need to get smooth colouring!

Wear Mandelbrot

The Mandelbrot Set is formed by iterating a seed x-y pair of numbers, which represent a complex number, using the quadratic equation: z=(z*z)+z. The number of iterations, or the amount of “time”, this loop is performed defines whether the two values represent a point that belongs in the Mandelbrot Set – and for graphical purposes which colour to render the given pixel as.

device-2014-09-09-165405

Mandelbrot Set with the ‘Rainbow’ colours

  • Reaching the maximum number of iterations
  • The calculated value z becomes greater than a ‘breakout value’

If the number of iterations tends to infinity then the x-y pair of values is said to belong to the Mandelbrot Set. Programmatically this can be said to happen if the iterative loop reaches a pre-defined maximum number of iterations (for example, 255).

But if you render the Mandelbrot Set just as pixels that are ‘in’ or ‘out’ then you end up with a monochrome image that looks a bit like an island in a sea. Much like a coastline it’s possible to zoom in on any section and find more and more detail as you move in. More visually impressive is if you render the number of iterations that it takes to exceed a ‘breakout value’ (typically 4) as a colour. As you tend away from the Set the number of iterations decreases and you see bands of colour, almost as though the Mandelbrot set was a high mountain and the height drops as you move away from the set itself.

The Android Wear app calculates the Mandelbrot initially with a zoom level which shows the whole set. Tapping the screen zooms in, making the new centre the position where you tapped. In essence you are taking a subset of the original image and blowing it up to the full resolution of the screen again. The fractal nature of the Mandelbrot Set allows you to do this infinitely, although in practise eventually computers lack the accuracy to represent the numbers involved precisely enough, meaning that there’s a practical limit. Double-tapping will undo the last zoom and pull you out from the Mandelbrot.

Corresponding Julia set

Swiping up and down on the watch app will cycle through a handful of pre-defined colour pallets that we chose to show off the Set in its best light. Swiping to the left will use the centre of the current Mandelbrot Set to calculate a Julia Set and display it. A Julia set is a similar fractal pattern that is based on using a seed pair of values from the Mandelbrot Set and essentially using the same equation but offsetting using the value pairs. Swiping right will put you back at the same place you were in the Mandelbrot Set.

In many ways the smart watch form factor is ideal for the Mandelbrot because the screen only has a relatively small number of pixels, making rendering a screen’s-worth of pixels only take a second or two. High-end phones have around 27 times more pixels so the app on the phone (which we’ve developed but not released yet) takes substantially more time to calculate the Mandelbrot Set.

Footnote: I find it stunning to see how fast these Android Wear watches can render the Mandelbrot Set. When I was at school we were drawing these fractals using the BBC Model B computer and you could sit and watch each line being rendered. Now a whole screen pops up after barely a couple of seconds. The raw processing power that’s paired with a full colour display and small enough to fit on your wrist is impressive. (Jonathan Howell)

Problem with starting Wear apps on an idle watch

If you have an Android app running on a phone that needs to start a Wear app on a smart watch then you’ll have a problem if the Wear device is in an idle state.

For example if the phone app uses this bit of code in order to start the Wear app (by launching a new activity):

Wearable.MessageApi.sendMessage(mGoogleApiClient, node.getId(), START_ACTIVITY_PATH, data);

Then if the watch screen is in idle mode and the app isn’t already running then the app won’t launch. On the LG G Watch there are no error messages shown on the screen, it just jumps to displaying the “Home screen”. If the screen was already unlocked then the Wear app would start normally.

The workaround for this involved starting another activity on the watch briefly before the main activity to turn the screen on. This couldn’t do be done from the main app for an unknown reason.

So in terms  of coding, I set up a new Window Activity:

import android.app.Activity;
import android.os.Bundle;
import android.view.WindowManager;

public class WindowActivity extends Activity {

  @Override
  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    getWindow().addFlags(WindowManager.LayoutParams.FLAG_TURN_SCREEN_ON);
    super.finish();
  }
}

And then made sure I called it in the WearableListenerService’s onMessageReceived method before trying to open the Wear app:

 if (path.equals(START_ACTIVITY_PATH)) {
   if (!started) {
     // reactivate the screen
     Intent startWindow = new Intent(this, WindowActivity.class);
     startWindow.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
     startActivity(startWindow);

     // now open the Wear app
     Intent startWatch = new Intent(this, Watch.class);
     startWatch.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
     startActivity(startWatch);
   }
   else {
     // screen already started so just open Wear app
     Intent startWatch = new Intent(this, Watch.class);
     startWatch.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
     startActivity(startWatch);
   }
 }

Android Wear app not installing

When using Android Studio to package up an Android Wear app into a signed APK a missing manifest permission in the phone app can break the Wear app installation.

There are no errors in the gradle build or when creating the APK. You can try to install the APK and it will succeed on the phone, but the corresponding Wear app won’t install on the watch. There will be no visible errors on the watch or phone to alert you to this.

If you are debugging your application then there are errors reported in the logging but they can be very hard to spot because of the constant flow of debug coming from the watch.

The solution turned out to be that you must replicate any android permissions that the Wear app needs in the phone app’s manifest file.

 

Android services – lost messages problem

When Android runs a service it’s necessary to bind it to a application, otherwise the operating system will clean the service up. This is useful because it reduces the waste of resources. Ireckon uses a wearable listener service which means that once it has interpreted the message from the watch and sent it back, the android garbage system destroys it quickly – but not quickly enough.

The problem for Ireckon was that if a message was sent from the watch to the phone whilst the OS was in the process of destroying the service then the message would be lost.

The solution to this was to check for a response from the phone. If there wasn’t a response it was necessary to back off for a few seconds and then send the message again, repeating this if necessary.