Wednesday, 28 January 2015

Smooth Videos - AKA Correcting NASA

What makes a video look smooth? Your eye is extremely sensitive to problems with videos, and for any video to look smooth it has to have:

  • A high frame rate
  • A steady camera
  • Roughly even brightness each frame

Normally these are easy to get. Any modern camera will give a decent frame rate, and the exposure time for each shot will be accurate, giving an even brightness of images each frame. Camera steadiness is more difficult, but a basic tripod will solve that.

This is a lot harder in space! For a NASA space probe floating through deep space, keeping a steady orientation is a challenge. Spacecraft can do this well quite well, using thrusters and reaction wheels. They still make some small mistakes though. Getting an even exposure time for each frame of a video is also harder in deep space, especially as it might take minutes or hours for radio commands to reach the space probe so you have to trust its autoexposure. Luckily, given ok starting material, correcting camera shake and frame brightness problems by image processing is quite easy.

NASA's Dawn space probe is currently approaching Ceres, getting sharper pictures of this dwarf planet than ever before. A series of these pictures even shows this tiny world rotating. Unfortunately, they didn't correct the shake or brightness problems in the video released to the press:

A quick fix in ImageJ to remove the shake and even out the frame brightness makes a (dwarf) world of difference:

As the probe gets closer and closer to Ceres its shots are getting more and more spectacular, but the videos still need shake and brightness correction.

Interested in improving some NASA videos? I did the corrections using the free scientific image editing software ImageJ, and these are two handy macro scripts for video corrections in ImageJ:

Image stabilisation
//Stabilise based on signal intensity centroid (centre of gravity)
//Stabilises using translation only, using frame 1 as the reference location
//This method is suitable for stabilising videos of bright objects on a dark background
for (z=0; z<nSlices(); z++) {
 //For each slice
 //Do a weighted sum of signal for centroid determination
 for (x=0; x<getWidth(); x++) {
  for (y=0; y<getHeight(); y++) {
   v=getPixel(x, y);
 //Calculate the centroid location
 if (z==0) {
  //If the first slice, record as the reference location
  print(rcx, rcy);
 } else {
  //Otherwise calculate the image shift and correct
  print(dx, dy);
  makeRectangle(0, 0, getWidth(), getHeight());
  makeRectangle(-dx, -dy, getWidth(), getHeight());
Brightness normalisation
//Normalise image brightness to reduce video flicker
//Scales intensity based on the mean and standard deviation, using frame 1 as the reference frame
//This method is suitable for reducing flicker in most videos
for (z=0; z<nSlices(); z++) {
 //For each slice
 //Find the signal mean and standard deviation
 run("Select All");
 getRawStatistics(area, mean, min, max, stdev);
 if (z==0) {
  //If the first slice, record as the reference signal mean and stdev
  print(rmean, rstdev);
 } else {
  //Otherwise calculate the brightness and scaling correction
  run("Macro...", "code=v="+rmean+"+"+rstdev+"*(v-"+mean+")/"+stdev);
  print(mean, stdev);

Software used:
ImageJ: Image corrections
GIMP: Animated gif file size optimisation

Thursday, 22 January 2015

Tengwar - Transliterating Font

This blog post is about a Tengwar font I designed. It automatically converts text as you type into accurate Elvish script. You can download it for free here.  Just make sure you enable ligatures, contextual alternates and kerning for best results!

While writing his Middle Earth books, JRR Tolkein invented an entire alphabet for the elves called Tengwar. His attention to detail was incredible, Tengwar is a fully functioning writing system. This is the famous Elvish writing seen all through Lord of The Rings and the Hobbit.

Tengwar is an alphabet, not a language, and can be used to write many languages. This is like, for example, Latin and Greek alphabets; the word English word “ring” is normally written in the Latin alphabet but could also be written in the Greek alphabet as “ρινγ”. The two sound the same, it is just a different way of writing the sounds of the word “ring”. The process of transferring a word between two different alphabets is called transliteration.

In Middle Earth, Tengwar is one of the major ways of writing. Many languages were written in Tengwar: two Elvish languages called Sindarin and Quenya, the Black Speech of Mordor (on the One Ring), and the language of men (English). Tolkein gave detailed notes on how to write English in the Tengwar alphabet. In Tengwar “ring” is written:

Writing in Tengwar follows simple rules but quickly gets complicated, so I designed a font that does it automatically! You can download it for free here.  As far as I know this font is unique, all other Tengwar fonts are just collections of symbols you have to manually mix and match.

To use this font you just need to download and install it. Once it is installed, just select it as the font and start typing as normal. The font will automatically transliterate the text you type into accurate Tengwar, based on Tolkein’s rules about writing English in Tengwar.

To make sure the font is working accurately you need to make sure three settings are enabled: kerning, contextual alternates and ligatures. For example, in Microsoft Word you can do this through the advanced font settings:

So how does it work? Basic Tengwar is similar to the Latin alphabet, with two classes of symbols representing the sounds of different consonants and different vowels. At the simplest level, to write the word “ring” the font just selects the four symbols for “r”, “i”, “n” and “g”:

Unlike the Latin alphabet, there are special rules for how vowels are written. Instead of always being a separate letter, if a vowel comes immediately before a consonant it is written as an accent on that consonant. In “ring” the “i” comes immediately before the “n” so the font writes it as an accent on the “n”:

There are some special rules to use for some consonants, depending on where they are in a word. “r” is one of these letters. If it is followed by a vowel then it should have a different symbol, which the font automatically selects:

Finally, some common combinations of consonants that have a single sound (like “th” as in “the”, “ch” as in “church” and “gh” as in “ghost”) have their own single symbol. “ng” is one of these pairs and, again, the font automatically makes this substitution:

And that is how the font automatically writes “ring” in Tengwar. These are not the only rules though, there are also other ones built into the font that involve double vowels, double consonants, the letter “n” preceding another consonant, whether a “y” is used as a vowel or a consonant, whether an “e” is voiced in a word or is silent at the end of a word, etc.

The key feature of my font is that it takes all of these rules into account automatically and lets you simply type away as normal and get an accurate, readable result in Tengwar. You can also just select an existing chunk of text and apply the font to it to transliterate it to Tengwar, but make sure the text is all lower case for best effect. It does make a few very small mistakes, but Tolkein would understand it!

Tengwar is a beautiful and concise alphabet. The way vowels, double letters and letter pairs combine make many words very short and elegant:

The overall flow of a paragraph is also excellent, with the letters falling into self-symmetric curves and alignments.

(This is the first paragraph of Lord of The Rings, converted to Tengwar by just changing the font to my Tengwar Transliteral font.)

If you are interested in playing with Tengwar text for any kind of design please consider downloading the italic and script versions of the font here. These cost a few pounds/dollars/euros.

If you are interested in reading Tengwar, or manually translating it, then the excellent “Tengwar Textbook” Chris McKay is available online for free: Tengwar Textbook.

There are also excellent simple guides on writing in Tengwar (like this one), but why do that when you could just download my font and type your name?

Software used:
Inkscape: Glyph design
Fontforge: Font design

Wednesday, 7 January 2015

Trypanosome Lego

Trypanosomes and Leishmania are the two tropical parasites that I do most of my research on. These cells seem to have a lot of modularity in controlling their shape, and have quite a lot of flexibility in reshuffling where particular structures (made up of many organelles) sit within the cell.

The base of the flagellum, the whip-like tail which the cell uses to swim, is also the site where the cell takes up material from its environment (essentially its mouth) and is linked with the Golgi apparatus (an important organelle in protein processing) and the mitochondrion (the powerhouse of the cell) and links to the mitochondrial DNA. It turns out reducing the level of just one protein in the cell can cause this entire complex structure to shift its position.

Cells are not quite as flexible as Lego, but it is still impressive that a single protein can have such a large effect on the organisation of a cell.

Monday, 29 December 2014

Forgotten Futures - New York

What if cities looked like this? The 1920s view of cities of the future was glorious; huge buildings towering into the sky, multi-layer roads, rail and pavements, airships and aircraft, and the bold geometry of art deco.

Sadly this world never came into existence. But what if it had? What would 1950s New York have looked like? I re-imagined this forgotten future based on the view from the Empire State building towards the Grand Central station and the Chrysler building in a world where the 1920s vision of the future came to be.

Software used:
Blender: 3D modeling, texturing, rendering, compositing.
Paint.NET: Final image tweaks.
Inkscape: Texture detailing.

Building a forgotten future; 7 days of 3D modelling in 20 seconds:

Friday, 12 December 2014


Ocean tides are one of the most amazing but overlooked natural wonders of our planet. As the Earth rotates relative to the sun and the moon, their gravity drags the Earth's water about, raising and lowering it in synchrony with the heavens. The importance of tides reaches further than just surfing, sunbathing and shipping: Tides are the reason the moon is drifting away from the Earth at 3.8 cm per year. Tides allow the formation of beaches with rock pools at low tide, that some biologists argue helped the evolution of early life. Tides (of the atmosphere) are the reason a satellite in a low orbit is more likely to burn up on the side of the Earth nearest or opposite to the moon. Tides even influence the time  earthquakes happen.

The explanation of why tides happen is classic high school geography/physics. The gravitational pull of an object is felt more strongly by something close to it. In the case of the Earth, this means that the oceans on the closest side of the Earth to the moon feels a stronger gravitational pull than the Earth as a whole, and the oceans on the far side feel a weaker pull. This means that the oceans on the near side of the Earth are pulled into a bulge (a region of high tide) and the oceans in the far side are also flung outward into a bulge (another region of high tide). This causes high and low tides twice per day. Throw in the similar contribution of the sun's gravitational pull, and it also explains spring tides around the time of the new and full moon.

Of course this is all a bit of a lie to simplify things. Many places have one  high and low tide per day, and a few places even have four. Some places have barely any tide, while others have very large tides where the water level can change by many metres. Why? Because the land gets in the way! It is impossible to have a bulge of water where Africa is, even if the moon was directly over the Sahara. So what does the pattern of tides actually look like?

Something like this:

[Watch in HD on YouTube]

This animation shows sea levels over the course of one day, where orange represents high water level, and blue represents low water level. Instead of the water levels changing because of two big bulges of water, there are instead complex patterns of water level change.

So, how does the simple rotation of the Earth relative to the sun and moon generate such complexity? It is easiest to think about the oceans as containers of water which gently slosh about as the water gets pulled by the gravity of the sun and moon. It is a bit like the sloshing of water you get carrying a glass of water, or when you climb out of a bathtub. The precise pattern of the sloshing depends on many things; the strength and direction of the gravitational force driving the sloshing, the depth of the water, and how the oscillating sloshing movement resonates when it gets trapped against the coastline.

The different water movements that make up the final tidal moment can be broken down by the force that generated them (the sun, the moon) and their frequency (once a day, twice a day). The two biggest contributing movements are a twice daily movement arising from the moon, and a once daily movement due to the combined action of the sun and moon.

These individual movements are mapped through their amplitude (how much the water changes height) and their phase (the relative time of high tide). These maps are surprisingly beautiful! Here are a couple of examples:

These are the patterns of movement of the "M2" part of tides, which is a twice daily water movement arising from the primary action of the moon's gravity. Brightness represents the amplitude, from black (zero amplitude) to white (5 metres amplitude). The coloured lines are a bit more complex. They represent the places where the highest water level occurs due to the M2 tidal component at different times, from red (at 0 hours) through the colours of the spectrum at 1 hour steps.

These are the patterns of movement of the "K1" part of tides, which is a once daily water movement arising from the combined action of the sun's and moon's gravity. Again, brightness represents amplitude, but the amplitudes are smaller and white represents only 2.5 metres. The coloured lines represent the time when highest water level due to the K1 tidal component occur, but this time separated by 2 hour steps.

These are just the two largest components of tides, there are many complex contributing factors: M2: principal semi-diurnal lunar, S2: principal semi-diurnal solar, N2: larger semi-diurnal elliptical lunar, K2: declinational semi-diurnal solar/lunar, 2N2: second-order semi-diurnal elliptical lunar, K1: principal diurnal solar/lunar, O1: principal lunar, P1: principal diurnal solar, Q1: larger diurnal elliptical lunar. Each of these components has similarly beautiful patterns of movement.

Software used:
ImageJ: HAMTIDE tital data plotting.

Tuesday, 8 July 2014

3D Wood Grain

Using a block of wood and a plane Keith Skretch made something amazing. He snapped a picture of the wood, then planed a thin layer off, snapped another picture, planed another layer off, and repeated this hundreds of times. In the resulting timelapse/stop motion video you fly through the wood structure, and can see knots and grain in the wood ripple by.

Waves of Grain from Keith Skretch on Vimeo
To my computational image analysis eyes, the truly amazing thing about this video is contains the detailed three dimensional map of the internal structure of blocks of wood; that these blocks of wood have been digitally immortalised!
Let's look at just one of the blocks of wood:
 The series of images 29-36 seconds through Waves of Grain

So what can you do with this data? Well you can reproject to give you a virtual view of what the left and the front sides of the blocks of wood would have looked like:

That's quite cool, but doesn't capture the power of having the full 3D information. The more powerful thing you can do is do a virtual cuts through anywhere you want in the block of wood. You can cut it somewhere in the middle to take a look at the internal structure... The yellow lines mark where the virtual slices were made:
That's also quite cool, but still doesn't capture the power of having all that 3D data. You can also reslice the image at any orientation that you want; it doesn't have to be neat orthogonal lines:

Again, quite cool. But you can still do more. Because this is now a purely digital representation of this block of wood you can display it in ways that would be physically impossible to make. Instead of just looking at the outside of the block...

... you can now look inside.

This 3D reconstruction lets you see how the growth rings appear in three dimensions, showing exactly where the grain runs. It lets you see how the knot, which is where a branch grew from the tree, cuts through the growth rings in a distinctive way. It lets you see pretty much everything about the internal structure of the wood!

This kind of approach is used all over biology, and is normally called something like serial sectioning. You can use it for everything from reconstructing a whole person by histology and a light microscope to a single cell by electron microscopy.

Software used:
ImageJ: 3D reconstructions