Calculated Images: imagej

Showing posts with label imagej. Show all posts

Wednesday, 28 January 2015

Smooth Videos - AKA Correcting NASA

What makes a video look smooth? Your eye is extremely sensitive to problems with videos, and for any video to look smooth it has to have:

A high frame rate
A steady camera
Roughly even brightness each frame

Normally these are easy to get. Any modern camera will give a decent frame rate, and the exposure time for each shot will be accurate, giving an even brightness of images each frame. Camera steadiness is more difficult, but a basic tripod will solve that.

This is a lot harder in space! For a NASA space probe floating through deep space, keeping a steady orientation is a challenge. Spacecraft can do this well quite well, using thrusters and reaction wheels. They still make some small mistakes though. Getting an even exposure time for each frame of a video is also harder in deep space, especially as it might take minutes or hours for radio commands to reach the space probe so you have to trust its autoexposure. Luckily, given ok starting material, correcting camera shake and frame brightness problems by image processing is quite easy.

NASA's Dawn space probe is currently approaching Ceres, getting sharper pictures of this dwarf planet than ever before. A series of these pictures even shows this tiny world rotating. Unfortunately, they didn't correct the shake or brightness problems in the video released to the press:

NASA/JPL-Caltech/UCLA/MPS/DLR/IDA/PSI

A quick fix in ImageJ to remove the shake and even out the frame brightness makes a (dwarf) world of difference:

Adapted from NASA/JPL-Caltech/UCLA/MPS/DLR/IDA/PSI

As the probe gets closer and closer to Ceres its shots are getting more and more spectacular, but the videos still need shake and brightness correction.

Adapted from NASA/JPL-Caltech/UCLA/MPS/DLR/IDA/PSI

Interested in improving some NASA videos? I did the corrections using the free scientific image editing software ImageJ, and these are two handy macro scripts for video corrections in ImageJ:

Image stabilisation

//Stabilise based on signal intensity centroid (centre of gravity)
//Stabilises using translation only, using frame 1 as the reference location
//This method is suitable for stabilising videos of bright objects on a dark background
for (z=0; z<nSlices(); z++) {
 //For each slice
 setSlice(z+1);
 //Do a weighted sum of signal for centroid determination
 sxv=0;
 syv=0;
 s=0;
 for (x=0; x<getWidth(); x++) {
  for (y=0; y<getHeight(); y++) {
   v=getPixel(x, y);
   sxv+=v*x;
   syv+=v*y;
   s+=v;
  }
 }
 //Calculate the centroid location
 cx=sxv/s;
 cy=syv/s;
 if (z==0) {
  //If the first slice, record as the reference location
  rcx=cx;
  rcy=cy;
  print(rcx, rcy);
 } else {
  //Otherwise calculate the image shift and correct
  dx=cx-rcx;
  dy=cy-rcy;
  print(dx, dy);
  makeRectangle(0, 0, getWidth(), getHeight());
  run("Copy");
  makeRectangle(-dx, -dy, getWidth(), getHeight());
  run("Paste");
 }
}

Brightness normalisation

//Normalise image brightness to reduce video flicker
//Scales intensity based on the mean and standard deviation, using frame 1 as the reference frame
//This method is suitable for reducing flicker in most videos
for (z=0; z<nSlices(); z++) {
 //For each slice
 setSlice(z+1);
 //Find the signal mean and standard deviation
 run("Select All");
 getRawStatistics(area, mean, min, max, stdev);
 if (z==0) {
  //If the first slice, record as the reference signal mean and stdev
  rmean=mean;
  rstdev=stdev;
  print(rmean, rstdev);
 } else {
  //Otherwise calculate the brightness and scaling correction
  run("Macro...", "code=v="+rmean+"+"+rstdev+"*(v-"+mean+")/"+stdev);
  print(mean, stdev);
 }
}

Software used:

ImageJ: Image corrections

GIMP: Animated gif file size optimisation

Friday, 12 December 2014

Tides

Ocean tides are one of the most amazing but overlooked natural wonders of our planet. As the Earth rotates relative to the sun and the moon, their gravity drags the Earth's water about, raising and lowering it in synchrony with the heavens. The importance of tides reaches further than just surfing, sunbathing and shipping: Tides are the reason the moon is drifting away from the Earth at 3.8 cm per year. Tides allow the formation of beaches with rock pools at low tide, that some biologists argue helped the evolution of early life. Tides (of the atmosphere) are the reason a satellite in a low orbit is more likely to burn up on the side of the Earth nearest or opposite to the moon. Tides even influence the time earthquakes happen.

The explanation of why tides happen is classic high school geography/physics. The gravitational pull of an object is felt more strongly by something close to it. In the case of the Earth, this means that the oceans on the closest side of the Earth to the moon feels a stronger gravitational pull than the Earth as a whole, and the oceans on the far side feel a weaker pull. This means that the oceans on the near side of the Earth are pulled into a bulge (a region of high tide) and the oceans in the far side are also flung outward into a bulge (another region of high tide). This causes high and low tides twice per day. Throw in the similar contribution of the sun's gravitational pull, and it also explains spring tides around the time of the new and full moon.

Of course this is all a bit of a lie to simplify things. Many places have one high and low tide per day, and a few places even have four. Some places have barely any tide, while others have very large tides where the water level can change by many metres. Why? Because the land gets in the way! It is impossible to have a bulge of water where Africa is, even if the moon was directly over the Sahara. So what does the pattern of tides actually look like?

Something like this:

[Watch in HD on YouTube]

This animation shows sea levels over the course of one day, where orange represents high water level, and blue represents low water level. Instead of the water levels changing because of two big bulges of water, there are instead complex patterns of water level change.

So, how does the simple rotation of the Earth relative to the sun and moon generate such complexity? It is easiest to think about the oceans as containers of water which gently slosh about as the water gets pulled by the gravity of the sun and moon. It is a bit like the sloshing of water you get carrying a glass of water, or when you climb out of a bathtub. The precise pattern of the sloshing depends on many things; the strength and direction of the gravitational force driving the sloshing, the depth of the water, and how the oscillating sloshing movement resonates when it gets trapped against the coastline.

The different water movements that make up the final tidal moment can be broken down by the force that generated them (the sun, the moon) and their frequency (once a day, twice a day). The two biggest contributing movements are a twice daily movement arising from the moon, and a once daily movement due to the combined action of the sun and moon.

These individual movements are mapped through their amplitude (how much the water changes height) and their phase (the relative time of high tide). These maps are surprisingly beautiful! Here are a couple of examples:

These are the patterns of movement of the "M2" part of tides, which is a twice daily water movement arising from the primary action of the moon's gravity. Brightness represents the amplitude, from black (zero amplitude) to white (5 metres amplitude). The coloured lines are a bit more complex. They represent the places where the highest water level occurs due to the M2 tidal component at different times, from red (at 0 hours) through the colours of the spectrum at 1 hour steps.

These are the patterns of movement of the "K1" part of tides, which is a once daily water movement arising from the combined action of the sun's and moon's gravity. Again, brightness represents amplitude, but the amplitudes are smaller and white represents only 2.5 metres. The coloured lines represent the time when highest water level due to the K1 tidal component occur, but this time separated by 2 hour steps.

These are just the two largest components of tides, there are many complex contributing factors: M2: principal semi-diurnal lunar, S2: principal semi-diurnal solar, N2: larger semi-diurnal elliptical lunar, K2: declinational semi-diurnal solar/lunar, 2N2: second-order semi-diurnal elliptical lunar, K1: principal diurnal solar/lunar, O1: principal lunar, P1: principal diurnal solar, Q1: larger diurnal elliptical lunar. Each of these components has similarly beautiful patterns of movement.

Software used:
ImageJ: HAMTIDE tital data plotting.

Tuesday, 8 July 2014

3D Wood Grain

Using a block of wood and a plane Keith Skretch made something amazing. He snapped a picture of the wood, then planed a thin layer off, snapped another picture, planed another layer off, and repeated this hundreds of times. In the resulting timelapse/stop motion video you fly through the wood structure, and can see knots and grain in the wood ripple by.

Waves of Grain from Keith Skretch on Vimeo To my computational image analysis eyes, the truly amazing thing about this video is contains the detailed three dimensional map of the internal structure of blocks of wood; that these blocks of wood have been digitally immortalised! Let's look at just one of the blocks of wood:

The series of images 29-36 seconds through Waves of Grain

So what can you do with this data? Well you can reproject to give you a virtual view of what the left and the front sides of the blocks of wood would have looked like:

That's quite cool, but doesn't capture the power of having the full 3D information. The more powerful thing you can do is do a virtual cuts through anywhere you want in the block of wood. You can cut it somewhere in the middle to take a look at the internal structure... The yellow lines mark where the virtual slices were made:

That's also quite cool, but still doesn't capture the power of having all that 3D data. You can also reslice the image at any orientation that you want; it doesn't have to be neat orthogonal lines:

Again, quite cool. But you can still do more. Because this is now a purely digital representation of this block of wood you can display it in ways that would be physically impossible to make. Instead of just looking at the outside of the block...

... you can now look inside.

This 3D reconstruction lets you see how the growth rings appear in three dimensions, showing exactly where the grain runs. It lets you see how the knot, which is where a branch grew from the tree, cuts through the growth rings in a distinctive way. It lets you see pretty much everything about the internal structure of the wood!

This kind of approach is used all over biology, and is normally called something like serial sectioning. You can use it for everything from reconstructing a whole person by histology and a light microscope to a single cell by electron microscopy.

Software used:
ImageJ: 3D reconstructions

Monday, 28 April 2014

A Year in The Life of a Computer

What does a year in the life of a computer look like?

Well, something like the map below! This is a map every bit of of mouse movement, every mouse click and every keyboard press I have done on my home and work computer over every day of a whole year.

2013-2014 [click for a bigger view]

To make it I wrote a little python script using pyHook to grab inputs in Windows, which I compiled to an .exe using py2exe. I set this up so that it starts recording the mouse movement, clicks, and keyboard presses after I log into my home or work computer. After 2 years it had collected nearly 10 Gb of data! This was far too much to look through by hand, so I wrote a second set of scripts to plot it to an image.

So what does it all mean? Well the map breaks down a bit like a normal calendar, with days of the week running from the top to the bottom of the map, and successive weeks running from left to right. The years and months are marked at the top of the map.

Within each day my computer activity is broken down by time. Time runs from the top to the bottom of each day, from midnight to midnight. Coloured speckles on the dark background indicate computer activity. It is easy to see that I use computers a lot, with a chunk of time from around midnight to 7 am when I am normally asleep, then smatterings of activity from around 8 am to midnight when I am at work or awake at home.

Different types of computer activity are shown in different colours.

The structure within each of the colours also contains information; distance in the horizontal direction corresponds to horizontal mouse position across my two screens (for mouse movement) which mouse button was clicked (for mouse clicks) and which key was pressed (for keyboard presses).

2012-2013 [click for a bigger view]

In these maps of usage some interesting structures jump out; you can spot the type of work I was doing with my computer based on the type of mouse and keyboard activity:

This is usage on a day where I was writing my PhD thesis. The keyboard (cyan) has loads of activity, while the mouse (magenta) did relatively little.

This is a day where I was mainly using Blender for 3D graphics. The mouse (magenta) has huge levels of activity, centred on just the left hand screen). The keyboard is hardly active except for the control and shift keys, which light up as a single column of bright cyan pixels.

It is quite scary how much information can be gleaned from these maps of computer activity. Without knowing which programs were open or which keyboard keys were being pressed it is still easy to work out where I have been, when I have been working, and the kind of things I was doing on my computer. Similar data can be collected remotely; particularly if an internet company tracks when and where you use the internet.

Stop for a second and think about the companies you interact with, and the data mining they can do. Think how much they can learn about you and your habits; Google and the websites you visit, your phone company and when and who you text and call, the supermarket you shop in and what you buy. These companies can work out what you are interested in, what you like and dislike, when you are awake and when you are asleep. This is big data, and it is valuable and it is powerful. Big data is how Target knew a man's teenage daughter was pregnant before he did!

Software used:

pyHook and py2exe: Data logging.

ImageJ: Data plotting.

Inkscape: Plot annotation.

Thursday, 10 April 2014

Cells and Worms - 2. The Shirt

Last post I talked about how seeing how many worms overlap if you drop them on a patch of ground, how (somehow) this was vaguely related to my scientific research, and that the simulation of this process even generates quite nice pictures. If you thought that was geeky, then this takes geekyness to a whole new level!

Part of my research has been into the shapes of trypanosome parasites. Trypanosomes that cause disease in people are fairly widely known (you might have heard of sleeping sickness, Chagas disease, or leishmaniasis) but trypanosomes don't just infect people. Trypanosome species have also been found infecting animals from sharks to penguins, crocodiles to elephants. There is even one species named after Steve Irwin (the crocodile hunter) that infects koalas!

A scanning electron microscope image of Trypanosoma brucei, the trypanosome which causes sleeping sickness.

In short, I did some research to test whether there were particular characteristic shapes of trypanosomes (length, width, etc.) that look like they might help the parasite survive in the bloodstream of different host animals. I made a big database of properties of trypanosome shape and, using the scripts I made to draw nicely tesselated trypanosome shapes I talked about in the last post, I put together a compelling summary of just how varied trypanosome shapes from different host species are are:

The science behind this picture suggests some interesting adaptation to help the parasites swim within their host bloodstream, but that's enough about the science. To me this pattern was just begging to be on a shirt, an abstract design with a biological twist!

Spoonflower is a fantastic online service where you can order customised fabric, wallpaper and other prints. So that is exactly what I did, and after some sewing (that I didn't do myself) I am now the proud owner of the world's only 100% scientifically accurate trypanosome shirt, featuring 27 different trypanosome species.

Scientists always say that research can take you down unexpected paths. This path from wriggly worms, through an image generating script, through research into trypanosome shape, to the world's only trypanosome shirt was quite an unexpected one!

Software used:
ImageJ: Automated trypanosome drawing.
Inkscape: Conversion to vector graphics for printing.

Wednesday, 9 April 2014

Cells and Worms - 1. The Theory

If you scatter 100 worms on a patch of soil 1 meter by 1 meter how many worms will fall on top of another worm? This might seem like a really pointless question, but it is surprisingly relevant to biological research using microscopes. It's also a surprisingly hard question to answer because worms are very wriggly! However, even this dry, theoretical, research problem provides the tools for making fun illustrations...

My work involves a lot of automated image analysis; taking a picture from a microscope and automatically analysing it to extract scientific data. To make sure an automated analysis is reliable you have to think about all the likely problems that might turn up, and with cells and microscopes a common problem is when two cells are lying on top of each other. The problems this causes are easy to imagine; if there are two cells with one nucleus lying on top of each other then it might look like one cell with two nuclei.

For some types of cells it is quite easy to work out how likely two are to touch or lie partly on top of each other when they are scattered randomly over a microscope slide. An example of an easy case is where all cells are circular and the same size; the approximate calculation is quite simple. Unfortunately the cells I work on are more worm-like in shape, about 17 microns long and 2 wide... if you scatter these cells over a slide how many will end up touching?

To work out the answer simulation is vital; the maths is just too complicated to do it analytically. A simulation of worm-like shapes proved to be quite simple:

Pick a random starting point, direction and curvature.
Start drawing a curved line from that point.
Occasionally re-randomise the curvature.
Stop once you have reached the length of the cell.
Draw the profile of the cell shape along that curve.

Following these simple rules and tweaking the parameters (e.g. the minimum and maximum curvature, frequency of randomising curvature, etc.) gives a simple algorithm for drawing a worm-like shape. With a bit of tweaking it could draw cells that look like trypanosomes. Using this drawing tool it was possible to measure the chance of a cell touching or lying on top of another cell already on the microscope slide. Just repeat the drawing process thousands of times and detect whether the newly drawn cell intersects with any previously drawn ones. Problem solved.

This process gave me the answer I needed, but it also provided a tool for drawing trypanosome-like shapes. Better than that, it was easy to adapt it to make sure no two cells overlapped and they fitted neatly together over the image... And just like that a dry, theoretical, research problem turned into a beautiful image:

This was also easy to adapt to other worm-like shapes, like earthworms:

Software used:

ImageJ

Friday, 28 March 2014

Monroe, Einstein and Visual Acuity

The recent Mirror newspaper advert in the UK has brought a classic optical illusion back into the public eye; a hybrid image of Marilyn Monroe and Albert Einstein which, up close, looks like Einstein but from further away, or with squinted eyes, looks like Marilyn.

Try it out! From close up Einstein's trademark hair and moustache jump out, but squint or stand back from the screen and you can see a classic shot of Marylin's curls, eyelashes and smile. A version of this illusion was first made by Aude Oliva for a feature in New Scientist, and it is a really striking example of a hybrid image illusion.

So what is your brain doing? And how can you make an image like this? Making an image is actually quite simple. First of all take pictures of these two pop icons with similar(ish) lighting and align them so their main features (eyes, mouth, overall face) are at the same size and position in the images:

The trick is then to to use a Fourier bandpass filter to filter out low frequency structure in the Einstein image, and filter out high frequency structure in the Marylin image. You can find Fourier bandpass filters (sometimes called FFT filters) for many image editing programs.

So what is a Fourier bandpass filter? Without diving into too much maths it is a way of separating out information based on its wavelength. Filtering out low frequency structure in an image leaves only the short wavelength features, i.e. fine lines and sharp edges, while filtering out high frequency structure leaves only the long wavelengths, i.e. the general brightness of different parts of the image.

Einstein with a <5px wavelength Fourier bandpass filter

Marylin with a >10px wavelength Fourier bandpass filter

Fourier bandpass filters are easier to intuitively understand with sound rather than an image. It might help to imaging using a Fourier bandpass filter on some music; a low frequency (long wavelength) bandpass filter would leave only the bassline and bass drum, while a high frequency (short wavelength) bandpass filter would leave vocal lines and high pitched instruments and drums.

If you are more mathematically minded it might be useful to imagine this through some graphs. These are plots how bright the image is as you go along a line across the middle of the two images. It is easy to see that the filtering of the Einstein image only leaves the short wavelength data, and the filtering of the Marylin image only leaves the long wavelength data:

You can also imagine a long wavelength Fourier bandpass filter as a blurring, and a short wavelength Fourier bandpass filter as the inverse of blurring; grabbing the details that are lost when the image is blurred.

Having made the two Fourier bandpass images it is simply a matter of averaging the two together to get the final product:

So how does it work? The trick is simply based on a limitation of how well you can see. From a greater distance your eyes are less able to see the fine detail of the image, so your brain interprets only the big structures. In this case this leaves your brain to latch onto the Marylin part of the image, helped by the fact that many of her photos are extremely recognisable.

From closer in your eyes can now resolve the fine detail in the image, and your brain does its best effort at interpreting a slightly messy image. Because both the photo of Einstein and Marylin are kind of similar (light skin on a dark background, with big hair) your brain can do a decent job of merging the fine detail of Einstein's face onto the general light and shadow of Marylin's face.

By switching the filtering of the two images you can get the reverse effect...

... although I do find Marylin's teeth in this photo quite terrifying!

Software used:

ImageJ

Monday, 24 March 2014

Finding Circles

Today's random image processing tip: finding circles in images. Circular structures pop up all over the place, from craters on Mars to cross-sections through microtubules inside cells, and automatically detecting them in an image is often really useful! Take this starting image:

An electron microscope image, with the circular crossection of a microtubule.

Using the right filter, circular structures like the microtubule pop out as bright spots. Bright spots are then easy to pull out for later automatic image analysis steps.

The electron microscope image after filtering.

The trick is use to use a kernel filter designed to find circles. Kernel filters are most commonly used for quickly filtering or sharpening a picture: this simple kernel finds edges in an image:

0 1 0

1 -4 1

0 1 0

Applying a kernel to an image is done by looking at each pixel in turn, and setting its new value according to the values in the kernel. In this example the new pixel value is equal to -4× the current pixel, plus 1× the pixel above, left, below and right of the current pixel.

To find circles the kernel should look like a circle. To find the microtubule in the electron microscope image this kernel, a circle the same size as the microtubule, was used:

This image can be represented as a matrix of numbers, which can be applied as a kernel:

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0

0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0

0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0

0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0

0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0

0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0

0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0

1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1