Micro 3D Scanning - 1 Focal Depth

3D scanning is a very powerful tool, and it's value isn't limited to the objects and scenes you interact with in everyday life. The ability to precisely determine the 3D shape of tiny (even microscopic) objects can also be really useful.

 The 3D reconstructed shape of a tiny (0.8 by 0.3 mm) surface mount resistor on a printed circuit board. This was made using only a microscope; no fancy laser scanning required!

3D scanning through a microscope is a bit different to normal 3D scanning; mostly because when you look down a microscope at an object it looks very different to what you might expect from day-to-day life. The most immediately obvious effect is that out of focus areas are very out of focus, often to the point where you can barely see what is there. This effect comes down to the angle over which light is collected by the lens capturing the image; your eye or a camera lens in everyday life, or an objective lens when using a microscope.

Three images of the surface mount resistor. The three pictures are taken at different focus distances so different parts of the image are clear and others blurred. The blurred parts are very blurred!

In every-day-life when using a camera or your eyes distance from the lens to the object is normally long, it may be several metres or more. As a result the camera/your eye only collects light over a small of angle, often less than one degree. In comparison microscopes collect light from an extremely large range of angles, often up to 45 degrees. The angle must be this large because the objective lens sits so close to the sample. A wider angle of light collection makes out of focus objects appear more blurred. In photography terms the angle of light collection is related to the f-number, and large f-numbers (which have a large angle of light collection) famously have very blurred out of focus portions of the image.

The upshot of this is that in a microscope image the in focus parts of an image are those which lie very near (often within a few micrometers) to the focal plane. It is quite easy to automatically detect in focus parts of an image by using local image contrast (this is actually how autofocus works in many cameras) to map which parts of a microscope image are perfectly in focus.

In this series of images the most in-focus one is image 6 because it has the highest local contrast...

 ... using edge detection to emphasise local contrast in the image really highlights which one is perfectly in focus.

In this series of images the most in-focus one is image 55 instead.

The trick for focus 3D scanning down a microscope is taking the ability to detect which parts of an image are in focus, and using this to reconstruct the 3D shape of the sample. Going to the 3D scan is actually really easy:

1. Capture a series of images with the focus set to different distances.
2. Map which parts of each of these images are perfectly in focus.
3. Translate this back to the focus distance used to capture the image.

This concept is very simple; if you know one part of an object is perfectly in focus when the focus distance is set to 1mm, that means it is positioned exactly 1mm from the lens. If a different part is perfectly in focus when the focus distance is 2mm, then it must be positioned 2mm from the lens. Simple!

It may be a simple idea, but this method gives a high quality 3D reconstruction of the object.

The reconstructed 3D shape of the resistor, using 60 images focused 0.01mm apart, mapped to a depth map image. The lighter bits stick out more from the surface, and the darker bits stick out less.

Using the depth map to reconstruct the resistor reconstructed in full colour in 3D! Pretty cool for something less than 1 mm long...

Does that seem impressive? Then check out the videos:

A video of the original focus series of images captured of the resistor.

The reconstructed 3D shape.

A 3D view of the resistor, fully textured.

This approach is, roughly speaking, how most 3D light microscopy is done in biological and medical research. It is very common practice to capture a focal series like this (often called a "z stack") to get this 3D information from the sample. 3D imaging is most useful in very thick samples where you want to be able to analyse the structure in all three dimensions, an example might be analysing the structure of a tumour. My research on Leishmania parasites inside white blood cells uses this approach a lot too. The scanning confocal fluorescence microscope was actually designed to maximise the value of this 3D effect by not only blurring out of focus parts of the image, but also eliminating the light all together by blocking it from reaching the camera.

Software used:

ImageJ: Image analysis.

Blender: 3D viewing and rendering.

Cell Biology of Infectious Pathogens - Ghana 2013

For the last four years there has been a cell biology workshop in West Africa, organised by Dick McIntosh, an intense two week course aiming to help young African scientists around the master's degree stage of their careers. This course ran again this year, and was the first organised by Kirk Deitsch (malaria expert and a regular from the previous courses) and I was fortunate enough to be invited to teach the trypanosome half of the course. For its fifth incarnation the course returned to a location where it has previously been held, the Department of Biochemistry, Cell and Molecular Biology in the University of Ghana, and was organised with Gordon Awandare.

[Interactive 360° Panorama]

The Department of Biochemistry, Cell and Molecular Biology - University of Ghana, Accra.

The focus this year was teaching basic cell biology and the associated lab techniques, emphasising how this helps understand and fight some of the major parasitic diseases in Africa: African trypanosomiasis (sleeping sickness), leishmaniasis and malaria. All three of these diseases impact Ghana and the surrounding countries and these diseases are of enormous interest to students embarking on a scientific career in Africa.

[Interactive 360° Panorama]

The teaching lab in The Department of Biochemistry, Cell and Molecular Biology.

Of the three diseases we were teaching about malaria is by far the most well known, both locally and internationally. It is caused by Plasmodium parasites (which are single cell organisms) which force themselves inside the red cells in the blood to hide from the host immune system. Malaria is often viewed as the iconic neglected tropical disease, however in the last 10 years or so the understanding of the disease and efforts to find a vaccine and new drugs has improved vastly. Unfortunately it is still very common (we had one case in the participants on the course in the two weeks), drug resistance is rising, and it places a huge cost and health burden on the affected countries. It also impacts a huge area; almost all of sub-Saharan Africa is at risk.

Looking at Leishmania. One of the lab practicals was making light microscopy samples from non-human infective Leishmania using Giemsa stain.

Leishmaniasis and trypanosomiasis are caused by two related groups of parasites, Leishmania and trypanosomes (also single cell organisms), and if malaria is a neglected tropical disease the these are severely neglected tropical diseases. The two parasites live in different areas in the host, trypanosomes swim in the blood while Leishmania live inside macrophages, a type of white blood cell that should normally eat and kill parasites. In comparison to malaria fewer drugs are available, the drugs are less effective and several have severe side effects. Even diagnosis is thought to often be inaccurate. The impact of these diseases is less than malaria; human trypanosomiasis is thought to be relatively rate and leishmaniasis is confined to a semi-desert band just to the south of the Sahara. Trypanosomiasis does have a huge economic impact though, as it infects cattle and prevents milk and meat production, and cases of leishmaniasis are probably under-reported.

Staining trypanosomes. One practical was making immunofluorescence samples. In this sample the flagellum of trypanosomes was stained fluorescent green using the antibody L8C4.

So what did we teach?

The teaching was a mixture of lectures, small group discussions, lab practicals and lab demonstrations and we taught for 14 hours a day for 11 days; we could cover a lot of material! All the teaching was focused on linking basic cell biology to parasites and to practical lab techniques. Topics taught included how parasites avoid the host immune system, molecular tools to determine parasite species, light microscopy techniques, using yeast as tool to analyse cell biology of proteins from other species, host cell interaction of parasites, and many more.

Detecting human-infective trypanosomes. This gel of PCR products shows whether the template DNA was from a human-infective or non-human infective subspecies of T. brucei. If there was a DNA product of the correct size (glowing green) then the sample was human-infective.

A great example of how all the teaching tied together was polymerase chain reaction (PCR) to determine species. Human-infective trypanosomes have a single extra gene which lets them resist an innate immune factor in human blood which would otherwise kill them, and I taught about why this is important for understanding the disease and how it was discovered. This gene can be detected by PCR, and this technique is used to tell if a particular trypanosome sample could infect people. We ran a practical actually doing this in the lab.

PCR is a simple, adaptable and easy technique for checking any parasite for a particular species-defining or drug resistance gene, and we also taught how to use online genome sequence data to design PCR assays. We even worked through PCR assay design for many individual participant's personal research projects, really transferring the skills we were teaching to their current research. Finally we looked at papers using PCR techniques to critically analyse the experiment and assay design to help people avoid pitfalls in their own work.

This was a great demonstration of how basic cell biology and lab techniques can have real practical application with medical samples and help with surveillance of a disease. We designed all of the teaching to have this kind of practical application.

7x speed timelapse video of fish melanophores responding to adrenaline.

One practical with massive visual impact was the response of fish melanophores to adrenaline/epinephrine. Fish normally use these cells to change colour in response to stimuli and melanin particles (melanosomes) inside specialised cells (melanophores) run along the microtubules which make up a large portion of the cell cytoskeleton. We used it to demonstrate signalling; adrenaline can be used to stimulate movement of the melanosomes towards the centrosome.

This is really flexible experimental system for demonstrating the functions of the cytoskeleton, motor proteins and signalling pathways because the output (movement of the pigment particles) is so easy to observe with a cheap microscope or even a magnifying glass. This experiment was particularly chosen as it is a useful and accessible teaching tool for cell and molecular biology, and many of the course participants had teaching obligation in addition to their research.

Western blots in Western Africa. 100x timelapse of loading and running a SDS-PAGE gel.

We aimed to cover all the major molecular and cell biology techniques and had practicals doing microscopy, immunofluorescence, growing a microorganism (in this case yeast), PCR, agarose gel electrophoresis SDS-PAGE and Western blotting. The yeast practicals were particularly cool; using genetically modified cell lines the students analysed the function of p53, a transcription factor with a major role in recognising genetic damage and avoiding cancer, and how well it promotes transcription from different promoter sequences. These practicals taught growing yeast, temperature sensitive mutants, several types of reporter proteins in yeast and Western blotting, all concerning a transcription factor with huge clinical relevance in cancer!

Exploring DNA and protein structures through PyMol in a bioinformatics session.

Practicals weren't just limited to lab practicals though. We also ran interactive bioinformatics sessions looking at the kinds of data which are freely available in genome and protein structure databases online. These were also very popular, especially as so much data is available for free online.

All in all the course was a great success. The participants were all extremely enthusiastic, hard working and scarily smart! Feedback so far has also been very positive. I feel that courses like this can have a huge impact on the careers of young African scientists, and I sincerely hope that funding can be secured to continue running this type of course in the future.

You can also read more about this course at the ASCB website.

Software used:

ImageJ: Image processing and timelapse video creation.

Tasker for Android: Timelapse video capture.

Pymol: Protein structure analysis

Valentine's Day Electron Microscopy

Electron microscopes are pretty impressive machines. By firing electrons at a sample they can generate images thousands, or even millions, times sharper than you can make with light. With the right sample you can spot individual protein molecules in a cell, and even individual atoms within a molecule. When using these microscopes to look at the structures within biological samples like cells there are two big problems though:

1. Proteins, sugars, fats, DNA, water, plastic, etc. all block electrons by about the same amount. 
2. Electrons can only travel very short distances through materials.

These two issues make looking at a cell tricky: firstly the cell is too thick for most electrons to travel through it. You can up the power of the electron beam, but then you just vaporise the cell. To solve this the cell has to be sliced up into very thin layers called 'sections'. The next problem is therefore how to slice up a cell, which is basically a bag of proteins dissolved in water. The answer is to replace the water with plastic, making a solidified version of the cell which can be sliced up into thin sections. Finally, to actually tell the difference between protein, fat and DNA in the sample, you need to stain them to make them stand out from the plastic in the background. This is done by using heavy metals, like osmium and lead. The huge positively charged nuclei in these atoms scatter electrons away from the the detector; this makes regions where heavy metals have bound to proteins, fats, etc. look darker.

A block of plasticated cells stained with heavy metals (the black bit) in an epoxy resin plastic (the amber bit) in a metal holder (the silver bit).

So how thin does one of these sections need to be? In short, very. Even travelling through air an electron may only go a couple of centimetres before an atom captures it or deflects it from its path. In a solid material like plastic the distance is far shorter, about 1 million times shorter, with electrons travelling less than 1 millionth of a metre before being scattered or captured. The sections a sample is currently in into must be very thin, about one ten millionth of a metre (100 nanometres) is common.

Cutting a sample into slices one ten millionth of a metre thick is a challenge. It is like cutting a human hair into 1000 slices, each section is about 2000 times thinner than normal office paper. A lab machine called an ultramicrotome is designed to do this, and can cut slices as thin as 30nm. This is a tiny distance, about the width of 15 DNA double helices. The next problem is; how do you know you have cut a slice of the correct thickness? The sections are far too thin to be able to pick them up and measure them by any normal method. Luckily a physical phenomenon called thin film interference gives an easy way to do this.

When light hits a thin film, like a thin layer of oil in water or a soap bubble, some light bounces off the front of the film and some off the back. If the film is a similar thickness to the wavelength of light (around 500 nm) then interference between the two paths the light takes will occur. The interference can be constructive (boosting the light intensity) or destructive (reducing the light intensity). Because different colours of light have different wavelengths, different colours will experience constructive or destructive interference, for example if blue and green wavelengths destructively interfere, but red does not, then only red light will be reflected. The film will look red, even if the material it is made of is totally transparent.

Thin film interference in a soap bubble. CC-BY-SA by link.

Sections of cells embedded in plastic for electron microscopy act just like an oil or soap film, and have distinctive colours based on their thickness. We use these colours to check that the sections we have cut are the correct thickness, without having to directly measure them. It really is very quick and easy!

Lots of electron microscopy sections of different thickness. From bottom to top: 30nm (grey), 50nm (white), 70nm (white), 100nm (gold), 150nm (purple), 200nm (blue), 250nm (yellow), 300nm (pink/red), 400nm (green), 500nm (purple).

So what does this have to do with Valentine's day?

As far as I can work out the sectioning process you use for electron microscopy is a great way to make the smallest possible valentines heart which is still has a vivid pink/red colour. 'All' you need is:

• An ultramicrotome
• A block of plastic 
• A razor blade 
• Steady hands
• About 10 minutes

Step 1:
Trim the plastic block to a flat face, then cut a heart shape into that face. This is the bit the steady hands and razor blade are needed for, the heart needs to be about 0.3 to 0.5 mm (250 to 500 um) wide! It also needs to have tidy flat edges so sections can be cut without damaging the knife blade.

The trimmed, heart-shaped, block of plastic.

Step 2:
Use the ultramicrotome to cut a section 300 nm (3 ten millionths of a metre) thick off the block, and catch it on the surface of a pool of water next to the knife edge. From the colour scale of thin film interference we expect a 300 nm section to be a vivid red/pink colour (even though the plastic itself is a transparent amber colour).

A 300nm thick sliver of the block, the tiniest Valentine's heart ever.

Step 3:
Bask in the glory of having made the smallest, and geekiest, valentines heart ever. It is so thin that even if you scaled it up to the size of an A4/letter sheet of paper it would still be 4 times thinner than cling film/saran wrap and 40 times thinner than paper. This tiny thickness gives it a tiny volume: it has a volume of about 3×10^-14 metres cubed, and a mass of about 3×10^-11 grams. That is similar to the mass of a single, average, human cell.

Software used:
ImageJ: Focus stacking.
Photomatix: HDR image processing.

Light vs. Microscopists

Light vs. Microscopists, my research comics feature in OUBS Phenotype, trinity term 2012. Everything you wanted to know about superresolution light microscopy in one fun package. Including a cute kitten. Check out the full issue here.

Software used
Inkscape: Document design and layout.
ImageJ: Micrograph simulation.

Diatomaceous Anaglyphs

I was trying to work out what the least friendly, most jargon rich blog title I could possibly write something interesting about... I think "Diatomaceous Anaglyphs" is a pretty good effort!

Diatomaceous means "of diatoms"; a type of single celled organism which grows beautiful shells made of silica. In this case these are the shells of dead diatoms from millions of years ago which settled out of the sea, made beds of diatom shells then got compressed into a soft rock. This rock is mined and processed for use in various areas of industry and is called diatomaceous earth.

Anaglyphs are the name of the red/blue 3D images you get which can be looked at using some glasses with coloured celophane instead of lenses. With the surge in 3D films at the cinema you can find more and more 3D stuff online using anaglyphs.

When I combined the I made diatomaceous anaglyphs, which are just plain beautiful (you can also view the whole album on my Flickr):

For those of you not geeky enough to have red/blue 3D glasses this is an animated version:

Software used:
ImageJ: Image alignment and processing

Diatomaceous Zoom

[watch in higher quality here]

I posted a video before about the magnification range of a scanning electron microscope... This is the update with a slightly more interesting sample! Last time I was using 10 micron beads to confirm the calibration of the scope, this time it is fossilised diatoms; tiny single celled plankton with beautiful shells.

The video starts at 25x magnification, the whole field of view is about 2mm across.



And it starts zooming in...



And in...



And in...



Until it gets to 200000x magnification, nearly 1000x greater than at the start of the video. Now the whole field of view is only about 2um across. That is 100 times narrower than a human hair.

Then it zooms back out again so you can see what you were looking at!

Software used:
ImageJ - Generating zoom series, converting individual images to a video.

Sublime - video

Sublime is a video combining music with science. Awesome.
 

[watch in higher quality here]

The story of this video is the phase of matter and how it changes. You will know the common phase changes of water; boiling, condensing, freezing and melting. These phase changes all involve a liquid step, and in this video liquids are not involved at any stage.

The phase changes here just involve solids and gases, either deposition (changing from gas directly to a solid) or sublimation (changing from a solid directly into a gas). The actual substances in the video are common, just dry ice (solid carbon dioxide) and water frost, the reason no liquids are involved comes down to the temperature and pressure. At normal atmospheric pressure dry ice does not melt, liquid carbon dioxide is not stable, so it changes directly into a gas. Dry ice is also very cold (-78°C/-108°F) meaning water vapour (a gas) in the air coming into contact with the dry ice directly deposits to form frost. In some places it also hits the cold carbon dioxide gas and solidifies from the air to form tiny snow particles.

The frost looks black and the dry ice looks white because of the way the sample was illuminated. It was lit up from the back and frost (which has lots of tiny crystals) scatters the light away from the lens, making it darker. Dry ice scattered the light less, i.e. light got transmitted straight through to the lens, so appeared lighter.

Over time the carbon dioxide gradually sublimates and the water frost gradually forms. Because the surface the frost is trying to grow on is constantly disappearing the frost constantly moves. As the carbon dioxide sublimates it also increases greatly in volume, blowing the frost about!

This video is of a microscopically small sample. The region the video shows is only about 1mm across, depending on the size of your screen you are seeing the frost and dry ice at 100× to 1000× its actual size. This high magnification lets us see individual ice crystals growing as the frost spreads. Initially the crystals in the frost are very small (5μm or so). Gradually as the dry ice sublimates the temperature increases and the water vapour in the air can take longer to deposit onto the growing ice crystals. Slower crystal growth leads to bigger crystals (up to 100μm).

Ice crystal shapes also vary depending on temperature and environment, in this video you can see both needle-shaped and flat ice crystals.

The frost grows and the dry ice sublimates quite quickly. All the clips in the video are at 4× real life, if you imagine everything happening at a quarter of the speed that is how fast the frost was growing and changing.

Software used:
ImageJ: Image editing, alignment, cropping and video making,
Audacity: Audio editing.
Windows Live Movie Maker: Final sequencing, adding captions.

Extended Depth of Field

One of the tricky things with microscopy and macro photography is the depth of field, as you start magnifying a sample you need to collect as much light as possible to generate the image with a sensible exposure time. Unfortunately this requires a large aperture, and this creates a very shallow depth of field...



This micrograph of a diatom clearly shows the problem, it is impossible to get the whole sample in focus in one image. Fortunately there are ways around it; by analysing the image for sharp edges it is possible to find which image is the most in-focus and the whole image can then be reconstructed only using the in-focus patches. This process is called focus stacking and generates an extended depth of field. Good free implementations of focus stacking are hard to come across, so I wrote one; you can download the ImageJ macro here.
Using the same technique on macro photography (processing the red, green and blue channels separately) gives a similarly impressive result. The three starting images:


And the extended depth of field result:
Software used:
Image processing: ImageJ

Webcam Microscopy

Just a very short post sadly, real life is pressing down on me again... But check out my latest Instructable; with a normal webcam and 5 minutes you can make a >100x microscope...