Tuesday, 16 July 2013

Micro 3D Scanning - 1 Focal Depth

3D scanning is a very powerful tool, and it's value isn't limited to the objects and scenes you interact with in everyday life. The ability to precisely determine the 3D shape of tiny (even microscopic) objects can also be really useful.

 The 3D reconstructed shape of a tiny (0.8 by 0.3 mm) surface mount resistor on a printed circuit board. This was made using only a microscope; no fancy laser scanning required!

3D scanning through a microscope is a bit different to normal 3D scanning; mostly because when you look down a microscope at an object it looks very different to what you might expect from day-to-day life. The most immediately obvious effect is that out of focus areas are very out of focus, often to the point where you can barely see what is there. This effect comes down to the angle over which light is collected by the lens capturing the image; your eye or a camera lens in everyday life, or an objective lens when using a microscope.

Three images of the surface mount resistor. The three pictures are taken at different focus distances so different parts of the image are clear and others blurred. The blurred parts are very blurred!

In every-day-life when using a camera or your eyes distance from the lens to the object is normally long, it may be several metres or more. As a result the camera/your eye only collects light over a small of angle, often less than one degree. In comparison microscopes collect light from an extremely large range of angles, often up to 45 degrees. The angle must be this large because the objective lens sits so close to the sample. A wider angle of light collection makes out of focus objects appear more blurred. In photography terms the angle of light collection is related to the f-number, and large f-numbers (which have a large angle of light collection) famously have very blurred out of focus portions of the image.

The upshot of this is that in a microscope image the in focus parts of an image are those which lie very near (often within a few micrometers) to the focal plane. It is quite easy to automatically detect in focus parts of an image by using local image contrast (this is actually how autofocus works in many cameras) to map which parts of a microscope image are perfectly in focus.

In this series of images the most in-focus one is image 6 because it has the highest local contrast...

 ... using edge detection to emphasise local contrast in the image really highlights which one is perfectly in focus.

In this series of images the most in-focus one is image 55 instead.

The trick for focus 3D scanning down a microscope is taking the ability to detect which parts of an image are in focus, and using this to reconstruct the 3D shape of the sample. Going to the 3D scan is actually really easy:
  1. Capture a series of images with the focus set to different distances.
  2. Map which parts of each of these images are perfectly in focus.
  3. Translate this back to the focus distance used to capture the image.
This concept is very simple; if you know one part of an object is perfectly in focus when the focus distance is set to 1mm, that means it is positioned exactly 1mm from the lens. If a different part is perfectly in focus when the focus distance is 2mm, then it must be positioned 2mm from the lens. Simple!

It may be a simple idea, but this method gives a high quality 3D reconstruction of the object.

The reconstructed 3D shape of the resistor, using 60 images focused 0.01mm apart, mapped to a depth map image. The lighter bits stick out more from the surface, and the darker bits stick out less.


Using the depth map to reconstruct the resistor reconstructed in full colour in 3D! Pretty cool for something less than 1 mm long...

Does that seem impressive? Then check out the videos:


A video of the original focus series of images captured of the resistor.



The reconstructed 3D shape.



A 3D view of the resistor, fully textured.


This approach is, roughly speaking, how most 3D light microscopy is done in biological and medical research. It is very common practice to capture a focal series like this (often called a "z stack") to get this 3D information from the sample. 3D imaging is most useful in very thick samples where you want to be able to analyse the structure in all three dimensions, an example might be analysing the structure of a tumour. My research on Leishmania parasites inside white blood cells uses this approach a lot too. The scanning confocal fluorescence microscope was actually designed to maximise the value of this 3D effect by not only blurring out of focus parts of the image, but also eliminating the light all together by blocking it from reaching the camera.

Software used:
ImageJ: Image analysis.
Blender: 3D viewing and rendering.

8 comments:

  1. great - but where do I find a tutorial / description how this is actually done in blender?

    ReplyDelete
    Replies
    1. Only the visualisation of the height map was done in blender, try out this tutorial if you're interested http://en.wikibooks.org/wiki/Blender_3D:_Noob_to_Pro/Making_Landscapes_with_heightmaps

      Delete
  2. Hi, it is an interesting approach and I would like to try it.
    Could you tell me how you performed local image contrast analysis in ImageJ ?

    ReplyDelete
    Replies
    1. I did it all with a couple of macros. The idea is just to use edge detection then find the slice of the stack with maximum edginess for each pixel. By using a maximum or median filter on the edge detection images you effectively increase the local contrast detection radius.

      Delete
  3. Hi i like this post cause very nice information
    i am a visitor blog, you can visits my blog to Obat Limpa Bengkak and you can review in the Ramuan Herbal Untuk Sakit Tenggrokan

    ReplyDelete
  4. Has this been combined with tilt-shift photography for doing 3D models of landscapes and cities? How accurate were the results?

    ReplyDelete
  5. LIVE Entry – 1 (March-21-2014)
    Problem: HP Printer not connecting to my laptop.
    I had an issue while connecting my 2 year old HP printer to my brother's laptop that I had borrowed for starting my own business. I used a quick google search to fix the problem but that did not help me.
    I then decided to get professional help to solve my problem. After having received many quotations from various companies, i decided to go ahead with Online Tech Repair (www.onlinetechrepairs.com).
    Reasons I chose them over the others:
    1) They were extremely friendly and patient with me during my initial discussions and responded promptly to my request.
    2) Their prices were extremely reasonable.
    3) They were ready and willing to walk me through the entire process step by step and were on call with me till i got it fixed.
    How did they do it
    1) They first asked me to state my problem clearly and asked me a few questions. This was done to detect any physical connectivity issues with the printer.
    2) After having answered this, they confirmed that the printer and the laptop were functioning correctly.
    3) They then, asked me if they could access my laptop remotely to troubleshoot the problem and fix it. I agreed.
    4) One of the tech support executives accessed my laptop and started troubleshooting.
    5) I sat back and watched as the tech support executive was navigating my laptop to spot the issue. The issue was fixed.
    6) I was told that it was due to an older version of the driver that had been installed.
    My Experience
    I loved the entire friendly conversation that took place with them. They understood my needs clearly and acted upon the solution immediately. Being a technical noob,

    sometimes find it difficult to communicate with tech support teams. It was a very different experience with the guys at Online Tech Repairs. You can check out their website www.onlinetechrepairs.com or call them on 1-914-613-3786.
    Would definitely recommend this service to anyone who needs help fixing their computers.
    Thanks a ton guys. Great Job....!!

    ReplyDelete
  6. Is Your VIRUS REMOVAL
    Computer Sluggish or Plagued With a Virus? – If So you Need Online Tech Repairs
    As a leader in online computer repair, Online Tech Repairs Inc has the experience to deliver professional system optimization and virus removal.Headquartered in Great Neck, New York our certified technicians have been providing online computer repair and virus removal for customers around the world since 2004.
    Our three step system is easy to use; and provides you a safe, unobtrusive, and cost effective alternative to your computer service needs. By using state-of-the-art technology our computer experts can diagnose, and repair your computer system through the internet, no matter where you are.
    Our technician will guide you through the installation of Online Tech Repair Inc secure software. This software allows your dedicated computer expert to see and operate your computer just as if he was in the room with you. That means you don't have to unplug everything and bring it to our shop, or have a stranger tramping through your home.
    From our remote location the Online Tech Repairs.com expert can handle any computer issue you want addressed, like:
    • - System Optimization
    • - How it works Software Installations or Upgrades
    • - How it works Virus Removal
    • - How it works Home Network Set-ups
    Just to name a few.
    If you are unsure of what the problem may be, that is okay. We can run a complete diagnostic on your system and fix the problems we encounter. When we are done our software is removed; leaving you with a safe, secure and properly functioning system. The whole process usually takes less than an hour. You probably couldn't even get your computer to your local repair shop that fast!
    Call us now for a FREE COMPUTER DIAGONISTIC using DISCOUNT CODE (otr214427@gmail.com) on +1-914-613-3786 or chat with us on www.onlinetechrepairs.com

    ReplyDelete