Due to the open nature of our driver system, it’s relatively straightforward to write a new driver file for almost any imaginable device in Nebula. However, driver development can be a pain, so I though I’d share some debugging tricks.
For all commands, we will navigate to the command line using windows Command prompt or powershell. Here I show command prompt syntax. Nebula is commonly installed under program files. So in the windows start bar type “cmd” and press enter. This should open the command prompt. The commands below navigate to the install location.
First, is the use of logging to the command line. When Nebula is launched from the command line, a series of optional switches can be used to report logged output. By opening this alongside the app, it’s very fast to figure out what or where something is going wrong. To see log output, use –printlog.
c:\Program Files\Nebula>Nebula.exe --printlog
This launches the app, but shows a live log output on the console at the same time.
Get a full list of Supported Drivers
This command will return a complete list of drivers found in the Nebula supported list. To print this list, use:
c:\Program Files\Nebula>Nebula.exe --listdrivers
Here is live example of that command and the expected result. A list of drivers is shown, press enter to close the listing.
Testing a specific Driver File
Next, let’s load up one of these drivers in the test utility. Now, this is where things get cool! The test utility can load a host device driver, inspect it’s shown devices, attempt to connect them, and even test every sub-device!
So, let’s assume we have a new camera driver we want to work on. The camera should do things like snap an image, show various properties etc. In my example I use a real UVC (web) camera, and inspect the driver. The driver actually pulls the real camera info and even snaps a test image. The goal of this function is twofold, first to validate a given driver, and second to validate the device itself.
For this command we will use –testdriver *drivername* and optionally –port *portname*, for example, COM5
Here we can watch the expected output in real time. Note a new window is spawned to show us the results for these tests.
Testing Host and Sub Devices
Let’s say we have a controller, for example, an ASI or Marzhauser box, which has a Z motor attached. In Nebula, the controller can be thought of as the Host (the resident controlling brain), and the device can be considered an appendage attached to the host. So all drivers must have a Host, but a Host may only have one device. In this example, we use a demo Focus device. First the host is loaded and analyzed, and then, you ca select to test any device provided by that host. I choose option # 1 (focus) and the test runs, even showing my driver has a small error I missed!
Obviously, if you have a driver which isn’t behaving, please contact us – yet I hope these options provide a first-line of interface for power users, so simple things can be corrected and analyzed quickly!
While this is an old subject in terms of technology, I find that every few years I end up describing it to a new group. Surprisingly to me, every time people seem to appreciate the information, and I think it’s just not covered often in broader imaging trainings. So today I’m writing about a favorite subject of mine – TTL control. To do this I’ll be referencing some fun technology – namely:
For illustration of the device control, I’ve captured a few real example frames from my oscilloscope, but also added some simulated oscilloscope examples that I hope will better depict the work behind the scenes for these devices.
I think it’s most important to frame this in a question,
When my software controls devices, how fast do they respond?
Assume we click a shutter button in any software. This can be for a laser, an LED, even an old mechanical shutter. In every case a bunch of things must happen for the end device to actually change it’s state. For that state (let’s say, “OFF” to change to “ON”,
the software must accept the command
and issue control signals to the device connection.
That connection talks to the device controller (normally, a microcontroller)
The microcontroller reads the command and does what it’s told
The state changes.
Now, along the way, each device has a communication bus, and each bus has a traffic control system to insure multiple communications don’t conflict. As a result, the timing from the start of the button click to the end is variable – sometimes, up to 20 milliseconds!
Now, because the software is typically programmed to “confirm” each thing it does, it doesn’t just shout orders into the dark. Instead, the software waits for a confirmation, or acknowledgment, of the command being received, and oftentimes it will confirm the new state is active.
Consider this means a new set of jobs which must be completed.
The device controller changes state
The device controller then replies, or reports state
The communication connection on the PC receives the reply
The software reads the reply, and confirms it’s correct
The software continues to the next job.
And so again, we can see a stack-up of not only delay, but a variable amount of delay.
If we visualize this in a graphic example like the one below, we can see the software code is sent to an LED microcontroller, the command is processed, and the led illuminates. However, we can also see there is a lot of room for improvement!
LED Control via Software (PC → LED Controller → GPIO)
A host PC issues serial commands. The LED Controller parses each command and drives a GPIO pin. The oscilloscope captures the resulting digital waveform on PA5.
OSCILLOSCOPE · CH1 — GPIO PA5RUN 1V / div · 500 ms / div
Another way to look at this control is what happens with a typical complex instrument. Below is a video where I control 3 devices. The PCO Camera, a Zaber filter turret, and the Tetrem. Notice how long it takes just to get 3 time points for 2 channels. The exposure is short, that's not the issue - the issue here is of course, the devices need to move, but also each device must receive + process + move + report positions, and this takes a long time.
TTL Control -> Old + Fast = Awesome
So what is TTL? Basically, nothing more than an on/off switch! Just like a light switch allows an electrical signal to feed a light, TTL uses a voltage to send a signal to another device. You can find a thorough history on this communication method here if interested.
What we care about is that we have an on/off control system. Because it's nothing more than a voltage, no crazy translations need to happen for things to immediately react. If a TTL control line is on, or "High", then the device should be doing whatever ON is. Many devices have this input, but few people even know about it! For example, most Focus controllers, XY Stages, Peristaltic Pumps, Perfusion systems, and many other devices like lasers, shutters, and actuators accept a TTL input for control.
TTL Logic typically comes in 2 flavors. Older systems will use 5V for "on" and 0V for "off". The device being controlled might use thresholds or a midpoint. Some devices, assuming you give them at least 2.5V, will go active. Others want at least 4V or 4.5V before they enable. Newer devices often use 3.3v control. This is the same concept as 5V, just with lower voltages. One word of caution here - before connecting things up, it's good to confirm the logic of your devices. Generally, the amount of power these use is quite low, and signal inputs won't break things - but in the case of an exposed microcontroller pin, you can definitely cook a 3.3v input with a 5V line!
TTL Blanking - A partial Solution
So what can be done about this overhead? First, we can consider that the only time we really need the illumination on, is when the camera is truly collecting data. One way to do this is to use the I/O control cables found on most scientific cameras. In my case I routed the expose signal from the edge camera into the Tetrem on a single channel, and then, told the tetrem in my software to use "TTL Control". With these changes, the LED source now waits for a TTL input to go high, before it enables it's emission.
So now, let's assume that when the camera is exposing, we can enable the LED line. The software can act as a logical "AND" if needed, but the line won't enable unless there is real collection of light by the camera. You can see an example of this below.
Software-Commanded vs. TTL-Triggered LED Control
A single event fires two paths simultaneously: the PC issues a serial command (software path), while a TTL line drives the controller directly (hardware path). The oscilloscope shows the resulting latency gap.
OSCILLOSCOPE · 3-CH · ALIGNED TIMEBASERUN 1V / div · 10 ms / div
For the example, the yellow line is the camera output, or trigger output, the teal line is the LED B brightness result, and the purple line is the software-only reaction. In this example, we can see the delay (latency) of the response, yet in most software, the purple line will illuminate before the camera exposure. What this means for research is your cells are being exposed to excitation energy, and you aren't even collecting that light onto a sensor! So, we are in effect just cooking cells for fun. NOT GOOD!
Oscilloscope reading TTL from PCO camera
Using the Triggerscope for State Control of Devices
This is great for a single LED, but how can we control more than 1 channel? For this, we would use a logic controller like the Triggerscope Mini! With the mini, the camera expose line is routed to it's TRIG input, and up to 4 output channels are connected to it's TTL control pins.
Triggerscope Mini with Tetrem Controller for intensity
With this done, we can assign an active line using the triggerscope. This provides fast easy state switching, and can even be done over multiple devices. Doing this will mean, for this example, we won't be moving our filter cube device. Instead, we would use a multi-band dichroic. However we can switch the LED excitation, and that's how we will gain some speed here.
In the example above, we see a high speed switch. This is simply by telling the Triggerscope Mini to change channels, instead of the LED. But - there is still more to do to improve our speed!
Streaming for the Win
The above example uses the triggerscope via USB, and while we only have 1 device, we are still using a "set + capture + set + capture" type of sequence. What if we just told the camera, "run as FAST as you can." Then, we used the output signal from the camera to count frames, and we told the triggerscope, "every other frame, switch from LED 1 to LED 2, then back to 1". THIS is the power of "Streaming", or also called "Sequencing". With this capability, we pre-load stuff to do into the triggerscope. Then, we arm the triggerscope. With that done, our triggerscope is sitting, locked into a state, simply waiting to receive camera frame TTL inputs. Below is a graphical depiction of this technique.
Now, we tell the camera, "Give us 2 channels * 10 frames = 20 pictures, and do so as fast as possible". In effect, "give us video of 20 frames".
In this method, there is zero control overhead. The camera free-runs, the triggerscope switches, and the LED reacts. Below is a video example of this, note also there is a small in-view live feed from a web camera (for all these video samples), and you can see that, indeed, streaming is crazy fast! For the below example, we've tied the camera "transfer" line into the triggerscope. This way, the triggerscope recieves a brief pulse before capture occurs. We enable on the "low" side of the signal, and illuminate. You can see for the CMOS style readout on the left of the example, once readout is done, the channel switches.
Trigger Switch — Camera-Driven LED Sequencing via Triggerscope
Each time the camera completes a frame, it emits a TTL pulse. The Triggerscope (Advanced Research Consulting) uses that pulse to advance its state machine and re-route its outputs — alternating TTL1 and TTL2 to the LED Controller so LED1 and LED2 illuminate consecutive frames.
OSCILLOSCOPE · 3-CH · CAMERA-SYNCHRONIZEDRUN 1V / div · ~16 ms / div
So what does the above look like in real life? Check out this example below! Note that I select in my "lambda" control box the "Stream" checkbox. Then once start capture is pressed, the triggerscope is armed, the camera is readied, and we snag 20 images in < 300ms!
Of course, we can extend this using the 5V DAC control line on the triggerscope for capture of Z as well as channel, and if using the more capable triggerscope 4B, we can control even more devices.
I hope this article has helped explain not only why I make devices such as the triggerscope, but also how it's important to use software that provides complete control for your light sources and other devices, as well as the ability to do things like this. My sincere thanks to Excelitas for lending me the Edge camera and Tetrem LED, both of which have worked flawlessly on my system. Nebula has drivers for both products fully integrated for streaming control if you have interest in using it.
Compact, Simplified TTL and DAC – Micromanager Native
I’m happy to share a new device I’ve been working on after encouragement from numerous clients. This new smaller version of the Triggerscope is intended to bridge the gap between large complex systems and more streamlined setups.
With the Mini you get:
4 TTL Outputs at 5V
2 DAC Outputs at 5V
1 3.3V to 5V input Trigger
Status LEDs for all connections!
A single USB-C Connection to the computer
Full supported Micromanager Triggerscope-Hub Behavior
This was a super fun project for me, as for the first time this board uses a bare MCU, which enabled my team to use any connector we wanted (hence the USB C) as well as full USB-Serial emulation on our own terms.
I’m also quite fond of the baby side-emitting LEDs on the outputs. These LED’s draw almost no current, but having them for all of the outputs makes it so easy to know whether a given control line is on or off. For the DAC LED’s, they change brightness with the output!
I tried to keep this design in-line with the current TG4, as it’s worked well for us over the years. This one is of course thinner, lighter and smaller, but in effect a truly miniaturized version of the larger model.
Let’s say you want to build an imaging system, but you want to attach 2 cameras for time-correlated 2 channel imaging. The limiting factor for such an arrangement is usually cost, that cost being $10,000 USD per camera means $20,000 for such a setup! A result of this is most researchers end up not using time correlation at such an accuracy, and resort to filter cube switching or emission wheels. Enter the new release of Micro-Manager drivers for Pt. Grey cameras. Through the hard work of Nico Stuurman, Micro-Manager now supports most of the low cost USB cameras offered by this machine vision camera manufacturer. Kurt Thorn has an excellent report on his blog with performance data from one of Pt. Grey’s CMOS cameras.
The key point? Most microscopy cameras start @ $6,000 USD, and go up from there. By comparison, Pt Grey cameras usually run less than $1,000 USD!
So why is it that such a camera can cost 1/10 of a typical microscopy camera? The major cause is the ongoing market cycle of entrenched mature products and services, vs. disruptive technologies and products. Clayton Christensen explains the nature of this process in his excellent book, “The Innovator’s Dilemma“, but let’s look at the top 2 reasons for the price delta.
Cooling: Most of the big camera manufacturers chase each other on performance specifications. If you’ve attempted to purchase one of these cameras through an on-site demo, you’ve likely heard the “spec war” sales pitch, wherein camera company A explains the important numeric advantages (quantum efficiency, read noise, dark current, etc)  over companies B, C etc. Of course, this rises out of a need to improve a camera’s performance as much as possible, to win sales. While this is all great for both product variation and for economic advantage (better product at a lower cost through competition), it assumes that the people driving these specs actually understand how important they are to the end user. In the past, dark noise was an easy problem to “one up” on performance. How? Simply cool the sensor! So peltier coolers (thermoelectric plates) were attached to the backside of sensors, with large radiators used to vent drawn heat on the back of the camera. This gave us heavy, large cameras with very low dark current of 0.03 electrons per pixel per second. But how does this relate to the average user? Normal exposure times are < 1 sec. Read noise levels on quiet cameras are ~ 3 electrons. So one would need to capture an image for an exposure time of 3 / 0.01 = 300 seconds, in order to reach a thermal issue with the camera! But the better spec gave an advantage to sales, and therefore this performance was considered standard in the industry, even though 99% of end users don’t need it.
Of course, other issues pop up when adding such extreme cooling. One is condensation. Because these chips would be cooled to -30c or lower, condensation loved to form on the face of the sensor, which obscures the image, and shorts the sensor. As a result, a vacuum sealed window must be placed around the sensor. Dry nitrogen may be back-filled into the chamber, but in the end, you have a chamber waiting for a leak. The manufacturing process is costly. All of this adds up to $$$$, which transfers down to the customer. Finally this trend is changing. Today, Hamamatsu, Pt Grey and others offer ambient (read un-cooled) camera options, resulting in substantial cost savings to customers.
Example of ice on a sensor. This is what a typical low temperature camera will produce when active, and when a seal is broken. condensation freezes due to the low sensor temp, producing images which look like crystals, or water droplets at lower temps.
Manufacturing: CCD sensors were initially produced on accident. RAM designers found that configurations of chip layouts were light sensitive, and as a result discovered they could be used to detect light in an array. Soon after CCDs began a strong fabrication growth, conventional memory switch from serial addressed layouts to dynamically addressed layouts. This resulted in CCD manufacturers being able to only build CCD chips, and not other, newer forms of memory. Single-use costs of such facilities caused prices to stay relatively high, as how many end-users require monochrome sensors, high sensitivity, precision, etc etc? Most consumers of sensor technology want to see a pretty picture on a phone! So CCD sensor production became a high cost problem. CMOS technology unlocked this problem, allowing conventional memory production houses to build CMOS sensors. As sensor production expanded, “modular” sensor packages went from the exception to the rule for the majority of cameras sold today. In the past, camera companies purchased a “chip” which had only the sensor, and access pins attached to it. It was required for the camera company to add a window, seal or other protective assembly, readout electronics, etc. Beginning with the ICX series from Sony, things changed. Instead of purchasing only a sensor, you could purchase a pre-built package right from the sensor manufacturer, including a complete window, and in some cases, the ADC and clocks all ready for use. Both of these changes in how sensors are produced have given rise to two new entrants in the camera market – low cost camera companies, who do as minimal work as required to get a camera out for sale, and more manufacturing houses capable of producing sensors. All this adds up to more choice for the consumer at a lower, more competitive price.
So what does this mean for you? Check out the Pt Grey camera line! It’ll save you a bundle on your imaging system, and may unlock your options for more multi-channel imaging!
Here’s a project I can finally talk about! This is one of my clients, PEOVI, who makes rugged, high speed camera mounts for anything from race cars, to airplanes. If you need to mount a Black Magic Micro, or GoPro, to your high speed vehicle, this is the product for you. The engineering team behind this company has decades of experience in manufacturing airborne cinematography platforms. No plastic on these bad boys 🙂
Here’s a link to the company store, where you can get anything from a simple fixtured mount, up to a tilt/roll/pitch enclosure.
Above is an example of the “speed ball” enclosure, which is used with GoPro cameras. The bottom mount us for a cinematic type tube, so it can be used with normal video production equipment if desired. Of course, it might also be attached to the strut of an experimental aircraft wing… 🙂
In 2011, I reviewed the Andor Neo, one of the first scmos cameras commercially available. I ended my report with this statement,
“…there is much life left in the good old frame transfer sensor…until someone cooks up a back thinned scmos…”
Recently, back thinned scmos camera have come to market, and Kurt Thorne at the UCSF NIC has an excellent report on Photometrics’ new Prime 95B camera. While there may be specific areas of research that can benefit from a BT EMCCD, it looks like the days of EMCCD are at an end for most of us. I guess the question now is, what comes next?
While I haven’t had a chance to test these for accuracy, I was excited to find them while looking for some tube lens specs! You can find the file set here. These designs include some of the common infinity corrected objectives, as well as tube lenses, from every major microscope manufacturer.
I made this video to show how to use both a state device shutter object and a group shutter object in micro-manager. Hopefully others can make use of it!
SO I’ve been working with AutoDesk’s Fusion360 for a while now, and have found it to be an exceptional tool for diving into CAD. If you’ve considered CAD before, but have been scared off by the learning curve, I recommend checking it out. One of the important functions of a truly parametric CAD program is the need to reference objects to other objects, and to reference measurements to variables or other object measurements. I knew Fusion360 could handle parametric work, but hadn’t been able to figure out a solution to my particular problem, so, here’s a video on how I did it. Not sure if this is an accepted method conventionally, but it worked for me!
The problem I had was how to set up a reflection line, which was dependent on 2 inputs, the angle of an input source, and the reflection angle of a mirrored surface. I wanted to be able to drive these 2 input values, and have either of them affect the resulting reflection line. Here’s how it worked out.
Over the past 5 years, almost every project I’ve worked on has required some level of optic design. In many cases, simple designs suffice. For example, projecting an LED can be accomplished at a basic level with an aspheric condenser. But when designs require flat fields, especially corrective elements, or apochromatic performance, it’s time to leverage commercial tools like Zemax. Luckily, I have one of the best Zemax gurus on the planet on hand for such work. But there is a middle ground where I want to work on designs I’d call basic+. This is where there has never been a clear solution. In such cases, the best compromise is to buy experimental grade lenses, and test them empirically.
Obviously this presents 3 problems:
shipping time + setup time = a long iterative design process.
when issues are found, the only solution is to snag a few more lenses and test, vs. Â modeling a solution.
The end result is “good enough” , but it’s unknown as to where other improvements may be found.
So imagine my surprise when I found an app for ray tracing and modeling, in the App store! Too good to be true – has to be a joke, right? Nope – Check out this short example, showing the Nikon 105mm SLR lens.
The basic app is ~ $2.00. Adding all of the features will run just under $30. Considering that the cheapest alternative runs ~ $1500, this is an amazing deal. Anyway – I hope others find this as useful as I have!
Oh one more example – here’s the optical design for a Zeiss 40x objective! Cool huh?!?!
I wrote this for a client, but figured others could make use of it. This code simply captures a number of images as defined by the user, and saves them to the specified folder without displaying them. This is useful if you don’t want all of the images open in memory during a timelapse.
Controlling a camera from an external trigger can be very useful. External triggering provides greater timing accuracy, direct control over exposure duration, and remote operation from TTL devices. This is a prime reason I made the Triggerscope controller, so I though it would be useful to explain how this works and what you can do with it.
What is “triggering”?
Most cameras built for scientific imaging or life science have a high density connector located on the back. 90% of the time, this is only used by savvy customers, or OEM integrators (companies buying a camera to place inside a machine for resale). The connector on the back of the camera will typically include 2 basic functions: Inputs (connections which send signals into the camera) and outputs (signals which activate when the camera is in a process or cycle).
Why use Triggering on a camera? Doesn’t the software control image capture?
Of course, software can tell a camera to snap an image. But when will that image be captured? Software needs to do more than say “snap” when you capture an image. Behind the scenes, a LOT of stuff happens!
the top level controls you see are converted into camera-specific commands via the camera “device driver” in your software.
The camera commands are fed into a control card or USB port and up to the camera.
The camera accepts the commands, which activate sets of tables in the camera firmware, to set up a sequence of turning on and off clocks and transistors at very high speed, all depending on your binning, ROI, etc.
The camera performs the above operations, and sends back an image.
What is quickly apparent is how much has to happen to get an image. In addition, the camera firmware will process some instruction sets faster than others! So some exposures might come back 200ms from issuing the “snap” button, while others will return in 130ms, 30ms, who knows!
So when precise timing is desired, and you really want that camera to fire at a specified time, triggering may be used. In this configuration, all software comms with the camera are pre-compiled, everything’s done. All of the steps above are performed, but the last step, the “camera performs operations above” sits….and waits…for the trigger line to run from 0 Volts to 5 Volts. As soon as that happens, an image is captured. Â For single frames, this is somewhat useful, but for sequences, this is great. We can capture images at some interval with 100% confidence timing is accurate.
Example
Let’s look at a typical camera, and pull the information we need from the user manual, in order to better understand triggered control. For this post, I’ll use the Photometrics CoolSNAP MYO camera, and as Photometrics makes it easy to read up on the user manual for their products, you can find the user manual here. Â Page 21 of the PDF describes the various operating modes for the trigger input. Note, the PDF also gives us information on EACH PIN used in conjunction with input and output signaling. There’s more than one!
Note, PM also provides short descriptions for each pin:
So – now we know which pins communicate which functions from the camera head connector – to connect to any of these, we’d simply connect a ground wire, and a positive pin wire to our chosen camera pin (for our needs, say Pin 1 – Trigger In)
Now – the most pertinent for us is the trigger in line, which runs in a few modes. Â Let’s review each, and why we might use them:
Trigger-First (sometimes called other things like “Fire”)
In this mode, the camera is set to snap some # of images (let’s say 10), but waits for a trigger input. Once that input goes high, the camera snaps 10 images as quickly as possible.
Strobe
In this mode, the camera is again configured for some # of images, but every image waits for a trigger.
Bulb
In this mode, the exposure time for each image is controlled by the input trigger. Again, multiple images are assumed (although Image qty of 1 is acceptable), and each frame waits for a trigger input pulse.
These three modes constitute the majority of uses for a typical triggered input line.
So, what does “receive a trigger” actually mean? Good question!
Triggering = TTL Control
Cameras, like almost all computer systems, are made to interact with external systems. One of the oldest, and most common methods, is referred to as “transistor transistor logic” or TTL. This boils down to the concept that if we specify an electrical voltage value of X as “off” and Y as “ON” then we can remotely flip a switch.
Many imaging devices use this communication method. A simple uniblitz shutter for example, can accept USB/serial data to open and close the shutter, but using TTL is faster and simpler. (just one thing to do, send 5Volts or 0Volts.) Below is an oscilloscope capture of such a signal. The vertical scale = Voltage level, and the horizontal scale from left to right = time elapsed. My annotations are in red. Note the yellow line, which indicates the signal.
Example Signal
Let’s say we have a device, for now, it’ll be the triggerscope 2, which is connected to the “TRIGGER” line of our camera. We want to signal the camera to capture immediately after we turn on an LED. In this case, we might use the triggerscope line 1 for the LED on and off, and line 2 for the camera signal.
In our software, we’d specify the camera exposure time, binning, and other usual parameters. In our controller, we’d set the output to high when we wanted to capture our image. Let’s use an example of 1 image:.
Tell the camera to acquire using software (nothing should happen yet!)
-Camera waits for a trigger signal
Tell our triggerscope to send a TTL high
-Camera captures an image.
Tell our triggerscope to send a TTL low
(nothing changes)
The above sequence can be used for a series of images. In that case, the acquisition would be configured for some # of frames (say 10), and the camera would capture 1 frame each time the signal swung from low to high.
To tie everything together, I’ve recorded  quick video explaining this on the scope. Check it out!
How fast can a typical imaging system really run? What factors affect this? While most research might not require rapid sequential capture, almost all experiments can benefiet from tight exposure/illumination timing, yet few microscope users are aware of timing delays injected into experiments by the control software they are using. So how much delay is caused by software? I’ve long wanted to look into this, and finally got the chance!
In order to accomplish this, I programmed a microcontroller to accept commands from the Lumencor Spectra-X. This high power/high speed LED engine is popular for fluorescent excitation, as a result, almost every available software application can control it. I set up my code to measure the time in which a known shutter, or known wavelength command was received on the serial line (well, USB serial).
The team at Technical instruments were kind enough to loan me the use of their equipment, so a generous thanks to Reese Allen and the entire Technical instrument staff!
Experimental conditions were as follows:
All software packages were installed on a clean OS install of Win7/x64
Test machine was a Dell Precision series
Tests were performed by configuring the camera to run ~100fps. This was accomplished using a 4×4 binning, and an exposure of 9.8mS.
All software applications were set to send at minimum one shutter and one wavelength change per acquisition cycle. (i.e. 1 picture from the camera, + 1 wavelength change + open shutter, then switch wavelength and cycle shutter).
All applications were first tested in a “free run” mode, to confirm the camera configuration was capable of 100fps at minimum.
No other devices were installed for these tests.
The results were quite interesting:
Software
NIS Elements V4.30
Micro-Manager V1.4.22
Metamorph V7.8.12
Metamorph Streaming
Average Overhead
60mS
71mS
65mS
11mS
Max FPS
16
14
15
90
Here are some videos showing each of the programs performing. You can see some timing variation (likely caused by my capture device) but the averages are easy to resolve, and they correlate to the speed seen on the captured image stacks.
NIS Elements
Micro-Manager
MetaMorph
What becomes quickly apparent is the influence of asynchronous device control using metamorph’s “Stream” function. This is making use of a patented computer-based device sequencing technique. This technique waits for a camera “event flag” (interrupt) to fire, and when it does, a pre-defined set of events occur which control devices. This is similar to what many external trigger devices do, such as a configured national instruments card , or a pulse oscillator, or the triggerscope. The key difference here is that an external card and/or device isn’t needed, this occurs within the PC itself.
Some further thoughts:
Micromanager is a great open-source competitor to other pay-for solutions. Good speed performance for a low up-front cost! (the cost of setting it up!)
It’s interesting that this never really seems to come up for 90% of customers who purchase “high speed” devices. Either the exposure times needed for acquisition of dim signal are so long as to never reach the speed thresholds shown here, or the capture frequency needed for a given study doesn’t require these speeds, or some other cause, but in my experience this is rarely a complaint on behalf of clients. Why isn’t this a bigger problem?
Many, many devices can’t reach the speeds shown here. In my tests I was using an LED driver. LED’s should have a switch time in the <5mS range. The slowest components usually found on a common automated microscope are the filter wheels and turrets, which usually run at the 50ms range when fully loaded with filters. (that’s on the fast side). So is this the reason the problem isn’t exposed more often?
By far, the slowest components you’ll find for microscopy are found on automated microscopes. Shutter open/close times on a common scope (i.e. big 4 name brand scope) can be in the upwards of 100ms! Moving a big filter turret usually takes 200mS. Again, this begs the question of how important speed really is for the common researcher….
For those who want the fastest possible speeds, have no fear! I’ll be announcing some major improvements to my triggerscope soon, to include programmable high speed sequencing capability!
Over 450 years have seen the compound microscope evolve into an incredible instrument. From simple contrast viewing, we’ve moved to super resolution systems capable of sub-diffraction accuracy. But for all this advancement, we’ve been stuck with the architecture of the microscope platform. Over 70% of labs end up buying both inverted and upright microscopes….until today!
I’m happy to introduce the Revolve Hybrid Microscope! A scope that combines both inverted and upright observation into one instrument. I’ve worked in this industry for over a decade, and this is by far the biggest evolution in the idea of a scope I’ve ever seen! I’m so excited to share this with the research community, and I hope you’ll enjoy learning more about it!
The Concept
We all know that uprights and inverts use similar objectives, illuminators, position systems and cameras. Why duplicate all of these expensive components? Can’t we find a way to merge these two systems into one unified instrument? Echo Labs has done just that, with the Revolve.
As you can see above, this a completely new way of approaching what a microscope should be! The revolve is two microscopes in one. It provides a fully capable inverted research instrument, AND a fully capable upright microscope, into one device. So the revolve is 2x the microscope at 1x the price, size and maintenance. While the revolve brings so much more capability, it’s also less, in all the ways less can be good. You’ll note the lack of a tower computer. It’s gone! The integrated iPad drives everything inside the scope. No more rat’s nest of wires on your lab bench. Of course, with the iPad, training isn’t required for new users. Both my boys (6 & 7yrs old) got a chance to use the fluorescent side of the app, and were capturing multichannel fluorescence with a few seconds. Using the touchscreen is simple and an easy extension to the other controls on the scope, so the keyboard and mouse won’t be missed. You may also note a lack of wires. In normal marketing pictures microscope manufactures won’t connect all the boxes, so things “look clean”, but the revolve IS clean. Only 1 power wire is required for operation, and all of the other brightfield, and fluorescent components are placed inside the body of the scope! To see these functions live, Echo Labs has produced a great website which includes demonstration videos, at www.echo-labs.comÂ
The Revolve Microscope
Let’s get right to the specs. This is a compound, infinity-path microscope. Current glass selection includes the entire line of Olympus objectives, with everything from long working distance phase to high NA oil immersion lenses. Both a high NA and long working distance transmitted light condenser are available to support phase and brightfield. The instrument uses the Apple iPad for control, interface, storage and display of images. The body and chassis are designed to be compact, fitting inside a fume hood or a lab bench with shelving installed. A single power supply runs all internal components, so there is only a single power connection required, and aside from that, no other cables need to be connected.
Brightfield
In brightfield mode the iPad camera is optically coupled to the lightpath, providing sharp, color balanced, auto exposed images. Optical coupling is set to provide the full objective field within the iPad camera FOV, with pinch+zoom available should the user want a traditional square view. On-screen controls are provided for color balance, brightness and contrast.
Due to the unique nature of the upright/inverted combination, both high res close working condensers as well as long working distance phase condensers are available. A high accuracy locking mechanism is used to securely hold the condenser assembly in place, while still allowing for easy removal by a single lever.
Fluorescence
In fluorescent mode the Revolve uses a high sensitivity quantitative monochrome camera, which is wirelessly connected to the iPad display, providing a no-latency view when scanning a fluorescent plate or slide. High power discrete LED’s provide fluorescence excitation. Fluorescent wavelengths are specified by “light cube”, with a cube and filter block set provided for popular fluorescent excitation/emission spectra.
The user interface for fluorescence includes enough control to be useful, but avoids the excessive detail usually found on older imaging programs. You won’t have to worry about parallel shift voltage settings, or pre/post sensor clearing options just to snap an image! On-image display scaling, a simple high/low gain selection, and attenuation control are easily controlled. The physical knob on the right side of the microscope controls the current fluorescent channel in parallel with the app, providing a bidirectional link for the user’s preferred mode of operation.
Mechanical
The body includes a USB hub which can accept common memory sticks, allowing for quick USB transfer of images obtained by the iPad app. The engineers spent an extensive amount of time working on the question of “what do we do with the images on the iPad?”. This is an important question for every microscope, yet it never seems to be completely addressed. How many times have you realized images you captured simply were lost, because someone deleted them from your scope’s hard drive! With the Revolve you can save your images almost anywhere: USB, AirDrop, DropBox, with more cloud options coming soon. Or, just take your iPad….and walk away!
The focus mechanism is duplicated for easy access in either upright or inverted configuration, providing a common coarse/fine knob adjustment. X/Y adjustment is provided by dual knob control placed close to the user’s hand. The stage is locked into the microscope with a secure, but easily removable cam lock and alignment pins. This makes removal of the stage easy for revolving the scope, but keeps the stage securely in place when in use. Check out this video for an example of each of the above functions!
Conclusion
During my time working with the Echo Labs team, I’ve seen the attention to detail put into the user-focused features of this microscope. My previous posts have hopefully allowed a sense of understanding as to why I am so excited to work with this team! This company is a completely new approach to an old industry. The management team consists of veterans in the microscopy industry with a passion for high end imaging. The engineers are an amazing cadre with broad experience in the music, medical, defense, and research industries. Watching this team put the combined skill and experience into this microscope has been simply incredible. I hope you get a chance to experience using this microscope, as you’ll never look at scopes the same way again! Of course, in keeping with the idea of a new-generation company, you don’t need to talk with anyone just to check out pricing, an online and interactive quote configurator is available, which allows you to easily build the revolve for your work.
I believe Echo labs is well on the way to completely changing the microscopy industry. The Revolve is only the first step in bringing easy to use, fair price research instruments into the lab. Welcome to the Revolution.
I guess I should continue providing lengthy explanations of this skunk works project, but I think it’s time to let this thing stand on it’s own merit. So…here’s a nifty countdown timer until the announcement goes up!
Every so often I will get the chance to cook up some code for one of the usual software scope control packages. But these days, I spend most of my time in programs like Fusion360, Fritzing, and the like. So I guess this is how I feel when using the typical scope control software. How about you?
I’ve used Elements, imagePro, MetaMorph, for years. It’s all good software. It’s powerful, and can do all sorts of crazy things that are super awesome. But what if I just want to obtain a 3 channel fluorescent image? Here’s what I end up looking at for a simple capture.
Screenshot from NIS Elements – Lots of options, but lots of buttons!
But this must be only one program, right? Surely other more capable systems have a current style of user interface, right? Here’s an example of the most capable (and IMHO fastest) software around, the venerable MetaMorph.
Another example of great software, Metamorph, that includes a large array of buttons….
For the typical person who just wants to snap an image, this is overkill. Now – I don’t intend to bash on these great software applications. For multi-dimensional timelapse, or other advanced work, there are few better tools available, and these do a great job of providing any feature one could want. But what percentage of labs have such complicated instruments, vs. the great number of simple, multichannel fluorescent microscopes around in every biology facility? How can it be that all these users must learn these behemoth software packages to capture some cultures? Other industries seem to have moved to workflow-based designs. You can find apps that perform full MIDI mixing on a tablet, heck, even airplanes have touchscreen controls now! Â Yet the microscopy industry seems to be locked into a slightly improved version of windows XP button mashup.
With all of the consumer products finding ways of simplifying control over hardware, isn’t there another way of doing this?
Good software design can perform a specified job, and provide users the controls they need to do that job, but excellent software does so without requiring any training. An example of this can be found below, showing a unified interface for shuttering, attenuation and capture. This is a very limited example of what’s coming up, but just from this clip I think the differences in design philosophy are evident.
Can this simple control drive shuttering, attenuation and capture?
I think Scotty wasn’t annoyed at the keyboard because he had to use it. He was annoyed because of what it represented: A barrier to getting his work done. Microscope time isn’t free time. There’s a lot of other, far more important work to be done than collecting the results of actual research. Cumbersome software results in less time to do the important stuff.
Complex software will always be required for highly complex jobs, but for everyone else, I’m happy to say that the days of “software training” will soon be behind us. My guess is that most people will have a hard time going back to click and hunt, after using the next generation of control software. I know my perspective has changed. I guess that’s why these days, when I sit down behind a button-crowded screen, all I can think of is Scotty cracking his fingers 🙂
Any user of a modern scope, or at least, a high end scope, considers the addition of new widgets. Whether it’s a better illuminator, adding a more sensitive camera, or even a motorized component, we think of scope upgrades,as…well…UP grades, yes? But in so many other areas of our lives, we’ve seen that less can really be more. How many of us still have a cable TV service, and a landline, and a fax line, and a fiber connection for internet? We’ve reduced all of these different services to a simpler and easier choice – wired of some type (Cable, fiber or whatnot) for high bandwidth, and wireless (cellular etc) for low bandwidth. I’d expect we’ll soon see the end of wired connections at all…. One could imagine the new apartment renter only 15 years ago, asking the question, “how many phone / fax lines do I need?” yet now, we find our less is better, is more. Less power consumption is better, yet 30 years ago every ad wasn’t about efficiency, it was about POWER. Now, don’t get me wrong, there’s nothing that stirs my soul like the sound of a Hellcat at the reno air races, pushing 500MPH and screaming out the gutteral sound of 2,000 horsepower only a few hundred feet away. Power is cool. But hey, I drive a hybrid. In many practical ways, less is more. It’s with this in mind that I ask, do you NEED that 30lb computer, sitting next to your scope? What if it wasn’t there? What could you do differently with your scope if, say, it was smaller, or lighter? What could you do with your scope if those boxes and wires hanging everywhere were….less?
Sometimes we don’t see the power in less. I love wrenching on scopes that, by all rights, may blow a breaker were everything fired up at the same time. I love the complexity, but I’m, for lack of a better term, an imaging gearhead. I love working on laser launches BECAUSE they might burn my fingers! If they can’t, it’s just not as cool for some reason. But at the end of things, I’m just a gearhead. Research is what matters, not my personal desire to melt fiber. Today more then ever we are presented with the challenge of better understanding our world, our environment, our universe, through basic research. We need affordable tools. We need tools that are easy to obtain, easy to use, easy to service. Tools that can be used for what they are. The research is the objective, but so many microscopy companies have fallen into the mistaken belief that the device they sell is the objective itself. A microscope is nothing more than 450 years of technological advancement beyond the hammer, but……it’s still a tool.
What if the tools you had in the lab weren’t aimed at being the most complex, all encompassing widget around, but instead were aimed at being useful for the intended purpose? Would less complexity equal more speed?
What if your microscope wan’t intended to have every cool new thing added to it? What if it were just made to have cool stuff, useful stuff, already inside? Would less wiring and add-on components equal more useable bench space? Or more potential places you could put the thing?
What if the scope you wanted to buy didn’t come with a 4 page list of crypto-part numbers, providing your sales rep, and you, an excellent way to miss detailed little parts on the system, only to figure that out when you receive the thing? Would less itemization result in more assurance in a simple, complete system?
What if you didn’t have to earn a dual Ph.D in computer science just to run esoteric software with a few hundred unintelligible icons scattered all over the place? Would less buttons equal more sanity?
You know, most of these questions aren’t new ones. There will always be gearheads who want a water cooled 220V laser simply because it needs water cooling, but in so many ways, the current microscope options available are built FOR the gearhead, and then reduced and passed to the regular researcher who just wants the tool to work.
I’ll always be a gearhead, but I’ve come to realize how powerful less can be, and it’s pretty freaking powerful….
One of the first known microscopes by Zacharias Jansen.
The foundational design of the compound microscope has, in many ways, remained locked in place for the 450 years of it’s existence. Combining an objective, eyepiece, and illuminator to provide a magnified view of a specimen has drastically improved. The illumination, staging, detection, optical design, and contrast methods have all evolved by leaps and bounds today. Super resolution will likely become the new normal in the next 10 years. But for all of this improvement, some of the basic limitations of the microscope remain locked in place. A further limitation is placed on development by the same names in the industry. Big name instrument manufacturers, with entrenched, slow, and top-heavy R&D divisions aren’t really set up to upset things, but instead to make small improvements over time. Sometimes it takes fresh eyes, a new way of seeing things, a new generation of builders, to break free of this incremental development cycle, to find a different way of solving the age old limitations we find in todays’ instruments.
Guess what this is 🙂
Over the past year, I’ve been privileged to find an amazing group of young entrepreneurs, who have tirelessly worked to bring a new microscope to life. I’ve been even more honored to play a small part along the way. I’ve been chomping at the bit to share the story of this group. I’ve watched as they struggled to solve extremely difficult problems. I’ve looked into eyes not seen since I was a young grunt in the infantry, the eyes of a man who hasn’t slept in days, fighting to figure out the answer to a seemingly unanswerable question. I’ve watched as we walked through success, through failure, through disappointment, watching as one man’s imagination was brought into reality, watching as each person contributed to make something bigger, better, than any of us could imagine.
On August 1, the company will announce it’s product, and I’ll be able to tell more of the story. For today, I just want to ask a series of questions, which I plan to further explore in the days leading up to release.
What can’t your microscope do?
Does the instrument you use today reflect an embodiment of available consumer-grade technology?
What is “ease of use”, how can such a phrase be quantified?
Is faster, bigger, heavier, stronger, always better?
Asking a tough question on mature markets and the voice of the customer.
I hope you’ll stay with me for the week. All I can promise is that will be worth the wait. I hope to at least provide an explanation on why I believe this group is on the right track, and why it’ll turn the microscopy industry on it’s head…..literally.
Here’s a simple method to deal with varying intensities in a large run of images, assuming you have a known number of cells in a field. If I assume I should see, say, more than X cells per area, I can adjust my threshold to slowly increase it’s aggressiveness until that minimum counted value is met. The code can be downloaded as an imageJ macro here. but here’s a walk through of this simple script. My comments are preceded by a *.:
rnum = 0; *this creates a new variable, or container, for our # of regions
tval = 2000; *this is the starting object threshold value.
while (rnum<500) *this is a "while" loop. The loop runs until the argument is satisfied.
{
setAutoThreshold("Default");
setThreshold(tval, 4095); *note that tval = the minimum threshold value
run("Analyze Particles...", "size=0-Infinity circularity=0.00-1.00 show=Nothing display clear add");
rnum = roiManager("count");
tval = tval - 100; *tval continues to become more aggressive in counting objects until the argument is met.
}
Simple routines like these can reduce your headache factor by a large margin if you learn how to harness the power of macros!
TIRF is a widely used tool for effectively creating a super resolution instrument out of a standard widefield microscope. The cool part of TIRF is that it bypasses the limits of axial resolution on the scope not by altering the optics per-se, but by altering what gets excited. In a normal widefield fluorescent microscope, photons are emitted from an arc lamp, LED, or other source that are of numerous polarization, phase and angular states. In effect you shower the specimen by flooding light through the objective, exciting any fluorescent molecule in the path of the light. The problem is that what gets excited may be inside, below or out of the objective’s focal plane. Any image you collect from such a setup represents both the “signal” (molecules in focus) and “noise” (molecules that aren’t in focus) in the field of view. More importantly, It’s common to have out of focus noise overpower in-focus signal, thereby reducing or restricting what information can be obtained from the microscope. My fancy cartoon below shows an example of a widefield system and it’s illumination path.
Typical fluorescent excitation
Of course, one simple solution, but an expensive one, to this problem is to add a pinhole-based confocal to the optical system. While a confocal does reduce or eliminate the out of focus haze in the image, it doesn’t provide any increase in Z accuracy. This axial resolution limit is still based on the objective’s performance.
TIRF avoids this limit by restricting the excitation field. In order to accomplish this, TIRF uses a laser, coupled to the microscope and most commonly delivered through the objective, to deliver excitation energy into the specimen. The trick is not that the laser excites the fluorescent molecules, the trick is that the laser, if set at a proper angle (critical angle) will bounce off of the bottom of a specimen container. When the laser bounces back there is a small electromagnetic field created, just on the specimen-side of the coverslip. This EM field is the same frequency as the bounced laser light, allowing it to excite fluorescent molecules. The field has an exponential decay as it extends from the coverslip, so it’s only strong enough to excite molecules that are very close to the interface of the coverslip and the specimen (usually the excitation field in TIRF is ~100nm of depth).
TIRF Cartoon example. Note the steep approach angle for the beam, which can only be used with high NA lenses (~1.49)
It’s hard to believe that light could be reflected back into a medium simply because of a refractive index change, but I was able to visually capture this effect on a large scale when installing a customer’s system at UC Davis. The image below is of a fluid filled jar, with some bits of glass tube floating in it. The cool part is the laser beam. You can see the beam emitting from the objective and passing through the fluid. When the beam reaches the air (lower refractive index), the refractive index change is at or beyond the critical angle the light can follow, so the light is reflected back into the previous, higher refractive index medium (the fluid). This is the fundamental effect that makes TIRF possible.
Laser emits from lens, but is trapped in fluid and reflected back down.
The limitations to this technique are that you need a laser, you need an optic that can reach the critical angle, and you need to have something in your research that can sit on the coverslip, i.e. this won’t work for anything farther than ~100nm from the coverslip. If everything lines up, you get an image representing roughly a 10x improvement in Z resolution over a confocal microscope, qualifying this as a “super resolution” technique.
If you’d like to learn more about the theory of TIRF microscopy I highly recommend the MicroscopyU explanation, which does a better job of explaining the physics than my overview here.
***Thanks to Professor Jawdat Al-Bassam of UC Davis for allowing me to demonstrate this effect on his instrument!
Andor’s Neo camera has been out for about a year now. I first wrote about using it back in February. The camera immediately showed the major difference in field of view between sCmos sensors and traditional interlines quite well, but the speed, binning, sub-array and bias clamp controls weren’t available. Over the past few firmware/SDK updates, Andor has steadily addressed these limitations. I had a chance to field test the latest combo of firmware and driver SDK. Basically I found a camera that runs at 100fps with roughly four times the field of view of a typical Sony285 type sensor. In my last post I noted that people would wait until end user software platforms could take advantage of the sCmos architecture. I think the wait might be over.
httpv://www.youtube.com/watch?v=RjBv9ChZknk
I’ll continue to report on this as I have an opportunity to put it through it’s paces. I’m curious to see how people respond to it’s performance…
Over the course of my career, I’ve flown across the US to attend a variety of imaging related instructional courses, all on the eastern seaboard. We are all familiar with the Woods Hole courses, the AQLM course, and so forth. What has boggled my mind is that here I am in the SF Bay area, surrounded by arguably the most densely populated and diverse centers of biological science innovation on earth, and there’s no major course?
I started pestering the owners of Technical Instruments about this one night, and they admitted the same question had long been on their minds. A few months later, we asked the NIC team at UCSF about it, and found to our surprise that they were already cooking something up. We offered to help as needed, and have been doing what we can to help support the course ever since.
If you are a regular reader of my blog, I’ll assume you’re involved enough in imaging that you would stand to gain much from attending the course. This won’t be a “sit in classes all day” kind of deal. The course has been designed around a kinesthetic approach, with lectures taking roughly 50% of the time, and the other 50% being hands-on bench work.
89 North has asked me to cook up a driver for the PhotoFluor 2, which I am now hosting under my Macros & Journals page, or you can use this link. This driver works in NIS Elements AR, and BR with Advanced Interpreter.
Please contact me if you have any trouble using it!
I’ve written a Lumencor Spectra Driver that works with NIS Elements B3.2+. (AR or BR + Advanced Interpreter) The driver provides integrated control into elements for both intensity of lines and channel. These appear as a shutter and two filter wheels, offering full integration into Elements’ Optical configurations. We are selling it for $750, including online install on your system. Please contact me if you’d like to try it out!
Walking through the alignment procedure of the DV 2, narrated by Jeff Harris from Photometrics . Great video and great explanation Jeff! For those of you who have a DualView, I’m certain this video will come in handy.
There are two docs published by Nikon regarding the use of oils. Most people think you can use any old oil for any microscope, but this is not the case. Zeiss, Leica, Nikon and Olympus all have specific chromatic, spherical and viscosity values, which are matched to work with the corrective properties of the lenses.
Oil comparison performed by Nikon Gurus
Thus one should keep Leica oil on Leica lenses, Nikon oil on Nikon lenses and so forth. Below are two comprehensive documents that describe which oils to use with Nikon systems, one that specifically covers the perfect focus. I’ve clipped a revealing example of how much oil can affect the performance of your instrument. Note the massive chromatic shift in Z when using non-specific oil on a 100x lens (btw this should be “Cargille”…guess I’m not the only one with typos –Â heh):
I’ve uploaded the two reference docs that go into specifics to my files and info page, or you can get them here:
Windows 7 has a new power management interface, and the design of modern motherboards, along with win7 Automatic Power Management (APM), can cause some insidious device-based software crashes if power options aren’t set correctly. Here’s how to make sure your system is running in the best mode.
Open the Windows Control panel
Search for and open “Power Options”
You’ll see a “power plan” like the one below. Click the “Show Additional Plans” button, and check the radio button for the “High Performance” plan.
Click on the “Change Plan Settings” link, then click “change advanced power settings”. This window is where the real control over the power plan exists.
Browse through each item provided, and set power management to either disabled, maximum performance, or off, depending on the item. Things like turning off the hard drives are a bad idea on an acquisition system, as this can cause problems with long term timelapse.
One key item is the PCI link state power management. This should be set to “Off”. This item selectively disables power to the PCI bus. Some of the cameras and devices used in acquisition systems rely on PCI / PciE based cards to operate, and cutting power to these cards can cause all sorts of havoc.
Another common problem, probably the biggest I’ve seen, is the USB suspend entry. Again this function turns off the USB bus when not in use. This has caused crashes in software when operating USB devices, so it should be set to “Disabled”.
Once you’ve investigated each item and made any desired changes, click OK, and then close out of the power option windows.
Making the changes noted above should make a big impact on the stability of your imaging system. Hopefully this guide helps you to avoid acquisition crashes on your system!
I posted briefly about this a while back, in my explanation of the demise of the parallel port. I finally got my hands on a National Instruments USB-6008 box, and have found the unit to be 100% compatible with any NI-DAQ controlling application. What follows is an example of how to set this up using NIS Elements.
First – what types of devices could be controlled with this?
AOTF Controllers (Spectral, Neos, Etc.)
Single Shutters (Uniblitz Etc)
Piezo Z Controllers (ASI, MCL)
Directly driven 0-5v lasers
Monochromaters
LED Light Engines (Lumencor/89 North)
So – this thing is great when compared to the PCI cards normally used for numerous reasons:
Cost – This is ~$160 USD, vs. the PCI/PCIe cards running $1,500+
Size – This box is quite small, doesn’t require a huge cable or breakout box, and doesn’t occupy a slot on the PC
Portable – this will work on a mini form factor PC, or a laptop (which is what I used for this post), instead of requiring a full size tower
Installation:
Install is a snap. The only hard thing to do is place the pin indication stickers in the correct orientation on the box. If you mess this up you can always refer to the manual, but the little sticker strips included with the box make field-modification much easier. After this run the NI installer and reboot. Finally, connect the USB cable. The box will appear, and if you like, you can launch one of the NI test apps to run the device.
In Elements, we’ll re-run the installer and check the “National instruments” hardware group, then select “NI DAQ”.
Next, open Elements, open the devices window, and add the NI Device.
Once we add the device, and click configure, we can define what each of the inputs and outputs do. Here I’ve selected a 2 line AOTF.
After you’ve dialed in the settings you want, close the window(s) and if you don’t see the new device control panel, right click and select “acquisition controls” to find it. In my case the AOTF Settings window popped up after closing the device window.
I also made a short guide to using this box in elements via the Youtube video below. ***In the video I mention this thing costs $130, which is incorrect. It’s listed on the NI site as $168 as of today. Also I have yet to test more than 1 unit, but will be doing so shortly…
httpv://www.youtube.com/watch?v=poKZjRgWdhQ
One point of discussion is devices that require a 0-10v range. I’m looking into using a small powered op-amp chip and maybe packaging it into a small box for sale, or at least I’ll write up a how-to on using one, for people that need this voltage range. Luckily most of the devices out there can accept 5v or 10v input. Overall I’m happy to see NI is moving beyond PCI and PCIe bus cards. USB based boxes like these lower the acquisition cost for the customer, provide everything we need in the life science world concerning speed, and open up access to a wide array of devices. I’m looking forward to using this box on future systems!
Nikon has redesigned and updates it’s classic C1 Confocal. The C1 was a great core facility confocal as it was a stable and reliable product. The one thing that held it back was the lack of access to NIS Elements software. With the release of the C2 this problem has been solved. So, along with a comprehensive timelapse multi site capability, users can also stitch using a high power objective, on multiple sites and with multiple channels.
Example showing large image capability
The C2 also brings spectral options and DIC options to the package, and it looks like Nikon is offering it at a very competitive price. It will be interesting to see how the access to Elements will affect the market performance of this instrument. I’m thinking it’ll do well.
When acquiring images in NIS Elements, it is possible to acquire each image with a color over that is approximate to the wavelength used for acquisition. This is the easiest way to view and relate multi-channel data, but when converting the images for export to other programs, it cam become a problem.
In the image below I have two channels acquired – DAPI and FITC. If you note the overlay each of the two channels are completely independent.
Original 2 Channel Image
If I now use the Image -> Convert -> Convert to RGB command, some of the green and blue signal will be mixed within each other, and some of each channel will mix into the red channel:
Image Converted to RGB and now color mixed
So what causes this? If you note the channel data in the original image, the color is not a pure green. The 488nm and 405nm alexa dyes actually emit in blue-green hues and green-orange hues. So if we were to attempt to measure signal intensities in the RGB image using another program, our measurements would be invalid. In order to avoid this, you can modify and use this macro, which will force the channels to pure Blue, Green and Red colors. With these color settings no color mixing will occur in the conversion.
For microscopy engineers and suppliers like me, the background produced in an image is of importance, if that background is created by the imaging system itself. What so many of us on this side of the scope (vs the user side) forget to train on and talk about is the specimen side of the background. So, how can you figure out where the background is coming from in your image?
Example of two noise sources, one distributed and the other localized.
Background Sources
Usually background sources mentioned are those produced by the imaging system. I’d like to cover instead those produced by the specimen and the imaging system. These can be:
Objective and other optical limitations (aberration/convolution etc.)
Camera measurement noise (A/D Read Noise)
Camera-produced background noise (heat that the camera sees as signal)
Specimen Background Sources
Auto-fluorescence produced by cells and/or media
Refraction/reflection of excitation light by specimen container (think plastic multi-well plate)
Ambient light in the room where the microscope is located
Byproduct remaining from sample preparation protocol (stain that remains after a rinse, etc.)
Identification and Removal
Instead of reviewing each noise source type and it’s cause, let’s instead perform a gross identification scheme to determine where the greatest noise source exists.
Method 1: System vs. Specimen Measurement
Configure your microscope and imaging system, and place your specimen on the scope.
Focus on your specimen.
Acquire 2-3 images of various locations of the specimen.
Remove the specimen from the microscope entirely, not changing the scope at all (don’t drop the focus or change filters).
Acquire 2-3 images with the scope in this state.
Using either the ROI tools, or other tools available in your software of choice, measure the average intensity value, for the non-exciting area of your 2-3 specimen-present images, that we acquired in step 3.
Measure the entire image field of the 2-3 images we acquired in step 5.
We now have an intensity measurement of the amount of background for the imaging system + specimen, and the imaging system only.
If the image intensities remain relatively unchanged between the two image groups, the source of noise is in the imaging system.
If the intensities from the images in step 5 are significantly lower than those from step 3, noise is being produced by the specimen, or specimen container.
In my next post I’ll cover how to further pinpoint noise sources, and how to eliminate them.
I recently slogged through a tough install with a customer (Thanks Sean!!) in an attempt to run an older Photometrics camera using a PCI card in windows 7/xp. What we found was that the latest driver, (as of this post v. 2,7,10) is not working with PVDS cards! The solution for now is to roll back one version earlier, then things should be OK.
You can basically tell if you’ll be affected by looking at the cable attached to your camera. an LVDS cable looks like this:
After a few requests for a report on x64 capability using this tethering technique, I tried installing the NK Remote software on a rather fresh copy of Windows 7×64 – it did not communicate with the camera at all. On the other hand, the open source application “DigiCamControl“, worked perfectly. Now, as far as micro-manager goes, there isn’t a connection from DigiCamControl to micro-manager. So – short term solution is to run the proposed setup below on an x32 system – if anyone gets x64 working please post here!!! -Austin
So I had a customer interested in a macro application for scanning. In some cases, applications like these don’t require a telecentric optical system such as the AZ-100, nor do they fit into a typical stereo microscope application. So, what to do? Well, when your object for imaging is too large for a microscope, one option may be a macro lens on a digital camera. Something like this:
Copy stand for SLR
This type of system will allow for a stable focusing and zoom setup, but is limited in magnification power and light control. In some cases though, this is the way to go.
The next problem is that of focal and positional control. No microscopy device control software available, such as elements or metamorph, supports non-scientific grade cameras. So, how do we position a specimen under this camera in an automated fashion? Well, it turns out micro-manager has a plugin available for such work, and it’s available in the nightly builds here .
What you end up with is a camera that can be controlled from a laptop, along with a stage, z motor and other automated imaging hardware. Here’s an example of my setup running on a windows 7 x64 laptop and a D80.
D80 and Win7 x64
Requirements
This setup requires the following items:
1. Suitable camera and lens (see below)
2. Copystand for camera.
3. Computer for control.
4. External power supply for camera or a battery extension (more for ease of use but I consider it a requirement).
5. USB Cable to camera.
6. Control Software for interfacing to uManager. This is pay-for software, found here. The software currently ranges from $90-180.
So, follow the instructions found here for setup. My summary of setup, which took a whopping 5 minutes (this is amazingly quick for open source software!) went as follows:
1. Download Breeze Systems trial for NKRemote & Install.
2. Download nightly uManager build & Install.
3. Set camera to PTP mode.
4. Run NKRemote and snap image to confirm communication with camera.
5. Run uManager. Edit hardware seup using the “Setup hardware” tool and removed the demo camera, and added the NikonDSLR.
6. started snapping images! Here is an example of running in uManager (click on the picture for a higher res version):
Nikon D-80 acquiring images in uManager
What cameras are supported:
All of the basic DSLR models from Nikon are included with the NKRemote software. Tethercam is the driver used for the uManager side, and it states an “incomplete list” of cameras as follows:
Some lenses that work as portrait lenses like a Nikon 50mm will work adequately as macro lenses. On the other hand, it’s recommended to shell out the extra cash for a well-made macro lens, such as this one. The nice thing about working with commonly available photo gear is that in many cases you can even rent lenses to see what works best for the job at hand.
This was a fun and quick test and I am excited to see how this capability is used for macro work in the future!
As anyone who is running NIS Elements v 3.20 + is aware, the ASI stage controllers of any MS-2000 series will not be usable once elements issues a connection command to the controller. Elements will control the stage properly, however the user will not be able to position the stage using the manual joystick. So, the guys at ASI have found the cause of the problem and have issued a series of firmware updates that will fix a variety of controllers.
***DISCLAIMER*** –
These files MUST be used on the correct controller. If you use the wrong file it could cause some serious problems for your controller/stage/software. So, call ASI or shoot me an email if you have any doubt whatsoever about this. Basically, if you had any custom option on your stage (inverted axis, non-standard lead-screw or other modification) CALL ASI and give them your serial # so they can make sure everything will work for you.
Or, better yet: make your direct or dealer rep do the patch. That way they can take the blame if they screw things up 🙂
– Restart the computer and restart the ASI controller. This just makes sure no other program is controlling the stage.
– Run the Updater5, and select the appropriate firmware files.
– apply the update.
– You should see a “firmware update complete” message when the update has finished.
– Power-cycle the controller, and check that the system responds to joystick commands etc. Check that the firmware version on stage start-up reads “8.8f”. This indicates that the patch was successfully applied.
– Run Elements – everything should work now, including the joystick.
Thanks to Gary at ASI for working on this and for providing a solution so quickly! This customer support is why you are on my Blogroll as the “favorite stage”! – Austin
I had this question asked at my recent Bio-X training, and wanted to provide a thorough answer to it, as this is a common need:
Assume we have multiple images, or multiple stacks, and we want to analyze something like the intensity through stack A vs. stack B. We don’t want to convert the stacks into an RGB image, as this loses flexibility and image resolution. Here is a step by step example of how to do this:
Create a Demo Image Set
We will be working in Fiji, which can be found here.
Select File -> Open Samples -> Fly Brain. This will give us a 2 channel stack image, that we will convert into an example image to import.
Select Image menu -> Color -> Split Channels. We now have Red, Green and Blue channels. Close the Blue channel as no blue data exists.
We are left with two z stacks of red and green, that are 8 bit. To prove our processing later on works in 16bit, let’s convert the two images into 16 bit and adjust their intensity values.
Step 4 - Two Channels Open
Select the Green image, then Image menu -> Type – 16 Bit. Next we will increase the numeric value of the image data, so that we can later confirm our measurement results are in 16 bit space.
Select Process -> Math -> Multiply – set the value to 255 and click OK. When prompted click OK to process all images in the stack.
Repeat this process for the Red image stack. Save both files as Green and Red.
We now have two stack files that are ready to be used as examples for color merging. Hopefully this also shows some use of the math and bit depth controls.
Merge Multiple Channels
Open both Red and Green stacks if you have closed them.
Click Image -> Color -> Merge Channels. You now have a multi-Z data set, with two 16 bit channels. The top slider controls the channel that you are working on or are measuring on.
Merged Two Channel Image
Note the intensity values reported in the main toolbar reflect the active channel selected.
Image 5-D
You may note that in the image stack we have created, the threshold tool doesn’t work. In order to threshold this image we will move it into the “Image 5-D” plugin. (included with Fiji)
With the Merged stack set open, select the  Plugins Menu -> Image5D -> Stack to Image5D. Note the resulting image can be altered for each color and scaled for display.
Image 5-D Convert Window
There is now a active-channel selection radio button, available on the right side of the 5D stack. Click on the channel you’d like to threshold, then try adjusting the threshold for the image.
Note you can adjust the threshold for channel A and then adjust for channel B.
Try an analysis using Analyze Particles. Note that the measurements provided for the active channel are in 16 bit values!
Analyze particles on Image5D Stack
Multi-channel visualization tools available in Fiji do require a few more operations than commercially available software, however the method used in Fiji is very straightforward. One can imagine the ease of script development using these tools, which is the optimum means of analysis. I am planning to explore other tools available in Fiji as time allows, and I encourage anyone using this version of imagej to do some thorough exploring of the menus, as it is extremely comprehensive and well thought out!
Due to it’s utility and ease of providing common tools in a variety of operating system choices, I’ll be running this course using the FIJI build of ImageJ. You can find Fiji here.
I’m very excited to announce Technical Instruments’Â first ever camera-specific calibration product! At left is an image of the CC-560, or Camera calibrator, which emits photons at 560nm.
All of the cameras in the microscopy world rely upon a single defining point that (theoretically) justifies the price premium we pay for them. That defining point is reproducible, quantitative linearity of intensity. Put simply, if I acquire some number of images, using the same camera settings, over a long period of time (days, weeks months etc), my camera should give me image intensities with the same brightness values, when provided the same amount of input light.
Note that linearity, as well as quantitative accuracy, are expected minimum specifications on camera info sheets. Some examples can be found here:
How does this quantitative/linear capability translate into real-world data for the end user? With difficulty! Upon purchasing a new camera, most new owners of a quality camera will find a unique set of conversion tables for their serial number camera. These tables note in part the individual electron to ADU conversion rates for each A/D readout speed, and Gain setting. This data can be used to discover how many electrons the camera collected for a given image. For instance one could have a 3.05 e/adu conversion rate at 10Mhz, as well as a 2.54x gain setting. Using these conversions we would find that a reading of intensity in an image, say with a value in the image of 100, would convert to ((100/adu)/gain) or (100/3.05)/2.54 = 12.9 electrons collected. Note this basic calculation does not include bias offset, which is another value provided in each camera’s specification sheet. The take home point here is that each camera manufactured will have it’s own, unique, set of values. You can use these values to convert your ADU’s (Analog to Digital Units) which are what you see as intensities in your image, back into electrons collected on the camera. This is especially useful when comparing images collected between multiple gain settings, or where camera speeds were changed and a corresponding change in image intensities was noted.
For some people, these series of calculations and corrections, as well as others, may be a required, however tedious, fact of life. On the other hand there are a vast majority of researchers who want answers to simpler questions, such as:
What is the comparative sensitivity difference between the camera I used to work with in the lab down the hall, and this new camera I am now using?
How do changes to my camera, such as binning, really affect my data?
What do the detailed controls do on my camera? (Such as changing “Clearing” settings, or Parallel Clock speeds and voltages)
Customers using EM-CCD cameras may have additional questions, such as:
Is the EM Gain slider on my camera (specifically first and second generation EM-CCD Cameras) linear?
The simplest answer for these questions was to have the customer find some photo-stable light source, and use that light source on the microscope to find the answer. The tough part was actually finding a useful photo stable source! As an answer to these challenges, a few colleagues and I have created this product, available from my employer, Technical Instruments. This calibrator is designed to give people simple and direct answers to these types of intensity based questions, with a minimum of hassle or headache. The product will work on ANY C-Mount camera, and can be used either for dedicated-use calibrations (i.e. install – calibrate – remove) or can be placed in between the camera and the microscope in a permanent installation (See example image of Andor camera and Nikon Ti-E microscope with CC-560 installed at right). I’ll be posting more soon on what this product can be used for, however I’m certain there are numerous uses for this that we haven’t considered. Please take a look at our new CC-560 Camera Calibrator.
Most of the time, I work with people who need measurable and accurate (quantitative) results from the images they obtain. While this constitutes the bulk of my daily job, there is a much larger group of people using cameras on microscopes for photomicrography. Or put simply – taking a picture of what you see through the scope for the
purposes of presentation or archiving.These people live in an a space completely underserved by the microscope imaging community. Here’s why:
C-Mount cameras are usually cooled (which isn’t needed for basic trnasmitted image capture)
C-Mount cameras are usually specified to be linear and quantitative (again unneeded for basic image capture)
The software used by these cameras has more features than are needed to snap an image
The production rates for these cameras and cost of scale equal an unjustified high price for basic image capture.
Now when I use the term C-Mount camera I’m referring to cameras that are dedicated for microscopy use, which have no glass lens assembly in front of them, and connect to the scope using an ISO C-Mmount.
So what options are available besides the traditional camera? Enter the venerable SLR. Before the digital camera era, everyone had a film body camera on their microscope if they were acquiring images. This system could produce beautiful images, but required all the usual steps in film photography. There are a wide array of mounts available for cameras such as these, as most miroscope optical systems were designed to work with SLR cameras at some level.
Today there are a wide array of DSLR (Digital Single Lens Reflex) cameras available. These cameras use similar (but not exactly the same) optical setups that will, in most cases, work quite well on a microscope. Why is this?
In general, we don’t want glass to be in front of the sensor. DSLR cameras, like C mount cameras, use a lens connection, allowing for the lens to be removed, and the camera to be placed on the microscope. So the microscope becomes the “lens”.
Modern DSLR cameras like the D90 from Nikon or the 50D from Canon include a “live view” function. This function allows the user to view the live image from either the camera body, or the method I recommend, through remote control software running on a computer.
DSLR cameras include incredibly powerful image processors in-camera. This means you won’t ahve to fiddle as much with white balance to get a high quality image that accurately depicts what you see under the scope. Pathologists are famous for seeing these color balance errors as they have the best trained eyes when it comes to minute color change on stained specimens.
Unlike film or digital still cameras, newer DSLR cameras can acquire HD live video. Here’s an example from a Nikon camera. One can imagine the use of this for live microscopy recording, all with the same camera…
So why not use any handheld camera? Because the typical lower cost camera will not allow you to remove the lens assembly. This is the primary benefiet of DSLR and C mount systems.
If you’ve read this far and are interested in using a DSLR for your microscope, you’ll need a short shopping list of components on order to make everything properly work. Here are the components you’ll need:
Camera Body (you don’t have to buy the camera with a lens if this will be a dedicated scope camera)
Camera -> Microscope Mount system.
You can use either a microscope manufacturer’s recommended system like the ones described here (Nikon F Mount options)
Once you have the parts together, you’ll need to set the camera up for the right shooting mode. On most cameras you’ll see an “M” mode. This is preferred for microscope use. M is full manual mode. There is no focus or aperture for the camera to control on the microscope, so those adjustments are not required. The remaining adjustments (exposure time and ISO) can be controlled manually. For low light fluorescent work, use high ISO. For normal transmitted light, use low ISO. You can use the metering system on the camera to determine the exposure time, and all of these settings can be controlled from the computer. Finally, make sure to enable “exposure delay” mode, which will avoid capturing any vibrations introduced into the scope when the shutter drops open!
As always if you have any questions on this technique you can contact me with questions!
A few of my work colleagues pointed out the well presented positive aspects of larger pixel, back thinned sensors. If anyone has heard me prattle on about this they’ll find it familiar! Fast forward to minute 1:00 for the points on the new iPhone 4 camera. Steve’s explanation is dead on!
This is useful for large screens that are using very high resolutions. Sometimes the icons are so small as to be hard to click and unreadable.
Open Elements and select the View Menu, then Layout, then open the “Layout Manager”.
Next select the “Toolbar” Section on the Left (Step 1), then Select the “Large Icons” Checkbox on the right (Step 2). Finally to set this as the default option for all modes right-click the Toolbar icon and select “Apply to All”. Finally save the setting to a file “Step 3”. You should now have large icons!
The Auto-calibration feature in Elements (beginning in version 3.1) is a great thing to have. You simply choose the objective you are working with, the software knows what camera you are running, so the calibration is calculated for you when you add the objective list.
While this is great for the compound scope systems out there, it’s not helpful for a variable-zooming stereo or macro system. Low and behold there is a nice feature in Elements to handle this, and here’s how it works:
The feature is called “Zoom Configuration” and can be found under the “Devices” menu.
To use this feature you’ll need to have at least one objective declared under the Calibration menu ->objectives list.
When you open the window you’ll see a configuration screen for the zoom optics. Configure as follows:
Also – If you want the Zoom to appear in the live window you’ll need to check the “Zoom is located in the camera light path” checkbox.
With this complete you can close the config window. You’ll see a new tool available on the toolbar which allows you to enter the zoom value before acquiring an image:
Note that for this to work you’ll need to have selected the main objective as active before you’ll see a calibration value in your acquired image.
This is an extremely useful tool and one I’m glad to see is available in Elements D up to AR!
Thanks to everyone who attended my first course on image processing (and acquisition as it turns out). As promised here are some links:
– MBF Tools for ImageJ. This is a suite of processing and analysis tools, provided by the McMaster Biophotonics Facility, which can be used for, among other things, splitting apart pigment based IHC specimen images.
***One of the course attendees asked how to merge multiple channels, without using primary color (RGB) values. For instance one may want to select “Red, Cyan and Green”. This function is available using the MBF Tools function found under Plug-Ins, then under the “Colour Functions” and finally using the “Colour Merge” command.
– This tool only merges two channels. It is possible to run this multiple times, so a merge can have a third channel added to it, and so-on.
– LOCI Formats Importer/Exporter is a tool which can be used to read and write many file types. It can also import metadata from many of the proprietary file types used today.
– I was not able to find a dynamic and interactive particle counting plugin for ImageJ, however when you have produced a count by selecting “Analyze particles” you can select the “Distribution” command from the Analyze menu to produce histogram distributions for every measurement type selected.
Please contact me if you have any other items you’d like me to cover. Thanks again!
If you want to merge multiple channels in Elements here is a quick solution:
1. Make sure you have the channel tabs activated. Do this by clicking the Edit menu, then Options. Select the “Appearance” settings. Make sure the “Show Channel Tabs” box is checked.
2. open or acquire the first image. In the lower left corner of the image is the channel tab. Right click this tab. If you have not configured color configurations for your system (optical configurations) you will need to select properties, then change the channel name and color to suit the properties of that image.
3. Right-Click again and select “Copy Selected Channel”.
4. Choose or acquire a second image. Again if needed on this new image set the channel properties (step 2). Place the cursor over the image and right-click. Select “Paste as New Channel”.
5. You should now have a single image with two channel tabs. You can now save this image to produce a composite. You can also repeat these steps and add more channels to the current merge. There is no realistic limit on the number of channels one image can have.
If there is one simple question I get during training sessions it’s “How do I save an image, so that I can open it using Photoshop or other programs, and have the image appear exactly as it does in Elements?”. The reasons for this are complicated. Suffice it to say if all you want is an image for display, and NOT for analysis, here’s what to do:
1. Set the image Zoom to 100%. You can also do this by pressing the 1:1 button in the upper right corner of the image window.
2. Under the Edit Menu, select “Create Full View Snapshot”.
3. Save the resulting image.
4. If you want to add a calibration mark to the image, make sure you set the position and size of the mark AFTER the image has been zoomed to 100%.
Why is the 100% zoom required? Saving”As displayed” refers to basically capturing a screen shot and copying it to a new document. While this preserves all overlays, calibration marks, regions and thresholds, it also includes the current magnification. So if you create the snapshot when the image is at 50% zoom, you will end up with a new image at 1/2 of the resolution of the original image. Conversely you can create a full view snapshot of an image at 200%, and the system will double the resolution in the resulting image.
In case you’d like to print this I’ve included a PDF version here.
Here is the latest build for Elements if you are running a Lumenera camera. Follow the directions explicitly if you plan on updating or installing without a local rep available!
Laura Sysko from Nikon sent us a How-To on outputting live graphs that can be included with a timelapse or other ND file. You can find the How To Here.
Believe it or not researchers at UC Berkeley found a way to make it happen. It uses a simple excitation diode and emission system, coupled to a cellphone’s camera.
microscopes around in every biology facility? How can it be that all these users must learn these behemoth software packages to capture some cultures? Other industries seem to have moved to workflow-based designs. You can find apps that perform full MIDI mixing on a tablet, heck, even airplanes have touchscreen controls now! Â Yet the microscopy industry seems to be locked into a slightly improved version of windows XP button mashup.
