Recently Tucsen had a presentation on their new Libra CMOS cameras which includes some nice improvements in sensor performance.
Key highlights were:
Improved QE at 92%
Large Full Well Capacity (48,000 e)
Native binning / debin system
In a recent presentation here a series of improvements were listed, among them a smaller pixel size selection option which supports a drop in pixel size to ~ 3.27um. From the explanation I assume the camera runs natively in a binned mode to produce 7.x um pixels, but can be optionally de-binned.
Each of the cameras (3 models announced) appear to have this feature, as shown below.
I’m more interested in the full well, as having a ~50k e full well seems a nice improvement in dynamic range over current options. Once item that was very interesting to me is that the full well capacity does NOT reduce when enabling the higher resolution mode. If this is correct, a 3.x um pixel with a 48ke full well would still be quite a nice bump in range and quite a feat!
For pricing, the presentation noted 0.4x the cost of a “classic CMOS”. I’m curious what that means in an actual dollar amount.
I’ve been working with Pt Grey (now FLIR) cameras for a few years now, and one nagging question in the back of my mind has been whether or not it is possible to cool them via external means. There are basically 3 categories of cooling when it comes to low light cameras:
no cooling whatsoever
cooling to + or – some delta from ambient
fixed-temperature cooling to some set point.
Of these methods, the fixed temp cooling is the most difficult to achieve, for a number of reasons, and I don’t believe these types of cameras would benefit greatly from such an attempt, so my idea was to simply answer the question, “Will cooling improve the performance of these cameras in a measurable way?”
Normally, the cooling of a sensor takes place as close as possible to the back side of the sensor itself. In this case, I started with an already-encased camera, the Pt Grey Chameleon 3. Because I didn’t have access to the sensor back directly, I figured making contact with the case, at as much surface as possible, would suffice. In order to do that I simply used 2 l channel strips of aluminum, and placed thermal tape on all 3 sides touching the camera. The graphic below shows what was added after the aluminum wrapper. First, a peltier (thermo-electric heat exchanger) was coupled to the aluminum. From there, a large CPU heat sink/cooler was attached to the hot side of the TEC. The general idea here is that the aluminum should absorb the heat from the camera. The peltier moves a lot of that heat to the CPU cooler, which radiates the heat into the air. This is a rather rudimentary setup, but it works for the proof of concept.
Below is an image of the assembled system. To provide the required 5V and 12V power to the cooler and peltier, I used my old reliable ATX Power supply. This assembly low cost components from Amazon and the like. I think the total cost was $27 or so. Maybe more including thermal tape.
So how did the system perform?
First I allowed the stack to heat up for ~ 30 minutes. I wanted the camera to heat-soak the wrapper, so that when cooling was applied any measurable change could be collected. I think the act of adding the non-powered stack aided in the camera performance a bit, as simply having the radiator attached means that the hot camera can cool more readily from greater surface area exposure. In any case, a thermal camera was used in timelapse mode to collect the change in temperature over time. The recording was started after 30 minutes of warm-up. Images from the thermal camera were captured during a 1 hour period, while the camera was run in timelapse mode using Micro-Manager 1.4. First, here’s a GIF showing a time-compressed thermal image.
Several things to note here. One is the large drop in camera body temp during the experiment. Another is the subsequent increase in radiator temperature, as it works to shunt the heat from the peltier. Also note the wings on each side of the camera body. They start at a similar temp as the camera body, then drop in temperature significantly (40-50°F) over the duration of the timelapse. It’s important to also note that the emissivity of the plastic camera body differs from that of the aluminum. As a result, even if the temperature is exactly the same, a slight difference in the IR image will be visible.
Below are the results of the image captures. First is a graph showing maximum measured intensity of any pixel in-frame, over the duration of the timelapse (120 frames @ 30Seconds interval). In the case of “hot pixels”, as the camera temperature decreases, the collected signal from those pixels drops.
Graph of measured 8-bit maximum intensity in image over time of experiment. Orange line represents a 20 point moving average trend.
Next, I wanted to measure the total # of hot pixels. Now, it’s somewhat nebulous to define a hot pixel, but I used anything 1.5x greater in intensity than the minimum background, or offset, of the image. On this experiment, the average background was ~ 14, resulting in a threshold of 21. I was really surprised by this one. There is a rather large spike in the data, and I’m not certain what that is from. However, this again shows that the cooling is indeed reducing the hot pixel count, as would be expected. Additionally, the count can be seen to normalize at around 1500 pixels, which I expect is where the system started to thermal stabilize.
For a final set of interesting images. Here’s a shot of the stack after the experiment finished. Note the low temp of the wings and the back wrapper block.
Another interesting shot, this one of the sensor face. Using the cooling system it was at 104°F.
Turning the system off produced a temp of ~ 130°F.
My conclusion is that it is possible to cool even encased cameras using external cooling methods, to improve the performance of the camera(s). I do not think such adaptation will yield comparable results to a camera designed with cooling from the start, but this is an interesting option to achieve better results from a camera with a low price point.
One of my recent projects which was very exciting and interesting was to build a custom imaging system for John Mclaughlin at Torch Therapeutics. This was a great build for a number of reasons, one of which is that Torch has allowed me to write a report on this build, which is greatly appreciated! From the tiny camera to similarly tiny NUC computer, this was a really interesting build, and I’m excited to share what I found during the testing and setup of the system. If you are looking for this type of custom integrated system, please take a look at what my company offers, as we’d love to build a system for you!
Scope Selection
What we were looking to build on this system was a low cost, yet fully automated imaging scope that would be easy to modify, simple to operate, and compatible with Micro-manager. In the short term, only Z control is required, but as the work grows, we wanted provision for XY automation, as well as autofocus tracking. For fluorescence excitation we an LED illuminator. After looking at several options, we selected the ASI RAMM microscope, with the inclusion of the Tiger unified controller, and the ASI LED excitation module. This system offers a complete microscope from one control box, on a scalable build platform, at a competitive price. I reviewed the RAMM scope a few years back, and happy to see that the quality and options have only increased over time. When I reviewed the system, it was required that I build the frame and align things correctly. The new systems ship completely assembled. I literally pulled it out of the box, plugged it in and things were operational. This out-of-box experience isn’t normally seen even in the “big 4” microscopes.
RAMM scope after unboxing
Dual lightpath setup for focus tracking – dichroic for epi on the left (thumbscrews shown) and provision for IR cut mirror on right.
Another feature of this scope is the inclusion of all needed driver components in one box, ASI’s Tiger controller. This is a modular control box which can accept riser cards as-needed to drive various components. The controller on this setup was actually the largest electronic box in the system! From left to right are installed driver cards for joystick/control input, Z stage, LED output, and filter wheel control. Another great feature of this box is that when installing in Micro-manager, only a single USB connection, and single hub device are required.
Tiger Controller, with intel NUC on top
LED Illumination
ASI offers a 4 channel LED illuminator, with option for almost any available LED module. This is a simple illumination setup using a common ladder-dichroic design. The open nature of the hardware allows for changes to the dichroics, or the LED modules, as may be needed by the customer as things change in the future. Each LED is driven using a simple audio style TRS cable, so connecting or disconnecting the LEDs from the driver is straightforward. On the output side of the module, an aperture diapragm is provided to avoid overfilling the back aperture of the objective.
Camera Selection
When selecting a camera, we wanted good QE, decent resolution, but didn’t require any advanced cooling or long exposures. Of the available options, the Pt Grey (FLIR) BlackFly camera fit the bill. Recently, Nico Stuurman @ UCSF wrote up a driver to support this camera in Micro-manager, and after working with it for w while I’m quite impressed. I was able to configure some tricky stuff, I tested various binning modes, subarrays, a wide range of exposure times, and things seemed stable. One suggestion if using this camera is to always snap an image after making changes, vs. simply clicking “live”.
The camera is unbelievably small, roughly the size of a 1″ cube. While cool at first glance, there are some drawbacks to such a design which I note below. I tested the camera at no-light conditions for 2 things – dark current over exposure time, and read noise. Read noise shows a 16-bit count standard deviation of 214.78. Value ranges on the noise floor ran 3264 counts max-min. This works out to a noise level of 4%. Calculation for this measurement was from 2 images in a sequence.
(Image1+2000) -image2 = 200 count centered noise level. (ranging around the 2k level).
Not the best performance from a typical sCMOS camera, but if we consider the next lowest-cost camera runs ~ $5,000, this is quite good in my opinion. There is some pattern noise apparent in the camera, below is an average of 20 exposures, all using 10ms exposure time. The image has been contrast enhanced and pseudo-colored for presentation.
Average of 20 images captured @ 10ms exposure time.
So it appears the taps are drifting a bit. Just for grins I tossed one of these images into an FFT and redisplayed the spectrum for view. Looking at the output, my gut says the image below results from temperature bias on the output taps, and this bias is distributed across the back side of the sensor, or at least wherever the taps are located on the readout layer of the PCB. The higher frequency events at the center look like the typical clock timing jitter seen on any ADC. But this is only my non-expert observation – I’d love for any uber-camera experts to weigh in!
I also wanted to look at how bad the hot pixel effects would be using longer exposures. For this test I ran a set of exposure times from 1ms to 3000ms.
Increase in dark current over time was about as expected.
I think the bottom line on this camera is that it runs as hot as any other camera, but has neither the active cooling nor the inherent surface area to radiate heat well. I took a pic of it using my Flir One camera, and it runs at about 95°F while idle.
Does this mean the camera isn’t suitable for microscopy? Not at all? This is an amazing piece of tech for a very low price. I can imagine buying 3 of these, one for each of 3 emission channels on a single scope! My goal in presenting this information is to share what the current limitations are for such a product. I think the raw QE and ~2.3MP resolution of the camera make it a perfect fit for most microscopy jobs. Of course, there will always be need for higher end cameras, but for many run of the mill operations, this is a nice solution.
NUC Computer
When searching for an “imaging computer” on this project, I had initially considered using a Mac Mini, dual booted with windows 10. I’ve used this for a number of other projects, and it has always run well in windows. John suggested looking at the NUC as an option, which provided later opportunity to upgrade the RAM and disk space. After building it up (which only required installing memory and an SSD), this little guy kicked butt!I ran Windows 10-Professional x64 for this build, and used the latest version of Micro-manager. Funny that a $500 computer and a $500 camera can be had for what used to run ~ $15k!. I successfully ran this on timelapse for 10 hours, and captured streams in MM of 100 frames. I didn’t push the buffer too much or analyze the USB-3 performance, but overall this is a great little machine for simple capture of images.
Building this system was a lot of fun, and I really enjoyed using non-traditional components for it. I think this type of system will become commonplace in the future, as more and more customers realize that traditional components, at a traditional high cost, aren’t always the best fit for their needs. If you are interested in this type of solution for your lab or company, please get in touch, as I’d love to build one just for you!
In 2011, I reviewed the Andor Neo, one of the first scmos cameras commercially available. I ended my report with this statement,
“…there is much life left in the good old frame transfer sensor…until someone cooks up a back thinned scmos…”
Recently, back thinned scmos camera have come to market, and Kurt Thorne at the UCSF NIC has an excellent report on Photometrics’ new Prime 95B camera. While there may be specific areas of research that can benefit from a BT EMCCD, it looks like the days of EMCCD are at an end for most of us. I guess the question now is, what comes next?
How fast can a typical imaging system really run? What factors affect this? While most research might not require rapid sequential capture, almost all experiments can benefiet from tight exposure/illumination timing, yet few microscope users are aware of timing delays injected into experiments by the control software they are using. So how much delay is caused by software? I’ve long wanted to look into this, and finally got the chance!
In order to accomplish this, I programmed a microcontroller to accept commands from the Lumencor Spectra-X. This high power/high speed LED engine is popular for fluorescent excitation, as a result, almost every available software application can control it. I set up my code to measure the time in which a known shutter, or known wavelength command was received on the serial line (well, USB serial).
The team at Technical instruments were kind enough to loan me the use of their equipment, so a generous thanks to Reese Allen and the entire Technical instrument staff!
Experimental conditions were as follows:
All software packages were installed on a clean OS install of Win7/x64
Test machine was a Dell Precision series
Tests were performed by configuring the camera to run ~100fps. This was accomplished using a 4×4 binning, and an exposure of 9.8mS.
All software applications were set to send at minimum one shutter and one wavelength change per acquisition cycle. (i.e. 1 picture from the camera, + 1 wavelength change + open shutter, then switch wavelength and cycle shutter).
All applications were first tested in a “free run” mode, to confirm the camera configuration was capable of 100fps at minimum.
No other devices were installed for these tests.
The results were quite interesting:
Software
NIS Elements V4.30
Micro-Manager V1.4.22
Metamorph V7.8.12
Metamorph Streaming
Average Overhead
60mS
71mS
65mS
11mS
Max FPS
16
14
15
90
Here are some videos showing each of the programs performing. You can see some timing variation (likely caused by my capture device) but the averages are easy to resolve, and they correlate to the speed seen on the captured image stacks.
NIS Elements
Micro-Manager
MetaMorph
What becomes quickly apparent is the influence of asynchronous device control using metamorph’s “Stream” function. This is making use of a patented computer-based device sequencing technique. This technique waits for a camera “event flag” (interrupt) to fire, and when it does, a pre-defined set of events occur which control devices. This is similar to what many external trigger devices do, such as a configured national instruments card , or a pulse oscillator, or the triggerscope. The key difference here is that an external card and/or device isn’t needed, this occurs within the PC itself.
Some further thoughts:
Micromanager is a great open-source competitor to other pay-for solutions. Good speed performance for a low up-front cost! (the cost of setting it up!)
It’s interesting that this never really seems to come up for 90% of customers who purchase “high speed” devices. Either the exposure times needed for acquisition of dim signal are so long as to never reach the speed thresholds shown here, or the capture frequency needed for a given study doesn’t require these speeds, or some other cause, but in my experience this is rarely a complaint on behalf of clients. Why isn’t this a bigger problem?
Many, many devices can’t reach the speeds shown here. In my tests I was using an LED driver. LED’s should have a switch time in the <5mS range. The slowest components usually found on a common automated microscope are the filter wheels and turrets, which usually run at the 50ms range when fully loaded with filters. (that’s on the fast side). So is this the reason the problem isn’t exposed more often?
By far, the slowest components you’ll find for microscopy are found on automated microscopes. Shutter open/close times on a common scope (i.e. big 4 name brand scope) can be in the upwards of 100ms! Moving a big filter turret usually takes 200mS. Again, this begs the question of how important speed really is for the common researcher….
For those who want the fastest possible speeds, have no fear! I’ll be announcing some major improvements to my triggerscope soon, to include programmable high speed sequencing capability!
Andor’s Neo camera has been out for about a year now. I first wrote about using it back in February. The camera immediately showed the major difference in field of view between sCmos sensors and traditional interlines quite well, but the speed, binning, sub-array and bias clamp controls weren’t available. Over the past few firmware/SDK updates, Andor has steadily addressed these limitations. I had a chance to field test the latest combo of firmware and driver SDK. Basically I found a camera that runs at 100fps with roughly four times the field of view of a typical Sony285 type sensor. In my last post I noted that people would wait until end user software platforms could take advantage of the sCmos architecture. I think the wait might be over.
httpv://www.youtube.com/watch?v=RjBv9ChZknk
I’ll continue to report on this as I have an opportunity to put it through it’s paces. I’m curious to see how people respond to it’s performance…
