Due to the open nature of our driver system, it’s relatively straightforward to write a new driver file for almost any imaginable device in Nebula. However, driver development can be a pain, so I though I’d share some debugging tricks.
For all commands, we will navigate to the command line using windows Command prompt or powershell. Here I show command prompt syntax. Nebula is commonly installed under program files. So in the windows start bar type “cmd” and press enter. This should open the command prompt. The commands below navigate to the install location.
First, is the use of logging to the command line. When Nebula is launched from the command line, a series of optional switches can be used to report logged output. By opening this alongside the app, it’s very fast to figure out what or where something is going wrong. To see log output, use –printlog.
c:\Program Files\Nebula>Nebula.exe --printlog
This launches the app, but shows a live log output on the console at the same time.
Get a full list of Supported Drivers
This command will return a complete list of drivers found in the Nebula supported list. To print this list, use:
c:\Program Files\Nebula>Nebula.exe --listdrivers
Here is live example of that command and the expected result. A list of drivers is shown, press enter to close the listing.
Testing a specific Driver File
Next, let’s load up one of these drivers in the test utility. Now, this is where things get cool! The test utility can load a host device driver, inspect it’s shown devices, attempt to connect them, and even test every sub-device!
So, let’s assume we have a new camera driver we want to work on. The camera should do things like snap an image, show various properties etc. In my example I use a real UVC (web) camera, and inspect the driver. The driver actually pulls the real camera info and even snaps a test image. The goal of this function is twofold, first to validate a given driver, and second to validate the device itself.
For this command we will use –testdriver *drivername* and optionally –port *portname*, for example, COM5
Here we can watch the expected output in real time. Note a new window is spawned to show us the results for these tests.
Testing Host and Sub Devices
Let’s say we have a controller, for example, an ASI or Marzhauser box, which has a Z motor attached. In Nebula, the controller can be thought of as the Host (the resident controlling brain), and the device can be considered an appendage attached to the host. So all drivers must have a Host, but a Host may only have one device. In this example, we use a demo Focus device. First the host is loaded and analyzed, and then, you ca select to test any device provided by that host. I choose option # 1 (focus) and the test runs, even showing my driver has a small error I missed!
Obviously, if you have a driver which isn’t behaving, please contact us – yet I hope these options provide a first-line of interface for power users, so simple things can be corrected and analyzed quickly!
Recently, I had two different requests to support the cameras manufactured by Touptek. Ash from Scientific Imaging was kind enough to let me borrow one for in-house integration and testing.
These cameras support Sony IMX-174 sensors, use a USB 3.1 (locking!) cable, and are straightforward to hook up and use. A GPIO connection is provided for I/O, and some models also have opto-isolated IO as well.
Binning, ROI and bit depth readout modes are also supported.
The camera I had hands on had a color version of the sensor, and while today has been a lot of rain, I was able to get a gloomy scene outside for my demo video below.
For questions on running this camera in Nebula, please contact us at www.starlyteimaging.com
If you are interested in these cameras, please contact Scientific Imaging.
Here is a brief video demo of the camera in Starlyte Nebula!
A few months back, I started promoting a new software application called Nebula. As I introduced it to people I realized that I’d not explained the development or release of it on my website, and so here I’ll take a minute to explain that.
The Why
20 years ago I fell into a career in the life science imaging industry, working for an integration and microscope dealer in southern California. One of my first jobs was installing a copy of software application called MetaMorph. This was an interesting thing to me, having a background in building computers from the days of ISA cards and DOS boot menus.
Metamorph could move things like a filter wheel (then a Sutter Instrument 10-2), and capture images from a camera like the then-new Photometrics 1300 YHS.
It fascinated me to watch the camera snap images with it’s mechanical shutter as the filter wheel switched positions, compiling images into a singular display. I was intrigued by this ability and also how it was used in research.
As time went on, I eventually moved from building the basic systems to maintaining them in-field, and then to offering complete hardware and software systems as a direct rep for the company who made this software, Universal Imaging Corporation.
The Need
What made MetaMorph appealing was the wide array of devices it could control at speed – from stages, to microscopes, to illuminators, basically anything. The company had no specific lockout, and would integrate device drivers for any product.
Over time, UIC was acquired by Molecular Devices, and after a long run ceased offering product sales in 2023. By that time, the industry was somewhat locked into the major microscope companies selling brand-specific products. Further, these companies would refuse support (understandably) for specific competing devices. Yet, this posed a genuine challenge for users:
Once invested into an automated microscope, I am forced to use proprietary software with restrictions, or use open-source software alternatives like MicroManager.
Now, MicroManager is an exceptional suite of software with device drivers, but for customers who want a simplified approach to control, with minimal hands-on configuration, and someone to call when things go wrong, there really aren’t many commercial applications filling the need. This situation led me to conclude that a new company was needed in the industry to:
Provide Custom Integration Support for existing microscopes
Offer hardware driver support for smaller and newer companies excluded by the major brands.
Wrap device control into a simple, easy to use interface
Provide exceptional support for products.
With this in mind, I engaged with my friend and colleague Jacob Gewerth, to launch Starlyte Imaging, and build our core product, Nebula. It’s been a long development process, but what Nebula does is something very special:
The What
We imagined, and have released, a software application geared to capture, and capture alone:
We are dedicated to controlling devices as the priority – it’s all about getting data
Our drivers are completely developed in Python, and our driver standard is open – meaning YOU or any company who wants to, can write device drivers for our software.
Our pricing is approachable and fair – no huge budget outlay for software – we offer fixed licenses, or service-based pricing.
We can support old as well as new devices – many older microscopes are perfectly suitable as a chassis to perform any modern study, they just need to be tied into the capture control scheme.
We also can support data collection and metadata recording from perfusion systems, peristaltic pumps, or any other environmental control device you want to talk to.
More than anything else, I wanted to bring the simple, fair and honest product support I’ve enjoyed providing to this industry for my career. Things fail, mistakes happen, but what makes the difference to me, is the dedication to see projects through, supporting your research, and seeing the project through.
I made a short video describing why laser pricing is so different. Much of it comes down to how precise the beam performance needs to be, and as with so many similar fields, as precision becomes better and better, pricing begins to log scale up as the challenges increase. Thanks for watching!
For some time now I’ve been using 8020 extrusion for quick prototyping of optical assemblies. It’s great for assemblies which require longer focal lengths and larger elements. For such systems, the choices are to use a breadboard or table, with posts – and then line everything up, having to worry at minimum about elevation and rotation of each element. Once lined up the parts aren’t quite easily moved around. For my parts, I simply created a negative profile for the “8020” 2525 brand of extrusion, then placed a 32mm circle at the center of this profile with vertical clearance. Likely I should have placed it higher, but for most elements it’s simple to create a clamp, send it to the 3-D printer, stick the element inside and voila, held at an axial center, generally en face, and very easy to slide back and forth.
For complicated systems which may use reflection or episcopic combined paths, additional pieces of extrusion can be cut and aligned to a core axis quite easily, and even attached in 3 rotations (left, right and “up” from the primary optical axis). Extrusion pieces can be ordered pre-cut, or easily cut to length using a hacksaw. Right angle clamps avoid any angular errors caused by bad cuts. Personally I use a mill to face off the ends, but most people don’t have access to a mill.
In this image I use the extrusion to align a large optic, with further parts out of frame.
To make things even easier, it’s nice to be able to mark off critical dimensions on the extrusion with a felt tip pen, which is easily later removed from the anodized surface with IPA.
In this example a high power excitation input was designed in Zemax to squeeze a bit more power from a fluorescent system. Before building the assembly the profile extrusion was used to verify the field collimation and flatness at the desired focal length.
I’ve added all of the designs I frequently start from, in STEP format. If someone needs another format please let me know and I’ll export one.
During a recent project for a client, I had to prepare a fluorescent bead slide for imaging. Beads require dilution, and a vortex mixer is recommended in order to provide ample separation of the individual beads. Not having one on hand, I figured there must be a way to achieve similar results, and found this used by a homebrewer for yeast prep. It was close for my needs, but I didn’t want to use wood due to the fabrication time and difficulty of install / removal. So I measured the screw hole spacing on the sander pad, and sketched one up in Fusion. A PLA print was made, and the result has been great for simple use.
One additional trick I use for mods like this, is a router speed control, available form Harbor freight or Amazon. It allows the motor speed to be adjusted. The little Ryobi is super fast, so I ended up suing the speed control on it’s lowest setting. These controls work well for most motors – some using “soft start” motors may not work, but it did great in this case!
If you would like to print your own (assuming the hole spacing is compatible), the STL file and STEP for modification are available on my Github page here.
One of my recent projects which was very exciting and interesting was to build a custom imaging system for John Mclaughlin at Torch Therapeutics. This was a great build for a number of reasons, one of which is that Torch has allowed me to write a report on this build, which is greatly appreciated! From the tiny camera to similarly tiny NUC computer, this was a really interesting build, and I’m excited to share what I found during the testing and setup of the system. If you are looking for this type of custom integrated system, please take a look at what my company offers, as we’d love to build a system for you!
Scope Selection
What we were looking to build on this system was a low cost, yet fully automated imaging scope that would be easy to modify, simple to operate, and compatible with Micro-manager. In the short term, only Z control is required, but as the work grows, we wanted provision for XY automation, as well as autofocus tracking. For fluorescence excitation we an LED illuminator. After looking at several options, we selected the ASI RAMM microscope, with the inclusion of the Tiger unified controller, and the ASI LED excitation module. This system offers a complete microscope from one control box, on a scalable build platform, at a competitive price. I reviewed the RAMM scope a few years back, and happy to see that the quality and options have only increased over time. When I reviewed the system, it was required that I build the frame and align things correctly. The new systems ship completely assembled. I literally pulled it out of the box, plugged it in and things were operational. This out-of-box experience isn’t normally seen even in the “big 4” microscopes.
RAMM scope after unboxing
Dual lightpath setup for focus tracking – dichroic for epi on the left (thumbscrews shown) and provision for IR cut mirror on right.
Another feature of this scope is the inclusion of all needed driver components in one box, ASI’s Tiger controller. This is a modular control box which can accept riser cards as-needed to drive various components. The controller on this setup was actually the largest electronic box in the system! From left to right are installed driver cards for joystick/control input, Z stage, LED output, and filter wheel control. Another great feature of this box is that when installing in Micro-manager, only a single USB connection, and single hub device are required.
Tiger Controller, with intel NUC on top
LED Illumination
ASI offers a 4 channel LED illuminator, with option for almost any available LED module. This is a simple illumination setup using a common ladder-dichroic design. The open nature of the hardware allows for changes to the dichroics, or the LED modules, as may be needed by the customer as things change in the future. Each LED is driven using a simple audio style TRS cable, so connecting or disconnecting the LEDs from the driver is straightforward. On the output side of the module, an aperture diapragm is provided to avoid overfilling the back aperture of the objective.
Camera Selection
When selecting a camera, we wanted good QE, decent resolution, but didn’t require any advanced cooling or long exposures. Of the available options, the Pt Grey (FLIR) BlackFly camera fit the bill. Recently, Nico Stuurman @ UCSF wrote up a driver to support this camera in Micro-manager, and after working with it for w while I’m quite impressed. I was able to configure some tricky stuff, I tested various binning modes, subarrays, a wide range of exposure times, and things seemed stable. One suggestion if using this camera is to always snap an image after making changes, vs. simply clicking “live”.
The camera is unbelievably small, roughly the size of a 1″ cube. While cool at first glance, there are some drawbacks to such a design which I note below. I tested the camera at no-light conditions for 2 things – dark current over exposure time, and read noise. Read noise shows a 16-bit count standard deviation of 214.78. Value ranges on the noise floor ran 3264 counts max-min. This works out to a noise level of 4%. Calculation for this measurement was from 2 images in a sequence.
(Image1+2000) -image2 = 200 count centered noise level. (ranging around the 2k level).
Not the best performance from a typical sCMOS camera, but if we consider the next lowest-cost camera runs ~ $5,000, this is quite good in my opinion. There is some pattern noise apparent in the camera, below is an average of 20 exposures, all using 10ms exposure time. The image has been contrast enhanced and pseudo-colored for presentation.
Average of 20 images captured @ 10ms exposure time.
So it appears the taps are drifting a bit. Just for grins I tossed one of these images into an FFT and redisplayed the spectrum for view. Looking at the output, my gut says the image below results from temperature bias on the output taps, and this bias is distributed across the back side of the sensor, or at least wherever the taps are located on the readout layer of the PCB. The higher frequency events at the center look like the typical clock timing jitter seen on any ADC. But this is only my non-expert observation – I’d love for any uber-camera experts to weigh in!
I also wanted to look at how bad the hot pixel effects would be using longer exposures. For this test I ran a set of exposure times from 1ms to 3000ms.
Increase in dark current over time was about as expected.
I think the bottom line on this camera is that it runs as hot as any other camera, but has neither the active cooling nor the inherent surface area to radiate heat well. I took a pic of it using my Flir One camera, and it runs at about 95°F while idle.
Does this mean the camera isn’t suitable for microscopy? Not at all? This is an amazing piece of tech for a very low price. I can imagine buying 3 of these, one for each of 3 emission channels on a single scope! My goal in presenting this information is to share what the current limitations are for such a product. I think the raw QE and ~2.3MP resolution of the camera make it a perfect fit for most microscopy jobs. Of course, there will always be need for higher end cameras, but for many run of the mill operations, this is a nice solution.
NUC Computer
When searching for an “imaging computer” on this project, I had initially considered using a Mac Mini, dual booted with windows 10. I’ve used this for a number of other projects, and it has always run well in windows. John suggested looking at the NUC as an option, which provided later opportunity to upgrade the RAM and disk space. After building it up (which only required installing memory and an SSD), this little guy kicked butt!I ran Windows 10-Professional x64 for this build, and used the latest version of Micro-manager. Funny that a $500 computer and a $500 camera can be had for what used to run ~ $15k!. I successfully ran this on timelapse for 10 hours, and captured streams in MM of 100 frames. I didn’t push the buffer too much or analyze the USB-3 performance, but overall this is a great little machine for simple capture of images.
Building this system was a lot of fun, and I really enjoyed using non-traditional components for it. I think this type of system will become commonplace in the future, as more and more customers realize that traditional components, at a traditional high cost, aren’t always the best fit for their needs. If you are interested in this type of solution for your lab or company, please get in touch, as I’d love to build one just for you!
The Waller Lab recently had a demo announcement of their awesome implementation of computational imaging. They have some examples on their lytro page, which look really cool! This got me thinking about the last 10 years or so, and how the combination of image capture with control over input and output image fields, has grown into a new force in microscopy today.
It’s interesting to consider what techniques fit into the field of computational imaging. Optical microscopy has pushed into an ever smaller domain of resolution, to the point that Abbe’s laws need to be avoided in order to pull resolution out of an image. Consider the initial techniques used for this – phase, DIC, confocal, TIRF, and the like are contrast enhancement techniques of enhancing an image’s contrast, by either increasing the signal intensity, or by reducing the surrounding noise/background intensity. Beginning with the Apotome, and maybe earlier, with deconvolution, a combination of using multiple image samples with post-capture image processing, to gather more information than can be obtained by a single image, started to emerge.
Building on these first techniques, we can see further sampling methods in the optical domain, like STORM/PALM, where optical control over emission is coupled with high sampling rates, and back-end processing, to statistically avoid Abbe’s laws. I’d imagine FLIM falls into this category as well, using time and multiple images, again to extract more data than may be obtained by a single sample. Non-microscopy applications for yet another method in this domain, the employment of coded apertures, similar to this work from MIT, are growing in a number of imaging fields.
This combination, of computing power + controlled image fields, can be employed to do all sorts of as-yet untouched things with a conventional microscope. I can imagine that very soon, we will see a few of the following products available for use:
phase contrast, and fluorescent, multi-capture resolution enhancement systems, all built into an “off the shelf” microscope
angular changes to fluorescence excitation, used to improve resolution
wavelength-based restriction inside an image field, to eliminate the potential for crosstalk in emission
multiple excitation power sampling, to pull greater dynamic range from images
multiple cameras used with restricting apertures, used to pull focus information, or multiple focal fields, from a single position in a sample
These are only a few ideas where this can go, but the general path I can see in the future is lower cost components, which are leveraged to do better and better work, which gives more imaging power to a greater number of researchers. At least – that’s a future I’d like to see 🙂
Proper optical performance should produce nice clear defocused rings as shown in this example
After all of the engineering and testing which goes into a microscope, you’d imagine that any manufactured scope would have perfect alignment of the optics. Believe it or not, in many cases things can be off! So how can you tell this? A simple test is to view a bead slide, or a grid slide, and run the focus up and down through the sample’s best focal plane. What you should see is clear defocused rings or bars which expand away from the object as you drive away from either side of the focal plane.
Nikon has a nice writeup on astigmatism here – click on the pic to view the artice.
Running through the focus quickly will reveal astigmatism:
Does the defocused haze seem to move from right to left as you drive into the focal plane, and continue traveling in the same direction as you focus past the focal plane? This can indicate astigmatism or sample alignment problems.
Do your out of focus regions seem to defocus into horizontal lines on one side of the sample, and then defocus into vertical lines on the other side? If so this can indicate astigmatism.
In either case this is a limitation to the amount of clarity your optics can resolve and should be addressed.
Simple example of astigmatic errors and the effect they cause on image clarity
You can further isolate the issue by switching objectives and running the test again.
If the problem remains, the problem is not related to a single objective, but is either due to the optical components inside the scope, or due to the stage not installed at perfect level (this is actually quite common!).
If the problem is only visible in one objective and not in any others, the problem is in the objective affected. (If it’s an oil lens first try giving it a good cleaning! old oil can build up on the lens and cause this type of behavior!
