Using an external master clock for hardware control of a stage-scanning high NA oblique plane microscope
Tutorial provided by qi2lab.
This tutorial uses Pycro-Manager to rapidly acquire terabyte-scale volumetric images using external hardware triggering of a stage scan optimized, high numerical aperture (NA) oblique plane microscope (OPM). The microscope that this notebook controls is described in detail in this preprint, under the stage scan OPM section in the methods.
This high NA OPM allows for versatile, high-resolution, and large field-of-view single molecule imaging. The main application is quantifying 3D spatial gene expression in millions of cells or large pieces of intact tissue using interative RNA-FISH (see examples here and here). Because the fluidics controller for the iterative labeling is also controlled via Python (code not provided here), using Pycro-Manager greatly simplifies controlling these complex experiments.
The tutorial highlights the use of the post_camera_hook_fn and post_hardware_hook_fn functionality to allow an external controller to synchronize the microscope acquisition (external master). This is different from the standard hardware sequencing functionality in Pycro-Manager, where the acquisition engine sets up sequencable hardware and the camera serves as the master clock.
The tutorial also discusses how to structure the events and avoid timeouts to acquire >10 million of events per acquistion.
Microscope hardware
Briefly, the stage scan high NA OPM is built around a bespoke tertiary objective designed by Alfred Millet-Sikking and Andrew York at Calico Labs. Stage scanning is performed by an ASI scan optimized XY stage, an ASI FTP Z stage, and an ASI Tiger controller with a programmable logic card. Excitation light is provided by a Coherent OBIS Laser Box. A custom Teensy based DAC synchronizes laser emission and a galvanometer mirror to the scan stage motion to eliminate motion blur. Emitted fluorescence is imaged by a Photometrics Prime BSI.
The ASI Tiger controller is the master clock in this experiment. The custom Teensy DAC is setup in a closed loop with the Photometrics camera. This controller is detailed in a previous publication on adaptive light sheet microscopy.
The code to orthogonally deskew the acquired data and place it into a BigDataViewer HDF5 file that can be read stitched and fused using BigStitcher is found at the qi2lab (www.github.com/qi2lab/OPM/).
Initial setup
Imports
[1]:
from pycromanager import Bridge, Acquisition, Core
import numpy as np
from pathlib import Path
from time import sleep
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
Cell In[1], line 1
----> 1 from pycromanager import Bridge, Acquisition, Core
2 import numpy as np
3 from pathlib import Path
ModuleNotFoundError: No module named 'pycromanager'
Access Micro-Manager core
[2]:
core = Core()
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[2], line 1
----> 1 core = Core()
NameError: name 'Core' is not defined
Define pycromanager specific hook functions for externally controlled hardware acquisition
Post camera hook function to start external controller
This is run once after the camera is put into active mode in the sequence acquisition. The stage starts moving on this command and outputs a TTL pulse to the camera when it passes the preset initial position. This TTL starts the camera running at the set exposure time using internal timing. The camera acts the master signal for the galvo/laser controller using its own “exposure out” signal.
[3]:
def post_camera_hook_(event, event_queue):
"""
Run a set of commands after the camera is started
:param event: current list of events, each a dictionary, to run in this hardware sequence
:type event: list
:param event_queue: thread-safe event queue
:type event_queue: multiprocessing.Queue
:return: event_queue
"""
# send Tiger command to start constant speed scan
command='1SCAN'
core.set_property('TigerCommHub','SerialCommand',command)
return event
Post hardware setup function to make sure external controller is ready
This is run once after the acquisition engine sets up the hardware for the non-sequencable hardware, such as the height axis stage and channel.
[4]:
def post_hardware_hook(event, event_queue):
"""
Run a set of commands after the hardware setup calls by acquisition engine are finished
:param event: current list of events, each a dictionary, to run in this hardware sequence
:type event: list
:param event_queue: thread-safe event queue
:type event_queue: multiprocessing.Queue
:return: event_queue
"""
# turn on 'transmit repeated commands' for Tiger
core.set_property('TigerCommHub','OnlySendSerialCommandOnChange','No')
# check to make sure Tiger is not busy
ready='B'
while(ready!='N'):
command = 'STATUS'
core.set_property('TigerCommHub','SerialCommand',command)
ready = core.get_property('TigerCommHub','SerialResponse')
sleep(.500)
# turn off 'transmit repeated commands' for Tiger
core.set_property('TigerCommHub','OnlySendSerialCommandOnChange','Yes')
return event
Acquistion parameters set by user
Select laser channels and powers
[5]:
# lasers to use
# 0 -> inactive
# 1 -> active
state_405 = 0
state_488 = 0
state_561 = 1
state_635 = 0
state_730 = 0
# laser powers (0 -> 100%)
power_405 = 0
power_488 = 0
power_561 = 0
power_635 = 0
power_730 = 0
# construct arrays for laser informaton
channel_states = [state_405,state_488,state_561,state_635,state_730]
channel_powers = [power_405,power_488,power_561,power_635,power_730]
Camera parameters
[6]:
# FOV parameters.
# x size (256) is the Rayleigh length of oblique light sheet excitation
# y size (1600) is the high quality lateral extent of the remote image system (~180 microns)
# camera is oriented so that cropping the x size limits the number of readout rows and therefore lowering readout time
ROI = [1024, 0, 256, 1600] #unit: pixels
# camera exposure
exposure_ms = 5 #unit: ms
# camera pixel size
pixel_size_um = .115 #unit: um
Stage scan parameters
The user defines these by interactively moving the XY and Z stages around the sample. At the edges of the sample, the user records the positions.
[7]:
# distance between adjacent images.
scan_axis_step_um = 0.2 #unit: um
# scan axis limits. Use stage positions reported by Micromanager
scan_axis_start_um = 0. #unit: um
scan_axis_end_um = 5000. #unit: um
# tile axis limits. Use stage positions reported by Micromanager
tile_axis_start_um = 0. #unit: um
tile_axis_end_um = 5000. #unit: um
# height axis limits. Use stage positions reported by Micromanager
height_axis_start_um = 0.#unit: um
height_axis_end_um = 30. #unit: um
Path to save acquistion data
[8]:
save_directory = Path('/path/to/save')
save_name = 'test'
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[8], line 1
----> 1 save_directory = Path('/path/to/save')
2 save_name = 'test'
NameError: name 'Path' is not defined
Setup hardware for stage scanning sample through oblique digitally scanned light sheet
Calculate stage limits and speeds from user provided scan parameters
Here, the number of events along the scan (x) axis in each acquisition, the overlap between adajcent strips along the tile (y) axis, and the overlap between adajacent strips along the height (z) axis are all calculated.
[9]:
# scan axis setup
scan_axis_step_mm = scan_axis_step_um / 1000. #unit: mm
scan_axis_start_mm = scan_axis_start_um / 1000. #unit: mm
scan_axis_end_mm = scan_axis_end_um / 1000. #unit: mm
scan_axis_range_um = np.abs(scan_axis_end_um-scan_axis_start_um) # unit: um
scan_axis_range_mm = scan_axis_range_um / 1000 #unit: mm
actual_exposure_s = actual_readout_ms / 1000. #unit: s
scan_axis_speed = np.round(scan_axis_step_mm / actual_exposure_s,2) #unit: mm/s
scan_axis_positions = np.rint(scan_axis_range_mm / scan_axis_step_mm).astype(int) #unit: number of positions
# tile axis setup
tile_axis_overlap=0.2 #unit: percentage
tile_axis_range_um = np.abs(tile_axis_end_um - tile_axis_start_um) #unit: um
tile_axis_range_mm = tile_axis_range_um / 1000 #unit: mm
tile_axis_ROI = ROI[3]*pixel_size_um #unit: um
tile_axis_step_um = np.round((tile_axis_ROI) * (1-tile_axis_overlap),2) #unit: um
tile_axis_step_mm = tile_axis_step_um / 1000 #unit: mm
tile_axis_positions = np.rint(tile_axis_range_mm / tile_axis_step_mm).astype(int) #unit: number of positions
# if tile_axis_positions rounded to zero, make sure acquisition visits at least one position
if tile_axis_positions == 0:
tile_axis_positions=1
# height axis setup
# this is more complicated, because the excitation is an oblique light sheet
# the height of the scan is the length of the ROI in the tilted direction * sin(tilt angle)
height_axis_overlap=0.2 #unit: percentage
height_axis_range_um = np.abs(height_axis_end_um-height_axis_start_um) #unit: um
height_axis_range_mm = height_axis_range_um / 1000 #unit: mm
height_axis_ROI = ROI[2]*pixel_size_um*np.sin(30*(np.pi/180.)) #unit: um
height_axis_step_um = np.round((height_axis_ROI)*(1-height_axis_overlap),2) #unit: um
height_axis_step_mm = height_axis_step_um / 1000 #unit: mm
height_axis_positions = np.rint(height_axis_range_mm / height_axis_step_mm).astype(int) #unit: number of positions
# if height_axis_positions rounded to zero, make sure acquisition visits at least one position
if height_axis_positions==0:
height_axis_positions=1
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[9], line 5
3 scan_axis_start_mm = scan_axis_start_um / 1000. #unit: mm
4 scan_axis_end_mm = scan_axis_end_um / 1000. #unit: mm
----> 5 scan_axis_range_um = np.abs(scan_axis_end_um-scan_axis_start_um) # unit: um
6 scan_axis_range_mm = scan_axis_range_um / 1000 #unit: mm
7 actual_exposure_s = actual_readout_ms / 1000. #unit: s
NameError: name 'np' is not defined
Setup Coherent laser box from user provided laser parameters
[10]:
core = Core()
# turn off lasers
# this relies on a Micro-Manager configuration group that sets all lasers to "off" state
core.set_config('Coherent-State','off')
core.wait_for_config('Coherent-State','off')
# set lasers to user defined power
core.set_property('Coherent-Scientific Remote','Laser 405-100C - PowerSetpoint (%)',channel_powers[0])
core.set_property('Coherent-Scientific Remote','Laser 488-150C - PowerSetpoint (%)',channel_powers[1])
core.set_property('Coherent-Scientific Remote','Laser OBIS LS 561-150 - PowerSetpoint (%)',channel_powers[2])
core.set_property('Coherent-Scientific Remote','Laser 637-140C - PowerSetpoint (%)',channel_powers[3])
core.set_property('Coherent-Scientific Remote','Laser 730-30C - PowerSetpoint (%)',channel_powers[4])
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[10], line 1
----> 1 core = Core()
2 # turn off lasers
3 # this relies on a Micro-Manager configuration group that sets all lasers to "off" state
4 core.set_config('Coherent-State','off')
NameError: name 'Core' is not defined
Setup Photometrics camera for low-noise readout and triggering
The camera input trigger is set to Trigger first mode to allow for external control and the output trigger is set to Rolling Shutter mode to ensure that laser light is only delivered when the entire chip is exposed. The custom Teensy DAC waits for the signal from the camera to go HIGH and then sweeps a Gaussian pencil beam once across the field-of-view. It then rapidly resets and scans again upon the next trigger. The Teensy additionally blanks the Coherent laser box emission between
frames.
[11]:
core = Core()
# set camera into 16bit readout mode
core.set_property('Camera','ReadoutRate','100MHz 16bit')
# give camera time to change modes
sleep(5)
# set camera into low noise readout mode
core.set_property('Camera','Gain','2-CMS')
# give camera time to change modes
sleep(5)
# set camera to give an exposure out signal
# this signal is used by the custom DAC to synchronize blanking and a digitally swept light sheet
core.set_property('Camera','ExposureOut','Rolling Shutter')
# give camera time to change modes
sleep(5)
# change camera timeout.
# this is necessary because the acquisition engine can take a long time to setup with millions of events
# on the first run
core.set_property('Camera','Trigger Timeout (secs)',300)
# give camera time to change modes
sleep(5)
# set camera to internal trigger
core.set_property('Camera','TriggerMode','Internal Trigger')
# give camera time to change modes
sleep(5)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[11], line 1
----> 1 core = Core()
2 # set camera into 16bit readout mode
3 core.set_property('Camera','ReadoutRate','100MHz 16bit')
NameError: name 'Core' is not defined
Setup ASI stage control cards and programmable logic card in the Tiger controller
Hardware is setup for a constant-speed scan along the x direction, lateral tiling along the y direction, and height tiling along the z direction. The programmable logic card sends a signal to the camera to start acquiring once the scan (x) axis reaches the desired speed and crosses the user defined start position.
Documentation for the specific commands to setup the constant speed stage scan on the Tiger controller is at the following links, - SCAN - SCANR - SCANV
Documentation for the programmable logic card is found here.
The Tiger is polled after each command to make sure that it is ready to receive another command.
[12]:
core = Core()
# Setup the PLC to output external TTL when an internal signal is received from the stage scanning card
plcName = 'PLogic:E:36'
propPosition = 'PointerPosition'
propCellConfig = 'EditCellConfig'
addrOutputBNC3 = 35
addrStageSync = 46 # TTL5 on Tiger backplane = stage sync signal
core.set_property(plcName, propPosition, addrOutputBNC3)
core.set_property(plcName, propCellConfig, addrStageSync)
# turn on 'transmit repeated commands' for Tiger
core.set_property('TigerCommHub','OnlySendSerialCommandOnChange','No')
# set tile (y) axis speed to 25% of maximum for all moves
command = 'SPEED Y=.25'
core.set_property('TigerCommHub','SerialCommand',command)
# check to make sure Tiger is not busy
ready='B'
while(ready!='N'):
command = 'STATUS'
core.set_property('TigerCommHub','SerialCommand',command)
ready = core.get_property('TigerCommHub','SerialResponse')
sleep(.500)
# set scan (x) axis speed to 25% of maximum for non-sequenced moves
command = 'SPEED X=.25'
core.set_property('TigerCommHub','SerialCommand',command)
# check to make sure Tiger is not busy
ready='B'
while(ready!='N'):
command = 'STATUS'
core.set_property('TigerCommHub','SerialCommand',command)
ready = core.get_property('TigerCommHub','SerialResponse')
sleep(.500)
# turn off 'transmit repeated commands' for Tiger
core.set_property('TigerCommHub','OnlySendSerialCommandOnChange','Yes')
# turn on 'transmit repeated commands' for Tiger
core.set_property('TigerCommHub','OnlySendSerialCommandOnChange','No')
# set scan (x) axis speed to correct speed for constant speed movement of scan (x) axis
# expects mm/s
command = 'SPEED X='+str(scan_axis_speed)
core.set_property('TigerCommHub','SerialCommand',command)
# check to make sure Tiger is not busy
ready='B'
while(ready!='N'):
command = 'STATUS'
core.set_property('TigerCommHub','SerialCommand',command)
ready = core.get_property('TigerCommHub','SerialResponse')
sleep(.500)
# set scan (x) axis to true 1D scan with no backlash
command = '1SCAN X? Y=0 Z=9 F=0'
core.set_property('TigerCommHub','SerialCommand',command)
# check to make sure Tiger is not busy
ready='B'
while(ready!='N'):
command = 'STATUS'
core.set_property('TigerCommHub','SerialCommand',command)
ready = core.get_property('TigerCommHub','SerialResponse')
sleep(.500)
# set range and return speed (25% of max) for constant speed movement of scan (x) axis
# expects mm
command = '1SCANR X='+str(scan_axis_start_mm)+' Y='+str(scan_axis_end_mm)+' R=25'
core.set_property('TigerCommHub','SerialCommand',command)
# check to make sure Tiger is not busy
ready='B'
while(ready!='N'):
command = 'STATUS'
core.set_property('TigerCommHub','SerialCommand',command)
ready = core.get_property('TigerCommHub','SerialResponse')
sleep(.500)
# turn off 'transmit repeated commands' for Tiger
core.set_property('TigerCommHub','OnlySendSerialCommandOnChange','Yes')
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[12], line 1
----> 1 core = Core()
2 # Setup the PLC to output external TTL when an internal signal is received from the stage scanning card
3 plcName = 'PLogic:E:36'
NameError: name 'Core' is not defined
Setup and run the acquisition
Change core timeout
This is necessary because of the large, slow XY stage moves.
[13]:
core = Core()
# change core timeout for long stage moves
core.set_property('Core','TimeoutMs',20000)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[13], line 1
----> 1 core = Core()
2 # change core timeout for long stage moves
3 core.set_property('Core','TimeoutMs',20000)
NameError: name 'Core' is not defined
Move stage hardware to initial positions
[14]:
core = Core()
# move scan (x) and tile (y) stages to starting positions
core.set_xy_position(scan_axis_start_um,tile_axis_start_um)
core.wait_for_device(xy_stage)
# move height (z) stage to starting position
core.set_position(height_position_um)
core.wait_for_device(z_stage)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[14], line 1
----> 1 core = Core()
2 # move scan (x) and tile (y) stages to starting positions
3 core.set_xy_position(scan_axis_start_um,tile_axis_start_um)
NameError: name 'Core' is not defined
Create event structure
The external controller handles all of the events in x for a given yzc position. To make sure that pycro-manager structures the acquistion this way, the value of the stage positions for x are kept constant for all events at a given yzc position. This gives the order of the loops to create the event structure as yzcx.
[15]:
# empty event dictionary
events = []
# loop over all tile (y) positions.
for y in range(tile_axis_positions):
# update tile (y) axis position
tile_position_um = tile_axis_start_um+(tile_axis_step_um*y)
# loop over all height (z) positions
for z in range(height_axis_positions):
# update height (z) axis position
height_position_um = height_axis_start_um+(height_axis_step_um*z)
# loop over all channels (c)
for c in range(len(channel_states)):
# create events for all scan (x) axis positions.
# The acquistion engine knows that this is a hardware triggered sequence because
# the physical x position does not change when specifying the large number of x events
for x in range(scan_axis_positions):
# only create events if user sets laser to active
# this relies on a Micromanager group 'Coherent-State' that has individual entries that correspond
# the correct on/off state of each laser. Laser blanking and synchronization are handled by the
# custom Teensy DAC controller.
if channel_states[c]==1:
if (c==0):
evt = { 'axes': {'x': x, 'y':y, 'z':z}, 'x': scan_axis_start_um, 'y': tile_position_um,
'z': height_position_um, 'channel' : {'group': 'Coherent-State', 'config': '405nm'}}
elif (c==1):
evt = { 'axes': {'x': x, 'y':y, 'z':z}, 'x': scan_axis_start_um, 'y': tile_position_um,
'z': height_position_um, 'channel' : {'group': 'Coherent-State', 'config': '488nm'}}
elif (c==2):
evt = { 'axes': {'x': x, 'y':y, 'z':z}, 'x': scan_axis_start_um, 'y': tile_position_um,
'z': height_position_um, 'channel' : {'group': 'Coherent-State', 'config': '561nm'}}
elif (c==3):
evt = { 'axes': {'x': x, 'y':y, 'z':z}, 'x': scan_axis_start_um, 'y': tile_position_um,
'z': height_position_um, 'channel' : {'group': 'Coherent-State', 'config': '637nm'}}
elif (c==4):
evt = { 'axes': {'x': x, 'y':y, 'z':z}, 'x': scan_axis_start_um, 'y': tile_position_um,
'z': height_position_um, 'channel' : {'group': 'Coherent-State', 'config': '730nm'}}
events.append(evt)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[15], line 5
2 events = []
4 # loop over all tile (y) positions.
----> 5 for y in range(tile_axis_positions):
6
7 # update tile (y) axis position
8 tile_position_um = tile_axis_start_um+(tile_axis_step_um*y)
10 # loop over all height (z) positions
NameError: name 'tile_axis_positions' is not defined
Run acquisition
The camera is set to
Trigger firstmode. In this mode, the camera waits for an external trigger and then runs using the internal timing.The acquisition is setup and started. The initial acquisition setup by Pycro-manager and the Java acquisition engine takes a few minutes and requires at significant amount of RAM allocated to ImageJ. 40 GB of RAM seems acceptable. The circular buffer is only allocated 2 GB, because the computer for this experiment has an SSD array capable of writing up to 600 MBps.
At each
yzcposition, the ASI Tiger controller supplies the external master signal when the the (scan) axis has ramped up to the correct constant speed and crossedscan_axis_start_um. The speed is defined byscan_axis_speed = scan_axis_step_um / camera_exposure_ms. Acquired images are placed into thexaxis of the Acquisition without Pycro-Manager interacting with the hardware.Once the full acquisition is completed, all lasers are set to
offand the camera is placed back inInternal Triggermode.
[16]:
core = Core()
# set camera to trigger first mode for stage synchronization
# give camera time to change modes
core.set_property('Camera','TriggerMode','Trigger first')
sleep(5)
# run acquisition
# the acquisition needs to write data at roughly 100-500 MBps depending on frame rate and ROI
# so the display is set to off and no multi-resolution calculations are done
with Acquisition(directory=save_directory, name=save_name, post_hardware_hook_fn=post_hardware_hook,
post_camera_hook_fn=post_camera_hook, show_display=False) as acq:
acq.acquire(events)
# turn off lasers
core.set_config('Coherent-State','off')
core.wait_for_config('Coherent-State','off')
# set camera to internal trigger
core.set_property('Camera','TriggerMode','Internal Trigger')
# give camera time to change modes
sleep(5)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[16], line 1
----> 1 core = Core()
2 # set camera to trigger first mode for stage synchronization
3 # give camera time to change modes
4 core.set_property('Camera','TriggerMode','Trigger first')
NameError: name 'Core' is not defined