A PC-based shutter glasses controller for visual stimulation using multithreading
For a more detailed version of this research report, please read the following publication:
Gramatikov, I.B., Simons, K., Guyton, D.L., Gramatikov, B.I. A PC-based shutter glasses controller for visual stimulation using multithreading in LabWindows/CVI. Computer Methods and Programs in Biomedicine (Elsevier), 143 (2017), 151-158. Uploaded online on 10 March 2017.
https://www.ncbi.nlm.nih.gov/pubmed/28391813
http://www.cmpbjournal.com/article/S0169-2607(16)31307-4/pdf
http://www.cmpbjournal.com/article/S0169-2607(16)31307-4/abstract
1. Introduction
In newborn children, the nerves and brain function that control eye movement and image processing are still developing. If sufficient binocular function is not achieved in the first several months of life, the eyes can drift out of alignment, most commonly crossing inward. This eye misalignment prevents the brain from receiving the aligned images needed for binocular depth perception. Another abnormality, namely different refractive error between the two eyes, causes blurred vision in one eye because human eyes cannot focus independently. If one eye is in focus, the other is not. In order to avoid confusion from double images or blurred vision in one eye, the young brain will suppress the non-dominant eye, relying only on the dominant eye. The suppressed eye often becomes “lazy” from lack of use, with poor visual acuity – an abnormality called amblyopia. It is a major public health problem, afflicting up to 3.6% of children, and will lead to lifelong visual impairment if not identified and treated in early childhood [1].
Amblyopia can be partly reversed at young ages by patching the strong eye, or “penalizing” it with blurring filters or atropine drops, to force the weak eye to be used. Surgery on the eye muscles is sometimes needed to bring the eyes back into reasonable alignment, but the brain may still suppress the non-dominant eye.
The older the child becomes, the harder it is to reverse amblyopia. Both patching and drops are bothersome, and compliance with treatment becomes all the more difficult. Better tolerated methods are needed to unsuppress the amblyopic eye. Liquid crystal shutter glasses may provide a good solution. These were originally developed for stereo TV and have recently been modified to treat amblyopia by intermittently occluding the dominant eye or by alternately shuttering the two eyes [2-4], with some success [5, 6]. Unfortunately there is yet no clarity regarding the best shuttering method, frequency, duty cycle (in the case of square waves), etc. Moreover, these parameters may need to be adjustable for best results in different patients. Research in this area can only be efficient if based on a system more flexible than a simple function generator. An integrated system is needed that will allow fast and easy adjustment of these parameters in the exam room or laboratory setting, and be able to transfer these individualized settings to portable rechargeable eyewear equipped with shuttering lenses and control circuitry. The objective of this work was to develop an inexpensive PC-based system that will allow fast and easy personalized adjustment of the shuttering parameters, and to enable initial testing of children with amblyopia in a clinical setting. The system is designed to serve both medical researchers and practitioners.
2. Methods
2.1. Hardware
The study was approved by the institutional IRB. Our goal was to develop the hardware and software for a low-cost, PC-centered system allowing eye stimulation by means of liquid-crystal-based shutter glasses. We direct-wired a pair of Xpand X103-M Active Shutter 3D Glasses, and added polymer quarter wave plates to convert their polarization to circular for less interference from head tilting when viewing circularly-polarized test targets having portions visible to the right eye and other portions visible to the left eye. Such LCD shutter glasses can be controlled with monopolar or bipolar electrical pulses of amplitudes in the order of several volts, depending on the type of the LCD and the level of occlusion desired. Further, it is highly desirable to provide feedback from the patient, indicating which shuttering frequency, duty cycle, and voltage amplitude are found to be optimal. This was achieved by including four potentiometers (one spare), providing an analog signal. Consequently, a digital-to-analog converter (DAC) and an analog-to-digital (ADC) converter were needed. We used an inexpensive multifunction (MF) ADC/DAC module, NI USB-6009 from National Instruments (NI), providing two analog outputs (AOs) of 12-bit resolution at an update rate of up to 150 Hz, and eight analog inputs (AIs) of 14-bit resolution at rates up to aggregate sampling rate of 48 kS/s (system dependent), not to exceed 12 kHz per channel. The AOs were used to control the shuttering voltage. Because of their limited output voltage (0-5V), level shifters and amplifiers/buffers were needed to produce amplitudes in the range of ±10V. The amplitude and the baseline were generated by software over the user interface (UIR), the voltage provided to the glasses being
Amplitudeglasses = (DACamplitude + DACbaseline – 2.5) x 4 [V] (1)
Baselineglasses = (DACbaseline – 2.5) x 4 [V] (2)
where DACamplitude and DACbaseline are generated by the software, the 2.5V are subtracted in the circuitry and Amplitudeglasses is what goes to the shutter glasses (Figure 1). The four potentiometers were connected to the first four AIs, providing voltages in the range (0 to 10 V). For controlling the USB-6009, NI provides a set of drivers called DAQmx, which in this case operates over USB 2.0.
Figure 1. Block diagram of the shutter glass controller
2.2. Software
We utilized the widely used LabWindowsTM/CVI (NI), basically an expanded ANSII C, which has become one of the few de facto standards for PC-based applications. Coupled with the company’s data acquisition hardware, drivers, and convenient GUI, the CVI provides a versatile development environment at a low cost, running under Windows.
The stimulation range is 0.3 Hz to 35 Hz for the frequency, 5% to 95% for the duty cycle, and -10V to +10V for the amplitude (including the baseline). The operator is able to change them at any time during shuttering using the GUI (Figure 2), and so is the test subject, using the potentiometers. The stimulation pulses (square or triangular waves) are displayed as a static plot and/or a running strip chart, which can be deselected at any time.
Unfortunately, USB 2.0 based systems operating under Windows, in conjunction with low-cost hardware, are not ideal for real-time operation. It can easily happen that the system must handle somewhere between 70 and 100 processes, running hundreds if not thousands of threads concurrently. This requires special attention to the way such systems are programmed, especially if analog outputs and inputs need to be serviced simultaneously.
Figure 2. The graphical user interface (GUI) of the application
2.3. Multitasking, Multithreading and Multiprocessing
The terms multitasking, multithreading, and multiprocessing all refer to distinctly different concepts, but are often used interchangeably [7]. Multitasking refers to the ability of an operating system to switch between tasks quickly to give the appearance of simultaneous execution of those tasks. In a single-threaded environment, such as the early Windows versions, a task is an application, i.e. Microsoft Word, Excel, PowerPoint, etc. These applications are written to share processor time with other applications. Each application performs operations and then gives up control of the microprocessor at certain points so that other applications can use it. This form of time sharing with the processor resources is also called non-preemptive multitasking, because the application itself controls how much time it uses and when it gives up control of the processor. Window NT and Windows 95 introduced the enhanced capabilities of preemptive multitasking, which transfers the control of processor time switching to the operating system, rather than relying on applications to yield voluntarily. This feature has a major impact on applications; when running in a preemptive multitasking system, applications can be suspended at any time. Therefore, they must be able to respond to changes in the system that occur while processor control is taken away from them in mid-operation. With multitasking working properly, all executing applications on the system appear to progress in parallel.
Multithreading extends the idea of multitasking to apply within applications, giving applications the ability to separate their own tasks into individual threads. The operating system can then divide processing time on these threads similar to the way it divided processing time between whole applications on the solely multitasking system. Thus, a multithreaded application can have multiple tasks progressing in parallel on the operating system along with other applications. While multithreading offers many advantages on single processor machines, implementing multithreading on machines with more than one processor is absolutely necessary to take advantage of the additional hardware for one’s application.
Multiprocessing refers to multiple processors in one computer, executing the threads that are ready to run. In a symmetric multiprocessing (SMP) system, the operating system automatically uses all of the processors in the computer to run any threads [8]. Once a multithreaded program is implemented, it can take full advantage of any number of processors within the system automatically. With multiprocessing power, a multithreaded application can truly run multiple threads simultaneously, finishing more tasks in less time. To achieve maximum performance from multithreaded operating systems and/or multiprocessor machines, an application must be multithreaded. Windows and some Unix operating systems, as well as other popular platforms, are multithreaded systems today. These operating systems run mostly on an increasingly popular multiprocessor desktop system. The multithreading software and multiprocessor computer technology are in place and readily available.
2.4. Multithreading Advantages for Instrumentation Users
Multithreading can offer significant benefits to the user, especially in PC-based data acquisition and instrument I/O applications, in cases where some tasks need a better deterministic outcome than others, or where user interface activity may run concurrently with hardware communication. On multiprocessor systems, a multithreaded application takes advantage of the additional hardware, usually resulting in greatly improved overall performance. Multithreaded applications can take full advantage of multiple processors to gain simultaneous execution of tasks. A well-implemented multithreaded application efficiently uses all the processors available for its own tasks and maximizes multiprocessor performance, whereas a single-threaded application must wait for each task to execute before continuing with the rest of the application. At no time can a single-threaded application execute on more than one processor in the system. Multithreading applications can help the operating system distribute the processor time more evenly among the different tasks it needs to complete, so that all tasks can move quickly towards resolution.
2.5. Building Multithreaded Applications in LabWindows/CVI
Building multithreaded applications can be difficult. Threads within a single application usually share data, so communication among the threads to coordinate data and resources is vital. If multithreading is not programmed properly, one may experience unexpected behavior. Conflicts can occur when multiple threads either request shared resources simultaneously, or share data space in memory. Describing the details of building multithreaded Windows programs in C is beyond the scope of this paper. We refer interested readers to the Microsoft web site for articles and technical papers on multithreading. However, to illustrate the benefits of multithreading for data acquisition or instrument control systems, we will review our software implemented in LabWindows/CVI, an ANSI C programming environment designed for scientists and engineers (http://www.ni.com/lwcvi/). LabWindows/CVI software has natively supported multithreaded application creation since the mid-1990s. Multithreaded libraries were included, that can be used with any C/C++ programming environment [9].
The LabWindows/CVI multithreading functions provide the following multiple performance optimizations over standard Interface to Win32 API threading functions:
§ Thread pools help schedule functions for execution in separate threads. They handle thread caching to minimize the overhead associated with creating and destroying threads.
§ Thread-safe queues abstract passing data between threads. One thread can read from a queue at the same time that another thread writes to the queue.
§ Thread-safe variables effectively combine a critical section and an arbitrary data type. One can call a single function to acquire the critical section, set the variable value, and release the critical section.
§ Thread locks simplify using a critical section or mutex by providing a consistent API and by automatically choosing the appropriate mechanism when necessary. For example, LabWindows/CVI automatically uses a mutex if the lock needs to be shared between processes or if threads need to process messages while waiting for the lock. A critical section is used in other cases because it is more efficient.
§ Thread-local variables provide per-thread instances of variables. The operating system places a limitation on the number of thread-local variables available to each process.
All of the multithreading functions in the LabWindows/CVI can be found in the Library Tree under Utility Library»Multithreading.
2.6. Protecting Data
A critical issue that one must address when using secondary threads is data protection. One must protect global variables, static local variables, and dynamically allocated variables from simultaneous access by multiple threads. Failure to do so can cause intermittent logical errors that are difficult to debug. Empirical evidence has shown that multithreaded programs with incorrect data protection typically run correctly during testing but fail immediately when installed on the target machine. Only data that is accessible from more than one thread in a program must be protected. Global variables, static local variables, and dynamically allocated memory are located in common memory areas that are accessible to all threads in a program. Data stored in these types of memory locations must be protected against concurrent access from multiple threads. LabWindows/CVI provides various high-level mechanisms that help to protect data from concurrent access [10]. The LabWindows/CVI Utility Library provides three mechanisms for protecting data – thread locks, thread-safe variables, and thread-safe queues.
A thread lock is a wrapper around a simple OS thread-locking object. The benefit of this approach is that the code is simpler and less error prone than an approach where you protect each piece of data individually. The drawback is decreased performance because threads in the program will tend to hold the lock longer than is actually necessary, which causes other threads to block (wait) longer than necessary to obtain a lock.
A thread-safe variable combines an OS thread-locking object with the data that it protects. This approach is simpler and less error prone than using a thread lock to protect a piece of data. One can use thread-safe variables to protect all types of data, including structure types. Thread-safe variables are less error prone than thread locks because one must call a Utility Library API function to get access to the data. Because the API function acquires the OS thread-locking object, one will not accidentally access the data without acquiring the OS thread-locking object.
A thread-safe queue is a mechanism for safely passing arrays of data between threads. One typically uses a thread-safe queue when the program contains one thread that generates an array of data and another thread that must operate on the array of data. In our case, the potentiometer-reading thread generates the data which is analyzed and displayed in the user interface by the main thread. A thread-safe queue has the following advantages over a thread-safe variable of an array type.
· Internally, thread-safe queues use a locking scheme in which one thread can read from the queue at the same time that another thread is writing to the queue (for example, reader and writer threads do not block each other).
· One can configure a thread-safe queue for event-based access. One can register a reader callback that is called when a certain amount of data is available in the queue and/or a writer callback that is called when a specified amount of space is available in the queue.
· One can configure a thread-safe queue to automatically grow if data is added when it is already full.
Using the LabWindows/CVI Utility Library thread-safe queue, we were able to safely pass data between threads. As mentioned before, it is most useful when one thread acquires the data and another thread processes that data. The thread-safe queue handles all the data locking internally. Generally, the secondary thread in the application acquires the data while the main thread reads the data when it is available and then analyzes and/or displays the data. The following code shows how a thread in our program uses a thread-safe queue to pass data to the main thread. The main thread uses a callback to read the data when available.
2.7. Running code in secondary threads in LabWindows/CVI
The thread in a single-threaded program is referred to as the main thread. The OS creates the main thread when the user tells the OS to begin executing a particular program. In a multithreaded program, the program itself tells the OS to create threads in addition to the main thread. These threads are referred to as secondary threads. A major difference between the main thread and secondary threads is where each type of thread begins execution. The OS begins execution of the main thread in the main (or WinMain)function. The developer specifies the function in which each secondary thread begins executing.
In a typical LabWindows/CVI multithreaded program, such as ours, we use the main thread to create, display, and run the user interface. It also runs the main function generation, which is timer--based. We use a secondary thread to perform other time-critical operations such as data acquisition from the potentiometers. The attempt to balance across the cores is shown in Figure 3.
Figure 3. Balancing across the processor cores using LabWindowsTM/CVI in a multithreaded application used to control shutter glasses.
3. Experimental results
This method was compared on five 64-bit machines running Windows 7, Windows 8.1, and Windows 10, three of them based on the i7 processor, and two on the i5 processor. The application was installed and run on all five computers, and the output pulses were measured with an oscilloscope. The frequencies and duty cycles measured were compared to the numerical settings that were selected by means of either the GUI or the potentiometers. The software was run first in GUI input mode, where the potentiometer inputs were blocked, i.e. no AIs were being read. Then the potentiometer inputs were enabled, and the AIs were being read concurrently with the stimulation. The simple GUI mode provided a much higher precision, approximately by an order of magnitude better than the potentiometer mode, where the secondary thread was enabled. Since the latter case was more important, providing feedback from the patient, we are giving the maximum errors for this mode in Table 1, at a shuttering frequency of 25 Hz, which is unlikely to be exceeded. The fluctuations are due to several factors: a) the timing resolution of the output pulses (1 ms), b) system delays due to the OS servicing other threads unrelated to this application, and c) delays introduced merely by the USB connection between the computer and the MF I/O module. While factor (a) can introduce errors up to ca. 0.6 Hz, factor (b) can introduce errors up to five times larger than the values in Table 1, if no multithreading is used, as was in earlier versions of our software. Factor (c) was not easy to assess, but it is known to play a role, because systems with MF modules interfaced directly via the peripheral bus and operating based on direct memory access (DMA) deliver significantly better time performance than USB based modules. Although the operating system did not appear to play a major role, the software was found to be most efficient on systems running Windows 8.1.
|
|
PC1 (desktop) i7, 3.4 GHz Windows 7 |
PC2 (laptop) i7, 2.6 GHz Windows 7 |
PC3 (laptop) i5, 2.6 GHz Windows 8.1 |
PC4 (desktop) i7, 3.4 GHz Windows 8.1 |
PC5 (laptop) i5, 2.4 GHz Windows 10 |
|
Max fs error |
0.9 Hz |
1.3 Hz |
1.1 Hz |
0.8 Hz |
1.6 Hz |
|
Max duty cycle error |
2.2% |
3.3% |
2.7% |
1.7% |
3.5% |
Table 1. Maximum errors measured at shuttering frequencies of fs=25 Hz, with potentiometers enabled.
4. Discussion
Apparently, the timing errors were due mainly to the number of tasks run by the OS. Our approach greatly improved real-time performance. Reducing the number of tasks initiated by other programs, including startups, generally helps in achieving predictable performance. An even more deterministic time behavior can be achieved by using symmetric multiprocessing (SMP), which is an operating system feature that allows multicore processors to run a single instance of an operating system, connect to a common main memory, and execute code in parallel. With this technology, one can move threads between processors to balance the workload efficiently. Most general-purpose operating systems such as Windows, Linux®, and Mac OS support SMP. However, real-time OS (RTOS) support for SMP is not trivial because the deterministic behavior of an RTOS must be preserved to meet hard real-time timing constraints while distributing threads across different processors. One of the few RTOSs that support SMP is the OS used by LabWindows/CVI Real-Time ETS targets. It helps to take full advantage of the latest multicore hardware by using processor affinity functions to control which cores are available for system use and which ones are reserved for specific threads [8].
There is little doubt that Windows OS is convenient to many users. Yet, Linux is known to have better real-time performance. National Instruments does provide run-time support on Linux for applications built with the LabWindows/CVI development environment for Windows. The tool is called LabWindows/CVI 2013 Run-Time Module for Linux. This means that applications developed with LabWindows/CVI on a Windows OS can be re-compiled and run on supported Linux distributions with the LabWindows/CVI Run-Time Module installed. The authors, however, believe that Linux is somewhat cumbersome to use with NI products, for reasons, such as lacking support for many Linux distributions, limited libraries support, limited support for NI devices, no LabWindows/CVI interactive development environment (IDE) for Linux, etc.
Multithreading offers significant benefits for data acquisition, instrument control, and general programming applications. As the computer industry continues to standardize multithreaded operating systems and symmetric multiprocessing machines, it becomes possible to implement multithreaded algorithms in inexpensive yet versatile medical instrumentation with only minimum requirements on the hardware.
REFERENCES
[2] S.H. Hay, L.N. Graham, Apparatus and method for treatment of amblyopia. United States Patent No. 6,511,175 B2, January 28, 2003.
[3] R. Herzog, O. Ben-Ezra, Y. Gross, Liquid-crystal eyeglass system. , PCT Patent Application WO 2005043224 A2. May 12, 2005.
[4] J. Sweis, V.L. Rice, D. Chao, Z. Guo, Shutter Eyewear. Unites States Patent No. 9,405,135 B2, August 2, 2016.
[5] A. Spierer, J. Raz, O. Benezra, R. Herzog, E. Cohen, I. Karshai, D. Benezra, Treating amblyopia with liquid crystal glasses: a pilot study, Invest Ophthalmol Vis Sci, 51 (2010) 3395-3398.
[7] National Instruments: Building Multithreading Applications with LabWindows™/CVI.
Last accessed on Nov 17, 2016
http://www.ni.com/white-paper/6728/en/
Last accessed on Nov 17, 2016
[9] National Instruments: Using LabWindows™/CVI Multithreading Functions.
http://zone.ni.com/reference/en-XX/help/370051V-01/cvi/programmerref/usingcvimultfuncs/
NI LabWindows™/CVI™ 2012 , Edition Date: August 2012, Part Number: 370051V-01
Last accessed on Nov 18, 2016
[10] National Instruments: Multithreading in LabWindows™/CVI.
http://www.ni.com/white-paper/3663/en/
Publish Date: Feb 02, 2012
Last accessed on Nov 18, 2016