0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Original Investigation |

Infrared-Based Blink-Detecting Glasses for Facial Pacing:  Toward a Bionic Blink FREE

Alice Frigerio, MD, PhD1,2; Tessa A. Hadlock, MD2; Elizabeth H. Murray, MS3; James T. Heaton, PhD3,4
[+] Author Affiliations
1Human Physiology Section, Department of Pathophysiology and Transplantation, Università degli Studi di Milano, Milan, Italy
2Facial Nerve Center, Carolyn and Peter Lynch Center for Laser and Reconstructive Surgery, Division of Facial Plastic and Reconstructive Surgery, Department of Otology and Laryngology, Massachusetts Eye and Ear Infirmary, Harvard Medical School, Boston
3Department of Communication Sciences and Disorders, MGH Institute of Health Professions, Boston, Massachusetts
4Department of Surgery, Harvard Medical School, Massachusetts General Hospital, Boston
JAMA Facial Plast Surg. 2014;16(3):211-218. doi:10.1001/jamafacial.2014.1.
Text Size: A A A
Published online

Importance  Facial paralysis remains one of the most challenging conditions to effectively manage, often causing life-altering deficits in both function and appearance. Facial rehabilitation via pacing and robotic technology has great yet unmet potential. A critical first step toward reanimating symmetrical facial movement in cases of unilateral paralysis is the detection of healthy movement to use as a trigger for stimulated movement.

Objective  To test a blink detection system that can be attached to standard eyeglasses and used as part of a closed-loop facial pacing system.

Design, Setting, and Participants  Standard safety glasses were equipped with an infrared (IR) emitter-detector unit, oriented horizontally across the palpebral fissure, creating a monitored IR beam that became interrupted when the eyelids closed, and were tested in 24 healthy volunteers from a tertiary care facial nerve center community.

Main Outcomes and Measures  Video-quantified blinking was compared with both IR sensor signal magnitude and rate of change in healthy participants with their gaze in repose, while they shifted their gaze from central to far-peripheral positions, and during the production of particular facial expressions.

Results  Blink detection based on signal magnitude achieved 100% sensitivity in forward gaze but generated false detections on downward gaze. Calculations of peak rate of signal change (first derivative) typically distinguished blinks from gaze-related eyelid movements. During forward gaze, 87% of detected blink events were true positives, 11% were false positives, and 2% were false negatives. Of the 11% false positives, 6% were associated with partial eyelid closures. During gaze changes, false blink detection occurred 6% of the time during lateral eye movements, 10% of the time during upward movements, 47% of the time during downward movements, and 6% of the time for movements from an upward or downward gaze back to the primary gaze. Facial expressions disrupted sensor output if they caused substantial squinting or shifted the glasses.

Conclusions and Relevance  Our blink detection system provides a reliable, noninvasive indication of eyelid closure using an invisible light beam passing in front of the eye. Future versions will aim to mitigate detection errors by using multiple IR emitter-detector units mounted on glasses, and alternative frame designs may reduce shifting of the sensors relative to the eye during facial movements.

Level of Evidence  NA.

Figures in this Article

Facial paralysis affects up to 0.3% of the population every year in western Europe and the United States.1 People usually experience unilateral facial paralysis, with preserved contralateral facial movement. One of the most bothersome issues is loss of eye blink due to paralysis of the orbicularis oculi muscle on the affected side. The perpetually open eye is at risk for exposure to debris and drying and results in chronic irritation and pain. In addition, the loss of eye blink may lead to permanent corneal damage from ulceration or infection. Aside from the functional impairment in blinking and other facial movements, facial paralysis is also a major psychological barrier to a healthy social life, and low thresholds for observer detection of blink asymmetry have been demonstrated.2,3 One possible way to restore blink is through a closed-loop neural prosthesis (facial pacing device) that detects normal blinking on one side of the face and simultaneously stimulates blinking on the paralyzed side. Because blinking is typically symmetrical, the healthy eye blink may serve as an appropriate trigger for prosthetically assisted blink on the contralateral side in facial paralysis.

One of the challenges in using healthy blinking as a trigger for induced blinking in facial palsy is to rapidly and accurately detect healthy blinks in a noninvasive and nondisruptive manner. The most frequently proposed method for noninvasively detecting blink on the intact side is through skin surface electromyographic recording of the orbicularis oculi muscle,47 followed by detection of periocular tissue movements through gyroscopes8 or accelerometers.9 Although these approaches demonstrate promise for accurately detecting blinks, they require placement of sensors on the face surface, which complicates device use and potentially influences dynamics of healthy blinks.

We have developed a novel approach to eye blink detection using infrared (IR) emission and detection as a safe, noncontact, invisible, and inexpensive indicator of eyelid closure over the eye. Our prototype system uses an IR light-emitting diode (LED) and an IR detector mounted on a pair of glasses. The LED and detector components are positioned at the nasal and temporal aspects of one eye, causing the IR beam to pass horizontally across the central portion of the palpebral fissure, just anterior to the corneal surface (ie, the pupil location when looking straight ahead). The beam remains unbroken when the eye is open but is interrupted by the eyelashes and/or eyelid tissue when the upper eyelid descends, causing the circuit receiving the IR detector signal to register a blink.

The aim of this study was to test our IR eye blink detection apparatus in healthy participants while they assumed multiple gaze positions (ie, primary, lateral, upwards, downwards) and facial expressions (broad smile, squinting, eyebrow raising). By comparing video-recorded blinks vs the output of our blink detection system, we sought to establish the sensitivity of the new system to complete and partial blinks during common gaze positions and facial expressions.

Blink Detection Prototype Glasses and Circuit

Institutional review board approval was obtained by Massachusetts Eye and Ear Human Studies Committee, and written informed consent was obtained for all participants.

The blink detection system uses IR LEDs and detectors mounted on 3 pairs of laboratory safety eyeglasses as a proof of concept, with each pair of glasses differing slightly in terms of the relative position of the IR emitter-detector units in relation to the nose bridge to accommodate a range of adult face shapes. The 3 pairs had beam positions that differed by less than 2 mm in relation to the lower rim and front surface of the glasses lens. The main difference among the glasses was that pair 2 had slightly narrower nose pad separation compared with the wider nose pads of pairs 1 and 3. In addition, pair 3 had the beam falling slightly closer to the lens surface (1.3 mm closer than pair 1 and 1 mm closer than pair 2).

For each prototype pair of glasses, the IR LED (QEC 123; Fairchild Semiconductor Corporation) has a peak emission wavelength of 880 µm, which is also the frequency of maximal sensitivity for the matching phototransistor (QSC 114; Fairchild Semiconductor Corporation). The LED and detector components are positioned at the nasal and temporal aspects of one eye, respectively, causing the IR beam to pass horizontally across the central portion of the palpebral fissure, just anterior to the corneal surface (Figure 1). The beam remains unbroken when the eye is open but is interrupted by the eyelashes and/or eyelid tissue when the upper eyelid descends, causing the circuit receiving the IR detector signal to drop in output voltage (Figure 2). The LED is powered with electrical pulses at a frequency of 5.2 kHz, and the detector compares the light received during pulses vs IR light detected between pulses as a means of rejecting signal artifact due to changes in ambient light levels. An opaque light shield is mounted on the detector to reduce the potential impact of IR light entering from glancing angles (Figure 1). The IR detection circuit functions well in typical indoor and outdoor lighting conditions, except when the phototransistor directly faces the sun or bright incandescent bulbs are in close proximity (ie, within a few feet).

Place holder to copy figure label and caption
Figure 1.
Prototype Blink Detection Pair of Glasses

Frontal (A) and lateral (B) views of a prototype blink detection pair of glasses with an arrow demonstrating the path in infrared (IR) light traveling horizontally across the corneal surface from the IR LED (nasal side) toward the IR detector (temporal side).

Graphic Jump Location
Place holder to copy figure label and caption
Figure 2.
Flow Diagram of the Eye Blink Detection System

The diagram shows the (1) infrared (IR) emitter and detector units in relation to the eye, (2) variable gain of the preamplifier receiving the phototransistor output, (3) signal filtering, and (4) final output of the circuit as a time-varying analogue voltage ranging from 0 to 5 V. An unbroken beam generates a 5-V output, whereas a completely obstructed beam generates a 0-V output, with partial obstruction producing voltages in between these extremes. LED indicates light-emitting diode.aFairchild Semiconductor Corporation.

Graphic Jump Location

Infrared light emitted from the LED in each prototype pair of glasses does not pose a known risk to the eye (eg, lens, cornea, retina). Maximal power output of the IR LEDs was 0.62 mW, as measured at the LED lens surface using an IR-sensitive light meter (Ophir Nova II Meter and Photodiode Power Sensor PD300 [Ophir Optronics Solutions Ltd], with appropriate filter setting). Even if the LED used in our system were inadvertently directed into the eye from the corneal surface for 8 consecutive hours, the amount of light energy delivered would be 2.13 log units below the maximal permissible exposure levels set forth by American National Standards Institute code Z136 for laser safety standards (ie, more than 2 orders of magnitude below a safe exposure level).

Blink Detection Participants

Twenty-four volunteers aged 22 to 62 years (10 male and 14 female [20 white, 4 Asian, and 1 African American]) were recruited from the Massachusetts Eye and Ear Infirmary (MEEI) community (eg, students, employees, and visitors). Individuals with a history of eye or retinal disease, neuromuscular disease (ie, myasthenia gravis, essential blepharospasm, facial palsy), or pathological exophthalmos (ie, Basedow-Graves disease) were excluded from participation. Seventeen individuals were younger than 40 years, 6 were aged between 40 and 59 years, and only 1 was older than 60 years (Table).

Table Graphic Jump LocationTable.  Age, Sex, and Race/Ethnicity of the Study Population
Blink Detection Apparatus and Data Collection Procedures

Blink detection system testing took place in a distraction-free room within the MEEI Facial Plastic Surgery Unit. Participants were briefly tested and screened with the 3 prototype pairs of eyeglasses to determine which pair provided the best blink detection given their particular orbital anatomy. They were then asked to stand 50 cm from a SMART Board screen (SBX880; SMART Technologies Inc) with their chin resting on an adjustable platform, which stabilized their head and oriented their forward gaze directly toward the screen. The testing session began with a 101-second instructional video projected in the center of the screen directly in front of the participant (Figure 3A and B). The video was a close-up view of woman’s face as she explained details of the testing session, including (1) the total session length (6 minutes), (2) the need to keep the head steady and to move eyes only when instructed (rather than moving the head), (3) the task of visually tracking a large dot in various locations on the screen, and (4) the task of mimicking various facial expressions being modeled on the screen. Data collection began at the start of the introductory video in order to sample spontaneous blinking during forward gaze.

Place holder to copy figure label and caption
Figure 3.
Blink Detection System Testing and Data Collection

A, A participant is shown standing in position for the 6-minute blink detection recording session, photographed during the initial 102-second instructional video. B, An adjustable chin rest helped participants maintain a forward head position, and a forward gaze was held through the instructional video (during which spontaneous blinks were detected). C, Photograph of a participant assuming an upper right gaze position as she looked at a target (blue dot). Targets appeared in the 4 corners of the smart board screen, in addition to center, upper center, and lower center (see Methods section for presentation sequence).

Graphic Jump Location

Data acquisition continued in the second part of the testing session, when participants were instructed to focus their gaze on a large dot appearing at various locations on the projection screen, without moving their head when changing their gaze position (Figure 3C). The dot changed position every 5 seconds, with an audible tone indicating when the dot was changing position. The sequence of dot locations was repeated twice and included the following sequence: center, center up, center down, center, left, left up, left down, center, right, right up, right down, center. This enabled testing of the effect of gaze direction on the blink detection circuit output, since upper eyelid position can potentially change based on gaze change and position (particularly for downward gaze).

In the third part of the testing session, participants were shown photographs of 4 different facial expressions (neutral, smiling, squinting, and eyebrows raising) and asked to mimic each expression throughout the 4 seconds each expression was shown. The neutral expression was mimicked first and also fell between each of the other 3 expressions. Each time the demonstration expression changed, an audible narration accompanied the visual cue, informing participants of the desired facial expression.

Recorded data included video and audio obtained by a video camera (Canon VIXIA HF R200; 1440 × 1080 pixels, 11 951 kbps video; 48 kHz, 256 kbps audio; Canon USA Inc) positioned on a tripod with a close-up frontolateral view of each participant’s eye being monitored and an output voltage from the blink detection circuit and room microphone digitized at 20 kHz using analogue-to-digital hardware (DigiData 1440 and a Cyberamp 380 programmable filter/amplifier; Molecular Devices) and software (Clampex 10.2; Molecular Devices). Video files were viewed on a computer using software (Corel VideoStudio Pro X5; Corel Corporation) that allowed the viewer to indicate each point in time (frame) when a blink or eye twitch began (29.97 frames per second [fps]). A blink was noted when the upper and lower edge of the pupil was at least partially covered by the eyelids, and a twitch was noted when there was a conspicuous movement of one or both eyelids without covering the upper and lower edge of the pupil. Each video was viewed a minimum of 2 times to ensure that all blinks were noted, and observers were blinded to the blink detection circuit output. The time locations for observed eyelid movements, gaze position changes, and mimicked facial expressions were compared with absolute value and rate of change of the blink detection circuit output.

Rate of signal change, represented by the signal’s derivative (ie, instantaneous velocity), was calculated in Matlab software using the “diff” function (MathWorks Inc) after down sampling from 20 kHz to 1 kHz. Signals that started from 3 V or more and had derivative values of either −0.1 or less or −0.2 or less were scored as blinks. Of these 2 levels, the particular derivative threshold applied for blink assessment was determined on an individual basis according to which level produced the least amount of detection error (ie, fewest false positives and false negatives). Once a threshold level was selected for an individual, that level was applied for blink detection throughout their recording session. Signal changes that occurred less rapidly than the applied threshold or that occurred from a staring level less than 3 V were not scored as blinks, nor were signals that occurred within 250 milliseconds (ms) of a detected blink (to avoid potential artifact associated with eye opening at the end of each blink).

To qualitatively relate blink detection circuit output (volts) with change in palpebral fissure during blinking, the blink detection system was also used for 2 participants (a man aged 43 years and a woman aged 32 years) in conjunction with high-speed video recording of eyelid movement (1000 fps, 240 × 256 pixels; KayPentax B/W System).

At least 1 of the 3 prototype pairs of eyeglasses fit each participant adequately for blink detection. The 3 different glasses versions (pairs 1-3) were worn by 12 (50%), 9 (38%), and 3 (13%) participants, respectively, with fairly even numbers of men and women wearing the 2 most commonly worn versions (pairs 1 and 2). Pairs 1 and 2 differed primarily in the position of the nose bridge pads, causing pair 1 to rest slightly lower on the face than pair 2 owing to differing gaps between nose bridge pads. Pair 3 had a beam falling 1.3 mm or 1 mm further from the eye surface than pairs 1 and 2, respectively, which provided the best blink detection during the initial testing and fitting in only 3 participants. In all 4 participants of Asian descent and 2 of the white participants, the lower rim of the glasses rested on the upper cheek (lower orbit and malar fat pad) rather than the bridge of the nose and therefore moved with changes in facial expression, potentially decreasing blink detection accuracy. At no time did participants report seeing light or feeling heat from the IR LED, consistent with the 880-µm peak wavelength and relatively low power output of the LED and driving circuit.

The airing of the instructional video provided an opportunity to measure spontaneous blinks from participants during a forward gaze. The 24 participants produced a range of 1 to 76 complete blinks (mean [SD], 27.3 [17.7]) during the 101 seconds of the instructional video.

A relatively unbroken light path between the IR emitter and detector caused a relatively high voltage output (typically 4-5 V) from the blink detection circuit. When the IR light path was broken, the circuit output voltage dropped toward a low state (typically 0-1 V; Video 1). Drops in the circuit voltage caused by eyelid obstruction of the IR light path during blinking were generally faster than when light path obstruction occurred during gaze changes and facial expressions (ie, squinting and broadly smiling), prompting us to consider not only the direction and magnitude of signal change, but also the rate of signal change when identifying blinks from the detection circuit output (Figure 4; Video 2).

Video 1.

Infrared (IR) Voltage Output During Forward Gaze and Gaze Movements

An unbroken light path between the IR emitter and detector causes a relatively high voltage output from the blink detection circuit. In forward gaze, at each blink event the IR beam is broken by eyelids/lashes and the circuit output voltage drops toward a low state. During gaze movements, the IR beam is occasionally broken (mainly during downward gaze and 1 leftward gaze event in this video).

Place holder to copy figure label and caption
Figure 4.
Examples of the Blink Detection Circuit Output for a Blink and Downward Gaze

The detector receives a relatively unobstructed beam during forward gaze, producing an output in the 4- to 5-V range (A, time zero), but drops in voltage when the beam is broken by the lowering upper eyelid during a blink or looking downward. This drop in voltage is typically more rapid for blinking than gaze-related eyelid movement, as shown by the slope (downward pointing arrows) and by the first derivative of the output signals (B). The difference in peak derivative for blink vs downward gaze is highlighted by the dotted lines and double-headed arrow.

Graphic Jump Location

Video 2.

Blink vs Downward Gaze

High-speed video recording of the circuit output during a blink event followed by downward gaze. Drops in the circuit voltage caused by interruption of the infrared light path during blinking are generally faster than when light path obstruction occurs during downward gaze. Thresholding on peak rate of signal change (first derivative) can help distinguishing blinks from slower, gaze-related eyelid movements.

Blink detection based on change in signal magnitude achieved 100% sensitivity in forward gaze but typically generated false detections on downward gaze and, to a lesser degree, other gaze changes as well. Blink detection based on the peak rate of signal change (first derivative) during the instructional video where participants maintained a primary (forward) gaze yielded an average correct detection of 87%, with 11% false positives, and 2% false negatives. Of the 11% false positives, slightly over half (6%) were associated with eye twitches (ie, partial eyelid closures). Therefore, the system had reasonably good sensitivity for blink detection during primary gaze (missing only 2% of observed blinks on average and missing no blinks in 14 of 24 participants), but the system falsely detected blinks when no visible eyelid movement occurred for 5% of the detected blink events. For the 21 gaze changes (starting from a primary gaze) tested in each participant, the resulting blink detection signal pattern was falsely identified as a blink 6% of the time for lateral eye movements, 10% of the time for upward movements, 47% of the time for downward movements, and 6% of the time for movements from an upward or downward gaze back to the central (primary) gaze.

Blink detection was not attempted during the mimicked facial expressions, but change in the signal baseline during these expressions would have often precluded blink detection owing to interruption of the IR light beam by the upper and/or lower eyelid. Interruption in the IR light path during these expressions was typically associated with partial eyelid closure (ie, during squinting or broad smile) but was also affected by shifting of the glasses by movement of skin at the bridge of the nose or upper cheek (ie, during eyebrow raising and broad smile). Specifically, the IR circuit output was reduced to 2.5 V (half) or less during broad smile in 14 participants (59%), during squinting in 23 participants (96%), and during eyebrow raising in 5 participants (21%).

A comparison between the blink detection circuit output and high-speed video (1000 fps) recordings of eyelid movement indicated that the initiation of signal change (deflection) from baseline was synchronous with visible eyelid movement (Video 3 and Video 4 show circuit output voltage during eye blink and twitch, respectively). The mean (SD) latency from the start of signal deflection during a blink to peak derivative was 45.6 (18.9) ms.

Video 3.

Infrared Signal of a Complete Eye Blink

High-speed video capture of the circuit voltage output during a complete blink. The blink closing phase causes gradual interruption of the infrared light path, with proportional drop of the circuit voltage toward the lowest state (complete beam interruption), followed by a gradual increase of the voltage during the eye opening.

Video 4.

Infrared Signal of an Eye Twitch

High-speed video capture of the circuit voltage output during an eye twitch. The onset of signal change is synchronous with visible eyelid movement. The shape of the signal reflects that the light path is only partially interrupted by the twitch movement. This drop in the voltage output is not detected as a blink event because it is under the threshold.

People typically experience facial paralysis on only one side of their face, with preserved contralateral facial movement. Since spontaneous mimetic movements are usually symmetrical, in cases of unilateral paralysis a device can record and process a biosignal when the healthy hemiface spontaneously moves. This approach to facial reanimation is referred to as closed-loop facial pacing.

With regard to the eye blink, the signal recorded from the healthy eye can be processed to trigger a train of electrical pulses that initiate an artificial blink from the paralyzed eye. We are developing a closed-loop facial pacing device to restore eye blink in individuals with unilateral paralytic lagophthalmos. Individuals affected by acute reversible facial paralysis with a prognosis for recovery within 12 months would benefit from an external apparatus to facilitate eye closure during waking hours, until the recovery process is complete. Individuals affected by irreversible facial paralysis would more likely benefit from a permanent, implanted neuroprosthesis10 using intramuscular stimulation, which could also be driven by biosignals from the healthy eye.

Our envisioned closed-loop eye blink pacing system for acute facial palsy is an assembly of 3 different components aimed at (1) detecting blink events on the healthy side, (2) processing the blink-related biosignal to activate a stimulation unit, and (3) pacing the contralateral facial nerve or orbicularis oculi muscle, thus eliciting a biomimetic eye blink. Both detection and stimulation hardware could ultimately be incorporated into the frame of glasses, but we expect that intermediate iterations will be tethered between the glasses and a small rechargeable signal processing unit worn under the clothes.

Herein, we describe a novel approach to eye blink detection that provides a robust and reliable indication of eyelid closure, while maintaining the advantages of IR emission and detection as a safe, noncontact, invisible, and inexpensive reporter of blink occurrence and duration. Our laboratory developed a blink detection system using an IR LED and a phototransistor (IR detector) mounted on laboratory safety eyeglasses as a proof of concept, but the device could be configured to clip onto nearly any pair of glasses. We mounted IR emitter-detector components in slightly different locations on 3 pairs of glasses to improve eyelid movement sensing across participants, with pairs 1 and 2 differing primarily on the vertical height of the IR beam in relation to the eye surface and pair 3 differing in the beam’s horizontal position. These slight variations helped us accommodate for differences in subject facial anatomy; however, clinically deployed versions of the glasses would need to provide adjustable positioning of the IR emitter-detector unit in relation to the frame (and therefore in relation to the user’s orbit) to accommodate a wider range of face shapes and achieve optimal blink detection.

Eye blink detection using IR light or miniature cameras is commonly performed in the context of gaze tracking (eg, for visual scanning research and eye-controlled systems) but requires bulky, conspicuous hardware that would be impractical for detecting and restoring blink throughout the day. These systems demonstrate the safe use of IR light for detecting eye and eyelid position, particularly considering that they typically shine IR light directly into the eye,11 whereas our proposed system directs the IR light in front of the eye surface.

Infrared light has also long been used to measure blinks in animal research by detecting the change in light reflectance from the eye surface vs the upper and/or lower eyelid surface during eyelid closure.1214 The globe of the eye is relatively translucent to IR light, causing little detected reflectance when an IR emitter-detector unit is positioned directly in front of the eye. In laboratory animals with fur-covered eyelids (eg, rats and rabbits), IR reflectance increases substantially when the eyelids close over the eye surface because of the high IR reflectivity of fur. Human eyelids are much less reflective to IR light, generating more subtle differentiation between open vs closed eyelids. Nevertheless, IR reflectance has been used to detect blink behavior in humans as an indicator of drowsiness in automotive drivers15 and aviators.16 Sleep Diagnostics Pty Ltd has developed such a method and a device (called Optalert) specifically for use while driving. This system provides a measure of the amplitude to velocity ratio for blinks and saccades,17 but it is unclear how rapidly the hardware is able to detect the onset of blink movement since a rapid response latency is not required for drowsiness assessment.

Successful application of IR-based blink detection for facial pacing requires minimal detection errors because both false triggers and missed blinks result in unilateral eye closure that can be conspicuous and bothersome. A challenge of the envisioned blink detection application is identifying the best trade-off between type I and II errors; in other words, between making the paralyzed eye spuriously wink vs missing true blink events, respectively. Our results demonstrated that blink detection based on signal magnitude achieved 100% sensitivity in forward gaze but generated false detections whenever participants looked downward due to the natural lowering of the upper eyelid (breaking the IR beam). Thresholding on peak rate of signal change (first derivative) helped distinguishing blinks from slower, gaze-related eyelid movements, correctly rejecting 54% of downward gazes as nonblinks. Analysis of blink detection during forward gaze revealed that 5% of the detected blink events did not correspond to any visible eyelid movement (type I error) and that only 2% of observed blinks went undetected (type II error). The application of error minimization algorithms on an individual basis could reduce this error by optimizing detection thresholds, or at least strike a better balance between error types.

Blink detection with the initial proof-of-concept configuration of a single IR LED (emitter) and phototransistor (detector) could be improved by adding 1 or more additional emitters and/or detectors. This would decrease the likelihood of nonblink eyelid movement (associated with change in gaze position) from generating false-positive blink events. Having a row of 2 or 3 opposing emitter-detector pairs mounted parallel to the eye surface may provide an alternative pathway for the IR beams to travel when the eyelids move up or down in concert, yet remain separated (eg, during downward gaze). Moreover, an array of emitters and detectors may make the detection system more robust to shifting of eyeglass position as people move their heads or move the skin on the bridge of their nose during facial expression.

A special requirement of the envisioned device is detecting blinks events early enough to trigger and pace synchronous movements. Human visual information updates at rates of approximately 30 to 33 Hz,18 allowing for 33 ms in right-left movement asymmetry before being noticed. However, eyelid closure during blink is not instantaneous, since the process of eyelid closing and reopening spans approximately 100 to 300 ms.19 Therefore dyssynchrony of one eye in relation to the other will produce an asymmetry in eyelid position lasting much of the blink duration plus the latency between the faster and slower sides. The longer the blink latency between the 2 eyes, the longer the asymmetry period and greater the discrepancy in right-left palpebral fissure size throughout the blink. In the present report, the mean (SD) latency from the start of blink movement to blink detection (ie, peak of IR signal derivative) was 45.6 (18.9) ms, indicating that stimulated blink would be conspicuously asymmetrical if triggered from our current detection system.

Our research group recently quantified the perceptual impact of side-to-side blink movement asymmetry using video files that systematically altered to differ in the latency of right vs left blink initiation (0-264 ms latency difference across 6 tested steps). The results showed that more than 65% of participants detected asymmetrical eye blinks at the shortest tested delay (1 video frame or 33 ms3). In addition, side-to-side asymmetry in blinking was perceived as unnatural sooner than other tested facial expressions (ie, smile, raising eyebrows, lower lip depression), demonstrating observers’ particular sensitivity to blink asymmetry compared with other facial movements. Further work is required to reduce detection latency in the proposed system, perhaps through detection threshold optimization and individualized sensor alignment, since asymmetrical eyelid movement and static position can easily draw unwanted attention.2,3

Electromyography has long been used as a neural interface for prosthetic control20,21 and represents the most popular recording technology applied to closed-loop facial pacing,47 holding potential for real-time eye blink detection and pacing. Indeed, electromyographic recording can detect the onset of the electrical activity of the orbicularis oculi muscle 11 ms before the eyelid movement starts for voluntary and spontaneous blinks.19 The downside of skin surface electromyography-based eye blink detection is its vulnerability to electrical cross talk with adjacent muscles; addressing this noise requires the application of software filters and implies approximately 20% missed blink events.9 Blink detection approaches that are IR-based, electromyography-based, and perhaps a combination of the two are likely to be fruitful toward the development of an external blink restoration system. Moving forward, blink elicitation studies in patients with unilateral facial palsy are necessary to complete the link between blink detection and generation in our envisioned biomimetic prosthesis.

Corresponding Author: Tessa A. Hadlock, MD, Facial Nerve Center, Division of Facial Plastic and Reconstructive Surgery, Department of Otology and Laryngology, Massachusetts Eye and Ear Infirmary, Harvard Medical School, 243 Charles St, Boston, MA 02114 (tessa_hadlock@meei.harvard.edu).

Accepted for Publication: December 6, 2013.

Published Online: April 3, 2014. doi:10.1001/jamafacial.2014.1.

Author Contributions: Drs Frigerio and Heaton had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Frigerio, Hadlock, Murray, Heaton.

Acquisition, analysis, or interpretation of data: Frigerio, Hadlock, Murray, Heaton.

Drafting of the manuscript: Frigerio, Hadlock, Heaton.

Critical revision of the manuscript for important intellectual content: Frigerio, Hadlock, Murray, Heaton.

Statistical analysis: Hadlock, Murray, Heaton.

Obtained funding: Hadlock.

Administrative, technical, or material support: Frigerio, Hadlock.

Study supervision: Frigerio, Hadlock, Heaton.

Conflict of Interest Disclosures: None reported.

Additional Contributions: The first author is grateful to Paolo Cavallari MD, PhD, director of the Human Physiology Section, Department of Pathophysiology and Transplantation, Università degli Studi di Milano, Milano, Italy, and Giacomo Colletti, MD, Maxillo-Facial Unit of San Paolo Hospital, Università degli Studi di Milano, for their inspiring influence to pursue facial pacing research. David Ahlgren, PhD, Department of Engineering, Trinity College, Hartford, Connecticut, designed and built the blink detection circuit.

Schrom  T, Bast  F.  Surgical treatment of paralytic lagophthalmos [in German]. HNO. 2010;58(3):279-288.
PubMed   |  Link to Article
Hohman  MH, Kim  SW, Heller  ES, Frigerio  A, Heaton  JT, Hadlock  TA.  Determining the threshold for asymmetry detection in facial expressions [published online July 30, 2013]. Laryngoscope. doi:10.1002/lary.24331.
PubMed
Kim  SW, Heller  ES, Hohman  MH, Hadlock  TA, Heaton  JT.  Detection and perceptual impact of side-to-side facial movement asymmetry. JAMA Facial Plast Surg. 2013;15(6):411-416.
PubMed   |  Link to Article
Chen  K, Chen  T-C, Cockerham  K, Liu  W. Closed-loop eyelid reanimation system with real-time blink detection and electrochemical stimulation for facial nerve paralysis. Paper presented at: International Symposium on Circuits and Systems (ISCAS 2009); May 24-27, 2009; Taipei, Taiwan. doi:10.1109/ISCAS.2009.5117807.
Kurita  M, Takushima  A, Muraoka  Y, Shiraishi  T, Harii  K.  Feasibility of bionic reanimation of a paralyzed face: a preliminary study of functional electrical stimulation of a paralyzed facial muscle controlled with the electromyography of the contralateral healthy hemiface. Plast Reconstr Surg. 2010;126(2):81e-83e.
PubMed   |  Link to Article
Frigerio  A, Brenna  S, Cavallari  P.  Surface electromyography recording of spontaneous eyeblinks: applications in neuroprosthetics. Otolaryngol Head Neck Surg. 2013;148(2):209-214.
PubMed   |  Link to Article
Yi  X, Jia  J, Deng  S, Shen  SG, Xie  Q, Wang  G.  A blink restoration system with contralateral EMG triggered stimulation and real-time artifact blanking. IEEE Trans Biomed Circuits Syst. 2013;7(2):140-148.
PubMed   |  Link to Article
Marcelli  E, Cavallari  P, Frigerio  A,  et al.  A new gyro-based method for quantifying eyelid motion. Int J Artif Organs. 2013;36(3):195-202.
PubMed   |  Link to Article
Frigerio  A, Cavallari  P.  A closed-loop stimulation system supplemented with motoneurone dynamic sensitivity replicates natural eye blinks. Otolaryngol Head Neck Surg. 2012;146(2):230-233.
PubMed   |  Link to Article
McDonnall  D, Askin  RE, Smith  CF, Guillory  KS. Verification and validation of an electrode array for a blink prosthesis for facial paralysis patients. Paper presented at: 2013 Sixth International IEEE/EMBS Conference on Neural Engineering (NER 2013). November 6-8, 2013; San Diego, CA. doi:10.1109/NER.2013.6696146.
Holmiqvist  K, Nistrom  M, Andersson  R, Dewhurst  R, Jarodzka  H, van de Weijer  J, eds. Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford, England: Oxford University Press; 2011.
Heaton  JT, Kowaleski  JM, Bermejo  R, Zeigler  HP, Ahlgren  DJ, Hadlock  TA.  A system for studying facial nerve function in rats through simultaneous bilateral monitoring of eyelid and whisker movements. J Neurosci Methods. 2008;171(2):197-206.
PubMed   |  Link to Article
Ryan  SB, Detweiler  KL, Holland  KH, Hord  MA, Bracha  V.  A long-range, wide field-of-view infrared eyeblink detector. J Neurosci Methods. 2006;152(1-2):74-82.
PubMed   |  Link to Article
Thompson  LT, Moyer  JR  Jr, Akase  E, Disterhoft  JF.  A system for quantitative analysis of associative learning, part 1: hardware interfaces with cross-species applications. J Neurosci Methods. 1994;54(1):109-117.
PubMed   |  Link to Article
Leder  RC, Stampi  C, Webster  JG.  Blink size and lid velocity from ambulatory subjects: an analog, retroflective eyelid sensor. J Sleep Res. 1996;(suppl 1):121.
Corbett  MA.  A drowsiness detection system for pilots: Optalert. Aviat Space Environ Med. 2009;80(2):149.
PubMed   |  Link to Article
Johns  MW.  The amplitude-velocity ratio of blinks: a new method for monitoring drowsiness. Sleep. 2003;26(suppl):A51-A52.
Rodriguez  M, Mendez  E.  Quantal processing of visual information in the brain. Neuroscience. 1998;84(3):641-644.
PubMed   |  Link to Article
Evinger  C, Manning  KA, Sibony  PA.  Eyelid movements: mechanisms and normal data. Invest Ophthalmol Vis Sci. 1991;32(2):387-400.
PubMed
Jacobsen  SC, Knutti  DF, Johnson  RT, Sears  HH.  Development of the Utah artificial arm. IEEE Trans Biomed Eng. 1982;29(4):249-269.
PubMed   |  Link to Article
Goldstein  EA, Heaton  JT, Kobler  JB, Stanley  GB, Hillman  RE.  Design and implementation of a hands-free electrolarynx device controlled by neck strap muscle electromyographic activity. IEEE Trans Biomed Eng. 2004;51(2):325-332.
PubMed   |  Link to Article

Figures

Place holder to copy figure label and caption
Figure 1.
Prototype Blink Detection Pair of Glasses

Frontal (A) and lateral (B) views of a prototype blink detection pair of glasses with an arrow demonstrating the path in infrared (IR) light traveling horizontally across the corneal surface from the IR LED (nasal side) toward the IR detector (temporal side).

Graphic Jump Location
Place holder to copy figure label and caption
Figure 2.
Flow Diagram of the Eye Blink Detection System

The diagram shows the (1) infrared (IR) emitter and detector units in relation to the eye, (2) variable gain of the preamplifier receiving the phototransistor output, (3) signal filtering, and (4) final output of the circuit as a time-varying analogue voltage ranging from 0 to 5 V. An unbroken beam generates a 5-V output, whereas a completely obstructed beam generates a 0-V output, with partial obstruction producing voltages in between these extremes. LED indicates light-emitting diode.aFairchild Semiconductor Corporation.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 3.
Blink Detection System Testing and Data Collection

A, A participant is shown standing in position for the 6-minute blink detection recording session, photographed during the initial 102-second instructional video. B, An adjustable chin rest helped participants maintain a forward head position, and a forward gaze was held through the instructional video (during which spontaneous blinks were detected). C, Photograph of a participant assuming an upper right gaze position as she looked at a target (blue dot). Targets appeared in the 4 corners of the smart board screen, in addition to center, upper center, and lower center (see Methods section for presentation sequence).

Graphic Jump Location
Place holder to copy figure label and caption
Figure 4.
Examples of the Blink Detection Circuit Output for a Blink and Downward Gaze

The detector receives a relatively unobstructed beam during forward gaze, producing an output in the 4- to 5-V range (A, time zero), but drops in voltage when the beam is broken by the lowering upper eyelid during a blink or looking downward. This drop in voltage is typically more rapid for blinking than gaze-related eyelid movement, as shown by the slope (downward pointing arrows) and by the first derivative of the output signals (B). The difference in peak derivative for blink vs downward gaze is highlighted by the dotted lines and double-headed arrow.

Graphic Jump Location

Tables

Table Graphic Jump LocationTable.  Age, Sex, and Race/Ethnicity of the Study Population

References

Schrom  T, Bast  F.  Surgical treatment of paralytic lagophthalmos [in German]. HNO. 2010;58(3):279-288.
PubMed   |  Link to Article
Hohman  MH, Kim  SW, Heller  ES, Frigerio  A, Heaton  JT, Hadlock  TA.  Determining the threshold for asymmetry detection in facial expressions [published online July 30, 2013]. Laryngoscope. doi:10.1002/lary.24331.
PubMed
Kim  SW, Heller  ES, Hohman  MH, Hadlock  TA, Heaton  JT.  Detection and perceptual impact of side-to-side facial movement asymmetry. JAMA Facial Plast Surg. 2013;15(6):411-416.
PubMed   |  Link to Article
Chen  K, Chen  T-C, Cockerham  K, Liu  W. Closed-loop eyelid reanimation system with real-time blink detection and electrochemical stimulation for facial nerve paralysis. Paper presented at: International Symposium on Circuits and Systems (ISCAS 2009); May 24-27, 2009; Taipei, Taiwan. doi:10.1109/ISCAS.2009.5117807.
Kurita  M, Takushima  A, Muraoka  Y, Shiraishi  T, Harii  K.  Feasibility of bionic reanimation of a paralyzed face: a preliminary study of functional electrical stimulation of a paralyzed facial muscle controlled with the electromyography of the contralateral healthy hemiface. Plast Reconstr Surg. 2010;126(2):81e-83e.
PubMed   |  Link to Article
Frigerio  A, Brenna  S, Cavallari  P.  Surface electromyography recording of spontaneous eyeblinks: applications in neuroprosthetics. Otolaryngol Head Neck Surg. 2013;148(2):209-214.
PubMed   |  Link to Article
Yi  X, Jia  J, Deng  S, Shen  SG, Xie  Q, Wang  G.  A blink restoration system with contralateral EMG triggered stimulation and real-time artifact blanking. IEEE Trans Biomed Circuits Syst. 2013;7(2):140-148.
PubMed   |  Link to Article
Marcelli  E, Cavallari  P, Frigerio  A,  et al.  A new gyro-based method for quantifying eyelid motion. Int J Artif Organs. 2013;36(3):195-202.
PubMed   |  Link to Article
Frigerio  A, Cavallari  P.  A closed-loop stimulation system supplemented with motoneurone dynamic sensitivity replicates natural eye blinks. Otolaryngol Head Neck Surg. 2012;146(2):230-233.
PubMed   |  Link to Article
McDonnall  D, Askin  RE, Smith  CF, Guillory  KS. Verification and validation of an electrode array for a blink prosthesis for facial paralysis patients. Paper presented at: 2013 Sixth International IEEE/EMBS Conference on Neural Engineering (NER 2013). November 6-8, 2013; San Diego, CA. doi:10.1109/NER.2013.6696146.
Holmiqvist  K, Nistrom  M, Andersson  R, Dewhurst  R, Jarodzka  H, van de Weijer  J, eds. Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford, England: Oxford University Press; 2011.
Heaton  JT, Kowaleski  JM, Bermejo  R, Zeigler  HP, Ahlgren  DJ, Hadlock  TA.  A system for studying facial nerve function in rats through simultaneous bilateral monitoring of eyelid and whisker movements. J Neurosci Methods. 2008;171(2):197-206.
PubMed   |  Link to Article
Ryan  SB, Detweiler  KL, Holland  KH, Hord  MA, Bracha  V.  A long-range, wide field-of-view infrared eyeblink detector. J Neurosci Methods. 2006;152(1-2):74-82.
PubMed   |  Link to Article
Thompson  LT, Moyer  JR  Jr, Akase  E, Disterhoft  JF.  A system for quantitative analysis of associative learning, part 1: hardware interfaces with cross-species applications. J Neurosci Methods. 1994;54(1):109-117.
PubMed   |  Link to Article
Leder  RC, Stampi  C, Webster  JG.  Blink size and lid velocity from ambulatory subjects: an analog, retroflective eyelid sensor. J Sleep Res. 1996;(suppl 1):121.
Corbett  MA.  A drowsiness detection system for pilots: Optalert. Aviat Space Environ Med. 2009;80(2):149.
PubMed   |  Link to Article
Johns  MW.  The amplitude-velocity ratio of blinks: a new method for monitoring drowsiness. Sleep. 2003;26(suppl):A51-A52.
Rodriguez  M, Mendez  E.  Quantal processing of visual information in the brain. Neuroscience. 1998;84(3):641-644.
PubMed   |  Link to Article
Evinger  C, Manning  KA, Sibony  PA.  Eyelid movements: mechanisms and normal data. Invest Ophthalmol Vis Sci. 1991;32(2):387-400.
PubMed
Jacobsen  SC, Knutti  DF, Johnson  RT, Sears  HH.  Development of the Utah artificial arm. IEEE Trans Biomed Eng. 1982;29(4):249-269.
PubMed   |  Link to Article
Goldstein  EA, Heaton  JT, Kobler  JB, Stanley  GB, Hillman  RE.  Design and implementation of a hands-free electrolarynx device controlled by neck strap muscle electromyographic activity. IEEE Trans Biomed Eng. 2004;51(2):325-332.
PubMed   |  Link to Article

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
Submit a Comment

Multimedia

Video 1.

Infrared (IR) Voltage Output During Forward Gaze and Gaze Movements

An unbroken light path between the IR emitter and detector causes a relatively high voltage output from the blink detection circuit. In forward gaze, at each blink event the IR beam is broken by eyelids/lashes and the circuit output voltage drops toward a low state. During gaze movements, the IR beam is occasionally broken (mainly during downward gaze and 1 leftward gaze event in this video).

Video 2.

Blink vs Downward Gaze

High-speed video recording of the circuit output during a blink event followed by downward gaze. Drops in the circuit voltage caused by interruption of the infrared light path during blinking are generally faster than when light path obstruction occurs during downward gaze. Thresholding on peak rate of signal change (first derivative) can help distinguishing blinks from slower, gaze-related eyelid movements.

Video 3.

Infrared Signal of a Complete Eye Blink

High-speed video capture of the circuit voltage output during a complete blink. The blink closing phase causes gradual interruption of the infrared light path, with proportional drop of the circuit voltage toward the lowest state (complete beam interruption), followed by a gradual increase of the voltage during the eye opening.

Video 4.

Infrared Signal of an Eye Twitch

High-speed video capture of the circuit voltage output during an eye twitch. The onset of signal change is synchronous with visible eyelid movement. The shape of the signal reflects that the light path is only partially interrupted by the twitch movement. This drop in the voltage output is not detected as a blink event because it is under the threshold.

Some tools below are only available to our subscribers or users with an online account.

282 Views
2 Citations

Related Content

Customize your page view by dragging & repositioning the boxes below.

See Also...
Articles Related By Topic
Related Collections
×
brightcove.createExperiences();