Automated Visual Acuity Evaluation Based on Preferential Looking Technique and Controlled with Remote Eye Tracking

Establishing a normal development of visual function in children is most likely when visual pathologies are detected and treated early in childhood prior to the time-limited critical periods of visual development being terminated [1]. Visual acuity (VA) measurement is an indispensable part of all current EU screening programs for visual function in children [2], as it enables detection of visual pathologies and has an important role in the later clinical management as well as a follow-up [3]. VA measurement can be acquired through various tests. The developmental stage, namely, child’s capacity of communication, is a determining factor when choosing the appropriate VA test [4, 5]. Evaluation of VA is especially challenging with infants and older children who have not yet acquired sufficient communication skills.

In this population, acuity cards, based on preferential looking (PL) technique, are the current gold standard test (ST) for routine clinical VA evaluation. Each card contains a grating pattern of specific spatial frequency on one side and a homogeneous gray field that matches the grating pattern in average luminance on the other side [6]. The infant shows a preference for the grating pattern over the homogeneous gray field when he can resolve the pattern [7]. Resolution VA is evaluated by the test investigator as the highest frequency grating pattern that a child can resolve and is based on infant’s behavioral response, in particular his eye and head movements toward the grating pattern as well as his facial and verbal expressions.

Despite yielding reliable and fast VA evaluation [4, 6, 8], the use of acuity cards remains limited, as they require experienced investigators [5, 9-11]. VA testing with acuity cards remains a subjective method. VA results can be altered when the investigator is familiar with the age and clinical history of the child tested [5]. Moreover, the distance between the investigator and the child and testing conditions are difficult to control, which may result in the misestimation of child’s VA.

Technological advancement recently enabled the researchers to develop automated visual acuity test (AVAT) and semi-automated visual acuity test for infants, which are combining gaze tracking [12-15] (see online suppl. material; for all online suppl. material, see www.karger.com/doi/10.1159/000512395) or touch screen [16] technologies with PL technique. Studies that aimed to assess grating acuity with gaze tracking system included healthy infants younger than 12 months [13, 14] and adults [12-14]. A combination of PL technique and gaze tracking has also been used for evaluation of oculomotor control and numerous visual functions, such as visual field integrity and contrast sensitivity [17] and for the assessment of visuospatial orienting functions [18].

The goal of our study was to observe the potential of an AVAT controlled with RET for evaluating VA in infants and children. A new AVAT controlled with RET technology and based on PL technique was established to evaluate VA in healthy infants, toddlers, and older children.

Thirty-six healthy children, aged between 5 months and 16 years, without any known ophthalmological conditions, except for possible mild (<2 spherical equivalents each eye) appropriately corrected refractive error, were included in our study. All children were appointed to the University Eye Hospital of Ljubljana in the period between April 4, 2019 and June 27, 2019. Children underwent a full ophthalmic examination, which comprised VA evaluation with the reference VA test. For each child, AVAT, controlled with RET technology, was performed to obtain a second VA evaluation.

Written informed consent was signed by the parents of children younger than 15 years or by children themselves when they were 15 years or older. The research adhered to the tenets of the Declaration of Helsinki and was approved by the Republic of Slovenia National Medical Ethics Committee (case number 0120-486/2018/10).

Two reference standard VA tests were matched with 2 different age groups. Monocular VA in 7 infants aged 6–34 months (aged 6, 13, 17, 19, 33, 34, and 34 months, respectively) was evaluated with a set of 10 Keeler Acuity Cards (Good-Lite Company Inc., Elgin, IL, USA), showing grating patterns of the following 9 spatial frequencies: 14.5, 9.6, 10.7, 6.5, 3.8, 2.1, 1.4, 0.36, and 0.29 cycles per degree (cpd), and an empty card. The test was performed by an experienced pediatric ophthalmologist from a distance of 38 cm. A child was considered to have resolved the grating pattern when he looked at it in 3 out of 4 or in 4 out of 4 essays. A staircase procedure was followed. LEA symbols (Good-Lite Company Inc., Elgin, IL, USA) were used in 9 children aged 37–76 months, where VA was tested from a distance of 3 m. Snellen letter charts (Good-Lite Company Inc., Elgin, IL, USA) from a distance of 6 m were used in 20 literate children aged 89–195 months.

AVAT equipment consisted of RET Tobii Pro X3-120 (Tobii AB, Stockholm, Sweden), which recorded binocular gaze data with a frequency of 120 Hz. RET was set below 15.6 inch LCD screen of laptop HP Zbook G5, graphic card: Intel UHD Graphics 630. The screen was calibrated before the study. Software comprised both Tobii Pro Eye Tracker Manager and Tobii Pro Lab. The test ran autonomously. All children were supervised by the same test investigator for their correct positioning, calibration, and gaze focus.

Ten digital stimuli were developed for AVAT. Each consisted of a gray background (330 × 185 mm) on which 2 circles (70 mm diameter, 1 mm white border thickness) were presented. Distance between the centers of 2 circles remained stable at 112 mm. Circles were set 37 mm away from lateral and at least 11 mm from upper and lower background borders.

One of the 2 circles always matched the gray background. The other (target) was filled with one of 9 possible grating patterns. Spatial frequencies of grating patterns corresponded to those of the 10 Keeler Acuity Cards used as ST for infants, except for the card with a grating pattern of 0.18 cpd, which was too large to fit into the circle of 70 mm diameter. Translated to approximate Snellen equivalents, 9 grating patterns represented the following logMAR values: 0.3, 0.5, 0.6, 0.7, 0.9, 1.1, 1.3, 1.0, and 2.0. Patterns were adjusted for looking from 65 cm distance, and the widths of black and white vertical lines were adapted to the resolution of HP Zbook G5 computer screen, the accuracy limit being pixel size. One stimulus was empty, with 2 circles matching gray background.

Six different layouts of the target circle were possible (Fig. 1). During AVAT, each of the 10 stimuli was displayed 3 times, and the target circle appeared in 3 out of 6 possible layouts.

AVAT was performed in a dark room. Testing conditions were comparable for all children. Infants were sitting in their parents’ lap. The computer screen was placed on the table in front of each child, and RET was set along the lower border of the screen (Fig. 2). All children and parents were encouraged to remain still throughout the test. Children were asked to keep their gaze fixated on the screen, and parents were asked to keep their eyes closed to prevent eye tracker form collecting parents’ gaze data. If the child lost interest and started looking away from the screen, the test supervisor encouraged him to look back to the screen.

Before testing, the distance between child’s eyes and the screen was set at 64–66 cm. During the test, which started with 5-point binocular calibration, RET was collecting binocular tracking data. After successful calibration [19], the test ran without interruptions for 3 min 24 s.

With 3 repetitions of each of the 10 stimuli, 30 stimuli were presented altogether. During each repetition, the stimulus appeared on the screen for 3 s. Stimuli were shown in random order. The test consisted of 9-s intervals comprising 3 consecutive stimuli. Attention grabber, including human figures, faces, and voice, was displayed during 11-s intervals that preceded each 9-s interval.

Areas of interest (AOIs) were defined as two 8.1 × 8.1° squares that were centered one above the empty and the other above the target circle. The child was considered to have looked on either circle if his gaze coordinates fell within borders of either AOI. For each of the 9 stimuli, total time spent on AOI above the target circle and total time spent on AOI above the empty and target circle were calculated for 3 stimuli repetitions. The ratio between total time above target circle and total time above both AOIs was then calculated for each of the 9 stimuli. Two thresholds for this ratio were set at 75% and at 60% to define the target circle as resolved. This meant that child was considered as having resolved the grating pattern of the target circle when he was looking at the AOI above the target circle for >75% or >60% of the total time spent looking at both AOIs.

IBM SPSS Statistics version 23.0, supported on Macintosh OS X, was used for statistical analysis. All ST VA thresholds were expressed in or transformed to logMAR units. Binocular VA evaluation, acquired on AVAT and expressed in cpd, was also transformed to logMAR units for result analysis. The correlation coefficient and Lin’s concordance correlation coefficient were used for the comparison of independent variables.

The flow and demographic characteristics of children participating in the study are presented in Figures 3 and 4. A child was considered to have successfully completed the study when he acquired VA evaluation with ST and completed AVAT when >50% of child’s tracking data were collected. We observed a higher percentage of tracking data acquired in elder children (r² = 0.25). The average age of healthy children was approximately 95 months (±56 months).

The VA value of the better eye on ST was compared with VA reference values for age, described in the literature for 4 age groups, respectively (Fig. 5). VA in infants aged 6–19 months was compared to values described by Salomao and Ventura [20], in children of 33–61 months to values described by Pan et al. [21], and in children of 63–142 months to values described by Langaas [22]. Adult VA reference values were used for children older than 142 months [23]. There was a good correlation between reference VA values and our VA evaluation on ST (r² = 0.83).

Figure 6 shows better eye VA value on ST and VA value on AVAT for 2 AVAT thresholds, 60% (Fig. 6a) and 75% (Fig. 6b). Both tests are compared in the interval of VA values, which comprised AVAT with the upper VA limit set at 0.3 logMAR. We observe a tendency of VA overestimation on AVAT for children who see >0.4 logMAR on ST (Fig. 6a, dotted line) and an underestimation for those who see ≤0.4 logMAR on ST (Fig. 6a, full line). Overestimation on AVAT is less prominent for threshold 75%, where the results of the 2 tests are less concordant. When the ST and AVAT were compared in the AVAT VA range (0.3–2.0 logMAR), all children aged 19 months or less had their VA overestimated on AVAT, whereas all children aged 33 months or more had their VA correctly estimated or overestimated on AVAT. Agreement between the results of an ST and AVAT was fair for the threshold of 60%, with Lin’s concordance coefficient being 0.53 (95% CI = 0.31–0.72). Figure 7 shows the agreement between ST and AVAT VA in the Bland-Altman plot.

The specificity of AVAT was calculated for all 9 stimuli. We were unable to calculate the sensitivity of AVAT for grating patterns that corresponded to less than approximately 0.7, since all children saw 0.9 or better on ST. Data are presented in Figure 8.

In the present study, an automated VA test with RET was developed, which yielded comparable VA results to the current gold standard tests established for infants and children. Moreover, for the first time, children older than 12 months had their grating acuity assessed with AVAT, controlled with RET technology. VA was successfully evaluated with AVAT in babies from 5 months on and in infants, who start showing an increasing interest in complex stimuli in their surroundings and whose cooperation in VA tests becomes most challenging [24].

The testability of infants and children with AVAT in the present study was 97%, which is comparable to the testability of children with Teller cards [25] as well as with ACTIVE procedure, developed by Jones et al. [14]. Even though AVAT and ST yield comparable results (Lin’s concordance coefficient being 0.53), a tendency toward VA overestimation on AVAT for children who see >0.4 logMAR on ST (Keeler Acuity Cards) and VA underestimation on AVAT for children who see ≤0.4 logMAR on ST was observed.

Overestimation could be, to some extent, explained by the difference in the precision of Keeler Acuity Cards and AVAT. All children with their VA estimation >0.4 logMAR were evaluated with Keeler Acuity Cards, a test that is based on subjective evaluation of the child, especially his head and eye movements toward the stimulus. The smallest angle, defined by the movement of infant’s eyes or head, which can be evaluated by an observer with certitude, is approximately 5° of visual angle [24]. RET, however, can detect much smaller angles, with an average calibration accuracy being 0.49° of visual angle (95% CI = 0.32–0.66° of visual angle). The tendency toward overestimation on automated test is also described by Jones et al. [14]. The authors try to explain it by the inexperience of the Keeler Acuity Card investigator. A tendency toward overestimation of VA thresholds for children is also observed in literature when acuity card test is compared to sweep visual evoked potentials (sVEP) test, an established objective VA test for infants [26]. Similar to our findings, Polevoy et al. [26] recently observed significantly better VA thresholds acquired by objective (sVEP) test in healthy infants compared to subjective acuity card test; the difference between sVEP and Teller acuity card VA thresholds decreased as a function of subjective acuity. Subjective VA acquired with Teller acuity cards was slightly exceeding sVEP acuity when it reached the value of approximately 0.6 logMAR.

VA underestimation on AVAT for children who saw ≤0.4 logMAR on ST was observed in the present study. Since AVAT is based on PL technique, we have to point out the major drawback of AVAT, interpretation of negative results. If a child does not look at the stimulus on PL test, this does not necessarily mean he or she does not see it. Looking at the stimulus is subjected to numerous (motor, motivational, cognitive, and other) factors, which may lead to VA underestimation [27]. The underestimation of VA on AVAT may be the result of the test being less attractive for older children (VA correlated well with age). Older children are used to see more complex stimuli, including voices, faces, and social interactions, which can diminish executive attention to more basic stimuli [28], such as grating patterns presented on AVAT.

The sensitivity of the test with AVAT was relatively good with expected higher sensitivity for the stimuli showing grating patterns of smaller spatial frequencies (corresponding to higher logMAR units). Since all children saw at least 0.9 logMAR on ST, specificity could only be calculated for stimuli corresponding to 0.7 logMAR or less. For more accurate estimation of AVAT sensitivity and specificity, a larger group of children should be tested.

The most important advantage of AVAT is the fact that it does not require an expert investigator for its manipulation, and it is easy to operate with. Other important advantages of AVAT are the objectivity of VA evaluation, high testability, and fast VA evaluation. Moreover, control of the standard distance between the child and the stimulus and standard testing conditions are easier to assure on AVAT than on the test with acuity cards. Digital stimuli are also more flexible; their characteristics such as contrast, size, and grating acuity are easier to manipulate. All of the mentioned features make the test interesting for implementation in clinical praxis.

Evaluation of VA on AVAT requires sufficient data collection. This could be a possible limitation to the use of AVAT in children presenting with medical conditions that influence their posture, head, and eye movements or their visual attention.

VA on AVAT was also tested in a small group of 4 children with nystagmus (aged 32, 87, 126, and 163 months), which to our knowledge has not yet been described in the literature but has been proposed [14]. The testability of the group was 75% and should be evaluated on a larger group. Lower testability in children with nystagmus is expected, since other clinical findings such as abnormal head posture, head shaking, flickering eyes, and other neurological conditions associated with nystagmus can influence eye tracking data quality [29, 30]. Lower testability of children with visual impairment (38%) compared to children with normal vision (64%) on 4 alternative choice PL task, assessed with gaze tracking, has been recently also demonstrated by Barsingerhorn et al. [15].

Better eye VA value on ST was used as binocular VA approximation on which may result in underestimation of real binocular value, which is the case when left and right monocular VA are matching [31]. When we compared AVAT and ST in the AVAT VA range (0.3–2.0 logMAR), a binocular VA on ST may have been underestimated in 5 children whose monocular VA matched and who saw <0.3 logMAR on ST. At the moment of the study, our technical equipment Tobii Pro X3-120 did not support monocular eye tracking, which would be essential to validate against ST before considering the implementation of any such test into clinical praxis. A newer version of RET that supports monocular eye tracking is now available (Tobii Pro Spectrum).

RET has been previously used as a tool for grating acuity evaluation [12-14], and a fully automated VA test with RET has been introduced by Jones et al. [14]. In the present study, a new version of AVAT with RET has been established, which does not depend on the experienced investigator. Healthy infants and children older than 12 months have been tested for the first time, showing high testability and comparable VA evaluation to ST. VA was also obtained in a smaller group of children presenting with nystagmus, indicating possibilities of automated VA evaluation in children with eye movement pathologies. Further studies need to be performed to confirm our observations, and monocular testing with RET should be carried out to validate the clinical applicability of AVAT further.

The authors thank Prof. Maja Zupančič at the Department of Psychology, Faculty of Arts, the University of Ljubljana, for advice regarding optimal characteristics of the attention grabber that appeared on automated visual acuity test. The authors also thank Matej Kopar at the IT company XLAB for his help with the development of digital stimuli for the automated visual acuity test.

Research complied with the guidelines for human studies in accordance with the World Medical Association Declaration of Helsinki. The research was approved by the Republic of Slovenia National Medical Ethics Committee (case number 0120-486/2018/10). Written informed consent was signed by the parents of children younger than 15 years or by children themselves when they were 15 years or older.

The study was partially supported by a University Medical Centre of Ljubljana research grant. None of the authors have any financial interest in the technology used in the study. The authors have no conflicts of interest to declare.

The authors received no specific funding for this work.

N.V.: study design, study performance, acquisition of data, data analysis, and writing of the paper; B.J.: study design; and M.T.P.: study conception and design, study performance, acquisition of data, research supervision, and critical revision.