Article Sidebar
Main Article Content
Pupillometry measures pupil size, and several open-source algorithms are available to analyse pupillometry data. However, only a few studies compared these algorithms' accuracy and computational resources. This study aims to compare the accuracy of computer vision-based algorithms (Swirski, Starburst, PuRe, ElSe, ExCuSe algorithms) and the machine learning algorithm, DeepLabCut, to the double-blinded human examiners (gold-standard). Training of DeepLabCut with different architectures and a variable number of markers (2-9 markers) was done on an open-source dataset. The duration of training was statistically longer for the ResNet152 model compared to the MobileNet model. The pupil diameters in computer vision-based software such as PuRe, Starburst, and Swirski were statistically different from human measurements. MobileNet 2 and 3 marker models were the closest to the human measurements. In conclusion, this work highlights the efficiency of lower marker models based on MobileNet architecture in DeepLabCut, which consumes fewer computational resources and is more accurate.
Article Details
A. Biró, A. I. Cuesta-Vargas, J. Martín-Martín, L. Szilágyi, and S. M. Szilágyi. Synthetized multilanguage OCR using CRNN and SVTR models for realtime collaborative tools. Applied Sciences, 13(7):4419, 2023. https://doi.org/10.3390/app13074419. (Crossref)
F. Boutros, N. Damer, K. Raja, R. Ramachandra, F. Kirchbuchner, et al. Iris and periocular biometrics for head mounted displays: Segmentation, recognition, and synthetic data generation. Image and Vision Computing, 104:104007, 2020. https://doi.org/10.1016/j.imavis.2020.104007. (Crossref)
Brain Initiative Alliance, Neuroscience. DeepLabCut: Neuroscience research initiative. https://www.braininitiative.org/toolmakers/resources/deeplabcut/, [Accessed: 3 Nov 2021].
Y. Chen, M. Adjouadi, C. Han, J. Wang, A. Barreto, et al. A highly accurate and computationally efficient approach for unconstrained iris segmentation. Image and Vision Computing, 28(2):261-269, 2010. https://doi.org/10.1016/j.imavis.2009.04.017. (Crossref)
W. Chinsatit and T. Saitoh. CNN-based pupil center detection for wearable gaze estimation system. Applied Computational Intelligence and Soft Computing, 2017(1):8718956, 2017. https://doi.org/10.1155/2017/8718956. (Crossref)
V. D. Costa and P. H. Rudebeck. More than meets the eye: the relationship between pupil size and locus coeruleus activity. Neuron, 89(1):8-10, 2016. https://doi.org/10.1016/j.neuron.2015.12.031. (Crossref)
W. Fuhl, T. Santini, G. Kasneci, and E. Kasneci. Pupilnet: Convolutional neural networks for robust pupil detection. arXiv, 2016. ArXiv:1601.04902. https://doi.org/10.48550/arXiv.1601.04902.
S. D. Goldinger and M. H. Papesh. Pupil dilation reflects the creation and retrieval of memories. Current Directions in Psychological Science, 21(2):90-95, 2012. https://doi.org/10.1177/0963721412436811. (Crossref)
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016), pp. 770-778. Las Vegas, NV, USA, 27-30 Jun 2016. https://doi.org/10.1109/CVPR.2016.90. (Crossref)
H. Himmel and T. E. Faelker. Pupillary function test in rat: Establishment of imaging setup and pharmacological validation within modified Irwin test. Journal of Pharmacological and Toxicological Methods, 99:106588, 2019. https://doi.org/10.1016/j.vascn.2019.106588. (Crossref)
J. P. Horwath, D. N. Zakharov, R. Mégret, and E. A. Stach. Understanding important features of deep learning models for segmentation of high-resolution transmission electron microscopy images. npj Computational Materials, 6(1):108, 2020. https://doi.org/10.1038/s41524-020-00363-x. (Crossref)
A. G. Howard. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv, 2017. ArXiv:1704.04861. https://doi.org/10.48550/arXiv.1704.04861.
R. A. Jeyarani and R. Senthilkumar. Eye tracking biomarkers for autism spectrum disorder detection using machine learning and deep learning techniques. Research in Autism Spectrum Disorders, 108:102228, 2023. https://doi.org/10.1016/j.rasd.2023.102228. (Crossref)
N. Kondo, W. Chinsatit, and T. Saitoh. Pupil center detection for infrared irradiation eye image using CNN. In: Proc. 2017 56th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), pp. 100-105. IEEE, Kanazawa, Japan, 19-22 Sep 2017. https://doi.org/10.23919/SICE.2017.8105630. (Crossref)
K. Kuraoka and K. Nakamura. Facial temperature and pupil size as indicators of internal state in primates. Neuroscience Research, 175:25-37, 2022. https://doi.org/10.1016/j.neures.2022.01.002. (Crossref)
R. S. Larsen and J. Waters. Neuromodulatory correlates of pupil dilation. Frontiers in Neural Circuits, 12:21, 2018. https://doi.org/10.3389/fncir.2018.00021. (Crossref)
M. C. Leong, D. K. Prasad, Y. T. Lee, and F. Lin. Semi-CNN architecture for effective spatio-temporal learning in action recognition. Applied Sciences, 10(2):557, 2020. https://doi.org/10.3390/app10020557. (Crossref)
M. W. Mathis and A. Mathis. Deep learning tools for the measurement of animal behavior in neuroscience. Current Opinion in Neurobiology, 60:1-11, 2020. https://doi.org/10.1016/j.conb.2019.10.008. (Crossref)
S. Mathôt. Pupillometry: Psychology, physiology, and function. Journal of Cognition, 1(1), 2018. https://doi.org/10.5334/joc.18. (Crossref)
P. R. Murphy, R. G. O'Connell, M. O'Sullivan, I. H. Robertson, and J. H. Balsters. Pupil diameter covaries with BOLD activity in human locus coeruleus. Human brain mapping, 35(8):4140-4154, 2014. https://doi.org/10.1002/hbm.22466. (Crossref)
A. Nasim, A. Maqsood, and T. Saeed. Multicore and GPU based pupillometry using parabolic and elliptic Hough transform. International Journal of Mechanical Engineering and Robotics Research, 6(5), 2017. https://doi.org/10.18178/ijmerr.6.5.425-433. (Crossref)
T. Nath, A. Mathis, A. C. Chen, A. Patel, M. Bethge, et al. Using DeepLabCut for 3D markerless pose estimation across species and behaviors. Nature Protocols, 14(7):2152-2176, 2019. https://doi.org/10.1038/s41596-019-0176-0. (Crossref)
N. O’Mahony, S. Campbell, A. Carvalho, S. Harapanahalli, G. V. Hernandez, et al. Deep learning vs. traditional computer vision. In: Advances in Computer Vision: Proc. 2019 Computer Vision Conference (CVC), vol. 943 of Advances in Intelligent Systems and Computing, pp. 128-144. Springer, LasVegas, USA, 2-3 May 2020. https://doi.org/10.1007/978-3-030-17795-9. (Crossref)
M. Privitera, K. D. Ferrari, L. M. von Ziegler, O. Sturman, S. N. Duss, et al. Author correction: A complete pupillometry toolbox for real-time monitoring of locus coeruleus activity in rodents. Nature Protocols, 16(8):4108, 2021. https://doi.org/10.1038/s41596-021-00493-6. (Crossref)
M. M. Raffaele, F. Carrara, A. Viglione, L. Lupori, L. L. Verde, et al. MEYE: Web-app for translational and real-time pupillometry. ENEURO, 8(5), 2021. https://doi.org/10.1523/ENEURO.0122-21.2021. (Crossref)
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen. MobileNetV2: Inverted residuals and linear bottlenecks. In: Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), pp. 4510-4520. Salt Lake City, UT, USA, 18-23 Jun 2018. https://doi.org/10.1109/CVPR.2018.00474. (Crossref)
T. Santini, W. Fuhl, and E. Kasneci. Pure: Robust pupil detection for real-time pervasive eye tracking. Computer Vision and Image Understanding, 170:40-50, 2018. https://doi.org/10.1016/j.cviu.2018.02.002. (Crossref)
T. Satriya, S. Wibirama, and I. Ardiyanto. Robust pupil tracking algorithm based on ellipse fitting. In: Proc. 2016 International Symposium on Electronics and Smart Devices (ISESD), pp. 253-257. IEEE, Bandung, Indonesia, 29-30 Nov 2016. https://doi.org/10.1109/ISESD.2016.7886728. (Crossref)
D. Sinha and M. El-Sharkawy. Thin MobileNet: An enhanced MobileNet architecture. In: Porc. 2019 IEEE 10th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON), pp. 0280-0285. IEEE, New York, NY, USA, 10-12 Oct 2019. https://doi.org/10.1109/UEMCON47517.2019.8993089. (Crossref)
S. Sirois and J. Brisson. Pupillometry. WIREs Cognitive Science, 5(6):679-692, 2014. https://doi.org/10.1002/wcs.1323. (Crossref)
M. E. Suryanto, F. Saputra, K. A. Kurnia, R. D. Vasquez, M. J. M. Roldan, et al. Using DeepLabCut as a real-time and markerless tool for cardiac physiology assessment in zebrafish. Biology, 11(8):1243, 2022. https://doi.org/10.3390/biology11081243. (Crossref)
Weights & Biases. The AI developer platform to build AI applications and models with confidence. https://wandb.ai.
P. Van der Wel and H. Van Steenbergen. Pupil dilation as an index of effort in cognitive control tasks: A review. Psychonomic Bulletin & Review, 25:2005-2015, 2018. https://doi.org/10.3758/s13423-018-1432-y. (Crossref)
Y.-H. Yiu, M. Aboulatta, T. Raiser, L. Ophey, V. L. Flanagin, et al. DeepVOG: Open-source pupil segmentation and gaze estimation in neuroscience using deep learning. Journal of Neuroscience Methods, 324:108307, 2019. https://doi.org/10.1016/j.jneumeth.2019.05.016. (Crossref)
B. Zandi and M. Lode. PupilEXT: Flexible open-source platform for high-resolution pupillometry in vision research. Frontiers in Neuroscience, 15:676220, 2021. https://doi.org/10.3389/fnins.2021.676220. (Crossref)