This paper describes a method for automatic recognition of isolated Swedish Sign Language signs for the purpose of educational signing-based games. Two datasets consisting of 51 signs have been recorded from a total of 7 (experienced) and 10 (inexperienced) adult signers. The signers performed all of the signs 5 times and were captured with a RGB-D (Kinect) sensor, via a purpose-built recording application. A recognizer based on manual components of sign language is presented and tested on the collected datasets. Signer-dependent recognition rate is 95.3% for the most consistent signer. Signer-independent recognition rate is on average 57.9% for the experienced signers and 68.9% for the inexperienced.
ASL. 2006. http://www.bu.edu/asllrp/cslgr/.
G. Awad, J. Han, and A. Sutherland. 2006. A Unified System for Segmentation and Tracking of Face and Hands in Sign Language Recognition. In International Conference on Pattern Recognition, volume 1, pages 239–242.
BSL. 2010. http://www.bslcorpusproject.org/.
Z. Cao, T. Simon, S.-E. Wei, and Y. Sheikh. 2017. Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. In IEEE Conference on Computer Vision and Pattern Recognition. H. Cooper, B. Holt, and R. Bowden, 2011. Sign Language Recognition, pages 539–562. Springer London.
DGS. 2010. http://www.sign-lang.uni-hamburg.de/dgs-korpus/.
DICTA. 2012. http://www.dictasign.eu.
R. Feris, M. Turk, R. Raskar, K. Tan, and G. Ohashi. 2004. Exploiting Depth Discontinuities for Vision-based Fingerspelling Recognition. In IEEE Conference on Computer Vision and Pattern Recognition, pages 155–155.
K. Fujimura and X. Liu. 2006. Sign Recognition Using Depth Image Streams. In International Conference on Automatic Face and Gesture Recognition, FGR’06, pages 381–386. IEEE Computer Society.
K. Grobel and M. Assan. 1997. Isolated Sign Language Recognition Using Hidden Markov Models. In IEEE International Conference on Systems, Man, and Cybernetics, volume 1, pages 162–167. IEEE.
R. Grzeszcuk, G. Bradski, M. H. Chu, and J. Y. Bouguet. 2000. Stereo Based Gesture Recognition Invariant to 3D Pose and Lighting. In IEEE Conference on Computer Vision and Pattern Recognition, volume 1, pages 826–833.
S. Hadfield and R. Bowden. 2012. Generalised Pose Estimation Using Depth. In European Conference on Trends and Topics in Computer Vision, ECCV’10, pages 312–325. Springer-Verlag.
S. Hong, N. A. Setiawan, and C. Lee, 2007. Real-Time Vision Based Gesture Recognition for Human-Robot Interaction, pages 493–500. Springer Berlin Heidelberg.
C.-L. Huang andW.-Y. Huang. 1998. Sign Language Recognition Using Model-based Tracking and a 3D Hopfield Neural Network. Machine Vision and Applications, 10(5):292–307.
K. Imagawa, S. Lu, and S. Igi. 1998. Color-based Hands Tracking System for Sign Language Recognition. In IEEE International Conference on Automatic Face and Gesture Recognition, pages 462–467.
T. Kadir, R. Bowden, E. J. Ong, and A. Zisserman. 2004. Minimal Training, Large Lexicon, Unconstrained Sign Language Recognition. In British Machine Vision Conference.
M.W. Kadous. 1996. Machine Recognition of Auslan Signs Using PowerGloves: Towards Large-Lexicon Recognition of Sign Language. In Workshop on the Integration of Gesture in Language and Speech, pages 165–174.
J.-S. Kim, W. J., and Z. Bien. 1996. A Dynamic Gesture Recognition System for the Korean Sign Language (KSL). IEEE Transactions on Systems, Man, and Cybernetics, 26(2):354–359.
S. Mitra and T. Acharya. 2007. Gesture Recognition: A Survey. IEEE Transactions on Systems, Man, and Cybernetics, 37(3):311–324.
R. Munoz-Salinas, R. Medina-Carnicer, F. J. Madrid-Cuevas, and A. Carmona-Poyato. 2008. Depth Silhouettes for Gesture Recognition. Pattern Recognition Letters, 29(3):319–329.
K. Murakami and H. Taguchi. 1991. Gesture Recognition Using Recurrent Neural Networks. In Conference on Human Factors in Computing Systems, CHI’91, pages 237–242. ACM.
L. Rabiner and B.-H. Juang. 1993. Fundamentals of Speech Recognition. Prentice-Hall.
L. Rabiner. 1989. A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE, 77(2):257–286.
S. S. Rautaray and A. Agrawal. 2015. Vision Based Hand Gesture Recognition for Human Computer Interaction: A Survey. Artificial Intelligence Review, 43(1):1–54.
J. Segen and S. Kumar. 1999. Shadow Gestures: 3D Hand Pose Estimation Using a Single Camera. In IEEE Conference on Computer Vision and Pattern Recognition, volume 1, page 485.
SIGNSPEAK. 2012. http://www.signspeak.eu.
SSL. 2009. http://www.ling.su.se/english/research/research-projects/sign-language.
T. Starner and A. Pentland. 1995. Real-Time American Sign Language Recognition from Video Using Hidden Markov Models. In International Symposium on Computer Vision, ISCV’95, pages 265–. IEEE Computer Society.
T. Starner, J. Weaver, and A. Pentland. 1998. Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12):1371–1375.
C. Vogler and D. Metaxas. 1997. Adapting Hidden Markov Models for ASL Recognition by Using Threedimensional Computer Vision Methods. In IEEE International Conference on Systems, Man, and Cybernetics, volume 1, pages 156–161.
C. Vogler and D. Metaxas. 1998. ASL Recognition Based on a Coupling Between HMMs and 3D Motion Analysis. In International Conference on Computer Vision, pages 363–369.
C. Vogler and D. Metaxas. 1999. Parallel Hidden Markov Models for American Sign Language Recognition. In International Conference on Computer Vision, pages 116–122.
M. B. Waldron and S. Kim. 1995. Isolated ASL Sign Recognition System for Deaf Persons. IEEE Transactions on Rehabilitation Engineering, 3(3):261–271.
J. Yamato, J. Ohya, and K. Ishii. 1992. Recognizing Human Action in Time-sequential Images Using Hidden Markov Model. In IEEE Conference on Computer Vision and Pattern Recognition, pages 379–385.
M.-H. Yang, N. Ahuja, and M. Tabb. 2002. Extraction of 2D Motion Trajectories and Its Application to Hand Gesture Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(8):1061–1074.
Z. Zafrulla, H. Brashear, T. Starner, H. Hamilton, and P. Presti. 2011. American Sign Language Recognition with the Kinect. In International Conference on Multimodal Interfaces, ICMI’11, pages 279–286. ACM.
J. Zieren and K.-F. Kraiss. 2004. Non-intrusive Sign Language Recognition for Human-Computer Interaction. In Symposium on Analysis, Design and Evaluation of Human Machine Systems, page 27.
J. Zieren and K.-F. Kraiss, 2005. Robust Person-Independent Visual Sign Language Recognition, pages 520–528. Springer Berlin Heidelberg.
