Читать книгу: «Serious applications of technology in games and health», страница 3

On the negative side, it could be said that many criticize Minecraft for its “square” aesthetics and extreme popularity. However, after understanding the possibilities of everything that can be created and done with this game, we must recognize that the “minecraft” aesthetic becomes a minor theme and sometimes just anecdotal. As Piaget (1968) would say: “knowing reality implies constructing systems of transformations that correspond, more or less adequately, to reality”. This statement is very close to the fact that Minecraft represents a more abstract view of reality. As Nicole Joseph (2018) states, “abstraction makes you work for the meaning of the piece, while realism shows off skill and technique”.

In short, Minecraft, appointed by some as the “game of life” or the “virtual lego”, is an “open world” game where the protagonist lives in a virtual world without rules or “levels” and no defined game goals. The constructions in Minecraft are defined in terms of “blocks” in a very similar way to how Lego toys are created, with the difference that being a digital tool there is no limit to the materials and objects that can be built, taking various forms depending on the preferences of the protagonist. Additionally, it can be said that, instead of rules, Minecraft is built providing mainly two contexts for player´s actions: the types of virtual worlds where the protagonist appears and the day and night cycle (good and evil cycle).

Indeed, Minecraft worlds can be configured in two ways:

Worlds in survival mode: where the protagonist can “die” and has to take refuge every night so as not to be attacked by creatures. This type of worlds can be configured in such a way that if the protagonist dies, he dies forever (HARDCORE). In survival mode the protagonist has no access to materials to improve his performance in the game and must get them for himself through mining, combat and survival in a hostile environment.

• Creative Worlds: in these worlds, the protagonist has all the powers and access to all construction materials. This type of worlds is ideal for modeling solutions or simply learning to use the game before facing a world in survival mode. In this type of world, the protagonist is immune to any attack and only dies if by bad luck and falls from the virtual world to do mining beyond the limit allowed.

Each day in a Minecraft world lasts 10 minutes. During the night, the protagonist must take refuge in his house and rest in a bed because randomly evil creatures appear that will attack him in the outside world. This is how Minecraft can be understood as a stage where the protagonists can define their own rules and objectives within very general contexts. Furthermore, to achieve its purposes, the protagonist must know four “arts” or basic skills of the game:

Survive

The protagonist will learn to move through the different dimensions of the game: the first dimension or “Overworld”, where there are different ecosystems or “biomes”, such as extreme hills, taiga, desert, forests, plains, jungle, ice plains, ocean, swamps or even imaginary ecosystems such as the biome of mushrooms. From Overworld, the protagonist can move (if it’s wished) to “Nether”, an infernal place or also enter to the world of the End, a world in the middle of nowhere, inhabited by the Ender Dragon. The protagonist will learn how to cultivate plants, domesticate and raise animals, get food and how to obtain materials and resources through mining for artisan elaboration in “work tables”.

Fight

The protagonist must learn to defend its home, build a fort, fight monsters, set traps, develop player vs player combats, avoid the death of hell and fight the dreaded dragon of the world of the end.

Build

The protagonist goes from building basic shelters, to mansions with beautiful gardens or elaborate amusement parks; can even build entire cities or countries and continents close to real life.

Create Technology (Redstone)

The Redstone metaphor is unique to the Minecraft worlds and is one of the most amazing features that make Minecraft completely different from other platforms; this ability offers the protagonist the tools to simulate almost any electrical or electronic circuit, and its applications are almost infinite. Redstone has created everything from basic electrical circuits for building lighting to complete digital computers, constructions that can take several months of work for the protagonist, and also interconnect real devices with Minecraft worlds (e.g. Arduinos, Raspberry PI), opening up new possibilities for learning by using these game components.

Among the most sophisticated constructions that have been developed on Redstone there are digital computers, chronometers, clocks, calculators, construction automation (elevators, access control systems, etc.), simulation of electronic components such as logic gates and transistors and all kinds of video screens, or explosion engines of several “cylinders” that function as power plants that feed cities of the Minecraft world. It is important to note that just as Redstone is vital for the creation of technological products, it is also the fundamental pillar for the creation of customized or custom-built worlds or maps. In fact, Redstone allows a player to create the essential mechanisms to include mini-games or maps with sophisticated scripts in Minecraft, so if the player needs to include very specialized functions that cannot be created using “Redstone”, they may have to resort to installing pre-made plugins or developing plugins directly in Java language (applies only to Minecraft Java edition). In the end, the protagonist ends up combining these four basic skills to create thousands of solutions among which you can count your own games, structures, constructions or immersive cities, ASCII ART or immersive virtual courses using the educational module known as “Minecraft Education Edition”.

Plugins and applications that extend Minecraft Java Edition

Minecraft has been developed thinking about its adaptation to diverse environments and needs. Its JAVA-based design facilitates extension with modules or “plugins” developed by third parties or by members of the extensive Minecraft fan community; this has allowed to include functionalities that do not exist in the default version of the game, such as including sophisticated vehicles (trains, planes, cars, etc.); making more accurate historical recreations (plugins that bring historical structures or vehicles and weaponry from world wars); or including fantastic animals and fauna to the game world. Below are some of the thousands of plugins and additional applications that are useful to enrich the experience with the Minecraft platform.

Plugins or “MODS”

DECOCRAFT.

Some Mods or plugins have brought Minecraft closer to behaving like traditional design tools; this is the case of DECOCRAFT, a plugin that allows to include in a Minecraft world a large number of decorative elements for buildings like electrical appliances, paintings, food, furniture, etc.

CustomNPCs.

It allows non-participatory characters to be incorporated into Minecraft worlds, among other equally valuable decorative elements. Characters serve to deliver important directions or messages to players or students.

Computercraft and OpenComputers.

Other applications of Minecraft are found in the teaching of engineering disciplines where the mods for learning to program (ComputerCraft) stand out, allowing to incorporate an endless number of technological implements in a Minecraft world: computers, wired and wireless networks, printers, screens, modems, PDAS and the recognized “Minecraft Turtles” that are nothing more than robots with a varied capacity of actions and that are left to program on the part of the student to fulfill diverse functions in the game; for instance, you can create and program turtles that make constructions, fight monsters, do mining or interact with Redstone devices in the Minecraft world. This is how the teacher can invent virtual worlds where the different components of ComputerCraft are integrated with the already recognized dynamics of Minecraft.

Computers can also be integrated to facilitate student dynamics, for example, by controlling access to homes. It is known that some programmers have even connected these virtual computers with real Arduino devices; each of the ComputerCraft devices (computers and turtles), has an Operating System “OSCraft” that works similarly to a MS-DOS system and has a compiler to develop programs in “LUA”, one of the most recognized programming languages when it comes to enabling software development in applications not originally intended for that. Its simple syntax makes it ideal for learning programming in early stages.

In addition to that, some fans have developed complete operating systems capable of replacing the limited “OSCraft” that comes pre-installed in ComputerCraft. This is how you can download the sources of many operating systems that bring Computer Craft computers closer to equipment with operating systems similar to the old Windows 3.1, created following the design principles of the real Operating Systems and being evident its potencial for academic use. Some outstanding Minecraft operating systems are CCWIN, OneOS, VoidOS, etc. Moreover, ComputerCraftEDU used to incorporate a compiler with graphic language (remote control) that hides the LUA instructions and masks them with icons in front of the student, facilitating even more the learning of the programming, if we remember that this is a technique that seeks to facilitate the learning of abstract concepts.

QCraft.

Though Qcraft is a MOD that brings the principles of quantum physics to the world of Minecraft, it is not a simulation, but provides analogies that attempt to show how quantum behavior is different from the experience of common physics, being useful for introducing and experimenting with principles of quantum physics.

Instant Structures Mod (ISM).

This MOD facilitates the extraction of parts of worlds to be reused in other worlds, and makes it easy to share structures through files that can be sent to other builders in Minecraft. ISM has a list of reusable structures that protagonists can copy into their worlds and modify if necessary, so players or builders can suggest to the ISM developer to include their structures in the downloadable database.

Minecraft Education Edition MAKECODE.

The version of Minecraft Education Edition has incorporated an additional module that facilitates the learning of computer programming for children from 8 to 16 years old. This module gives each player a robotic “Agent” who can be programmed to interact with the Minecraft world through a programming language with block type instructions (equivalent to tools such as Scratch, Blockly, etc.); through this, the teacher can create activities and complete courses to learn algorithmic thinking through the basic concepts of programming. It is worth noting that a complete example of the curriculum for CSTA (Computer Science Teachers Association) exists and was implemented using this tool.

Texturizers and Shaders.

The graphical presentation of Minecraft can be considerably upgraded incorporating modules for the handling of shadows and textures that improve the graphical presentation of Minecraft. There are hundreds of libraries intended for this purpose, with surprising results when compared to the proposal that comes with the basic version of the game. To get the most out of these resources, it is recommended to install the OptiFine module.

Complementary applications

Mineways

An independent developer has created a free use module called Mineways, that allows the creation of 3D renders of parts of Minecraft worlds and print them on 3D printers.

WorldPainter and MCEdit

Worldpainter, used to model the “World of San Cayetano”, is able to read images of real terrains, and if they have relief information, it can create Minecraft worlds very close to reality. Furthermore, MCEdit allows the editing and creation of Minecraft worlds, facilitating the terrain modeling and including an undo function in case of any error (a function that does not exist directly in the game). Both editors make it easy to model Minecraft world terrains.

Conclusions

In conclusion, it can be said about the Minecraft platform, that it has defined a new standard for virtual worlds with high possibilities of being applied in educational contexts. These virtual worlds, or voxel classrooms, allow the teachers to involve different ingredients in the student’s training, facilitating more meaningful and realistic experiences for them. In 2016, 32 students of the Piloto University modeled their projects of the subject “Design I” through the Minecraft platform. In the second semester of the same year, teacher Adriana Gómez’s course was reformulated, incorporating additional learning activities into the final project, which accompanied the students from the earliest phases of this subject. The testimonies of the students indicate that the platform has facilitated the formulation of the projects through a construction “from within”, which in some way could change the traditional process of design in the first semesters of Architecture. It is still too early to indicate what the future of this tool will be, even more taking into account the profound changes that Microsoft is applying in its new Windows 10 version; what is clear is that Minecraft has defined a model or “standard” for the creation of virtual worlds, which has many points related to the didactic and project needs of teachers of different disciplines.

References

BBC. (2014). The Culture Show: Lego-The Building Blocks of Architecture [Video]. Youtube. https://www.youtube.com/watch?v=Fb_GCNV6qmA

Bringuier, J. (1980). Conversations with Jean Piaget. The University of Chicago press.

Cheshire, T. (22 of november of 2012). Want to learn computer-aided design (CAD)? Play Minecraft. Wired. http://www.wired.co.uk/magazine/archive/2012/11/play/minecrafted

Donati, P. (2015). The educational challenge: analysis and proposals. Educ, 18 (2), 307-329 https://doi.org/10.5294/edu.2015.18.2.7

Figueroa, J. (12 de agosto de 2015). El aprendizaje inmersivo. Educadictos http://www.educadictos.com/el-aprendizaje-inmersivo/

Gómez, F. & Cifuentes, O. (2012). Diagnóstico sobre la calidad de los entornos de Aprendizaje y su relación con los objetivos de aprendizaje para el desarrollo de competencias específicas del Programa de administración de empresas de la universidad Piloto de Colombia. https://core.ac.uk/download/pdf/47071347.pdf

Joseph, N (2018) Realism vs. abstraction in art. https://canfieldcardinal.com/opinions/2018/09/24/realism-vs-abstraction/

Méndez, P. (2014). Building a community of practice in Second Life for higher education learning, EAN University, Faculty of Studies in Virtual Environments. TICAL2014, IV Conferencia de Directores de Tecnologías de Información y Comunicación (TIC), At Cancun, Mexico.

Piaget, J. (1968). Genetic Epistemology-First Lecture. Columbia Univesity Press

Piaget, J. (1972). To Understand Is to Invent. The Viking Press, Inc.

Serrano, N. (2014) The pedagogical opportunities of immersive spaces: the benefits of the fulldome as a learning tool. Revista Didáctica, Innovación y Multimedia, (30), 1-16. http://dimglobal.net/revistaDIM30/docs/OC30espaciosinmersivos.pdf

Smeaton, D. (2012). The Minecraft teacher, An anthropological exploration of the pedagogy behind Minecraft as a teaching tool. https://www.academia.edu/9237685/The_Minecraft_Teacher_An_anthropological_exploration_of_the_pedagogy_behind_Minecraft_as_a_teaching_tool

Triana, M. (2016). El juego en la pedagogía como lenguaje. Revista Magisterio. https://www.magisterio.com.co/articulo/el-juego-en-la-pedagogia-como-lenguaje

Unipiloto virtual (7 september 2016). Moodlemoot 2016 Universidad Piloto de Colombia. YouTube. https://www.youtube.com/watch?v=Wsrgm2Yx-KU

Wikipedia (15 august 2019). Minecraft definition. Wikipedia. https://en.wikipedia.org/wiki/Minecraft

3. Workshop on Computing and Technology in Health-CoTH 2019

The deployment of new Information and Communication Technologies (ICT) for health have improved healthcare and quality of life. This is a major research topic that continues attracting cross-disciplinary research groups, whose combine emerging technologies around the development of software and hardware to change the experiences for patients and their families, enhance healthcare access, and improve medical processes. The purpose of this workshop was to bring together researchers interested in the application of information and communication technologies to healthcare, to present and exchange ideas of non-ready publications and projects in progress within the area. The workshop papers were mainly addressed to young researches, like Ph.D. or MSc students or researchers who wanted to share the ideas they are developing in their ongoing projects.

Serious game for learning sign language in Colombia

Oscar David Imbachí S.³, Sebastián Pimentel González⁴, Gabriel Elías Chanchí G.⁵, Katerine Marceles Villalba⁶ Engineering Faculty Colegio Mayor del Cauca, Popayán-Cauca

Abstract

Sign language is the means by which people with hearing and vocal disabilities communicate, in such a way that being a natural language has its own complexity of learning for those with a disability and those who handle verbal communication. Thus, in this article we propose as a contribution the construction of a serious game for the learning of the basic signs of the colombian sign language, such as the most important letters of the dictionary, aiming to support in a playful way the learning process of people with hearing and voice disabilities. The proposed videogame was developed taking into account the design thinking methodology, as well as the unity3D development environment, the OpenCV libraries for sign detection and TensorFlow for the creation of the recognition dataset.

Keywords: Hearing disability, learning, serious games, sign language, OpenCV, vocal disability.

Introduction

Communication is considered a fundamental priority in the social development of human beings, becoming a process of constructive interaction that favors the learning of each person (Chacón, 2013). Globally, there are two ways of communication: verbal, characterized by emitting spoken sounds is distinguished, and non-verbal, represented through gestures and body movements (Muñoz, 2014). In spite of this, not all individuals have the necessary conditions to make proper use of verbal communication, due to some disability related to making it difficult to pronounce and listen correctly to the language; therefore, sign language is one of the most widespread methods of communication to this population, which involves different types of movements and expressions through the hands, eyes, mouth and face. However, this language is not universal because its structure depends on the geographical location, making it a challenge to learn and understand for people with hearing disabilities and their family or friendship circle (Valencia & Villa, 2014).

According to the statistics provided by DANE (2018), in Colombia there are 317.195 people with hearing and vocal disabilities, which is equivalent to 1% of the population. In this sense, with the aim of facilitating an adequate communication, it is necessary to provide tools and technological resources to reduce the complexity of the sign language learning process in the colombian context. Among the tools that enable a ludic and effective learning in different application contexts are serious games, tools designed to achieve one or more objectives beyond fun like skills training or the acquisition of knowledge (Rodríguez, 2018; Matas, 2019). In order to this, in this article we propose as a contribution the design and construction of a serious game as a support to the learning process of basic elements of sign language, formed by three levels that allows the user to study and practice the interpretation of the letters corresponding to the Colombian sign dictionary and built taking into account the design thinking methodology, as well as the unity3D development environment, the OpenCV libraries for sign detection and TensorFlow for the creation of the recognition dataset. The rest of the article is organized as follows: second section presents the methodology used for the construction of the videogame; third, describes the functional design of the proposed videogame, as well as the final prototype; fourth, presents the usability test of the game; and fifth, the conclusions and future works obtained from this investigation are presented.

Methodology

For the development of the game, it was carried out an adaptation of the design thinking methodology with the following stages: empathize and define, ideate, prototype and test (Steinbeck, 2011).

Figure 1.

Considered methodology

Source: the authors.

In the first phase, an exploration of sign language was carried out in the Colombian context in order to know its grammar, syntax and vocabulary; from there, the letters E, I, O, L, N, R, Y, and W of the dactylological dictionary were selected as the first step to teach this language and introduce it into the video game. Second, user stories and mockups of characters, rooms and colors were generated, and in the third phase, the prototype of the video game was generated from the designs obtained in phase two. This test also demonstrates the contribution allowed with games in learning sign language.

Construction of the video game

Based on the problem of not evidencing the existence of a technological tool for the teaching-learning of sign language, in this section we present the high-level design and prototype generated from the video game as an educational support, focused on the ludic learning of the letters of the basic dictionary of the Colombian sign language.

Video game design

Using the free online tool DoppelME, the game has 2D characters with the function of guiding the user through the different levels proposed.

Figure 2.

Avatars mockups

Source: the authors.

On the other hand, Figure 3 shows the map of the building in which the different levels are framed, generated using the SweetHome3D tool, which allows exporting the design of the building to the Unity3D platform, being possible to perform 3D animation. In level one of the game, the users will be able to visualize the different signs of the colombian sign language, while in levels two and three they’ll have to interpret the signs learned in the first level by means of the movement of the hands in front of to the camera. It is important to note that at the end of the third level, the player accesses a last room of the building, in which the different achievements during the game are presented.

Figure 3.

Building design

Source: the authors.

Video game prototype

In this section, we describe the video game prototype; thus, Figure 4 shows two of the four avatars designed in 3D through Autodesk character generator, which can be chosen by the player as the main character within the video game. The movements of these characters were reused from the free features of the Unity3D environment.

Figure 4.

Some aspects of the game. A) Videogame characters. B) Level 1 of the videogame. C) Level 2 of the videogame. D) Detection of dictionary letters. E) Award message. F) Test users

Source: the authors.

As previously mentioned, the proposed game has three levels, where the player must familiarize with the colombian sign language and later interpret its signs through the movement of the hands in front of the camera; regarding level one, it has three sub-levels whose general objective is to appropriate the learning of the letters of the dictionary. In this sense, as the player advances through the rooms of the building, it is possible to observe the different sign language symbols arranged on the walls and in other parts of the rooms. According to the information provided by the game guide, the user must collect the indicated symbols, so that in case of hitting the player will receive as rewards gold coins and in case of failure, will lose one of the three lives assigned; the information of the coins and lives in turn will be available at the top of the game (game status bar). So, in sublevel one the user must collect only the signs corresponding to the vowels in the sign language dictionary in a given time, without losing the three lives available and collecting the coins available in the different rooms associated with the level. Similarly, on sublevel two (Figures 4B and 4C), the user must collect within the game rooms the consonants between the letter B to the M, while on sublevel three the symbols between the letter N to the Z. In this sense, the video game allows the player to identify the correct symbols by means of a blue rectangle and the incorrect ones by means of a red rectangle.

Once the user has visualized some of the signs, in level two the player must interpret the vowels E, I and O with hand movements in front of the camera.

In accordance with the above, in level three the player has a greater challenge: the interpretation by the movement of the hands in front of the camera of the letters L, N, R, Y and W. Considering that the environment in which the video game was developed does not have the direct support for the recognition of images, a communication was made via TCP socket with a Python script for the detection of the user’s gestures, using the OpenCV library and the TensorFlow tool which allows the creation of a dataset of 54.000 images, where each letter of the dictionary has a total of 3000 images; in the same way inside the dataset, the images that represent the absence of signs are included. Finally, it is important to mention that the interpretation of gestures by users in levels two and three of the game, must be done inside a blue rectangle, in a room with good lighting, white background and without the user having objects in the hands(Figure 4D). In this way, a better recognition of the signs within the proposed video game is guaranteed.

Finally, once the player has overcome the challenges proposed in the three levels, the video game shows a message of congratulations and indicates the level at which it was left based on the number of coins collected and challenges fulfilled correctly (Figure 4E). In the same way, through the message shown on the screen, the player can restart the game.

Video game evaluation

The proposed video game was evaluated from user tests developed in the usability laboratory of the Colegio Mayor del Cauca, a test that evaluates the capacity of a product to meet the expectations of a user through observation and analysis of the experience of use. In this sense, in user tests the most appropriate users are chosen depending on the type of application and a set of tasks to be developed within the software (Matas, 2019). According to this, five users (three men and two women) between the ages of 25 and 27 years participated in the usability test developed on the proposed video game (Figure 4F), structured in four phases: confidentiality agreement, informing that the data obtained in the test will be used only for academic purposes; pre-test questionnaire, where the profile of the user is characterized; task list, defined by the test coordinator, where the users develop in the usability laboratory a set of specific tasks within the evaluated software; and post-test questionnaire that evaluates user’s perception of the interaction with the game. In this specific case, the five users developed tasks corresponding to the exploration of the instructions of the video game and the compliance with levels one, two and three thereof.

Based on the results obtained by the users in the post-test questionnaire of the usability test, Figure 5 presents the level of acceptance with respect to different elements of the evaluated video game.

Figure 5.

User acceptance level

Source: the authors.

It is possible to see that the lowest grades correspond to the aspects of Complete tasks and Orientation, given that users found it difficult to complete task one, so they repeated on several occasions the first level. Similarly, among the most positively valued aspects of the game are the user’s adaptation to the use of sign language (4.2), the functionality of the game (4) and the possibility of playing again (4.4).

Conclusions and future work

Based on the need to facilitate the learning of sign language in the colombian context for people with hearing and speech disabilities, this article proposed a video game to facilitate the appropriation of the letters in the dictionary of sign language in a playful way, taking advantage of the level of image recognition provided by the OpenCV library and its integration to the Unity3D platform, in order to enrich the interaction between the user and the game. Though the usability test carried out on the game made it possible to show that the interest and level of acceptance of the users are adequate, it is important to improve the orientation and use of contextual instructions within it, in order to enrich the interaction with the user.

This work aims to serve as a reference for the creation of future projects in the field of video games involving people with voice and/or hearing disabilities as a target audience, which make use of the advantages of image recognition in real time. As future work, it is intended to expand the functionality of the proposed prototype, through the inclusion of words or phrases typical of colombian sign language and adapt the game to the environment of mobile applications, in order to facilitate access to a wider number of people.