Virtual Worlds for Language Teaching and Learning

April 6, 2023April 25, 2023 Paul Raine Tech Tips, TEFL

Introduction

The COVID-19 pandemic has forever changed the way that we work and learn. The “new normal” in the 21st century is for students to engage with their teachers and peers in both physical and virtual learning environments. In the summer of 2022, Paul Raine and Raquel Ribeiro explored 6 different virtual worlds, and evaluated their viability for language teaching and learning. Only virtual worlds that run in a web-browser and have a free trial were selected for this project. In this article, we present the results of our investigation, with the hope that this information will be of interest to other language teachers looking to teach all or some of their classes online. A series of YouTube videos to accompany this article can be found here.

Overview

The following virtual worlds are review in this article:

*In April 2023, Wonder announced it would be shutting down. We list the platform here for informational purposes only.
**In December 2022, Orbital announced it would be shutting down. We list the platform here for informational purposes only.

	Spatial	Gather	Wonder	Orbital	Mozilla Hubs	Kumospace
Perspective	3D	Top-down	Top-down	Top-down	3D	Top-down
Audio & Video Chat	Yes	Yes	Yes	Yes	Yes	Yes
Text Chat	Yes	Yes	Yes	Yes	Yes	Yes
Screen Sharing	Yes – Embedded	Yes	Yes	Yes	Yes – Embedded	Yes
Customizable Avatar	Yes	Yes	No	No	Yes	No
Sticky Notes	Yes	No	Yes	Yes	No	No
Object Picker	Yes	Yes	No	Yes	Yes	No
Interactive Environment	Yes	Yes	No	No	Yes	No
Emoji Reactions	No	Yes	Yes	Yes	Yes	Yes
Mini Games	No	Yes	No	No	No	Yes
Web App	Yes	Yes	Yes	Yes	Yes	Yes
iOS App	Yes	No	No	No	No	No
Android App	Yes	No	No	No	No	No
Forever-free Plan	Yes	Yes	No – Free Trial	Yes	Yes	Yes
Best Feature	Dance moves	Mini Games	Randomise Users	Private Island	Laser Pen	Piano Music

Common Features

Perspective

The virtual worlds (VWs) we evaluated for this report came in two different perspectives: top-down and 3D. Four of the six worlds had top-down perspectives, and two offered a full 3D experience.

Figure 1: The top-down perspective of Kumospace

Figure 2: The 3D environment of Spatial

Audio and video chat

All the VWs we evaluated had the ability to chat via live video and audio with other members. In some VWs, the video stream was rendered as the user’s avatar, and in other VWs, the video was rendered above or to the side of the environment.

Figure 3: In Orbital, the user’s video stream is rendered as their avatar.

Figure 4: In Wonder, the user’s video stream is rendered above or to the side of the environment.

Text chat

All of the VWs we investigated in this study offered the ability to send text-based messages to other members of the environment. We found that the text chat was a very useful supplement to audio-visual teaching methods, especially when teaching new words with unfamiliar pronunciations.

Screen Sharing [Embedded]

All of the VWs in this study offered the ability to share a screen with other users in the environment. In Mozilla Hubs and Spatial, the shared screen was “embedded” in the environment such that users could choose to either continue interacting with each other, or view the shared screen from a variety of perspectives.

Figure 5: Sharing a screen in Spatial

Custom Avatar

Some of the VWs we investigated allow the user to customise their avatar in various ways. The most advanced and personalised customization was offered by Spatial, which provided a way to convert a digital photograph into a 3D head for a user’s avatar.

Figure 6: Spatial offers the ability to convert a photo into a 3D head for a user’s avatar

Sticky Notes

Most of the VWs we investigated offered the ability to add “sticky notes” to the environment. These came in useful when teaching new words or phrases.

Figure 7: The “sticky note” function in Orbital

Interactive Environment

Some of the VWs we investigated offered the ability to interact with one’s environment. For example, by picking up and moving objects around. This feature could be used for teaching prepositions, by instructing learners to “put the goldfish on the wall” for example.

Figure 8: Interacting with a 3D goldfish in Mozilla Hubs

Mini Games

Two of the VWs we investigated featured “mini games” such as chess, which were completely contained within the virtual world. It is possible that these mini games could be used for spoken fluency practice by higher level language learners.

Figure 9: An invitation to play chess within the Kumospace virtual environment.

Object Picker

In addition to being able to interact with one’s environment, some VWs also offer an “object picker”, “designer”, or “build tool” that allows the user to add, remove, or change objects in the environment. This could either be used for introducing new vocabulary items, or for making the environment a more conducive space for language learning.

Figure 10: The object picker within Gather allows the user to add a wide range of weird and wonderful items to their virtual environment.

Emoji Reactions

Most of the VWs we investigated allow the user to react with a variety of emojis. These could be used for expressing comprehension, interest, or confusion when learning a language in a virtual environment.

Figure 11: Reacting with a “heart” emoji in Gather

General Suitability for Language Learning

The authors found that in general, it was possible to learn new foreign words and phrases inside of the VWs investigated in this study. This was verified in a rudimentary way by learning words and phrases in Portuguese and Japanese. One author had a native level of Portuguese, and attempted to learn some basic Japanese. The other author had an intermediate level in Japanese, and attempted to learn some basic Portuguese. Both authors were complete beginners in the language being taught to them.

It was found that the fidelity of the audio stream was of paramount importance in the language teaching and learning process. Where the quality of the audio was bad (such as in Spatial) it was sometimes not possible to distinguish between similar consonant sounds, such as “d” and “b”. For instance, when the Portuguese word for “chair” was first introduced, it was initially pronounced by the learner as “cabeira” whereas the correct pronunciation is “cadeira”. The authors found that the chat function could be used to clarify the pronunciation of unfamiliar words when the audio was insufficiently clear.

Specific Methodologies

Because both authors were complete beginners in the languages they were learning (Portuguese and Japanese) simple “show and tell” and “listen and repeat” methodologies were the main ones adopted in this preliminary investigation. In addition, Total Physical Response (TPR) was also briefly trialled, with one author being instructed by the other to “go closer to the tree” in Portuguese.

It is expected that, in reality, learners using VWs would not be complete beginners in the languages they are studying. Therefore, it seems reasonable that methodologies such as Communicative Language Teaching (CLT) or perhaps even Task Based Language Teaching (TBLT) could be adopted, and that this would result in improvements to communicative and pragmatic competence in a similar way that it would in real, physical classrooms.

Issues and Limitations

There were several issues and limitations with the current study. Firstly, the authors involved were living on opposite sides of the world, with a 12 hour time difference. This sometimes made it difficult to find a suitable time to meet up. It also caused occasional network issues.

In addition, the authors encountered some audio fidelity problems in the Spatial virtual environment, which interfered with the ability to clearly understand the correct pronunciation of unfamiliar foreign words.

Finally, only the two authors in this study were able to participate in the environments investigated. In real life situations, it is highly likely that there would be more participants in a language learning environment, including the teacher and perhaps 10 to 20 students. The effect that this number of users would have on the quality of the language learning experience is not known, and should be further investigated. Many of the VWs investigated in this study were specifically designed to handle a large number of concurrent users, and it would be interesting to see how the affordances of these virtual environments could be leveraged for larger classes.

Conclusion

Although Zoom has become the de facto application for online synchronous communication, it is not the only way to connect with people in remote locations in real time. The authors found many of the above virtual worlds to be just as reliable as Zoom and in most cases more visually engaging and stimulating. Language teachers might like to consider one or more of the above options in addition to or instead of Zoom for their online language classes.

First Steps with the Meta Quest 2

December 28, 2022December 28, 2022 Paul Raine Tech Tips

As I donned the rather hefty Head Mounted Display (HMD) for the first time, I was half expecting to be whisked away to a minimalistic dojo where Laurence Fishbourne would slowly convince me that the world I had always known was an illusion.

That did not happen, and in fact there are a few little annoyances about using a Meta Quest 2 HMD to remind you that you are not in fact a lightsaber wielding Jedi. Getting the right level of tension on head straps is one challenge. Tight enough to block out the light that creeps through the nose hole, but slack enough to avoid completely crushing your cheekbones. Also, you’ll need a physical space at least a few meters squared (with no furniture) if you want to wield your virtual lightsabers without without destroying real world objects with your flailing arms. Indeed, the first order of business having put on the HMD is to set up your “Guardian” – a virtual boundary that reflects the limits of the physical space you are in, and is visible during gameplay as a futuristic neon mesh. However, I also recommend having a trusted friend or family member to act as a human guardian to ensure you don’t inadvertently walk into a table and bruise your shins. One of the handy features of the Meta Quest 2 HMD is the “passthrough” feature which allows you to see the real physical world at the same time as experiencing the virtual one. But for most games and apps, you’ll want full immersion to really appreciate the experience.

*The First Steps app walks you through the Meta Quest 2 controls and how to use your “virtual hands”.*

The first app you should try is appropriately called “First Steps”, and it is included with the Meta Quest 2 (MQ2). This app walks you through all of the buttons on the two hand-held controllers. The controllers also double as your VR hands, and provide you with the ability to pick things up and throw them around, and also give thumbs up and thumbs down (kudos to the Meta company for finally allowing a thumbs down reaction!). The first experience of the First Steps app is one of the most memorable, impactful, and engaging. It will make you laugh out loud at the pure peculiarity of being able to pick up and stack virtual blocks, knock them over with a virtual paper airplane, and then pilot a virtual airship with a virtual controller. Yes, in some games you are given a virtual controller that you control with your actual controller. It’s a little confusing but also a stroke of genius.

There are also a few other freebies included with the MQ2. Some VR TV programs that offer experiences like being an astronaut on the International Space Station, and some virtual animations that make you feel part of the action. In some of the VR TV programs, when the narrator looks directly at you, it is a very disconcerting experience indeed. It puts you well and truly in the uncanny valley. While one part of your brain is doing its best to remind you that it’s all VR, the social anxiety inducing part often remains unconvinced and it is bizarre having someone sit so close to you and look directly at you when they talk to you.

My MQ2 purchase also included Beat Saber, which I suppose is to the MQ2 as Sonic was to the Mega Drive or Street Fighter was to the Super Nintendo, i.e. the game that comes with the console and the one everyone plays first. Beat Saber is a VR version of one of those mobile games where you tap colored blocks to the time of a dance track. It makes you feel less like a Jedi and more like a baton-wielding cheerleader. In fact, I have a strong suspicion that those with a background in dance would actually be very good at the game, requiring as it does coordinated and formulaic movements to the time of music. One thing I can say for sure about the MQ2 — it’s an excellent riposte to those who (rightly) harp on about video games being one of the causes of obesity in the modern age. Most of the games I have played so far on the MQ2 require a lot of physical movement (especially with the arms) to slice, climb, or shoot your way to the end of the level.

*Beat Saber requires you to slice cubes by dual wielding “lightsabers” to the time of the music*

One of the things you’ll definitely want to set up early on is “Casting”, which enables you to see what the MQ2 user is seeing in Virtual Reality on your phone or computer screen. You can pair your MQ2 device with your smartphone via the Meta app. Doing this also allows you to open MQ2 apps from your smartphone, which is really useful if you are guiding someone else through the device for the first time. It’s also incredibly entertaining to watch someone else jerking their arms around in VR at the same time as seeing what’s happening within VR. At least, those jerking movements make a little bit more sense with context.

*“The Climb” has breathtaking graphics, but is not for those who suffer from vertigo.*

After you have exhausted all the freebies (or the freebies have exhausted you), you can fire up the MQ2 App Store, and get ready to input in your credit card details (in case you could ever forget that the MQ2 was developed by a profit seeking enterprise). Since it’s hard to read your credit card number whilst fully immersed in VR, this step can also be done via the Meta smartphone app. Most of the paid-for apps for the MQ2 retail for between ¥500 to ¥5000, and while there are currently far fewer apps than there are in the iOS store for example, I can only see this number increasing in the future. Apps and games cover a range of different categories, from simulations such as fishing and cooking, to brand name shooters such as Medal of Honor. Among the games I purchased were a climbing game aptly named The Climb, and a flying game named Ultrawings. The Climb has breathtaking graphics, and is not for those who suffer from vertigo. Ultrawings, on the other hand, gave me nausea almost instantly. I have since discovered that any game that requires the character to move around is nausea inducing for me, whether flying or walking, my brain just does not like the feeling of being told it is moving when it knows it is physically stationary. Other physical side effects also include eyestrain, which sets in for me after 30 mins or so.

*The Meta Quest 2 App Store features a number of brand name and independent titles – but some of them can be nausea inducing.*

All in all, I would say that my initial experience with the MQ2 was breathtaking and astounding, while also at times being disorienting and nauseating. I have no doubt that the MQ2 has educational applications that go beyond pure entertainment. As of now, I have not tried any applications that allow interaction with other MQ2 users, but I imagine it will be these interactions that will provide the basis for language learning opportunities. In addition, you could definitely have students write or speak about their experiences with the VR world: “I was a Jedi knight.. I went climbing in the Grand Canyon… I battled German soldiers”. It will certainly lead to more interesting speaking and writing assignments than the oft repeated “nothing special”.

15 Tech Tips from JALT CALL 2021

June 6, 2021July 19, 2022 Paul Raine Tech Tips

The JALT CALL 2021 online conference took place from 4th to 6th of June 2021.

In case you missed it, here are some useful Tech Tips I picked up over the course of the weekend!

SPARQL is a semantic query language for databases which is able to retrieve and manipulate data stored in Resource Description Framework (RDF) format
According to the founders of Xreading, the following are both fallacies: “Cheaters are only hurting themselves” and “Japanese students don’t cheat”!
GIGAProject is a partnership between Japan and U.S. companies to help put the latest mobile devices in the hands of every Japanese student.
Some of the physical impacts of prolonged online learning include: eyestrain, backache, and fatigue:

Image from “Transitions and connections: student reflections on emergency remote teaching and learning (ERTL) in 2020” (Sandra Healy)

Universal Design is the design and composition of an environment so that it can be accessed, understood and used to the greatest extent possible by all people regardless of their age, size, ability or disability.
Miro provides an engaging, intuitive, in-person collaboration experience
Most teachers surveyed at the JALT 2021 conference believe there will be some limited teaching and learning online after the COVID pandemic subsides:

Image from “Optimizing the future of language teaching with technology in Japan” (Betsy Lavolette)

Students have various concerns about learning online, including Physical, Pedagogical, Social, Technological, and Content-related concerns:

Image from “EFL students’ perceptions and preferences of online learning: a Japanese higher education context” (Satchie Haga)

Spatial.io allows you to create a lifelike avatar and work as if your are next to your colleagues, utilizing VR and Augmented Reality

Gather is a video-calling space that lets multiple people hold separate conversations in parallel, walking in and out of those conversations just as easily as they would in real life.

Wonder is a space where you can connect to others in a spontaneous and fluid way by moving around freely between groups. It’s fun, creative and energizing.

Run The World is a one-stop solution for virtual social gatherings, webinars and conferences that deliver engagement.
The ISTE Standards for Students are designed to empower student voice and ensure that learning is a student-driven process
VocabLevelTest.Org allows you to easily create and administer meaning-recall and form-recall vocabulary levels tests

Image from “Self-marking online form-recall and meaning-recall vocabulary tests” (Stuart McLean)

LingoBingo.Live allows students to practice listening and speaking in a live online game

Image from “Fun listening & speaking practice with LingoBingo.Live” (Oliver Rose)

Learning Languages with AI-powered Chatbots

March 19, 2021November 9, 2021 Paul Raine TEFL

Advances in both Natural Language Processing (NLP) and Automatic Speech Recognition (ASR) have raised the question of whether AI-powered chatbots could be an alternative or supplement to flesh-and-blood human teachers in some situations. Can these tools really help learners acquire foreign languages?

From the malevolent Hal 9000 in Stanley Kubrick’s 2001: A Space Odyssey to the charming Samantha in Spike Jonze’s Her, computers that talk have shocked and seduced us in popular culture for many decades.

Spike Jonze’s *Samantha* (pictured) developed an intimate relationship with its (her?) owner

When Apple officially incorporated their voice assistant Siri into iOS in 2011, the reality of having an intelligent assistant that understood and obeyed our every word seemed one step closer for everyone.

The Amazon Echo “smart speaker” hit the market in 2014, and has been the dominant device in that field ever since. Other products in the same space include Google’s Nest, and Apple’s HomePod. Social networks also started to jump on the AI assistant bandwagon, with Facebook incorporating chatbots into its Messenger platform in 2016, and LINE releasing the Clova assistant in 2017.

Smart Speakers such as Amazon’s Echo (pictured) have been gaining popularity since 2014

Advances in both Natural Language Processing (NLP) and Automatic Speech Recognition (ASR) have raised the question of whether AI-powered chatbots could be an alternative or supplement to flesh-and-blood human teachers in some situations. Can these tools really help learners acquire foreign languages?

General purpose AI assistants for language learning

The applicable theory of language learning to bear in mind here is interactionism – the idea that languages are acquired by interacting with other speakers of those languages.

Even though smartphone and smart speaker based AI assistants haven’t usually been designed specifically with language learning in mind, some innovative teachers and researchers have used them for these purposes. One of the major issues to overcome here is the fact that these assistants aren’t optimized for non-native speech, and may struggle to correctly transcribe or understand it.

Research has shown that the tech behind these devices can recognize non-native speech to some extent (with Google Assistant recognizing 87% of learner utterances and Apple’s Siri recognizing 67% in one Japan-based study).

So if a language learner is able to speak clearly enough for an AI assistant to understand them, what kinds of activities can be done to bring about further gains in language ability?

The applicable theory of language learning to bear in mind here is interactionism – the idea that languages are acquired by interacting with other speakers of those languages.

Interacting with another person with language involves taking turns, negotiating meaning (figuring out what the other person is trying to say), and an information gap (transferring information from one speaker to another). There is no reason in theory why the tenets of interactionism cannot apply to human-computer interaction as well as human-human interaction.

General purpose AI assistants can stand in for human interlocutors in interview or quiz type activities, especially where the learner is asking the questions. However, the interaction tends to become one-sided, because AI assistants don’t ask questions unless programmed to do so.

And while learners may receive implicit feedback on pronunciation or grammatical form where the AI doesn’t understand the question that has been uttered, they won’t receive explicit feedback unless they are using an app that has been specifically designed for language learners.

AI assistants specifically designed for language learning

There have been several attempts to develop AI assistants and other interactive speech apps specifically for language learners. Here we will take a look at some of these products and services, and evaluate their usefulness and effectiveness.

Duolingo Chatbots

Duolingo launched chatbots for its iOS app back in 2016, promising to help users “come up with things to say in real-life situations”. Although the feature seemed to be well received by its users, it quietly slipped away and there is no sign of it returning yet.

Duolingo’s chatbots included a “Help me reply” feature, which would suggest words and phrases for the learner to use in their responses. The interactions with the chat bots would become more advanced as the users’ level progressed. There were some limitations to Duolingo’s chatbots though. For example, they only offered structured dialogues, as opposed to open-ended speech.

Duolingo’s chat bots (iOS only) promised to help users “come up with things to say in real life situations”.. but the feature quietly slipped away and shows no sign of returning

We hope to see a new version of these Chatbots from Duolingo as they have shown promising results for language learning outcomes.

Elai (ETS)

In December 2020, ETS released Elai (iOS/Android), an app that allows users to practice speaking about a range of topics and receive feedback on their speech.

Elai includes model answers from other learners and native speakers, and also provides tips for learners who want to repeat the same exercise.

Unlike Duolingo chatbots, Elai’s focus is on open speech. Users must respond to a prompt and record their answers within a 30 second time limit.

Elai offers a variety of feedback on learner speech, including the extent to which the learner repeated the same words; how often the learner paused in during their speech; and whether or not the learner used a lot of “filler” words, e.g. “uh”, “erm”, “ah”.

Elai attempts to improve the speaker’s vocabulary knowledge by providing a list of higher level words at the end of the exercise, which could also be used to respond to the prompt.

Elai is still in Beta status, and the extent to which it will be embraced by learners, teachers, and researchers is still an open question, but being developed by one of the world’s largest English language testing companies (ETS is behind the TOEFL and the TOEIC) certainly puts it in a strong position from the outset.

Buddy.ai

Buddy.ai (iOS/Android) focusses on the young English language learner market, and promises to “[provide] unlimited practice of spoken English.. to millions of students”.

The app offers a variety of language games and activities, including listen and repeat, question and answer, and interactive videos.

One of the drawbacks of the app, however, is that it only supports users with Russian, Spanish, Turkish and Polish as a first language. The app has a bilingual interface, and if the user has a first language other than one of these four, they will struggle to understand the instructions.

ELSA

Elsa (iOS/Android) is a mobile app that focuses specifically on improving the users pronunciation to help them “speak like an American” (although proponents of TEFL Equity might have something to say about this – should “American” be the target for all English learners?).

Through listen-and-repeat and interactive dialogue type exercises, Elsa teaches the user how words are blended together in casual speech, which in turn helps to improve the user’s fluency.

Summary

The principles of interactionism suggest that language learners can improve their skills simply by conversing with another speaker of the target language. However, there are issues to be overcome when using AI-powered virtual assistants for language learners, including lack of optimization for non-native speech, and lack of true discourse-level interaction.

Apps that specifically target language learners can do better when it comes to recognizing non-native speech, and offering more life-like interactions.

English Central, for example, is one of the leaders in the recognition of non-native speech, and gives users instant feedback on their pronunciation and fluency while speaking lines from a library of thousands of videos.

However, many of the other apps discussed here focus on either niche segments of learners (e.g. Russian and Polish speaking children) or niche language language skills (such as fine-grain pronunciation problems).

There is yet to emerge an artificially intelligent chatbot which can be used by all levels and all ages of learners that offers true human-like interaction and feedback.

In addition, student reactions to the recent COVID pandemic have shown that many students value face-to-face learning over online methods. Although chatbots and smart speakers could be a useful supplement to face-to-face or online learning with a human teacher, it seems unlikely that they will be a complete replacement for human teachers any time soon.

Introducing Learn-English.Org!

February 10, 2021June 6, 2021 Paul Raine Tech Tips, TEFL

What is Learn-English.Org?

Learn-English.Org is a free website for learners of English to practice listening, speaking, reading, and writing online, anywhere, anytime!

How do I use Learn-English.Org?

Find an activity you would like to study by using the navigation panel on the left. There are three ways to navigate the activities on this site: by category, by skill, and by level.

Can I track my progress?

Yes, you can track your progress on this site by creating an account and then checking your progress report.

Who is behind Learn-English.Org?

This site is produced and developed by English language and Ed-Tech experts, and powered by TeacherTools.Digital, an innovative digital assignment creation platform for language teachers.

paulsensei.com

Japan-based EFL teacher, presenter, author, and developer since 2006.