Category Archives: meetings

Captions and Craptions for Academics

-Michele

In recent years, to my delight, captions have been showing up in more and more places in the United States. While I’ve been using captioning on my home TV for decades, now I see open captioning on TVs in public places, many internet videos, and most recently, in academic presentations. Everyone benefits from good captioning, not just deaf/HoH or folks with an auditory processing disorder. Children and non-native English speakers, for example, can hone their English skills by reading captions in English. And nearly everyone has trouble making out some dialogue now and then. But not all captioning is the same. While one form of captioning may provide fabulous access for deaf/HoH, another is useless. To ensure that our materials optimize inclusion, we need to figure out how to invest in the former and avoid the latter.

craptions

To unpack this a bit, I’m going to distinguish between 4 types of captioning that I’ve had experience with: 1) captions, 2) CART (communication access real-time translation), 3) auto-craptions, and 4) real-time auto-captions with AI. The first two are human produced and the last two are computer produced.

Captions: Captions are word-for-word transcriptions of spoken material. Open captions are automatically displayed, while closed captions require the user to activate the captions (click the CC option on a TV). To make these, a human produced script is added to the video as captions. Movies and scripted TV shows (i.e. not live shows) all use this method and the quality is usually quite good. In a perfect world, deaf/HoH academics (including students) would have access to captioning of this high quality all the time. Stop laughing. It could happen.

CART:This real-time captioning utilizes a stenotype-trained professional to transcribe the spoken material. Just like the court reporters who document court proceedings, a CART professional uses a coded keyboard (see image at right) to quickly enter phonemes that steno machineare matched in the vocabulary database to form words. The CART transcriptionist will modify the results as they go to ensure quality product. While some CART transcriptionists work in person (same room as the speakers), others work remotely by using a microphone system to listen to the speakers. Without a doubt, in-person CART provides way better captioning quality than remote CART. In addition to better acoustics, the in-person service can better highlight when the speaker has changed and transcriptionists can more easily ask for clarification when they haven’t understood a statement. As a cheaper alternative to CART, schools and universities sometimes use C-Print for lectures, where the non-steno-trained translators capture the meaning but not word-for-word translation. In professional settings, such as academic presentations, where specific word choice is important, CART offers far better results than C-Print but requires trained stenographers.

Some drawbacks of CART are that the transcription lags, so sometimes the speaker will ask “Any questions?” but I and other users can’t read this until the speaker is well into the next topic. Awkward, but eventually the group will get used to you butting in late. CART also can be challenging with technical words in academic settings. Optimally, all the technical vocabulary is pre-loaded, which involves sending material to the captionist ahead of time for the topics likely to be discussed. Easy-peasy? Not so fast!  For administrative meetings of over 10 people, I don’t always know in advance where the discussion will take us.  Like jazz musicians, academics enjoy straying from meeting agendas. For research presentations, most of us work on and tweak our talks up until our presentation. So getting advance access to materials for a departmental speaker can be… challenging.

Craptions:These are machine-produced auto-captions that use basic speech recognition software. Where can you find these abominationsless-than-ideal captions? Many YouTube videos and Skype use this. We call them ‘crap’tions because of the typical quality. It is possible that craptions can do an okay job if the language is clear and simple. For academic settings, these auto-craptions with basic speech recognition software are pretty much useless.

IMG_3588

The picture at right shows auto-craption for a presentation at the 2018 American Geophysical Union conference about earthquakes. I know, right]Yes, the speaker was speaking in clear English… about earthquakes. The real crime of this situation is that I had requested CART ahead of time, and the conference’s ADA compliance subcontractor hired good quality professional transcriptionists. Then, the day before the conference, the CART professionals were told they were not needed. Of course, I didn’t know this and thought I was getting remote CART. By the time the craptions began showing up on my screen, it was too late to remedy the situation. No one that I talked with at the conference seemed to know anything about the decision to use craptions instead of CART; I learned all of this later directly from the CART professionals. The conference contractor figured that they could ‘save money’ by providing auto-craption instead of CART. Because of this cost-saving measure, I was unable to get adequate captioning for the two sessions of particular interest to me  and for which I had requested CART. From my previous post on FM Systems, you may remember that all of my sessions at that conference were in the auxiliary building where the provided FM systems didn’t work. These screw-ups meant it was a lousy meeting for me. Five months have passed since the conference, and I’m still pretty steamed. Mine is but one story; I would wager that every deaf/HoH academic can tell you other stories about material being denied to them because quality captioning was judged too expensive.

Real-time auto-caption with AI: These new programs use cloud-based machine learning that goes farbeyond the stand-alone basic speech recognition of craptioning software. The quality is pretty good and shows signs of continuous improvement. Google slides and Microsoft office 365 PowerPoint both have this functionality. Link to a video of Google Slides auto-caption in action.You need to have internet access to utilize the cloud-basedScreen Shot 2019-04-24 at 5.26.12 PM machine learning of these systems. One of the best features is that the lag between the spoken word and text is very short. I speak quickly and the caption is able to keep up with me. Before you start singing hallelujah, keep in mind that it is not perfect. Real-time auto captioning cannot match the accuracy of captioning or CART for transcribing technical words. Keep in mind that while it might get many of the words, if the captions miss one or two technical words in a sentence, then deaf/HoH still miss out. Nevertheless, many audience members will benefit, even with these missing words. So, we encourage all presenters to use real-time auto caption for every presentation. However, if a deaf/HoH person requests CART, real-time auto caption, even it is pretty darn good, should never be offered as a cheaper substitution. Their accommodation requests should be honored.

An offshoot of the real-time auto-caption with AI are apps that work on your phone. Android phones now have a Google app (Live Transcribe) that utilizes the same speech recognition power used in Google Slides. Another app that works on multiple phone platforms is Ava. I don’t have an Android phone and have only tried Ava in a few situations. It seems to do okay if the phone is close to the speaker, which might work in small conversation but poses problems for meetings of more than 3 people or academic presentations. Yes, I could put my phone up by the speaker, but then I can’t see the captions. So yeah, no.

What are your experiences with accessing effective captions in academic settings? Have you used remote captioning with success? For example, recently, I figured out that I can harness google slides real time auto-caption to watch webinars by overlapping two browser windows. For the first time, I can understand a webinar.  I’ve got a lot of webinars to catch up on! Tell us what has worked (or not worked) for you.

Understanding unfamiliar accents

-Ana

I wrote this post on an airplane coming back from an international conference I attended in Thailand. Because of the distance involved, participation at this meeting was pretty light on scientists from North and South America, but had a lot of participants from Europe (primarily the Netherlands, France, Spain, and Belgium) and Asia (primarily Thailand, China, Japan, Taiwan, but several other countries too). It was a wonderful conference: great venue, warm hosts, cutting-edge talks, great food, new people to meet, and some fun sightseeing thrown in. It also brought with it the usual challenges of trying to understand talks and poster presentations and network with important scientists in noisy settings. But this conference also brought home a specific problem that has stymied me throughout my career: understanding unfamiliar accents.

Deaf/HoH academics who depend on oral communication will likely be familiar with the problem that, even in an optimal hearing environment, we better understand those who speak like we do. Unfamiliar or “foreign” is relative, of course. I speak English and Spanish, but, due to the particularities of my upbringing, my best shot at hearing/understanding Spanish is with people who speak Colombian Spanish, or even more, the version of Colombian Spanish spoken in and around Bogotá (indeed, that is the accent I speak with – immediately recognizable to most Latin Americans). My Argentinean and Mexican friends can attest to how obnoxious I can be asking them to repeat themselves. Likewise, for English, I fare best with a northern Midwestern US type of English; Australian, British, Indian and many other accents will leave me floundering. I imagine that the same is true for other deaf/HoH academics, but with different versions of their language they are most used to.

Scholarly research, of course, is a global venture, and it is wonderful that many luminaries in my field hail from around the world. I’m already incredibly lucky that most professional communication is conducted in English, a language I happen to know. But, while hearing people can be quite understanding of my communication difficulties in suboptimal environments, it seems cruel (and professionally unwise) to tell colleagues that I can’t ‘hear’ them because of their accents—especially because many such colleagues have worked hard to acquire their English skills, thus going the extra mile to ensure communication. Because of globalism, the problem with understanding unfamiliar accents goes beyond conferences and professional networking. Many of my undergraduate and graduate students are also from various international locations. I am heartbroken every time I feel that my difficulty understanding my students negatively affects my ability to mentor them.

I have not found ideal strategies to deal with the challenges of unfamiliar accents. Every accent becomes a little more familiar with constant exposure, so I do understand my graduate students (with whom I communicate almost daily) better as time goes by. But it never stops being a challenge, and I sometimes have to resort to written communication in our one-on-one meetings. Since the undergraduates I teach change each semester, I don’t have similar opportunities to become familiar with their accents. For conferences and professional networking, I imagine that real-time captioning would be the ideal solution; but such a resource is not available at all conferences (though it should be!) and is generally not an option for networking. I’ve been excited by the recent advances in speech recognition software, such as that demonstrated by Google Slides, and wonder both if the technology can accommodate a range of accents and, if so, if it could ever become a portable “translator” for deaf/HoH individuals (I know some portable translator apps exist, but haven’t tried them and don’t know the scope of their utility; perhaps some readers can share their experiences?). I’m also curious whether unfamiliar accents are ever a challenge for deaf/HoH academics who rely on sign language interpreters. What other strategies have deaf/HoH academics employed to help navigate the challenge of unfamiliar accents in a professional setting?

How to work with ASL-English interpreters and Deaf academics in academic settings

Just like their non-Deaf colleagues, Deaf academics teach students, discuss and present their research, attend various professional meetings, and give media interviews. Communicating and sharing knowledge with others is a critical part of academia. However, not everyone has had experience communicating with somebody using sign language, and many non-signers are unfamiliar with the protocols of working with ASL-English interpreters. Ashley Campbell, the staff ASL-English interpreter at Saint Mary’s University in Halifax, Nova Scotia, and Linda Campbell, a Senior Research Fellow of Environmental Science at Saint Mary’s, have put together a rich set of resources: a series of tip sheets on how best to work with interpreters in various academic scenarios. By sharing these resources with The Mind Hears, Ashley and Linda provide quick reference tools that will simultaneously educate and lessen any stress around facilitating communication through interpreters. Though originally written to facilitate ASL-English communications, these tip sheets can be applied to any settings that incorporate signed language-spoken language communications.

The tip sheets can be found at:

https://smu.ca/academics/departments/environmental-science-work-with-interpreter.html

Do you have ideas on further tip sheets to add to this resource? Are there other recommendations that you would add to the existing tip sheets? Please let us know what strategies you have found useful in educating non-signers, and help Ashley and Linda expand the reach and utility of the resources they have created. Write to Ashley at Ashley.N.Campbell@smu.ca or share your thoughts in the comments below.

 

Ashley Campbell

Since 2015 I have been the staff ASL-English interpreter within the Faculty of Science at Saint Mary’s University in Halifax, Canada. My first exposure to sign language was in Belleville, Ontario where I lived for a short period early in life. Many years later I took ASL night classes for enjoyment and through learning the language and culture I became interested in studying it more formally. I graduated from an interpreting training program in 2010 and along with interpreting have volunteered for both provincial and national interpreting association boards. I have a passion for sharing knowledge with the mentality of “each one, teach one”. When I’m not working I am a mom to a very active toddler, cooking feasts for my family, and enjoying the odd Netflix program.

Linda Campbell

Dr. Campbell is a Professor and a Senior Research Fellow at Saint Mary’s University in Halifax.  She moved to Halifax from a Canada Research Chair (Tier II) faculty position at Queen’s University in Kingston. Her research and teaching at Saint Mary’s University focus on contaminants in the environment and on sustainability / resilience issues, with emphasis on aquatic ecosystems and water resources. Currently, Dr. Campbell’s research group is examining environmental contaminants across the Maritimes and around the world, with projects looking at impacts of legacy gold mine tailings from the 1800’s and contaminant transfer in aquatic food webs, birds, bats and humans.