Subtitles taken to the next level

[skipro33]

I've had hearing loss beginning when I was in the Air Force working the flight line back in the 70's. I have hearing aids but I don't like them. After a short wear period, my ear itches inside and it drives me nuts! Recently I saw a video about some glasses that you wear that have microphones some sort of method of projecting a heads-up display of the audio captured. In essence, subtitles. You look at a person, or a movie screen for example, and the subtitle pops up on the glasses and seems to be projected out in space. Sounds (get it?} great!
I've read of them referred to as 'subtitle glasses, 'live captioning glasses' and 'augmented reality glasses (AR glasses)
Anyone out there know about these and if they are ready for prime time?

Also, there's a version that translates.

 

[GrayHare]

Interesting, might even work as a translator of foreign languages. They might not be viable if you are prone to motion sickness, however.

 

[skipro33]

Interesting, might even work as a translator of foreign languages. They might not be viable if you are prone to motion sickness, however.

I updated my post with a link for translation glasses. Check it out

 

[braumeister]

FWIW, my iPhone works very well as a real time translation device. And even better, it works with spoken language (in both directions) as well as showing the text.

 

[Koolau]

I wonder if the person you're talking to can see the heads up (in reverse.)

Personally, I don't like sub titles on a TV screen. They're too distracting. However, I do kinda like the eye-glasses heads up idea.

 

[sengsational]

FWIW, my iPhone works very well as a real time translation device. And even better, it works with spoken language (in both directions) as well as showing the text.

Same on Android. You can put your phone into "conversation mode" and both people can look at the phone to see their own language. I've used it while traveling with occasional complete success, but more often partial success.

As for the glasses, if you wanted to use them for travel, you'd need to have two pairs of glasses, and get the foreign language speaker to dawn a pair of the glasses. It might be weird to ask the waiter or uber driver to put on a pair of glasses. You could have a pre-printed card in their native language that says "put these on to read the translation of what I'm saying."

Alternatively, only you could wear the glasses and, if it was typing out the foreign language of what you said in English, you might be able to get good at reading the foreign language. So you would say "I understand you need to stop for gas first, that's fine", then try to read "Entendo que você precisa parar para abastecer primeiro, tudo bem" (but for me, it would be better to have it type it out phonetically if I'm going to stand a chance of pronouncing it). ETA: phonetically would take a lot of practice too "ẽ.ˈtẽ.du kɨ vu.ˈse pɾɨ.ˈsi.zɐ pɐ.ˈɾaɾ pɐ.ɾɐ ɐ.bɐʃ.tɨ.ˈseɾ pɾi.ˈmɐj.ɾu, ˈtu.du bɐ̃j." Better, have a device read the translated English in the other language. So they'd ask you a question, they'd hear the answer first in English, then hear something out of my phone. Not as good as both wearing the glasses, which is much closer to the Star Trek "universal translator".

 

[audreyh1]

This is kind of funny. But the captions on the TV work perfectly fine for us.

 

[swakyaby]

I’ve tried out those subtitle glasses in a movie theater, and they worked great. When Christopher Nolan’s Tenet came out in 2020, I went to see it in an IMAX theater. I had read reviews that the dialogue was sometimes hard to understand because of the movie’s background action noise, so I asked if the theater could supply an assistive subtitle device. The theater manager handed me a special glasses with a cable running to a small transducer that clipped to your shirt. He programmed it for the auditorium I was seeing the movie. The glasses were branded Sony. The subtitles appeared as if projected on the bottom of the screen, and the optics were excellent.

 

[CaptTom]

...The theater manager handed me a special glasses with a cable running to a small transducer that clipped to your shirt. He programmed it for the auditorium I was seeing the movie. The glasses were branded Sony. The subtitles appeared as if projected on the bottom of the screen, and the optics were excellent.

Did it do real-time speech recognition, or did it just connect to a transmitter sending out the pre-prepared subtitles? I suspect the latter, since it had to be set for the specific movie being shown.

Of course, to have a real Star-Trek universal translator, they'd also somehow have to get everyone's mouth to move correctly for English speech. And the natives you're speaking to would also have to see your mouth moving in their language, if you wanted to fool them into thinking you're a native.

 

[JoeWras]

Did it do real-time speech recognition, or did it just connect to a transmitter sending out the pre-prepared subtitles? I suspect the latter, since it had to be set for the specific movie being shown.

I like subtitles for scripted shows or movies. These subtitles are usually scrubbed and cleaned, with emphasis and even sarcasm added if necessary, along with sound effects.

The on-the-fly titles you see on live shows can be comical. Most of it is good, but some of the titles come across completely wrong, and usually humorous.

 

[swakyaby]

The glasses provided in the theater appeared to provide prepared subtitles. I prefer that since on-the-fly translations often misinterpret what is actually said and becomes distracting.

 

[Koolau]

I resent it when movies (or TV) "need" subtitles just to understand my native language. I suppose having the ability to use subtitles is a good thing, but it should not be necessary IMHO. The producers of a movie should favor dialog clarity over the "noise" of the action (or worse: The dramatic music.)

Full disclosure: I am occasionally also distracted by the reality of such "good diction" and "clarity" in an action movie (For instance, I know that NO one can hear you talk in a helicopter unless you wear noise-cancelling ear phones - that is, unless you scream.)

 

[CaptTom]

The producers of a movie should favor dialog clarity over the "noise" of the action (or worse: The dramatic music.)

OMG Yes!!!

I guess movie people are the "artistic" type who are more into getting the "mood" right than being able to actually understand what's going on. They seem very proud of all that awful background noise which they call "music" but which only irritates the audience. Likewise with those dramatic scenes filmed so dark that you can't make out what's happening. Sometimes whole series are done that way.

 

[Koolau]

OMG Yes!!!

I guess movie people are the "artistic" type who are more into getting the "mood" right than being able to actually understand what's going on. They seem very proud of all that awful background noise which they call "music" but which only irritates the audience. Likewise with those dramatic scenes filmed so dark that you can't make out what's happening. Sometimes whole series are done that way.

Yes. "Dark" productions make me think they are tying to hide their inability to make a good movie!

 

[Music Lover]

I resent it when movies (or TV) "need" subtitles just to understand my native language. I suppose having the ability to use subtitles is a good thing, but it should not be necessary IMHO. The producers of a movie should favor dialog clarity over the "noise" of the action (or worse: The dramatic music.)

Inaudible dialogue and fast cuts have made movie watching unbearable in most cases. I hate CC because my eyes gravitate towards it and I miss what's happening on screen.

And I find it incredibly annoying when they change camera shots every 2 or 3 seconds.

 

[sengsational]

I also can't tear myself away from the words, if they are present. I only have captions intensionally on when it's in a different language (including UK English). It's helpful if I'm watching a Mystery, because it helps with US English but mainly helps me remember names.

 

[MichaelB]

I always have subtitles on now. It’s probably an age related thing. I’m starting to struggle with accents, which never have been a challenge ‘til now.

This only voices I hear clearly are the ones in my head.

 

[Koolau]

I always have subtitles on now. It’s probably an age related thing. I’m starting to struggle with accents, which never have been a challenge ‘til now.

This only voices I hear clearly are the ones in my head.

How do you get sub-titles for that? Smart Glasses??

 

[EastWest Gal]

Did it do real-time speech recognition, or did it just connect to a transmitter sending out the pre-prepared subtitles? I suspect the latter, since it had to be set for the specific movie being shown.

Of course, to have a real Star-Trek universal translator, they'd also somehow have to get everyone's mouth to move correctly for English speech. And the natives you're speaking to would also have to see your mouth moving in their language, if you wanted to fool them into thinking you're a native.

All you need is a Babel fish.

"The Babel fish is small, yellow, leech-like, and probably the oddest thing in the Universe. It feeds on brainwave energy received not from its own carrier, but from those around it. It absorbs all unconscious mental frequencies from this brainwave energy to nourish itself with. It then excretes into the mind of its carrier a telepathic matrix formed by combining the conscious thought frequencies with nerve signals picked up from the speech centres of the brain which has supplied them. The practical upshot of all this is that if you stick a Babel fish in your ear you can instantly understand anything said to you in any form of language. The speech patterns you actually hear decode the brainwave matrix which has been fed into your mind by your Babel fish."
-From The Hitchiker's Guide to the Galaxy.

 

[Koolau]

All you need is a Babel fish.

"The Babel fish is small, yellow, leech-like, and probably the oddest thing in the Universe. It feeds on brainwave energy received not from its own carrier, but from those around it. It absorbs all unconscious mental frequencies from this brainwave energy to nourish itself with. It then excretes into the mind of its carrier a telepathic matrix formed by combining the conscious thought frequencies with nerve signals picked up from the speech centres of the brain which has supplied them. The practical upshot of all this is that if you stick a Babel fish in your ear you can instantly understand anything said to you in any form of language. The speech patterns you actually hear decode the brainwave matrix which has been fed into your mind by your Babel fish."
-From The Hitchiker's Guide to the Galaxy.

Touché!

 

[GaryInCO]

Of course, to have a real Star-Trek universal translator, they'd also somehow have to get everyone's mouth to move correctly for English speech. And the natives you're speaking to would also have to see your mouth moving in their language, if you wanted to fool them into thinking you're a native.

The future is here! Here’s a short demonstration of someone using a translation app that also creates the video of the appropriate mouth movements. I suspect you could lip read off the generated video!

AI-Assisted Language Translation of Speaking, Including Mouth Movements

Ok, this is a little bit bonkers: HeyGen’s Video Translate tool will convert videos of people speaking into videos of them s

kottke.org

 

[jblack]

I always have subtitles on now. It’s probably an age related thing. I’m starting to struggle with accents, which never have been a challenge ‘til now.

If you're streaming you're not alone and not age related. Due to a lack of standard industry audio formats (and desired data reduction to transmit) the result has been lower quality overall for everyone and increasing dependence on subtitles.

I hate it and can't believe a group consortium sponsored by the major players hasn't come together to solve this yet, esp as bandwidth continues to increase dramatically. I can handle subtitles if they are small and below the video but each streaming service has their own formats and many are large & very distracting. But the quality is still often just too bad. As crazy as it seems if you still have DVDs, Blueray, or (gasp) VHS it can be a better experience.

Can’t Hear the Dialogue in Your Streaming Show? You’re Not Alone. (Published 2023)

Many of us stream shows and movies with the subtitles on all the time — and not because it’s cool.

www.nytimes.com

 

[JoeWras]

I think this is an excellent article that addresses the many varied reasons that dialog has gotten more unintelligible. It matches what the above NYT mentions, then adds many other reasons.

Here's Why Movie Dialogue Has Gotten More Difficult To Understand (And Three Ways To Fix It) - SlashFilm

/Film spoke with several Oscar-winning sound designers, editors, and mixers to learn why it has become tougher to understand what characters are saying.

www.slashfilm.com

My guess is the big one is style of acting. People made fun of William Shatner's delivery, but you could understand it. It came from his stage experience. "These kids today" come up differently and mumbling is good.

The article mentions a lot of about mixing but I'm surprised it doesn't get into the ADR process. And perhaps, that's the problem. They "fix it in post" by twiddling knobs instead of doing Automated Dialog Replacement. It requires the actors to re-do their lines in the booth.

ADR is too expensive, too time consuming, and so on. And that's ultimately the big reason. Expense and time to market. Shove it out the door ASAP. Who cares about the mix and sound? Visuals are what it's about. Problem with dialog? Just turn up the music. Nobody will notice.