macOS Monterey: How to use Live Text and Look Up

Live Text is a new feature in macOS Monterey that allows you to use the text in an image. It’s a feature that’s quite helpful—for example, if you’ve ever been in a meeting or lecture, and took a picture of a whiteboard filled with written information, you can now select the text in the photo, copy it, and then paste it into a document.

There are a lots of ways you can use the Live Text feature, and it’s a huge time-saving production tool. Live Text is also in iOS 15 and iPad OS 15, and because you probably use the cameras on those devices often, you might use Live Text on your iPhone and iPad more than you would your Mac. But it’s coming to the Apple silicon Macs in macOS Monterey this fall and works in the Photos app, Safari, Quick Look, and the Screenshot function.

Here’s a look at how Live Text works on the Mac, using the Public Beta on macOS Monterey. According to Apple, Live Text requires at least an M1 processor. Compatibility with Intel-based Macs is not listed on Apple’s website.

Using Live Text in Photos, Quick Look, and Safari

The Live Text function in the Photos app, Safari, and Quick Look work in a similar way. (At the time of this writing, Live Text did not seem to be active using the Screenshot app in the Public Beta.) When you are looking at an image, move your cursor over the text, and the text selection tool appears. You can then select the text, and you can copy it and then paste it into a document. You also have to option to grab the text in the image and drag it over to another app, and then drop the text, as you can see in the gif below.

Live Text does a very good job with its character recognition. In the above gif, you can see that Live Text was able to recognize the accent over the letter “O”. You may also notice that there are a couple of marks in the corner of the sign before the “S” in SOY; Live Text interpreted that as a bullet. Also, Live Text translated the “O” in BASURERO as a “D”, but it’s hard to fault Live Text for that, because it does resemble a “D” in the image.

Macworld’s Glenn Fleishman took a look at Live Text and compared it to other character recognition tools, such as PDFpen, Google Docs, and Microsoft OneNote, and found that Live Text produced the best results.

Using Look Up

With text selected, you can right-click, bring up the contextual menu, and you have the option to Look Up the text. (If you’re using Safari, the Look Up selection in in the third section of the contextual menu, under the Save Image and Copy Image sections.) 

IDG

Three Look Up options are offered:

Siri Knowledge, which displays information based on the context of the selected text.

IDG

Maps, which does a search in Maps based on the text and displays a location.

IDG

Siri Suggested Websites, which does a web search using the selected text and offers a website you can peruse.

IDG