Seeing AI is a free app developed by Microsoft designed for people who are blind. It will read out text, describe images, and even recognise people.
On our old website, we posted a series of short posts about how to use Seeing AI and its different modes (called channels). Rather than bringing them all over one by one, we thought we’d put them all into one to create a guide to the app.
Seeing AI has for a long time been an iOS only app, but is now available on Android as well with similar features. I have only used the iOS version so this article is primarily about the iOS version.
If you’re interested in having a go with Seeing AI, you can download it here.
Short Text
Seeing AI has several modes called channels. In this section we will focus on the short text channel. This is the first channel you are on when you open the app.
This is the only channel that can be used without internet.
It will read out print text that your phone can see through it’s camera. You could use this to help you read signs, letters and other short bits of text. The first time you will use this channel (as with all channels), you will be shown a tutorial screen. Click “Done” when you’re ready.
Once you are on the short text screen, just hold your phone in front of the text and it should start to read it.
Keep in mind that this is not very good for long sections of text as the reading is interrupted after a small movement of the phone. The next channel is the document channel, which is better for long bits of text.
Document channel
The document channel is great for recognizing large parts of text (like print documents).
You can take a picture of one page at a time. It works great for restaurant menus or printed letters.
To open the Document channel, just swipe through the short text screen until you hear “Channel” and then swipe up. If this is your first time using the document channel, you will see a tutorial. When you’re ready, click “Done”.
To take an accurate picture, put your phone in the middle of the page and slowly rise it up.
You should hear guidance such as “Top edge not visible” Move your phone in the dirction it says (for example if you hear top and left edge not visible, move your phone left and forward.
Once the whole page is in view, you should hear “Hold steady” followed by a tune which indicates the picture is being processed. If your phone fails to automatically take the picture, double-tap the “Take picture” button.
This channel does require internet access to function, unlike Short Text which can work offline.
Product
The product channel helps you to work out what something is based on its barcode.
It is especially helpful for tinned food.
To find it, swipe through the Seeing AI home screen to get to the “Channel” option, and then swipe up twice.
Once you hear “Product” You are on the right channel.
If this is the first time using this channel, you will be presented with a tutorial. Read through it and click “Done” when you’re ready.
To use this channel, just move the item around to try and get a barcode in view of the phones camera. There are beeps to help guide you (the closer the beeps, the closer you are).
Once the barcode is captured, you will hear a tune to ingcate it is being processed. After that, you will see information about the product. You can click “More details” if it is available.
Person
To get to the person channel, swipe up through the “Channel” list until you find “Person”.
The person channel helps you recognise people.
Once you take a picture of someone’s face, it will estimate various things about them.
To be honest this channel is more of something to play around with than something to help you in everyday life.
Warning: Do NOT use estimations from this to influence real life decisions. Based on personal experience it is more something to joke about than trust.
You can also make Seeing AI recognise who the person is by adding them using the “Face Recognition” option on the person channel screen.
You will need to get their consent before doing this. To add someone, take 3 photos of them (or let them take them) and then enter their name.
After that, they can be recognized using the person channel.
A more practical application for this is that you can check whether that person appears in a photo in your photo library.
To do this, you can either browse your library within Seeing AI using the “Browse photos” option in the Menu, or you can share a photo to “Recognize with Seeing AI”.
The face recognition feature is more accurate than the estimations (e.g. age, gender) it makes about people.
Currency
The currency channel is very useful. It will tell you what denomination a coin or note you have is.
To use it, first swipe up through the “Channel” list until you find Currency.
After that, select the currency you have (e.g. Pounds, Euros) from the “Recognizing” list. After that, just hold your phone above the coin or not
and it will announce its denemination.
The notes now used in the UK have tactile markers which provide a quicker way to identify them, but this channel is still very helpful for other currencies like Euros.
Scene
The scene channel can give you a basic description of a scene.
To be honest it isn’t very useful in real life because you can quite easily find out this information yourself.
It is more useful when analysing photos, as you can get a general idea what the photo is of.
To use the scene channel, just find Scene, preview in the “Channel” list and take a picture of the scene.
If you would like to use this to analyse a photo, either use the “Browse photos” option in the Menu or select “Recognize with Seeing AI” from your share menu.
World
The world channel uses LIDAR to tell you what’s around you, and estimate how far away things are. Therefore lots of the core functions of it only work on iPhones with LIDAR support (e.g. Pro models in recent years).
This channel also has an indoor navigation option in a “Actions” menu, which can be used to create routes through a building and later follow them with spatial audio uidance.
Colour
The colour channel will tell you what colour things are.
Sadly, it is not very accurate so will
recommend using a colour meter if you have one.
To use this channel, just find “Colour, preview” in the Channel list.
We recommend you do not use information from this channel to influence serious decisions, like what clothes you wear, as the recognition is not very accurate.
You cannot use this channel to analyse the different colours in your photos.
Light
This channel is very simple, but nevertheless useful. It will play a tone to show you how much light is in the area. The higher the tone, the brighter it is.