Computer Vision: A Sharper Competitive Edge for Photo Libraries & Media Organisations

Hosting computer vision software on your own system, rather than the cloud, offers greater control while opening the door to new commercial opportunities, Dr Appu Shaji, Founder of Mobius Labs, argues in this latest guest article.

What does an office look like? Sounds like a strange question, but it is especially important for photo libraries during the coronavirus lockdown. The pandemic saw a surge in searches for ‘home office’ images to illustrate articles and advertising. Libraries that could return high-quality, relevant images were in a far better position to capitalise on the growth in demand.

Of course, it is not just the home office. At a time when news cycles move more quickly than ever, photo libraries are turning to computer vision software which automates the classification of images faster and more accurately than a human. Improving search matches, even by a fraction of a percent, can make the difference between commercial success and failure.

Based on machine learning and artificial intelligence, most computer vision tools require substantial processing power to manage archives that often contain millions of images. This typically requires support from one of the big cloud providers such as Amazon, Microsoft and Google who undertake machine learning training and classification based on photos and other media uploaded to their networks.

This approach has several sticking points. Some cloud providers will negotiate price based on extracting metadata from uploaded media. They then use this information to train their own machine learning algorithms. Photo libraries that have spent decades building up their business are reluctant to turn over such valuable, proprietary information, especially when it could eventually enable one of the cloud giants to muscle in on their own sector.

Then there is the quality of the connection with your cloud provider. If your service depends on data transfer between the cloud, your own systems and the client, how do you ensure a consistent, split second response to a search query? Any delay or break in service risks both income and long-term customer loyalty.

Finally, there are the rules that govern the sharing of data with a third party. In the face of GDPR and other legislation, it is increasingly complex and expensive to exchange data with another organisation, especially when it relates to sensitive information such as facial recognition.

So how do you bypass the processing power of a cloud provider? Media organisations are increasingly turning to computer vision software that can be deployed securely on their own systems. This model, sometimes called ‘edge computing’, means that proprietary data and content never leaves their servers, whether hosted on premise or with a co-hosting partner.

In this scenario, the software is delivered in the form of an SDK (solution development kit) which includes compressed algorithms that run in the background of the existing media library or digital asset management tool. Early adopters include DPA and ANP, the largest press agencies in Germany and the Netherlands respectively, who have deployed an SDK developed by Berlin-based Mobius Labs.

The most recent edge solutions come pre-loaded with thousands of tags, including abstract concepts such as emotions and actions as well as physical objects. Mobius Labs, for example, includes facial recognition and video classification, as well as aesthetic ranking models which identify the best content in a collection.

In addition to such ‘out of the box’ tagging, the latest SDKs enable non-technical users to train the software and create bespoke tags, based on a relatively small number of images. This gives photo libraries, for example, a significant advantage in the race to find images that satisfy the evolving tastes of publishers and fast-moving news cycles. Many are also re-classifying content to reflect changing cultural attitudes towards society, diversity and the environment.

Taken as a whole, the software enables photo libraries to remain in step with the needs of their customers, while branching out to a wider audience and retaining control over their data. More bluntly, if a customer cannot find a photograph, they cannot pay for it. By deploying computer vision on local computers, photo libraries, press agencies and other media organisations are better equipped to compete in a world that is drowning in images and video.