Digital Innovation & i media cities
The i media cities platform has implemented several innovative tools and digital technical solutions in order to improve the way all our visitors find, watch and interact with the films and images on our platform. The project has been build on three key digital strategies, aimed at maximising impact and helping users: Open Source / Machine learning / Linked Open Data.
Open Source Software
We believe open source software is good and necessary for the free development and evolution of digital services. This is why, in agreement with the developer of the software, CINECA, we have released our code under a CC BY-SA license. The source code can be downloaded from our Github page
Machine learning & A.I.
The platform uses a series of machine learning tools, all provided by Fraunhofer IDMT. These tools activate computers to process the films and images automatically, creating a large number of data, which are then translated into interface solutions
Automatic shot segmentation
This tool segments the films automatically into shots by having a computer automatically detect so-called cuts and transition frames.
As soon as these frames are detected, the data is extracted and as a result you will be able to find an automatically generated shotlist next to every film in our platform.
Concept detection and classification
Computers automatically scan and analyse all images and all frames of every film to determine whether they contain specific urban related concepts, such as person, tram, train, clock … . The computer can currently detect nearly a hundred concepts, and will be trained for more in the years to come.
As soon as a concept is detected, it is generated as a keyword and added to the corresponding image or frame (indicated by an ‘a’).
At the same time the computer also stores some other useful information, such as the region of the image or the shot the concept is detected in (graphically indicated by coloured borders), and a percentage that indicates how certain the computer of his detection. Below you can find a side by side comparison of the same shot. On the left the computer is 20% sure of the concepts he has detected. For the shot on the right he is 70% sure.
It is not always easy to find the correct balance, and as such in the platform we have chosen to display all concepts for which the computer is at least 65% sure he is correct. We are constantly monitoring and improving this feature to better serve you, but note that mistakes might still be detected.
Building and Monument Recognition