DBMO.ai / AI product development
Making multimedia searchable with AI.
We worked with DBMO’s founders to build a multimedia analysis product that connects transcripts, image recognition and searchable metadata.
Explore the story ↓
01 / The challenge
A media library is full of information.
Videos, podcasts and written content carry useful information in different formats. DBMO’s founders saw a gap between the volume of published media and the tools available to understand and organize it.
More about the challenge
The product needed to make analysis useful to creators and publishers. That meant deciding which information mattered and giving different kinds of media a shared representation.
- Project
- DBMO.ai
- Our contribution
- Product research, design & engineering
- People using it
- Content creators & publishers
- Engagement
- AI product development
02 / A product decision
Image recognition
Since auto-transcription captures only the audio portion of multimedia content, the DBMO product also needed visual interpretation abilities to provide high quality video analysis. We incorporated image recognition features able to supplement voice transcriptions to provide a more complete catalog of where entities, themes, and topics are present throughout a piece of video content.
03 / Inside the work
Give different formats a shared language.
We combined the founders’ industry knowledge with product research to identify useful categories: entities, topics, themes and keywords. Features were prioritized around how those outputs would be used.
Design and development proceeded iteratively. As analysis capabilities evolved, the interface changed with them so the product could connect technical output to a recognizable task.
04 / A product decision
Text as an anchor
To create one tool capable of analyzing various media types, our first priority was converting them to a common denominator: text. Written language was already the primary information vehicle for blogs and news outlets, and videos with audio and podcasts could be automatically converted into time-stamped transcripts that captured most of their meaning. Homogenizing the content’s format freed us to refine an AI application designed to pull relevant information out of text-based files (and to deliver its findings in an actionable, versatile format).
05 / A product decision
Entities, topics & themes
We combined the text/transcript and image data streams and used AI analyses to identify entities (location names, people), topics (events, news items), themes (media genre). Using similarity matching via Doc2Vec and other text-focused AI recognition and evaluation techniques, we trained DBMO to generate time-associated metadata ready to be leveraged for search engine and content optimization.
06 / A product decision
Actionable insights
Time-associated metadata gives creators and publishers a way to search, tag and reference material. Transcription and image analysis can also support captions and descriptive text.
07 / Our contribution
Media analysis that reached
a working product.
DBMO’s founders brought publishing and media knowledge. Twenty Ideas contributed research, design and engineering across the analysis pipeline and the interface for exploring its output.
The tool ran in production with a major U.S. sports organization for about a year. The product connected transcripts and image analysis with searchable, time-associated metadata.
Under the surface: how this project was built
Language analysis
Named entity recognition, tokenization, part-of-speech tagging and Doc2Vec similarity helped turn text into searchable metadata.
Connected data
The project used GraphQL with Hasura to connect analysis data and the frontend.
Processing workflows
Apache Airflow and Google Cloud Composer coordinated video, audio and text-processing workflows.
08 / Product work
Find the meaning inside the media.
Transcripts, image analysis and searchable metadata connect different ways of understanding content.
Work with Twenty Ideas
Bring us the work
you want to make possible.
Tell us who it is for, where the experience breaks down and what needs to change.
How we build and evolve products