The Imprint Protocol: Rethinking Visual Data with Architecture-Driven Storage

Oct 29, 2024

3 min read

Our code is open-sourced, and you can check out the project on our GitHub repository: AEC Hackathon QR Killer. You can also view our presentation here.

A Technical Dive into the Imprint Protocol: A QR Code Alternative

The Imprint Protocol offers a pioneering new approach to visual data storage, embedding data directly into architectural visuals and design elements, removing the need for QR codes. This approach relies on a carefully curated technology stack, leveraging both client and server resources for data registration, retrieval, and processing. Below, we’ll delve into the technical mechanisms behind Imprint and the stack powering this innovative approach.

Core Architecture: Removing QR Codes Through Feature-Rich Images

Imprint employs architectural drawings as data storage by embedding encoded information within the visual features of existing design graphics. This human-friendly approach integrates information naturally into visuals that are easy to understand while avoiding the clutter and distraction that come with QR codes.

The Technology Stack: Breaking It Down

Frontend Client: The client side of Imprint serves two main functions: Registration and Lookup.
- Data Registration: The client gathers data to be stored (markup and files) and sends it through a POST API call to the backend. This backend process registers the data in the database, encoding it and returning a unique identifier (UUID) for each asset. The UUID is relayed back to the client as a reference.
- Data Lookup: The user can upload an image or use a camera to capture the relevant drawing. The client then performs a GET API call to the backend, which processes the uploaded image by extracting its features and querying the database for a match. If the encoding aligns, the system returns a link to the content, redirecting the user to relevant data.
Backend Service: The backend is responsible for data encoding, image processing, and storing. Key image preprocessing techniques include:
- Data Preparation: This critical component of Imprint’s backend processing uses data augmentation techniques such as grayscale conversion, threshold masking, Gaussian blur, and Canny edge detection. These steps are used to create noise-free, marker-aligned images creating a valid embedding function that can be used for data matching.
- Image Embedding and Cosine Similarity Function: To confirm data matching, the backend employs a Cosine Similarity Function, which calculates the similarity between the uploaded image and stored images. With a similarity score of 0.97, this approach proves highly accurate.
- Database and Storage: For database storage, Viktor is used as a repository, with ASP.NET Core powering server interactions and data storage.
Feature Detection and Matching: One of the key features we tried integrating was feature extraction. This matches key points within architectural drawings. Drawing inspiration from existing feature detection methods, as detailed in Deepanshu Tewari's article on feature detection and matching, the protocol identifies distinctive image points such as corners and textual markers. By using these features as anchor points, the protocol can detect and align points with correct image orientation, enhancing accuracy and reliability. QR codes use similar anchor points to create finder and alignment patterns for identification and matching.
Integration with Revit: A Revit plugin was developed to integrate architectural software with Imprint’s data ecosystem. Floor plans and blueprints developed in Revit can interact directly with the backend, allowing architects to register, store, and retrieve data without disrupting visual design elements.

Image Processing: Preparing for Accurate Embeddings

The backend applies a series of preprocessing steps to prepare images for embedding. These techniques, including grayscale, threshold masking, Gaussian blur, and Canny edge detection, allow for cleaner, noise-free images that simplify embedding and comparison.

Cosine Similarity Scoring: High-Accuracy Image Matching

The Cosine Similarity Function is crucial in measuring the degree of similarity between captured images and stored data. Achieving a 0.97 similarity score, the Imprint Protocol surpasses conventional matching accuracy, enhancing reliability and usability across various industries.

Expanding Beyond Architecture

The Imprint Protocol has potential applications in industries beyond architecture, including retail, social media, and healthcare, where aesthetic appeal and functional efficiency are valued. By eliminating QR codes in favor of integrated design graphics, Imprint enhances both user experience and operational functionality.

Conclusion

By integrating architectural elements directly into digital data processes, the Imprint Protocol redefines how information can be stored and accessed within existing design frameworks. We invite developers and architects alike to explore and contribute to this innovative protocol on GitHub.