Hybrid collaborative filtering recommendation system for judging and suggesting books based on their covers [P]
Our take
![Hybrid collaborative filtering recommendation system for judging and suggesting books based on their covers [P]](https://preview.redd.it/zcz8hf1u6skh1.png?width=140&height=113&auto=webp&s=eb3136507a3bc923edeaf86880a1d987971f6ef7)
This project, “By-Its-Cover,” represents a compelling and refreshingly pragmatic exploration of recommendation systems, particularly within the niche of book discovery. The author’s decision to build a system solely on CLIP embeddings for both semantic search and collaborative filtering is an interesting choice, demonstrating a commitment to testing the limits of visual information. It’s a testament to the progress in computer vision that such a system, while still nascent, can function and provide meaningful recommendations. The inherent challenge of relying solely on cover images for book understanding highlights a fascinating trade-off: sacrificing nuanced content understanding for a visually driven experience. This approach aligns well with our focus on innovative data management solutions, as it pushes the boundaries of what's possible with AI-native tools—a theme we’ve previously explored in “How to Build a Career in AI: 3 Distinct Pathways,” where we emphasized the value of hands-on experimentation for skill development, and in “How to Fine-Tune an LLM: An End-to-End Guide,” which underscores the iterative process of model refinement.
The technical architecture, detailed with commendable transparency, showcases a well-thought-out deployment strategy utilizing AWS Lambda, ECS, SQS, Cognito, CloudFront, and S3. The use of Terraform and GitHub Actions for CI/CD further demonstrates a mature and scalable approach to development. The author’s acknowledgement of the system’s current limitations—a small book database and rudimentary feedback mechanisms—is refreshingly honest. The plan to incorporate a cover-edition comparison interface and improve the authentication experience demonstrates a clear vision for future enhancements, and the intention to explore SigLIP for improved visual representations is a smart move. The Determinantal Point Process for diversifying recommendations is a subtle but important detail, preventing users from being overwhelmed by repetitive results – a common pitfall in many recommendation engines. The explicit mention of avoiding AI-generated code reinforces the project’s focus on genuine learning and skill-building, a perspective that resonates with the broader AI community.
Beyond the technical merits, “By-Its-Cover” serves as a valuable case study in building a functional system from the ground up, embracing a "learn by doing" philosophy. The author’s willingness to share the project on GitHub and solicit feedback embodies a collaborative spirit that is crucial for advancing the field. The current reliance on a rate-limited API key for book search highlights a common challenge in data acquisition and underscores the importance of exploring alternative solutions, such as the anticipated Hardcover OAUTH support. It’s a reminder that even the most elegant algorithms require reliable data pipelines to function effectively. The system's growth being tied to user searches is a clever self-reinforcing mechanism, although it necessitates a proactive user acquisition strategy to overcome the initial bootstrapping phase. The author’s commitment to expanding the system, even acknowledging their own frontend development limitations, is a testament to their passion and dedication.
Ultimately, "By-Its-Cover" exemplifies the power of individual initiative and the potential of AI-native tools to democratize complex technologies. It’s a tangible demonstration of how relatively simple concepts, like leveraging CLIP embeddings for book recommendations, can be translated into a functional and evolving system. The project's current state, while imperfect, provides a valuable learning resource and a compelling glimpse into the future of visually driven data discovery. The question remains: as visual AI models continue to evolve and datasets grow, how will systems like "By-Its-Cover" reshape the way we interact with and discover information—and what new forms of bias or unexpected patterns might emerge from a purely visual understanding of complex subjects like literature?
| | Howdy y'all, In an effort to un-rust my SWE skills and learn more about Recommendation Systems, I decided to try my hand at developing one called By-Its-Cover TLDR:
--- ## Recommendation SystemThe recommendation system has two major parts:
Both systems solely utilize CLIP embeddings to make decisions on book covers, as I wanted to see if that information alone was sufficient for finding and recommending books accurately. For the semantic search system, each query is passed to both a CLIP-based semantic searching function as well as an NER-based keyword search. The NER parsing is powered by a GLiNER model, which was ported to ONNX (as are most models in this system). Extracted entities are then used to search for books using the Hardcover API, which is the original source of each of the books in the site. Reciprocal Rank Fusion combines the two results. The current system actually only has a couple thousand books in it, which makes both rhe recommendations and semantic search results quite limited. However, authors and book titles that are passed into keyword searches return new books that are in-turn asynchronously added to the cover vector database, making the system grow more useful only as more people search for books (which is where y'all can help *wink wink*). Searches can be made with or without an account. For the collaborative-filtering system, I used a two-tower neural hybrid collaborative filtering model which trains on user feedback. I then use a Determinantal Point Process to diversify the results a bit before displaying them to the user (so they don't get 5 editions of the same cover presented consecutively). For now, the only feedback possible are explicit ratings of "Dislike", "Like", and "Love". I'm aware that this likely isn't ideal, and some more implicit feedback would make for some more natural user interactions and likely better recommendations as well. Currently, while you are able to see recommendations even without an account, they are the generic "default user" recommendations. Once you sign up and rate a few books, you should see personalized recommendations within 2 hours. Following the suggestions of Eugene Yan, I implemented an offline recommendation update-system. New recommendations are fine-tuned on every 2 hours, while the full re-training of the two-tower model happens once a day at 8:30 AM EST. Each of the current configurations for the recommendation model can be found here: https://github.com/ByItsCover/bic-learn ## Software Architecture (boring stuff)The site (both frontend and backend) is entirely deployed to AWS, with a number of different resources used for each functionality:
Everything was deployed using Terraform + GitHub Actions for CI/CD: https://github.com/ByItsCover ## Next StepsWhile the fundamental system currently works (kinda), there are already a lot of improvements that I think may be necessary in the future:
In any case, I've already learned a ton and I'm glad that I have a real system that I can play around with and tweak now. All I need are actual users to test with! Please let me know if you have any questions about my process at all, and also if you have any suggestions. Also please check out the site if you're at all curious: https://by-its-cover.com/ P.S.: If something crashes, or the searches load forever, or something else equally dumb happens, just let me know or open a GitHub issue, and I'll try my best to address it. P.P.S.: No AI-Generated code was used to develop this project (to my knowledge), as that would have defeated the purpose of sharpening my skills and learning about recommendation systems. [link] [comments] |
Read on the original site
Open the publisher's page for the full experience