Back to Portfolio Projects
PROJECT 01RAG & Search

Semantic Search Analyzer

Vector Embeddings, GMM Clustering & First-Principles Semantic Caching

Built a scalable semantic search system over the 20 Newsgroups corpus using vector embeddings and GMM-based clustering, featuring a custom first-principles semantic cache.

PythonFastAPIVector EmbeddingsGMM ClusteringSemantic CachingREST APIs
SYSTEM ARCHITECTURE & EXECUTION PIPELINEVERIFIED SPECIFICATION
1
20 Newsgroups Corpus Input
2
Dense Vector Embedding Pipeline
3
GMM-Based Gaussian Clustering Engine
4
First-Principles Custom Semantic Cache
5
FastAPI Scalable REST Endpoints

01. The Problem Statement

Standard keyword search fails to capture contextual intent, while traditional in-memory caching solutions like Redis/Memcached require external dependency management and lack semantic similarity awareness.

02. Architectural Solution

Designed and deployed REST APIs using FastAPI for an end-to-end embedding → clustering → indexing pipeline, accompanied by a custom zero-dependency semantic cache built from first principles.

Key Engineering Features & System Capabilities

First-principles semantic cache with zero external dependencies (no Redis required)
GMM soft clustering for multi-topic probabilistic document categorization
FastAPI asynchronous REST endpoints for ingestion, embedding, and query retrieval
Vector distance similarity metrics optimized for high dimensional space
Engineered by Nikhil Agarwal (VIT Chennai)View Source Code on GitHub