Exploration Projects
RAG-Laravel
A local Retrieval-Augmented Generation stack built on Laravel Sail, pgvector and Ollama
Description
A Laravel Sail project extended with two extra services on top of the default stack: a pgvector-enabled PostgreSQL image for storing embeddings, and an Ollama service for running local LLMs and embedding models. The goal was to explore building a Retrieval-Augmented Generation pipeline entirely with local, self-hosted tooling instead of external APIs.
Stack
| Category | Detail |
|---|---|
| Backend | Laravel, Laravel Sail |
| Vector store | PostgreSQL with the pgvector extension |
| LLM runtime | Ollama (llama3.2:3b for generation, nomic-embed-text for embeddings) |
| Tooling | Docker Compose, Composer, pnpm |
What I learned
- Extending a Sail
compose.yamlwith extra services (pgvector, Ollama) - Docker networking: using service names (
pgsql,ollama) instead oflocalhostfor inter-container calls - Enabling the
vectorPostgres extension via a Laravel migration - Calling the Ollama HTTP API directly for text generation and embeddings
- Benchmarking local LLM generation and embedding latency
LLM Performance Metrics
Benchmarks captured against the local Ollama instance:
Generation (llama3.2:3b)
| Metric | Value |
|---|---|
| Total duration | 655.19 ms |
| Load duration | 3.10 ms |
| Prompt eval duration | 83.24 ms |
| Prompt eval count | 32 tokens |
| Eval duration | 567.11 ms |
| Eval count | 8 tokens |
Embedding (nomic-embed-text)
| Metric | Value |
|---|---|
| Elapsed time | 13.45 ms |
| Vector length | 768 |