AI Evals For Engineers & PMs: A Retrospective
7 min
The initial cohort of the AI Evals for Engineers & PMs course is now in the books. Below are some of my…
Jun 15, 2025
ohmeow · Full-stack AI products, built end to end
Available for new engagements
AI products that actually ship
I design, build, and deploy LLM-powered products end to end: eval-first AI pipelines, production-grade APIs, and polished front-ends. Backed by 25+ years of shipping software for companies of every size.
LLMs, evals & ML FastAPI · Vue · React · Postgres Author of blurr · fast.ai community leader
25+Years shipping software
10+Companies served, startups to enterprise
End-to-endStrategy, models, APIs, UI, deployment
Eval-firstAI you can measure, trust, and improve
What I do
Services
Practical, production-minded help for teams that want AI features and applications that work in the real world, not just in a demo.
AI / ML Application Development
Applied AI tailored to your business, from first prototype to a monitored, production system.
- LLM apps, RAG, agents & NLP workflows
- Forecasting & classification models
- Eval-first pipelines with measurable quality
Full-Stack Engineering
Robust, scalable web platforms engineered with modern tooling and a strong DevOps mindset.
- Vue, Quasar, React, Next.js front-ends
- FastAPI, ASP.NET Core, Node.js back-ends
- PostgreSQL, Docker & cloud deployments
AI Strategy & Enablement
Hands-on guidance so your team adopts, evaluates, and operates AI with confidence.
- Technical evaluations & model benchmarking
- AI-driven product architecture
- Team training & workshops
How I work
From idea to production, without the guesswork
A simple, transparent process that keeps risk low and momentum high.
01
Discover
We scope the problem, the users, and what “good” looks like, then agree on a plan measured in weeks, not quarters.
02
Prototype
A working slice of the product, fast. Real data, real users, real feedback before big investments are made.
03
Evaluate
Evals and error analysis drive every iteration, so quality is something we can see and improve, not just feel.
04
Ship & support
Deployed, monitored, documented, and handed off cleanly, with ongoing support when you want it.
Track record
Clients & employers
I’ve had the pleasure of building with great companies across research, agriculture, e-commerce, education, and enterprise software.
Architect and lead developer of DTAM, an AI-powered digital threat assessment and management platform helping schools identify and respond to safety concerns.
Worked with the Logfire team to redesign the evals and observability experience, improving features and leading QA to make it a solid option for evaluating and monitoring LLM applications.
Contributed to core libraries and tutorials driving open-source innovation in LLM tooling.
Delivered ML pipelines and FastAPI backends to power cutting-edge conversational AI tools.
Created educational content training ML developers to use fast.ai and blurr with Hugging Face transformers across NLP tasks.
Architected and built a production-ready AI platform to support real-time agricultural insights.
Built an enterprise survey delivery and analytics platform, an AI pipeline for qualitative data, and forecasting models for HR employment needs.
Cotelligent
Provided enterprise-grade software development and architecture consulting.
Epson
Helped streamline development workflows with tailored software solutions.
Car Crazy Central
Built branded digital experiences and backend infrastructure for media-rich applications.
BuyProduce.com
Built and led a dev team creating a flexible e-commerce engine for the produce industry.
AltaVista / Shopping.com
Rebuilt e-commerce systems and led backend dev teams at the forefront of early internet retail.
From the blog
Latest writing
Notes from the trenches on evals, LLMs, fast.ai, and full-stack engineering.
No matching items
Who I am
About me
Wayde Gilliam
Founder of ohmeow · Full-stack & AI/ML engineer
I’m a full-stack web application and AI/ML developer with well over 20 years in the business. I’ve led teams and shipped systems for early-internet retail giants, universities, startups, and research labs, and these days I spend most of my time building LLM-powered products that are measured as carefully as they are engineered.
I’m the author of blurr, a Python library that lets fast.ai developers train Hugging Face transformers for core NLP tasks, and a proud fast.ai community leader, contributing to the library and helping folks on the forums and Discord.
Outside of work I enjoy hanging out with my two labradors, reading, working out, and above all else, learning.
PythonPyTorchfast.aiHugging FaceLLMs & evalsFastAPIVue / QuasarReact / Next.jsASP.NET CorePostgreSQLDockerCloud / CI
Have an AI project in mind?
Whether it’s an LLM-powered product, an ML pipeline, or a second opinion on your AI strategy, let’s talk about what I can build for you.