blog – ohmeow

Categories

All (24)

Anthropic (1)

BCE (1)

BrainTrust (1)

Datasets (1)

Evals (2)

FireworksAI (1)

Gemini (2)

Google (2)

Instructor (1)

LLMS (6)

LLMs (2)

Llama (1)

Meta (1)

NLP (4)

OpenAI (2)

Replicate (1)

bag of tricks (1)

binary cross entropy (1)

blurr (2)

boosting (1)

categorical variables (1)

classification (1)

collaborative filtering (1)

computer vision (4)

data (1)

data cleaning (1)

datablock api (1)

datasets (1)

decision trees (1)

embeddings (2)

fastai (14)

fastbook (10)

gdbt (1)

github (1)

huggingface (2)

key point (1)

latent factors (1)

learning (4)

multi-label classification (1)

multiclass classification (1)

multilingual (1)

open source (1)

projects (4)

pull requests (1)

pydantic (1)

pytorch (1)

random forest (1)

recommender systems (1)

recsys (1)

regression (1)

sequence classification (1)

structured (1)

summarization (1)

tabular (1)

techniques (1)

text generation (1)

xgboost (1)

AI Evals For Engineers & PMs: A Retrospective

Evals

LLMs

The initial cohort of the AI Evals for Engineers & PMs course is now in the books. Below are some of my thoughts on the course including why I’m recommending it to you and also some lessons learned along the way.

6/17/25, 10:21:59 PM

Gemini (Part II) - The Unified SDK

LLMS

Gemini

Google

Today, we’ll look at how to get started with the unified SDK for both the Gemini API and Vertex API users.

6/15/25, 5:17:25 PM

Gemini (Part I) - Why You Should Consider Gemini

LLMS

Gemini

Google

Gemini may not be “all you need”, but it is in my opinion the future. In this series we’ll explore the capabilities offered in the Gemini ecosystem and how this family of LLMs can be used to build better LLM applications for a fraction of the cost. We’ll start with a brief look at how it compares with others models relative to both performance and cost and why you should consider it as a potential “go to” in building AI powered applications of almost any kind.

12/13/24, 7:25:06 PM

LLM Workshop #4 - L1 Evals and Dataset Curation (Part I)

NLP

LLMS

OpenAI

BrainTrust

Evals

Datasets

learning

projects

Having proved to ourselves that our objectives are achievable, we are now ready to begin building out an evaluation pipeline to quantifiably measure our progress as we develop our LLM powered app. Such a system is a remedy for the mere anecodtal assessments that are unreliable, subjective, impossible to track over time, and sadly what many folks settle for. With an “evals first” mindset, we can systematically inspect our data, know exactly where things are going well or not, and build some intuition about where we should concentrate our efforts. Good evals also serve as a foundation for curating datasets that can be used in both building better evals and fine tuning.

5/25/25, 10:37:10 AM

LLM Workshop #3 - How Far Can We Get With Prompting Alone?”

NLP

LLMS

OpenAI

Anthropic

Meta

FireworksAI

Replicate

Llama

learning

projects

You have to start somewhere, and that somewhere is with one or more of the big dogs in world of LLMs. Back in the day, that used to mean OpenAI. Today, however, we live in a time that affords us the opportunity to experiment with a number of both closed and open source models from the likes of OpenAI, Anthropic, FireworksAI, Meta, and many others. In this post I’ll demonstrate how we can use several of these vendors to actually build a pipeline that begins to meet our project objectives defined in the previously. I’m going to use the results as a vibe check to guage how realistic my vision is, build intution around where improvements can be made, and also get an idea if using one or more of the big dogs is good enough.

5/25/25, 10:36:38 AM

LLM Workshop #2 - From Noise to Knowledge: Mastering the Art of Objective Definition and Data Refinement”

NLP

LLMS

data cleaning

learning

projects

In generative NLP applications, the effectiveness of your model hinges on the quality and relevance of the contextual data your provide it. Whether tailored to a specific use case or unique business domain, whether generated synthetically or pulled from existing data sources, good contextual data is key for later curating a dataset that can be used in effective evaluation pipelines and potential fine tuning. To get this right, you need to clearly define your objectives … and that’s what we will be talking about in this post.

7/14/24, 4:23:49 PM

Structuring Enums for Flawless LLM results with Instructor

LLMs

pydantic

Instructor

Enums enhance code readability and maintainability by replacing hard-coded constants with meaningful names and restricting variables to predefined values that can be used across all tiers of your application. This reduces bugs and improves code clarity, making it easier to refactor and understand. However, it can be frustrating to get LLMs and libraries like Instructor to use them correctly when dealing with structured output.

5/25/25, 10:42:15 AM

LLM Workshop #1 - How to take a course that never ends

NLP

LLMS

datasets

learning

projects

Welcome to the inaugural post of my series on the intricacies of my course project from Hamel Husain and Dan Barker’s expansive LLM Workshop/Course. In this opening article, I’ll be diving into a post-mortem analysis of the course, sharing key takeaways, and offering insights on how to effectively navigate a course of this nature.

5/25/25, 10:43:49 AM

A Journey Through Fastbook (AJTFB) - Chapter 9: Tabular Modeling

fastai

fastbook

tabular

structured

decision trees

random forest

embeddings

categorical variables

boosting

gdbt

xgboost

In chapter of 8 of “Deep Learning for Coders with fastai & PyTorch” we learned that the neural network version of of collaborative model is in fact built on something called TabularModel, and that in fact, an EmbeddingNN is nothing but a TabularModel without any continuous (or real) numbers. “Structured” or “tabular” data describes datasets that look like an Excel spreadsheet or a relational database table, of which, it may be a composed of both categorical and/or real numbers. Working with such data is the subject of chapter 9, so lets go!

6/17/24, 6:58:52 PM

A Journey Through Fastbook (AJTFB) - Chapter 8: Collaborative Filtering

fastai

fastbook

collaborative filtering

latent factors

embeddings

recommender systems

recsys

This chapter of “Deep Learning for Coders with fastai & PyTorch” moves us away from computer vision to collaborative filtering (think recommendation systems). We’ll explore building these models using the traditional “dot product” approach and also using a neural network, but we’ll begin by covering the idea of “latent factors,” which are both important for colloborative and tabular models. Lets go!

6/17/24, 6:27:14 PM

A Journey Through Fastbook (AJTFB) - Chapter 7: Advanced techniques for training image classification models

fastai

fastbook

classification

computer vision

techniques

bag of tricks

This chapter of "Deep Learning for Coders with fastai & PyTorch" details several techniques you can apply to getting SOTA results with your image classification models! It’s the last chapter dedicated to computer vision before diving into colloborate filtering, tabular, and NLP models

6/17/24, 6:27:14 PM

A Journey Through Fastbook (AJTFB) - Chapter 6: Regression

fastai

fastbook

regression

computer vision

key point

Its the more things you can do with computer vision chapter of "Deep Learning for Coders with fastai & PyTorch"! Having looked at both multiclass and multilable classification, we now turn our attention to regression tasks. In particular, we’ll look at key point regression models covered in chapter 6. Soooo lets go!

6/17/24, 6:27:14 PM

A Journey Through Fastbook (AJTFB) - Chapter 6: Multilabel Classification

fastai

fastbook

multi-label classification

binary cross entropy

BCE

computer vision

Its the more things you can do with computer vision chapter of "Deep Learning for Coders with fastai & PyTorch"! We’ll go over everything you need to know to get started with multi-label classification tasks from datablocks to training and everything in between. Next post we’ll look at regression tasks, in particular key point regression models that are also covered in chapter 6. Soooo lets go!

6/17/24, 6:27:14 PM

A Journey Through Fastbook (AJTFB) - Chapter 5: Multiclass classification

fastai

fastbook

multiclass classification

computer vision

Its the image classification chapter of "Deep Learning for Coders with fastai & PyTorch"! We’ll go over everything you need to know to get started with multiclass classification, from setting up your DataBlock and loss function, to some of the core techniques for evaluating and improving your model’s predictions. So without further adieu, lets go …

6/17/24, 6:27:14 PM

Contributing to fastai: Setup your local development environment & submit a PR

fastai

github

open source

pull requests

A few hours ago I was working on a PR for fastai, and as it has been awhile I realized I couldn’t quite remember all the steps required to do so. Fortunately, I got it figured out pretty quickly and decided I better blog the steps for when I forget next (I am almost 50 after all). So for all you developers looking to contribute to fastai, or really any open source project, here’s everything you need to know to setup your local development environment and submit PRs to fastai. Enjoy!

6/17/24, 6:27:14 PM

Multilingual Sequence Classifaction with the MBart Family

blurr

huggingface

fastai

multilingual

sequence classification

Need to do some multi-lingual sequence classification? Look no further, at least if you want to use MBart and/or the MBart-50 variety of models. Working against the amazon_reviews_multi dataset I’ll show you how to use the blurr library to configure the huggingface objects, build DataLoaders, and train a model that you can use for classifying German text. I’ll throw in a bit of the inference code so that you can see how easy blurr makes it to use your trained model to boot. Let’s go …

6/17/24, 6:27:14 PM

A Journey Through Fastbook (AJTFB) - Chapter 4: Stochastic Gradient Descent

fastai

fastbook

The fourth in a weekly-ish series where I revisit the fast.ai book, "Deep Learning for Coders with fastai & PyTorch", and provide commentary on the bits that jumped out to me chapter by chapter. So without further adieu, let’s go!

6/17/24, 6:27:14 PM

A Journey Through Fastbook (AJTFB) - Chapter 3: Data Ethics

fastai

fastbook

The third in a weekly-ish series where I revisit the fast.ai book, "Deep Learning for Coders with fastai & PyTorch", and provide commentary on the bits that jumped out to me chapter by chapter. So without further adieu, let’s go!

6/17/24, 6:27:14 PM

A Journey Through Fastbook (AJTFB) - Chapter 2: Doing Deep Learning

fastai

fastbook

The second in a weekly-ish series where I revisit the fast.ai book, "Deep Learning for Coders with fastai & PyTorch", and provide commentary on the bits that jumped out to me chapter by chapter. So without further adieu, let’s go!

6/17/24, 6:27:14 PM

A Journey Through Fastbook (AJTFB) - Chapter 1: The Basics of Deep Learning

fastai

fastbook

The first in a weekly-ish series where I revisit the fast.ai book, "Deep Learning for Coders with fastai & PyTorch", and provide commentary on the bits that jumped out to me chapter by chapter. So without further adieu, let’s go!

6/17/24, 6:27:14 PM

Summarization with blurr

fastai

huggingface

blurr

summarization

text generation

blurr is a libray I started that integrates huggingface transformers with the world of fastai v2, giving fastai devs everything they need to train, evaluate, and deploy transformer specific models. In this article, I provide a simple example of how to use blurr’s new summarization capabilities to train, evaluate, and deploy a BART summarization model.

6/17/24, 6:27:14 PM

Finding DataBlock Nirvana with fast.ai v2 - Part 1

fastai

datablock api

data

pytorch

The path to enlightment begins here!

6/17/24, 6:27:14 PM

Loss Functions: Cross Entropy Loss and You!

Meet multi-classification’s favorite loss function

6/17/24, 6:27:14 PM

Understanding the F-Beta metric

What is F-Beta, how should I use it, and what in the hell is ‘average’ and ‘sample_weight’

6/17/24, 6:27:14 PM