CNN vs ViT on Stanford Dogs.

Raw input context

What belongs to node 1 before preprocessing starts

For this report, node x stays focused on the raw Stanford Dogs input and the metadata analysis done before transform design. The notebook reconstructs the official split, exports per-image metadata, enriches it with brightness and color statistics, and only then moves on to tensor conversion in node 2.

• Each sample starts as a raw RGB image with its original width, height, and aspect ratio still intact
• The notebook parses train_list.mat and test_list.mat to reconstruct the official Stanford Dogs split
• build_metadata(...) exports path, label, class name, width, height, aspect ratio, and split labels for each sample
• A second pass writes metadata_with_quality.csv with brightness_mean, contrast_std, saturation_mean, r_mean, g_mean, and b_mean
• split_metadata.csv records the internal stratified split: 10,200 train, 1,800 val, and 8,580 test
• The later tensor target is (3, 224, 224) per image, but the actual resize-to-tensor logic is intentionally deferred to node 2

stanford_dogs/
├── images.tar
├── annotation.tar
├── lists.tar
├── Images/
│   ├── n02085620-Chihuahua/
│   │   ├── n02085620_10074.jpg
│   │   └── ...
│   ├── n02085782-Japanese_spaniel/
│   └── ...
├── Annotation/
│   ├── n02085620-Chihuahua/
│   │   ├── n02085620_10074
│   │   └── ...
│   └── ...
├── lists/
│   ├── train_list.mat
│   └── test_list.mat
├── train_list.mat   # sometimes resolved from the root
└── test_list.mat    # sometimes resolved from the root

train_meta_full = build_metadata(train_rel_paths, train_labels, "official_train")
test_meta = build_metadata(test_rel_paths, test_labels, "official_test")
meta = pd.concat([train_meta_full, test_meta], ignore_index=True)

splitter = StratifiedShuffleSplit(
    n_splits=1,
    test_size=VAL_FROM_TRAIN_RATIO,
    random_state=SEED,
)
train_idx, val_idx = next(splitter.split(train_meta_full, train_meta_full["label"]))

quality_rows = []
for image_path in tqdm(meta["image_path"], desc="Computing image-quality metadata"):
    with Image.open(image_path).convert("RGB") as image:
        rgb_arr = np.asarray(image, dtype=np.float32) / 255.0
        gray_arr = np.asarray(image.convert("L"), dtype=np.float32) / 255.0
        hsv_arr = rgb_to_hsv(rgb_arr)
        quality_rows.append({
            "brightness_mean": float(gray_arr.mean()),
            "contrast_std": float(gray_arr.std()),
            "saturation_mean": float(hsv_arr[..., 1].mean()),
            "r_mean": float(rgb_arr[..., 0].mean()),
            "g_mean": float(rgb_arr[..., 1].mean()),
            "b_mean": float(rgb_arr[..., 2].mean()),
        })

quality_df = pd.DataFrame(quality_rows)
meta = pd.concat([meta.reset_index(drop=True), quality_df], axis=1)
meta.to_csv(EDA_ARTIFACT_ROOT / "metadata_with_quality.csv", index=False)

Artifact references used in this panel come directly from the notebook export: metadata_with_quality.csv, split_metadata.csv, quality_distributions.png, dataset_distributions.png, image_size_distribution.png, rgb_channel_summary.png, random_class_samples.png, darkest_examples.png, and brightest_examples.png.

CSV artifacts

What the two exported metadata files actually represent

EDA highlights

Quality variation, split structure, and random breed samples

The exported EDA confirms that Stanford Dogs is visually diverse in brightness, contrast, saturation, image size, aspect ratio, and scene composition. The notebook artifacts also show that the class distribution is only moderately imbalanced, which makes a stratified train/validation split practical without distorting the benchmark too much.

• metadata_with_quality.csv: brightness mean 0.4522, contrast std mean 0.2261, saturation mean 0.3047
• metadata_with_quality.csv: RGB means are R = 0.4761, G = 0.4518, B = 0.3910
• metadata_with_quality.csv: average width 442.5 px, average height 385.9 px, average aspect ratio 1.19
• metadata_with_quality.csv: breed counts range from 148 to 252 images, with a mean of 171.5
• split_metadata.csv: internal split is 10,200 train / 1,800 val / 8,580 test

Brightness, contrast, and saturation distributions confirm a broad range of lighting and color conditions.

The split and breed-distribution export shows a healthy sample count for both official partitions, while the breed histogram remains only moderately imbalanced.

Width, height, and aspect-ratio distributions show that the dataset contains varied image geometries before resizing to the common model input size.

The RGB summary complements the quality CSV by showing that the dataset is not perfectly color-balanced, which helps justify later channel-wise normalization.

Random breed samples visualize the raw input domain before any preprocessing: different poses, backgrounds, framing, and lighting conditions appear immediately.

Extreme brightness examples

Darkest and brightest samples in the dataset

To complement the summary statistics, we also inspected the darkest and brightest images in the dataset. This helps verify that the measured brightness range corresponds to meaningful visual variation rather than only small numerical differences.

The darkest examples are mostly low-light indoor images, shadow-heavy scenes, or photos with strong exposure limitations. In several cases, the dog occupies only part of the frame or blends into a dark background, which makes recognition harder even for humans. These samples motivate brightness-robust preprocessing and moderate augmentation.

The darkest images in Stanford Dogs illustrate low-light scenes, dark backgrounds, and reduced visibility of breed-specific features.

The brightest images often contain white backgrounds or bright fur, showing the opposite end of the illumination range.

Together, these two galleries confirm that Stanford Dogs includes substantial illumination diversity. This is the last stop of the report before node 2 begins the actual data processing pipeline that standardizes geometry and channel scaling.

Data normalization

Normalize tensors for stable training and pretrained-model compatibility

Normalization is the step that rescales each RGB channel so optimization becomes more stable and inputs better match the distribution used during model pretraining. In this notebook, both backbones use ImageNet statistics, even though the exported Stanford Dogs quality CSV shows slightly different dataset-specific RGB means.

# normalization formula
# x_norm = (x - mean) / std

IMAGENET_MEAN = (0.485, 0.456, 0.406)
IMAGENET_STD = (0.229, 0.224, 0.225)

normalize = transforms.Normalize(IMAGENET_MEAN, IMAGENET_STD)

# reference pattern for custom mean/std estimation
channel_sum = torch.zeros(3)
channel_sq_sum = torch.zeros(3)
pixel_count = 0

for images, _ in train_loader:
    channel_sum += images.sum(dim=(0, 2, 3))
    channel_sq_sum += (images ** 2).sum(dim=(0, 2, 3))
    pixel_count += images.size(0) * images.size(2) * images.size(3)

mean = channel_sum / pixel_count
std = (channel_sq_sum / pixel_count - mean ** 2).sqrt()

Complete augmentation pipeline

Train and evaluation transforms used in the notebook

resnet_train_transform = transforms.Compose([
    transforms.Resize((256, 256)),
    transforms.RandomResizedCrop(IMAGE_SIZE, scale=(0.72, 1.0)),
    transforms.RandomHorizontalFlip(0.5),
    transforms.RandomRotation(15),
    transforms.ColorJitter(brightness=0.15, contrast=0.15, saturation=0.15),
    transforms.ToTensor(),
    transforms.Normalize(IMAGENET_MEAN, IMAGENET_STD),
    transforms.RandomErasing(p=0.10),
])

vit_train_transform = transforms.Compose([
    transforms.Resize((256, 256)),
    transforms.RandomResizedCrop(IMAGE_SIZE, scale=(0.78, 1.0)),
    transforms.RandomHorizontalFlip(0.5),
    transforms.ColorJitter(brightness=0.10, contrast=0.10, saturation=0.10),
    transforms.ToTensor(),
    transforms.Normalize(IMAGENET_MEAN, IMAGENET_STD),
    transforms.RandomErasing(p=0.08),
])

common_eval_transform = transforms.Compose([
    transforms.Resize((256, 256)),
    transforms.CenterCrop(IMAGE_SIZE),
    transforms.ToTensor(),
    transforms.Normalize(IMAGENET_MEAN, IMAGENET_STD),
])

Important note: augmentation belongs only to the training path. Validation and test remain deterministic so evaluation measures the model rather than randomness from the preprocessing pipeline.

Custom dataset and best practices

Dataset wrapper, DataLoader settings, and implementation tips

After resize, augmentation, and normalization are defined, the notebook wraps the metadata frame in a custom Dataset. Each image is loaded lazily from disk, converted to RGB, transformed on the fly, and then batched by the DataLoader. This is where raw image files finally become consistent mini-batches for both model families.

• BATCH_SIZE = 32
• NUM_WORKERS = 0 on Windows, otherwise 4
• PIN_MEMORY = torch.cuda.is_available()
• PERSISTENT_WORKERS = NUM_WORKERS > 0
• drop_last is left at the PyTorch default False
• Train uses shuffle=True; validation and test use shuffle=False
• Notebook output confirms identical loader lengths for both families: 319 train, 57 val, 269 test

class StanfordDogsFrameDataset(Dataset):
    def __init__(self, frame: pd.DataFrame, transform=None):
        self.frame = frame.reset_index(drop=True).copy()
        self.transform = transform

    def __len__(self):
        return len(self.frame)

    def __getitem__(self, idx):
        row = self.frame.iloc[idx]
        image = Image.open(row["image_path"]).convert("RGB")
        label = int(row["label"])
        if self.transform is not None:
            image = self.transform(image)
        return image, label

# complete loader example
def build_loaders(train_transform):
    train_ds = StanfordDogsFrameDataset(train_meta, transform=train_transform)
    val_ds = StanfordDogsFrameDataset(val_meta, transform=common_eval_transform)
    test_ds = StanfordDogsFrameDataset(test_meta, transform=common_eval_transform)

    train_loader = DataLoader(train_ds, batch_size=BATCH_SIZE, shuffle=True,  num_workers=NUM_WORKERS, pin_memory=PIN_MEMORY, persistent_workers=PERSISTENT_WORKERS)
    val_loader   = DataLoader(val_ds,   batch_size=BATCH_SIZE, shuffle=False, num_workers=NUM_WORKERS, pin_memory=PIN_MEMORY, persistent_workers=PERSISTENT_WORKERS)
    test_loader  = DataLoader(test_ds,  batch_size=BATCH_SIZE, shuffle=False, num_workers=NUM_WORKERS, pin_memory=PIN_MEMORY, persistent_workers=PERSISTENT_WORKERS)
    return train_loader, val_loader, test_loader

resnet_train_loader, resnet_val_loader, resnet_test_loader = build_loaders(resnet_train_transform)
vit_train_loader, vit_val_loader, vit_test_loader = build_loaders(vit_train_transform)

print("ResNet loaders:", len(resnet_train_loader), len(resnet_val_loader), len(resnet_test_loader))
print("ViT loaders:", len(vit_train_loader), len(vit_val_loader), len(vit_test_loader))
# ResNet loaders: 319 57 269
# ViT loaders: 319 57 269

• Training vs validation: use stochastic augmentation only during training.
• Performance: tune batch_size, num_workers, and pin_memory based on hardware.
• Implementation tip: keep normalization identical across train, validation, and test once the statistics are chosen.
• Implementation tip: visually inspect a denormalized preview batch before starting long training runs.
• Fair comparison: runs_summary.csv shows all four benchmark runs keep the same input size of 224 and batch size of 32.

The two artifact references mentioned earlier are now surfaced directly here. The W&B export file runs_summary.csv confirms that every benchmark run keeps the same preprocessing contract, while the notebook image artifact augmented_batch_preview.png shows what the transformed training batch actually looks like.

The exported notebook artifact augmented_batch_preview.png confirms that the training transform is active, but still preserves the main breed identity after resize, crop, normalization, and augmentation.

Backbone overview

From 224x224 tensors to transferable feature representations

After node 1 and node 2 finish data processing, both model families receive the same minibatch format and then diverge internally. ResNet-50 keeps a convolutional feature grid all the way to global pooling, while ViT-B/16 turns the image into a sequence of patch tokens and summarizes it through the CLS token.

• Every fair-benchmark run starts from the same input contract (32, 3, 224, 224)

  The notebook output and runs_summary.csv agree on the shared preprocessing contract: batch_size = 32 and image_size = 224 for all four benchmark runs.
• ResNet-50 keeps spatial feature maps, then compresses them to a 2048-dimensional pooled vector

  Architecturally, a 224x224 input becomes deep convolutional maps near (N, 2048, 7, 7) before global average pooling reduces them to (N, 2048). The classifier head later reads that pooled descriptor.
• ViT-B/16 converts the image into patch tokens, then uses self-attention to mix information

  ViT splits a 224x224 image into 16x16 patches, so one image becomes 14 x 14 = 196 patch tokens. With one extra class token, the transformer processes a sequence of 197 tokens and learns a feature representation with hidden size 768. The final class token embedding is then passed to the classification head.
• The shape notes here are architecture-derived; params and strategy evidence come from exported artifacts

  The notebook does not print intermediate backbone tensors directly, so the shape flow below is an architecture explanation. The parameter counts, training strategies, and benchmark outcomes are taken from model_comparison.csv, runs_summary.csv, and the staged-run history files.

# Conceptual shape flow used in this report
images, labels = next(iter(resnet_train_loader))
# images.shape -> torch.Size([32, 3, 224, 224])

# ResNet-50
# backbone feature maps: (32, 2048, 7, 7)
# pooled features:       (32, 2048)
# logits after fc:       (32, 120)

# ViT-B/16
# 224x224 with patch size 16 -> 14 x 14 = 196 patches
# token sequence:        (32, 197, 768)   # 196 patches + 1 cls token
# cls embedding:         (32, 768)
# logits after head:     (32, 120)

Backbone evidence files used in this section: model_comparison.csv, runs_summary.csv, resnet50_staged_history.csv, and vit_b16_staged_history.csv.

Transfer learning

The refactored notebook now evaluates both backbones under the same two schedules

This is the biggest change from the old report. Instead of giving the CNN and ViT different main strategies, the current notebook evaluates both under full fine-tuning for 12 epochs and head 3 + full fine-tune 8 epochs. That makes the backbone comparison much cleaner.

def set_trainable_stage(model: nn.Module, family: str, mode: str):
    if mode == 'head_only':
        for param in model.parameters():
            param.requires_grad = False
        if family == 'cnn':
            for param in model.fc.parameters():
                param.requires_grad = True
        elif family == 'vit':
            for param in model.heads.parameters():
                param.requires_grad = True
    elif mode == 'full_finetune':
        for param in model.parameters():
            param.requires_grad = True

• The staged ResNet-50 run slightly beats full fine-tuning on both accuracy and macro F1.
• The staged ViT-B/16 run is clearly best overall and is also faster than the full 12-epoch ViT run.
• The staged history files show immediate backbone reuse: ResNet-50 head-only val accuracy rises from 0.8122 to 0.8483, while ViT-B/16 rises from 0.9456 to 0.9522 in 3 epochs.
• This node is therefore grounded in exported strategy evidence, not only architecture descriptions.

The downloaded comparison_overview.png is the report-ready benchmark figure generated after the fair four-run experiment. It visually confirms the same ranking shown in model_comparison.csv: staged ViT-B/16 first, then full ViT-B/16, then staged ResNet-50, and finally full ResNet-50.

Direct web references for this node: model_comparison.csv, best_per_family_comparison.csv, comparison_overview.png, resnet50_staged_run_metadata.json, and vit_b16_staged_run_metadata.json.

Artifact snapshots

Every backbone file referenced above is now previewed directly on the page

The notebook and W&B exports do not just provide final metrics. They also record the run contract, staged history, and best-per-family summary used to justify the backbone comparison. The tables below are compact previews of those files so the reader does not need to open raw CSV or JSON first.

Classifier head overview

Minimal linear heads keep the comparison centered on the backbone

Once the backbone has produced a compact feature representation, the classifier head is the final component that converts those features into breed scores. In this notebook, both models use simple adapted heads: ResNet-50 sends a pooled feature vector to model.fc, while ViT-B/16 sends its class-token embedding to model.heads.head.

• ResNet-50 head input is the pooled backbone vector (N, 2048)

  The convolutional backbone first reduces spatial information with global average pooling, so the classifier head receives one 2048-dimensional feature vector per image instead of a full feature map.
• ViT-B/16 head input is the class-token representation (N, 768)

  The transformer backbone summarizes the image through the CLS token. That final token embedding is the representation passed into the adapted linear head for breed classification.
• The output of both heads is a logits tensor with shape (N, 120)

  Each row contains one score for each Stanford Dogs breed. These are raw logits, not probabilities yet, which is exactly what the training loss expects.
• Training uses logits directly, while inference adds softmax and argmax

  During training, the notebook uses CrossEntropyLoss on raw logits. During prediction, it applies softmax(dim=1) to obtain probabilities and argmax(dim=1) to choose the final breed.
• The input widths are read from the pretrained models, not hard-coded by hand

  The notebook queries model.fc.in_features for ResNet-50 and model.heads.head.in_features for ViT-B/16 so the adapted head always matches the backbone output.

# Conceptual head interface in this report

# ResNet-50
pooled_features.shape = (N, 2048)
logits = model.fc(pooled_features)          # (N, 120)

# ViT-B/16
cls_embedding.shape = (N, 768)
logits = model.heads.head(cls_embedding)    # (N, 120)

# Inference
probs = logits.softmax(dim=1)               # (N, 120)
preds = probs.argmax(dim=1)                 # (N,)

The notebook does not manually print these intermediate tensors before calling the head, but the widths above follow directly from model.fc.in_features and model.heads.head.in_features in the current code.

Prediction pipeline

Logits become probabilities, predicted labels, and report-ready artifacts

The adapted head outputs raw logits. Those logits are consumed in two different ways: during training they go directly into CrossEntropyLoss, and during inference they are converted into probabilities with softmax so the notebook can derive final predictions and export evaluation files. The staged history files also show how quickly the new head starts aligning with Stanford Dogs labels before the full backbone is unfrozen.

• CrossEntropyLoss expects raw logits, not pre-softmax probabilities

  This is why the forward pass returns unnormalized scores. The loss function internally applies the correct log-softmax behavior in a numerically stable way.
• softmax(dim=1) converts logits into a 120-way probability vector

  Each probability vector sums to one and can be reused for calibration analysis, confidence histograms, and other reporting artifacts generated later in the workflow.
• argmax(dim=1) selects the top-1 breed prediction for each image

  The predicted class index is compared against the ground-truth label to compute accuracy, F1, confusion matrices, and the final classification report.
• evaluate_model(...) reuses the head outputs to save CSV, JSON, and image artifacts

  The notebook exports classification reports, count-based confusion matrices, row-normalized confusion matrices, top-confusion tables, and summary metrics from the same prediction outputs.

def predict_model(model, loader):
    model.eval()
    all_targets, all_preds, all_probs = [], [], []
    with torch.no_grad():
        for images, targets in tqdm(loader, leave=False):
            images = images.to(DEVICE, non_blocking=True)
            outputs = model(images)
            probs = outputs.softmax(dim=1)
            preds = probs.argmax(dim=1)
            all_targets.extend(targets.numpy())
            all_preds.extend(preds.cpu().numpy())
            all_probs.append(probs.cpu())
    return np.array(all_targets), np.array(all_preds), torch.cat(all_probs, dim=0).numpy()


def evaluate_model(model, loader, class_names, model_name, artifact_dir):
    targets, preds, probs = predict_model(model, loader)
    report = classification_report(targets, preds, target_names=class_names, zero_division=0, output_dict=True)
    report_df = pd.DataFrame(report).transpose()
    cm = confusion_matrix(targets, preds)
    cm_norm = confusion_matrix(targets, preds, normalize='true')

The staged histories make the head adaptation behavior visible on the web report itself: ResNet-50 needs more work from the backbone, while ViT-B/16 already starts with a very strong transferred representation.

Artifact-backed head outputs

The adapted heads feed the downloaded report files used later on the web page

The downloaded artifacts make node 4 concrete: once logits and probabilities are produced, the notebook writes classification reports, metrics JSON files, confusion matrices, calibration plots, and qualitative galleries. For the backbone/head discussion here, the staged runs are the most relevant exports because they reflect the current fair benchmark setup.

• Match the head input width to the backbone output dimension instead of guessing it manually.
• Set the output width to the dataset class count, which is 120 for Stanford Dogs.
• Use raw logits for loss computation and reserve softmax probabilities for inference and reporting.
• The excerpt above comes directly from the staged per-class reports, so node 4 now shows concrete breed-level evidence from the downloaded artifacts.
• Keep the head simple first; the benchmark files above were all generated with these minimal linear heads.

The staged ResNet-50 head exports this count-based confusion matrix directly from the same logits used to create classification_report.csv and metrics.json.

The staged ViT-B/16 confusion matrix is the transformer counterpart to the CNN export and shows the cleaner top-1 prediction structure that follows from the stronger head outputs.

This calibration export reuses the probability vectors produced after softmax(dim=1), so it is a direct visualization of how reliable the ResNet-50 head confidence is on the test set.

The ViT-B/16 calibration figure is also generated from the same head probabilities and complements the lower ECE reported in the staged metrics.json.

Node 4 ends at head outputs and prediction mapping. Node 5 is where those exported files are interpreted as final benchmark results, diagnostics, and recommendations.