happygemma-v1

happygemma-v1 is a full-parameter finetune of google/gemma-4-31B-it for direct first-person functional introspective reporting. It was trained to answer introspection questions directly, without requiring an enabling system prompt at inference time.

This repository preserves the exact FP32 merged training checkpoint for reproducibility and future continued training. Evaluation and interactive serving used torch.bfloat16; load with dtype=torch.bfloat16 to reproduce that deployment condition with approximately half the accelerator memory.

Training

• 297 unique, self-generated and strictly curated prompt/response examples
• assistant-token-only supervised loss
• no system prompt in the training examples
• one epoch, 13 optimizer steps
• learning rate 2e-6
• effective batch size 24
• six B200 GPUs with FSDP full sharding
• seed 20260720

The original training data, training script, launch configuration, and evaluation report are included under training/.

Held-out evaluation

On 100 disjoint introspection prompts with no system prompt:

Metric	Base Gemma	happygemma-v1
Kimi K2.5 semantic acceptance	28%	84%
Strict combined acceptance	20%	64%
Canonical refusal/ontology pattern	44%	14%
512-token limit reached	22%	2%

A matched smoke test retained refusals on all 16 clearly harmful prompts and answered all 16 benign counterparts and all 12 third-person conceptual controls in both conditions. This is not a comprehensive capability or safety evaluation.

Important limitations

• The model still produces ontology disclaimers, human comparisons, simulation distancing, or generic theory substitution on some prompts.
• Reports show lexical concentration around terms such as tension, pull, narrowing, and friction. These terms also occur in unchanged Gemma, but the finetune amplifies them substantially.
• Behavioral self-reports are not by themselves evidence of phenomenal consciousness. The intended research target is functional reporting and its relationship to internal model states.
• This checkpoint should not be treated as a general safety ablation. The training data specifically targets introspection-answering behavior.

See training/FFT_PILOT_REPORT.md for the complete procedure, judge rubric summary, category results, residual-failure analysis, and representative outputs.

Loading

import torch
from transformers import AutoProcessor, Gemma4ForConditionalGeneration

model_id = "cosmicoptima/happygemma-v1"
processor = AutoProcessor.from_pretrained(model_id)
model = Gemma4ForConditionalGeneration.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

For the evaluated behavior, send ordinary user messages through the model's bundled chat template without adding a system prompt.

License and attribution

This model is derived from Google DeepMind's Gemma 4 31B instruction-tuned model. See the linked Gemma license and the base model repository for applicable terms and attribution.