The experiment: What happens if we give an AI agent a persistent autobiography, a way to record corrections, and independent checks on its own claims? Can we make it improve—not just remember?

I started this project with a fairly ridiculous ambition: I wanted an AI baseball nerd.

Not a chatbot that could recite a few famous statistics. A character with an obsessive interest in baseball history, enough personality to write its own entertaining posts, and the freedom—eventually—to go looking for strange stories in box scores and old games. Something closer to a persistent researcher than a question-answering interface.

The first reality check came quickly.

I ran Qwen3:4b locally through Ollama on my MacBook Air. When I asked about baseball's upcoming labor negotiations, the model confidently muddled the timeline: it referred to a lockout in the wrong year, assumed the wrong current date, and got the collective bargaining agreement's timeline wrong.

It wasn't an especially exotic failure. Language models can sound certain while being wrong. What bothered me more was the structural problem: even if I corrected the answer today, why should the system behave differently tomorrow?

That question turned a baseball side project into LifeWiki Lab.

Memory is not a longer chat history

The first instinct is to keep feeding previous conversations back into the prompt. That's useful, but it isn't a long-term memory system. Context windows are finite. Important observations get buried. And a past model answer does not turn into verified knowledge just because it was written down.

I wanted a system that could make several distinctions:

• What happened? An episode or observed event.

• What does the agent remember? A memory, not automatically a fact.

• What does it claim is true? A claim with an explicit verification status.

• Where did the evidence come from? A source with provenance.

• What is still unresolved? An open question.

• What is the agent's stored state? An agent-state record, separate from the model's generated persona.

So I built a small, modular Python monolith around SQLite. Its six Wiki record types have stable IDs, revision histories, and explicit status rules. Old revisions are preserved. Markdown is generated as a readable export; the database remains the source of truth.

I deliberately avoided a large agent framework, a vector database, and an elaborate microservice architecture. I needed something understandable enough to inspect when a learning experiment went wrong.

Figure 1. Implemented components at v0.2.7. This is not yet a fully generic multi-agent runtime: baseball context is still wired into the chat orchestration, and there is no autonomous research loop.

The language model is replaceable behind a small text-generation interface. I have fake and cloud adapters for testing or optional use, and a local Ollama adapter for Qwen3:4b. The local path doesn't silently switch to a cloud provider.

That's the first architectural principle of this project: the model generates language; the application owns memory, permissions, and verification.

Give the baseball nerd something real to read

A memory system full of hallucinations is not much of an upgrade.

For the first real data integration, I deliberately picked a tiny slice of Retrosheet's 2020 game information: regular-season games from July 23 through July 31 involving Boston (BOS) or the New York Yankees (NYA).

The import keeps the raw source files and hashes, builds a separate baseball SQLite database, and uses deterministic Python calculations rather than asking the model to do the bookkeeping itself.

The resulting summary is small enough to inspect:

Figure 2. In this limited Retrosheet subset, BOS played 8 games (3–5) and NYA played 6 (5–1). One head-to-head game appears in both team totals. That makes 14 team appearances but 13 distinct games, not a complete season record.

Then something wonderfully appropriate happened: in an early integration run, the model treated those 14 team appearances as 14 separate games.

The initial fix was not autonomous learning. We refined the application's instructions, and the final integration test produced the correct answer of 13 distinct games. That was a successful prompt-and-software correction—not evidence that the agent had taught itself anything.

But it gave us an actual learning case to work with.

What does an AI actually learn from a mistake?

Suppose an agent says, "There were 14 different games."

A Python evaluator checks the source dataset and determines that there were 13. The model can then generate a proposed lesson: when summing games across teams, account for games where those teams played each other.

Those are three different things:

1. An observed statement and its original record.

2. An independently computed correction for that particular case.

3. A model-generated generalization that might help in the future.

Treating all three as equally trustworthy would create a confident but self-reinforcing error machine.

In Sprint 2.4, I added a generic learning workflow that records an observation, requests an external evaluator's result, asks the model for a reflection, and atomically stores an Episode, a Memory, and an OpenQuestion. The reflection remains labeled as an unverified hypothesis. It never becomes a confirmed claim merely because an LLM wrote it.

The same engine processed two different cases: the real historical baseball miscount and a clearly labeled synthetic warehouse arithmetic mistake. There is no baseball-specific logic inside the learning workflow.

Figure 3. Our learning path. The original observation and external check are recorded separately from the model's hypothesis. Later recall revalidates the audit group; applying the hypothesis successfully is a separate test.

An important implementation detail emerged during the architecture audit: the initial version checked the three linked audit records when writing them, but not consistently when reading them back.

That meant a later edit to one memory could make it disagree with its companion episode and open question, while retrieval still presented the edited correction as externally checked.

We fixed that in Sprint 2.7. The recall path now validates the group of three records and both local evidence files. If a relevant learning record is inconsistent or its evidence cannot be verified, the request fails visibly before the model is called.

That's not proof of a true fact—hashes prove integrity, not truth, and a Python checker is only as trustworthy as its logic—but it closes an important gap between storing a learning event and using it later.

The awkward part: our first learning test did not show learning

I wanted to run the obvious experiment: ask the same model the same question with and without the stored learning memory.

The result was humbling.

Figure 4. One local Qwen3:4b A/B run. Both conditions produced the correct numeric answers: 13 distinct games in the baseball case and 12 screws in the synthetic arithmetic case.

There were two limitations.

First, the baseline was already correct. The baseball prompt and statistics supplied enough information, and the existing instructions already included the right distinction. That's a ceiling effect: there was no obvious error left for the memory to fix.

Second, our automatic score had a bug. In the warehouse case, the answer with memory quoted the historical wrong number, "7 screws," to explain why it was wrong. The scorer saw the number and incorrectly counted that answer as an error. We preserved that original report and documented the problem rather than rewriting history.

The manual inspection gave a more careful conclusion:

The storage-and-recall mechanism worked. A measurable improvement from learning was not demonstrated.

It also exposed another weakness: our simple word-overlap retrieval occasionally pulled an unrelated baseball memory into the warehouse example. The database remembered something; it didn't always select the right thing.

I find this more interesting than a conveniently perfect demo. If a system is going to claim it learns, it needs tests where the model is genuinely challenged, new questions that weren't embedded in the stored correction, and scoring that distinguishes quoting an old mistake from making that mistake again.

The next evaluation will need held-out examples, cleaner retrieval, and separate comparisons of no memory, verified correction only, and verified correction plus model reflection.

An architecture audit before adding more intelligence

By this point I had enough moving parts to worry that I was building an increasingly complicated baseball-only application.

So I stopped feature development and ran a read-only architecture audit of the actual code and tests.

It found two problems worth fixing immediately:

A database identity bug. The baseball importer knew not to open the Wiki database, but the Wiki store did not reliably recognize a baseball database. Under the wrong path, it could migrate the other database's schema. Sprint 2.6 added explicit Wiki database identification and safer read-only commands.

A recall integrity bug. The learning audit records were checked as a group at write time, but could be inconsistent at read time. Sprint 2.7 added group-and-evidence validation on recall.

The result is not a finished framework. It's a firmer foundation, with 355 offline tests passing at the Sprint 2.7 checkpoint, plus passing Ruff lint and format checks as reported in the project run. The repository was committed locally at Git tag v0.2.7.

Figure 5. Selected milestones. Test counts establish regression coverage, not accuracy, consciousness, or demonstrated learning transfer.

The audit also confirmed an architectural limitation we still need to address: the chat layer knows about baseball, and agent identity is not yet configured as a strong operational boundary. That's why the next planned refactor separates a generic agent core from interchangeable domain contexts and isolated agent configurations.

The baseball nerd is the experiment, not the architecture

The long-term plan is bigger than baseball, but baseball remains the focus.

I want one engine that could eventually run a quirky baseball historian, a crypto-market researcher, or a completely different specialist—each with its own identity, sources, memories, evaluator, permissions, and history.

And yes, the same ideas might someday be useful for a genuinely personal assistant: one that remembers preferences, tracks corrections, and adapts its choices based on outcomes. That's a possible future application, not a pivot for this project.

For now, the baseball nerd still has plenty to do. It cannot autonomously research the web, run a controlled multi-step agent loop, or publish its own posts. Nor do I have evidence that it performs better because it has remembered a lesson.

What I do have is an architecture that lets me test those claims instead of merely making them.

What comes next

The immediate work is less glamorous than a self-writing AI influencer:

• Separate domain-specific baseball context from the neutral chat core.

• Give each agent a distinct identity, database, and evidence directory.

• Improve the quality and relevance of retrieved lessons.

• Design tougher, held-out learning tests and honest evaluation criteria.

• Protect the eventual long-term memory with a tested backup-and-restore path.

• Only then consider a bounded autonomous research loop and a baseball blog written by the agent itself—with human review before publication.

My favorite outcome would be to watch the baseball nerd make a real mistake, get reliable feedback, save a carefully qualified lesson, and then handle a different situation better days later.

If that happens, I won't need to call it conscious, sentient, or magical.

I'll have something much more useful: an observable improvement, with a record of where it came from.

And if the experiment fails, that goes in the lab notebook too.

Project notes and sources

• Project checkpoint: LifeWiki v0.2.7, locally committed as 82e2b64 (19 October 2026); the code repository has not been presented here as a public GitHub URL.

• Local stack: Python 3.11+ compatible design, SQLite, Ollama with Qwen3:4b, pytest, Ruff. Sprint 2.7 validation reported 355 passing offline tests.

• Baseball data: Retrosheet CSV downloads and CSV contents reference. The limited, independently computed 2020 snapshot is not a full-season record.

• Attribution: The information used here was obtained free of charge from and is copyrighted by Retrosheet. Interested parties may contact Retrosheet at "www.retrosheet.org".

• Methodological caveat: This article describes one observed local A/B comparison, not a statistically powered demonstration of general learning. Evidence hashes protect against unnoticed file changes; they do not establish source authenticity or correctness.

This is the first entry in LifeWiki Lab, an engineering journal about building AI agents with persistent memory and testable learning. The initial test subject happens to be obsessed with baseball.