Independent · free · vetted · no vendors

info@pharmaaicollective.org

Pharma AI Collective
Menu
Week 1 · unit 1 of 7

  1. Brief

    1. The confident intern
    2. From answering to acting
    3. Where it lands in pharma
    4. Follow the money

  2. Try

    1. Same question, three ways

  3. Ask

    1. Questions for Monday

  4. Build

    1. Page 1: the opportunity map

The confident intern

Introduction

What a language model actually does, why fluency isn't accuracy, and why it can ace a hard test and fail an easy one.

Reflection

Where you are today

Rate each statement from 1 (not yet) to 5 (I could do it in front of the executive committee). You'll rate them again at the end of the week.

I can explain to my boss, in two sentences each, what a language model does and doesn't do, and how an agent differs from a chatbot.
I can name where AI changes decisions, and where it only changes the work, across the pharma value chain.
I can size an AI opportunity from our own P&L and balance sheet instead of a vendor's slide.

Exercise

Where did Section 6.3 come from?

What is the most likely status of that section reference?

Your first week as VP, Operations Excellence at Meridian Biologics. The digital lead shows you the enterprise assistant approved last quarter. She types a question about line clearance in Kingsford's downstream suites. In seconds a clear, well-structured answer appears, ending with "as required by Section 6.3".

This assistant is not connected to Meridian's document system, and nobody pasted in the SOP. It has only the question and what it learned in training.

Where did Section 6.3 come from?
How confident are you?

Show answers
It can't look up what it isn't connected to, and this one isn't. If you assume it searched, an unchecked reference can travel into a deviation report with your name on the approval.
Yes. The model generates text that fits the pattern of SOP answers, and section numbers are part of that pattern. It might happen to match the real SOP, but nothing in the process checked. Until someone opens SOP-MFG-0147, treat it as unsupported.
It has seen a lot of public pharma text, which is why the shape of the answer is convincing. Meridian's controlled documents are not public, so whatever "6.3" says, the model didn't get it from them.
An overcorrection that will cost Meridian real gains. The same model is useful for drafting and summarising material you supply. The skill is knowing what it was looking at when it answered.

Fluency shows the model has seen many answers like this one. It is not evidence that this answer is true, and neither is a citation until someone checks it. The first question about any AI output: what was it looking at?

A large language model is trained on a vast amount of text, then refined further by its maker. When you use it, it generates its answer one token at a time, each chosen to fit everything in its context window: your question, anything supplied or retrieved, and what it has written so far. It doesn't consult a source unless one is supplied or connected. Some tools flag uncertainty or run checks, but neither fluency nor a citation proves an answer correct. Fluent output that is false or unsupported is called a hallucination.

Hence the title of this unit. The assistant behaves like a well-read new hire: fast, articulate, good at drafts and at summarising what you hand over, and with no knowledge of your SOPs or systems unless you supply them.

How a model writes its answerDiagram: the question and the text written so far fill the model's context window. The model scores candidate next words, such as '6.3', picks one and appends it to its answer. Meridian's document system sits outside, unconnected: it is not consulted unless supplied or connected.Context windowworking memoryYour questionAnything suppliedor retrievedempty in this exampleWhat it has writtenso far…as stated in SectionModelPatterns learnedin training, up toa cutoff datePossible next tokens6.35.24Illustrative likelihoods.The longest is picked.The chosen token is added to the end, and the cycle repeatsMeridian’s document systemNot consulted unless supplied or connected How a model writes its answerDiagram: the question and the text written so far fill the model's context window. The model scores candidate next words, such as '6.3', picks one and appends it to its answer. Meridian's document system sits outside, unconnected: it is not consulted unless supplied or connected.Context windowworking memoryYour questionAnything supplied or retrievedempty in this exampleWhat it has written so far…as stated in SectionModelPatterns learned in training,up to a cutoff datePossible next tokens6.35.24Illustrative likelihoods. The longestis picked and added to the end ofthe answer, then the cycle repeats.Meridian’s document systemNot consulted unless suppliedor connected
Why a fluent answer can cite a section nobody looked up. The likelihoods are illustrative.
Text description

Context window. Holds three things: your question, anything supplied or retrieved (empty in this example), and what the model has written so far, which ends "…as stated in Section".

Model. It draws on patterns learned in training, up to a cutoff date, and scores the candidate next tokens: 6.3 highest, then 5.2, then 4. These are illustrative likelihoods, not measurements.

The pick. The likeliest token is chosen and added to the end of what it has written. Then the cycle repeats.

Meridian's document system. Sits outside, unconnected. The model doesn't consult it unless a document is supplied or the system is connected.

Four beliefs to retire

"It's a search engine that writes nicely."

Unless the tool is connected to a source and shows you what it retrieved, it answers from patterns learned in training, which stop at a cutoff date. Meridian's controlled documents reach it only when you supply them or someone connects them.

"If it beats experts on a hard test, it will handle our easy task."

Capability is uneven, the jagged frontier. A model scored gold at the 2025 International Mathematical Olympiad, yet the best model read analog clocks correctly only about half the time, against about 90% for people (Stanford AI Index 2026 (opens in a new tab)). Test it on your task, not on a leaderboard.

"The new reasoning models don't make things up."

A reasoning model works through intermediate steps before answering. That can help on complex, multi-step problems, but it doesn't guarantee a truthful answer, and it still works from learned patterns and whatever it is given. On the AI Index 2026's general-knowledge benchmark, rates of unsupported answers across 26 mixed models ranged from 22% to 94% (AI Index 2026 (opens in a new tab)). That is not a rate for SOP questions; it shows how widely models differ, so test yours on your task.

"With a million-token window, we can paste in everything."

The context window is the model's working memory: the conversation, anything supplied or retrieved, and its own output, which for some models includes their reasoning steps (Google, Gemini long-context guide (opens in a new tab)). As reported in the AI Index 2026, some models now accept a million tokens or more, but there is a wide gap between what a model accepts and what it uses well, and Chroma's July 2025 tests of 18 models (opens in a new tab) found accuracy falling, unevenly, as inputs grew. Give it the right three pages, not all 9,000 controlled documents.

The jagged frontierSchematic: a jagged line separates tasks a model usually does well from tasks where it gives plausible, wrong answers. Olympiad maths, which looks hard, is inside. Reading an analog clock, which looks easy, is outside.Reading an analog clock:about half right(people about 90%)Summarising a documentyou hand overQuoting a section of anSOP it was never givenOlympiad maths:gold, 2025 IMOOutside the frontier:plausible, wrong answersInside the frontier:usually done wellLooks easy to a personLooks hard to a person The jagged frontierSchematic: a jagged line separates tasks a model usually does well from tasks where it gives plausible, wrong answers. Olympiad maths, which looks hard, is inside. Reading an analog clock, which looks easy, is outside.Top: looks easy to a person.Bottom: looks hard to a person.Outside:plausible, wronganswersInside:usually done wellReading an analogclock: about halfright (peopleabout 90%)Summarising adocument youhand overQuoting a sectionof an SOP it wasnever givenOlympiad maths:gold, 2025 IMO
Schematic: positions are not measurements. Sources: Stanford AI Index 2026 (opens in a new tab); summaries, Van Veen et al., 2024 (opens in a new tab); unseen sources, Dahl et al., 2024 (opens in a new tab).
Text description

Tasks run left to right by how hard they look to a person. A jagged line crosses them. Below it is inside the frontier, where the model usually does well; above it is outside, where it gives plausible, wrong answers.

Reading an analog clock looks easy and sits outside: about half right, against about 90% for people. Olympiad maths looks hard and sits inside: gold at the 2025 International Mathematical Olympiad.

Summarising a document you hand over is inside: in a 2024 study, physicians rated the best model's summaries of clinical text as good as or better than experts' in 81% of cases. Quoting from a document the model was never given is outside: asked specific questions about real court cases, public models answered wrongly at least 58% of the time.

Key idea

Sources for this unit

  • The AI Index 2026 Annual Report — Stanford Institute for Human-Centered AI, April 2026. Read the source (opens in a new tab) Independent annual review. Read the Top Takeaways, then the Economy and Medicine chapters.
  • Adapted large language models can outperform medical experts in clinical text summarization — Nature Medicine (Van Veen et al.), 2024. Read the source (opens in a new tab) Ten physicians judged the best model's summaries equivalent (45%) or better (36%) than experts' on four clinical tasks.
  • Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models — Journal of Legal Analysis (Dahl et al.), 2024. Read the source (opens in a new tab) Asked verifiable questions about real court cases, public models answered wrongly at least 58% of the time.
  • Context Rot: How Increasing Input Tokens Impacts LLM Performance — Chroma Research, 2025-07-14. Read the source (opens in a new tab) 18 models tested: accuracy falls, unevenly, as inputs get longer.
  • Long context — Google AI for Developers (Gemini API documentation), accessed 2026-11-02. Read the source (opens in a new tab) Vendor documentation: one model's million-token window is shared across input, thinking and output tokens.