What Would a CEG Foundation Model Actually Look Like?

1. What is a foundation model?

The term "foundation model" was introduced by the Stanford Center for Research on Foundation Models in 2021 [1]. It refers to a large model trained on broad data via self-supervised learning, designed to be adapted to a wide range of downstream tasks rather than solving any single one out of the box. Three properties define the category. First, broad pretraining: the model is trained on massive, diverse, mostly unlabelled data using self-supervised objectives such as next-token prediction, masked reconstruction, or contrastive learning. Second, scale: typically hundreds of millions to hundreds of billions of parameters. Third, adaptability: the same base model is reused across many downstream applications via fine-tuning, prompting, retrieval augmentation, or other adaptation methods.

Examples span modalities. GPT-4 and Llama [2] cover text; CLIP [3] handles image-text; Whisper [4] processes speech; AlphaFold 2 [5] predicts protein structure. The unifying idea is not the architecture or the modality but the training paradigm and the intended role.

1.1 How do LLMs relate?

Large Language Models are a subset of foundation models: specifically those whose training data is text and whose objective is language modelling. Every LLM is a foundation model, but not every foundation model is an LLM. Vision-only models, protein models, audio models, and multimodal models all count. In casual usage the terms are often conflated because text-based LLMs dominate current attention; the distinction matters, however, when one starts thinking about foundation models for other kinds of structured objects, such as Chain Event Graphs.

2. Building a foundation model from scratch

Before asking what a CEG foundation model would be, it helps to understand what building any foundation model actually entails. The pipeline is essentially sequential but iterative: data collection feeds into cleaning and filtering, which feeds into tokenisation and encoding; this in turn supports architecture and infrastructure setup, which enables pretraining, followed by post-training (supervised fine-tuning, RLHF, safety alignment), and finally evaluation and deployment. Each stage feeds back into earlier ones during iteration, and the loop rarely runs cleanly the first time.

The dominant cost is compute for pretraining, but the dominant risk is data quality and pipeline engineering. Frontier LLMs are trained on trillions of tokens with dense Transformer or Mixture-of-Experts architectures [6], typically decoder-only, using AdamW-style optimisation over thousands of accelerators for weeks. Post-training, comprising supervised fine-tuning followed by reinforcement learning from human or AI feedback [7], turns a raw next-token predictor into something usable. For all but a handful of well-funded organisations, true from-scratch pretraining is prohibitive; the practical alternative is to adapt an existing open base model (Llama, Mistral, Qwen) via fine-tuning.

3. Chain Event Graphs: a brief recap

A Chain Event Graph (CEG) is a graphical statistical model developed primarily by Smith and colleagues at Warwick [8], [9]. It generalises Bayesian Networks by natively representing context-specific and asymmetric conditional independence. The construction proceeds in four stages: one starts with an event tree describing every possible unfolding of a process; identifies positions (vertices whose downstream subtrees are stochastically equivalent); groups positions with equivalent transition probabilities into stages; and finally collapses the tree into a coloured directed graph where root-to-sink paths correspond to atomic event histories.

CEGs are used in forensic reasoning, medical decision-making, public health, and criminology: domains where the process is inherently sequential and where Bayesian networks force uncomfortable symmetry assumptions [10].

4. What could a "foundation model for CEGs" mean?

The phrase does not map cleanly onto the standard foundation-model concept, because a CEG is a structured statistical object rather than a raw data modality. At least four interpretations are worth distinguishing.

4.1 Interpretation A: a model that infers CEGs from event data

Input: structured event-history data (sequential categorical observations, possibly with covariates). Output: a CEG comprising the event-tree topology, staging, and conditional probabilities. This is the most direct analogue of AlphaFold: an amortised inference model that learns to produce a specific structured object. It would replace or accelerate existing CEG structure-learning algorithms based on MCMC, agglomerative clustering, or dynamic programming [11].

4.2 Interpretation B: a model whose internal representation is a CEG

Here CEG structure serves as an architectural inductive bias for sequential reasoning, rather than being the output. This is a research programme, not a product; attractive for interpretability but with no obvious training recipe at present.

4.3 Interpretation C: a general event-sequence foundation model with a CEG interface

Pretrain a large model on diverse event-sequence data (electronic health records, judicial pathways, industrial logs, customer journeys). One of its adaptation modes is producing CEG views of the learned dynamics. The CEG becomes a legible output surface rather than the whole model.

4.4 Interpretation D: an LLM fine-tuned on CEG literature and tooling

Take an existing LLM and fine-tune it on CEG papers, textbooks, and code (e.g. the ceg R package [12]). This produces a specialist assistant, not a foundation model in the strict sense.

5. Narrowing down: the two useful interpretations

Interpretations B and D are, respectively, too speculative and too shallow to be the interesting framing. The real candidates are A and C. Interpretation A is the cleanest scientific target: an amortised structure-learning model. It has a serious bottleneck, however: (event-data, CEG) pairs barely exist as a curated resource. Interpretation C is broader and more feasible on the data side, because event-sequence data is genuinely abundant even if labelled CEGs are not.

Neither, however, addresses what may be the most practically valuable version of the problem: producing CEGs from natural-language descriptions of processes such as case reports, clinical narratives, judgments, and incident writeups. This is where the hybrid framing enters. Notably, there is already exploratory work in this direction: Yu and Smith [13] demonstrate hierarchical causal analysis of natural-language documents on a CEG, providing an initial proof of concept.

6. The hybrid: a text-to-CEG foundation model

The proposal is to take an existing LLM as the base foundation model, and adapt it to output CEGs from natural-language input describing an event process. This inherits the language-understanding capability of a modern LLM and repurposes it to produce a specific structured object.

6.1 Why the hybrid framing is attractive

Three reasons stand out. First, the input modality is well-served: text is exactly what LLMs already handle well. Second, the data problem is more tractable, because descriptions of processes are abundant even where labelled CEGs are not, and synthetic (text, CEG) pairs can be manufactured by generating CEGs and then prompting an LLM to write plausible narratives that would produce them. Third, existing infrastructure applies: structured-output fine-tuning, constrained decoding [32], and self-consistency methods developed for code and knowledge-graph generation [14] transfer directly.

6.2 Where the difficulty is real

Three genuine difficulties temper the optimism. The first is data scarcity for (text, CEG) pairs: perhaps a few hundred examples exist in published form, so bootstrapping via synthetic generation is necessary, with the usual caveats about distribution shift to real text. The second is ambiguity of ground truth: two experts reading the same case description will typically draw different CEGs, so evaluation must accept distributions over CEGs rather than single answers. The third is that staging requires statistical reasoning that text underspecifies. A narrative tells you what could happen; the staging is a conditional-independence claim usually justified by data. Realistically the model produces the event tree confidently and proposes stagings as hypotheses.

6.3 Is this really a "foundation model"?

Strictly, no; it is a fine-tuned specialist built on a foundation model. But the loose usage matches how the field talks, and if the model is trained such that it can be adapted to multiple downstream CEG-related tasks (extraction, scoring, comparison, editing, explanation), it earns the label in spirit.

7. Computational and infrastructure estimates

Any cost estimate for a project that does not yet exist is speculative, and the numbers below should be read as such. What we can do, however, is anchor each estimate to something measurable: published fine-tuning benchmarks, cited cloud-pricing snapshots, canonical papers on the training method we plan to use, and analogous structured-output projects with documented costs. Because the same project can be executed in very different ways depending on funding structure, this section presents two contrasting scenarios: a commercial setup (well-funded startup, industrial research lab, or venture-backed team) and a research setup (the actual situation for the present project, in which the researcher and PI are already funded and Eddie provides substantial free compute). The scenarios differ not only in cost but in what they can plausibly deliver, and it is important to be honest about the trade-off.

7.1 Method: fine-tune, do not pretrain

The cost regime of the project is set by a single decision: adapt an existing open-weight base model rather than pretrain from scratch. Chinchilla-optimal training of a 7B model requires on the order of 140 billion tokens [15], and Kaplan-style pretraining scaling laws [16] make clear that the compute is dominated by the pretraining step itself. Frontier-scale pretraining is out of scope for any university lab and only just within reach for the best-funded commercial teams. Post-training adaptation, however, is now a mature and inexpensive craft; we lean specifically on Low-Rank Adaptation (LoRA) [17] and its 4-bit quantised variant QLoRA [18]. Dettmers et al. [18] show that a 65B parameter model can be fully fine-tuned on a single 48GB GPU in 24 hours, and that QLoRA preserves 16-bit full-fine-tuning task performance; at the 7B–13B scale we propose, the resource envelope shrinks accordingly. Both scenarios below adopt this method.

7.2 Model and data

The base model will be a 7B to 13B parameter open LLM (Llama-3, Qwen, or Mistral); this size is the empirical sweet spot for LoRA fine-tuning on a single high-end GPU [17], [18]. Training data will consist of 50,000 to 500,000 (text, CEG) pairs. Given the near-total absence of naturally occurring (text, CEG) pairs, most of these will be synthetic, following the Self-Instruct paradigm [19] in which a strong teacher LLM generates training examples that a smaller student is then fine-tuned on. Scale AI's cost-effectiveness analysis of synthetic-data generation strategies [20] provides useful priors: for structured-output tasks, they find that the quality of the teacher model and the choice of augmentation strategy matter more than raw volume beyond a threshold of a few tens of thousands of examples, so our range is defensible without escalating further.

The evaluation set will comprise 50 to 200 gold-standard (text, CEG) pairs annotated by multiple domain experts. Following the model of the CUAD legal contract-review dataset [21], multi-expert annotation is essential because it allows us to quantify the inter-rater ceiling (the upper bound on model performance, beyond which further "improvement" cannot be distinguished from noise in the ground truth). The commercial and research scenarios diverge sharply on how this annotation is procured, discussed below.

7.3 Fine-tuning compute

Published fine-tuning benchmarks converge on remarkably consistent numbers at the 7B scale. A QLoRA run of 2 to 3 epochs on a 7B model with 10,000 to 15,000 examples takes 3 to 5 hours on a single A100 80GB [22], and a 65B QLoRA run completes in 24 hours on a single 48GB GPU [18]. For our POC (see Section 8), which uses 5,000 to 20,000 examples, a single fine-tune therefore consumes a few hundred A100-hours. For the full project, iteration across 20 to 50 experiments (varying prompt structure, CEG serialisation, LoRA rank, and data composition) brings the total to 1,000 to 20,000 A100-hours across all runs, dominated by iteration rather than any single training run. These figures cover supervised fine-tuning only; any RLHF or preference-based refinement is on top, and its contingency budget is discussed separately in Section 7.10.

The two scenarios differ substantially in the hardware assumed. A commercial team would run the entire pipeline on H100 or H200 GPUs from the outset, both to compress the timeline and to leave headroom for larger base models if the initial results warrant scaling up. H100 is roughly 3× the throughput of A100 for typical transformer fine-tuning workloads, and the 3× cost premium is more than repaid by the speed advantage in a commercial iteration cycle where researcher time is expensive. A research team using Eddie has access to A100 hardware only; H100 access requires either a national-scale allocation (Cirrus, EIDF, JADE 2, Baskerville, or a Tier-1 grant on ARCHER2's GPU testbed) or a modest commercial-cloud budget for specific experiments [23], [24]. This is a real limitation, not just a cost difference, because it caps the base model size the project can practically explore.

7.4 Scenario A: commercial setup

Assume a small commercial team of three to five people (technical lead, two ML engineers, a data engineer, and a part-time domain-expert consultant), running on hyperscaler or specialist-cloud H100 infrastructure, with commercial-rate annotation and a target of a production-ready system in 12 to 18 months. Costs decompose approximately as follows.

Compute is a real but bounded cost. Fine-tuning across 20 to 50 experiments at H100 rates of $3 to $6 per GPU-hour on hyperscalers [25], [26], or $2 to $3 on specialist clouds [23], gives a fine-tuning compute budget of roughly $30,000 to $120,000 for the full project including buffer for larger-model experiments and RLHF-style refinement. These rates have fallen substantially through 2025 following price cuts from major hyperscalers [27], so the estimate is likely to look conservative rather than optimistic by the time this project runs. Teacher-LLM API access for synthetic-data generation of 500,000 pairs at $0.02 to $0.05 per generation adds $10,000 to $25,000, potentially more if higher-tier models are used throughout. Total compute and API costs sit at roughly $50,000 to $150,000.

Staffing dominates the budget. Fully loaded costs for commercial ML engineers in the US or Western Europe run at $250,000 to $400,000 per person-year including salary, benefits, and overheads; a small team of four for 15 months therefore costs $1.25 million to $2 million. Expert annotation of a 200-pair evaluation set at commercial specialist rates of $150 to $500 per hour [21], [28], with three annotators per pair at 30 minutes per annotation, costs $45,000 to $150,000 (dwarfed by staffing but a real line item). Add another $50,000 to $150,000 for office space, software licensing, legal and compliance work, and general operating overhead.

A commercial full-project budget therefore sits at roughly $1.5 million to $2.5 million (approximately £1.2M to £2M) over 12 to 18 months. What this buys is a production-ready system: robust codebase, hardened inference pipeline, formal benchmark release, dedicated engineering support, and a timeline compressed enough to matter commercially. It does not, in itself, buy scientific credibility with the CEG methodology community; that requires the peer-reviewed evaluation and expert engagement that academic projects specialise in.

7.5 Scenario B: research setup (our actual case)

Our situation is unusually favourable in the categories that dominate the commercial budget. The researcher (a PhD student with prior senior-engineering experience) is already funded through an existing scholarship. PI time is committed in kind through the ordinary academic supervision relationship. Eddie provides substantial GPU compute free at point of use to accepted allocations. Domain-expert annotation is handled through the CEG methodology community and the PI's academic network at academic-honorarium rates rather than commercial specialist rates. This eliminates the salary, cloud, and annotation costs that make up the bulk of the commercial estimate.

What remains is a small and specific set of line items. Teacher-LLM API access for synthetic-data generation is the largest, at roughly £3,000 to £15,000 depending on whether frontier commercial teachers are used throughout (upper end) or open-weight teachers running on Eddie are used for bulk generation with a smaller commercial-teacher seed set (lower end). Expert annotation honoraria for external annotators recruited outside the co-investigator network sit at £3,000 to £8,000 at academic rates of £20 to £50 per hour. Conference travel and publication fees add roughly £4,000 to £8,000 across the project. A modest commercial-cloud contingency for H100-class experiments that Eddie cannot supply, or for specific overflow when Eddie queues become blocking, is prudent at £2,000 to £10,000. Aggregating these gives an incremental cash requirement of approximately £12,000 to £40,000 across 30 months, well within reach of a small internal grant, a school pump-priming award, an industry gift, or a modest responsive-mode application.

Compute deserves a more careful treatment than "Eddie is free". Eddie has 28 A100 GPUs shared across the entire University research community [29], and while the researcher's default school allocation is generous relative to a hobby project, it is far from unlimited for a production ML workload. Actual priority varies by school and by the group's existing usage, so the practical envelope should be confirmed with the ECDF service desk before the scale-up phase begins. In practice, the POC phase (Section 8) runs comfortably on Eddie. The scale-up phase runs on Eddie with growing tolerance for queue delays. The final phase, particularly if it involves 30B+ base models, larger training-data sweeps, or any RLHF component, is likely to hit the practical limits of Eddie's shared allocation and require either a national-scale HPC grant (Cirrus additional time, EIDF allocation, or a Baskerville / JADE 2 access application) or a modest commercial-cloud spend. This is why the £2,000 to £10,000 cloud contingency line item exists: it is a placeholder for the transition point at which Eddie stops being sufficient, and its size reflects a realistic scenario in which perhaps 5% to 15% of total compute happens off-Eddie.

7.6 What the research setup cannot deliver

Being honest about the trade-off matters. The research setup delivers a scientifically credible artefact: peer-reviewed evaluation, expert-endorsed benchmark construction, and integration with the CEG methodology community that a commercial team could not easily replicate. What it does not deliver is what commercial money buys. There are four specific things the research setup will not produce within the £12,000 to £40,000 envelope.

First, there will be no production-hardened system. A commercial team ships a documented API, a maintained inference stack, a monitoring and observability layer, and a support contract; a research team ships a research prototype, a paper, and a public code release with best-effort documentation. Second, there will be no formal industrial partnerships or third-party integrations built in during the project; those require dedicated business-development and engineering effort that a research team does not have and would not want. Third, the timeline is longer, meaningfully so: a research setup taking 30 months delivers what a commercial setup delivers in 15 months, which matters if there is any competitive pressure. Fourth, and most importantly, the model size and RLHF sophistication the project can practically explore are capped by compute availability. A commercial team can pivot to a 70B base model with reward-model refinement mid-project if the initial results warrant it; a research team hitting Eddie's limits at that decision point has to either apply for further HPC time (adding six months to the schedule) or accept the smaller-model result and publish it. The academic project is scientifically honest but methodologically less ambitious than the commercial equivalent.

For the specific goal of this project, which is to establish scientific credibility for a text-to-CEG foundation model and to build the evaluation methodology that underpins any future work in this direction, the research setup is entirely adequate. If the goal were to ship a commercial product in this space, it would not be.

7.7 Infrastructure options at Edinburgh

Eddie (ECDF) is the workhorse for the research route. It offers over 15,000 cores and 7 GPU nodes with a total of 28 NVIDIA A100 devices [29]. This is sufficient for LoRA and full fine-tuning of 7B to 13B models, and for all inference and evaluation workloads, subject to queue contention. Cirrus (EPCC Tier-2) provides 36 GPU nodes each with 4 NVIDIA V100s, giving 144 GPUs total [30], and is useful as overflow. EIDF is a newer GPU facility offering A100 and H100 access, and applying for an EIDF allocation is the natural first move when Eddie's practical capacity is exhausted. Nationally, JADE 2 and Baskerville offer larger GPU pools accessible through responsive-mode research council applications, and ARCHER2 has a small AMD MI210 testbed [31] that is not a primary target for this project but is worth noting. Tursa is ring-fenced for DiRAC particle physics [31] and not accessible.

7.8 Funding summary

Table 1 summarises the two scenarios side by side across the cost categories that matter. The point of the table is not to argue that the research route is universally better (it is not: it is slower and more limited) but to make clear what each route buys and what it costs. For the scientific goal of this project, the research route dominates on cost-effectiveness and adequacy; for a commercial product goal, the commercial route dominates on speed and completeness.

7.9 What Eddie cannot do

The pretrain-from-scratch version (Interpretation A at frontier scale, or Interpretation C with a bespoke architecture) is not realistic on Eddie under either scenario. Following the Chinchilla-optimal training regime [15], even a 7B model trained from scratch requires computing on the order of 10²² to 10²³ FLOPs, which translates to tens of thousands of A100-days: an order of magnitude beyond Eddie's total capacity, before accounting for the fact that pretraining a single model would monopolise the cluster for months. The hybrid text-to-CEG fine-tuning approach, by contrast, sits comfortably within reach for both scenarios, with the differences described in Table 1.

7.10 Uncertainty and what would change the estimate

Three factors could shift the research-setup estimate substantially. First, if a much larger base model turns out to be necessary for CEG output quality (say, 70B parameters), the compute at that scale exceeds Eddie's comfortable single-user allocation and the project needs a serious national HPC grant or £10,000 to £20,000 in additional commercial cloud spend. Second, if synthetic (text, CEG) pairs generated by a teacher LLM prove insufficiently diverse or realistic (a common failure mode of self-instruct pipelines when the target distribution differs materially from anything the teacher has seen [20]), the fallback is heavier reliance on expert-generated pairs, pushing annotation honoraria upward by £5,000 to £15,000. Third, if RLHF or preference-based refinement is needed on top of supervised fine-tuning (which we exclude from the POC but is plausibly required in the full project [7]), reward-model training and rollout collection add another 3,000 to 15,000 A100-hours (Eddie may or may not sustain this) plus a further round of preference annotation of £3,000 to £8,000.

None of these would derail the project; each would shift the research estimate within, or slightly outside, the £12k to £40k range quoted above. In the commercial setup, the corresponding contingencies are absorbed easily within the existing budget. The takeaway is that the numbers here are grounded but not sharp: they are useful for scoping the project and for planning the annotation and HPC-application programme, not for line-item budgeting until the POC has actually been run.

8. A proof of concept before committing to the full project

Section 7 makes clear that this project sits in an unusually favourable position within the research scenario: the researcher is already funded, PI time is committed in kind, and Eddie provides free-at-point-of-use compute for the early phases of work. The incremental cash requirement for the full project of £12,000 to £40,000 is small enough that it does not require a major grant competition to secure, and can plausibly be supported through small internal awards, industry gifts, or a modest responsive-mode application. What still needs to be de-risked, however, is the scientific bet itself: does a fine-tuned LLM actually produce useful CEGs from natural-language descriptions? This is what the POC is for. A negative POC redirects the PhD before its middle years are committed to an unworkable idea. A positive POC provides the empirical foundation for the full project and, should the group later choose to pursue commercial or larger academic scaling, becomes the central evidence in any subsequent funding application.

8.1 What the POC needs to demonstrate

Three questions must be answered, in decreasing order of importance. First: can a fine-tuned LLM produce syntactically valid CEGs from natural-language descriptions? This is the lowest bar and almost certainly achievable; if it fails, the direction is unworkable. Second: are the produced CEGs semantically reasonable, i.e., do domain experts recognise them as plausible representations of the described process? This is the real scientific question. Third: does the model generalise across domains, at least modestly? A model that only works on the domain it was trained on is not evidence of a foundation-model-like capability; it is a specialist tool. Some cross-domain transfer, even weak, is the signal that the broader project is worth pursuing.

8.2 POC scope

The point of the POC is to answer the three questions above cheaply, not to build a production system. Concretely, the base model should be a single 7B open LLM (Llama-3-8B, Qwen-2.5-7B, or Mistral-7B) with no comparison across base models; pick one and move on. The domain focus should be one primary domain plus one secondary domain for cross-domain testing: healthcare pathways and forensic case reasoning are natural candidates because published CEG examples exist in both. Training data should consist of 5,000 to 20,000 synthetic (text, CEG) pairs, plus 100 to 300 curated examples from the CEG literature; the synthetic pipeline is built once and reused for the full project. The CEG serialisation should be a single JSON schema for representing positions, stages, edges, and probabilities, with constrained decoding [32] enforcing syntactic validity. The fine-tuning method should be LoRA or QLoRA rather than full fine-tuning: faster, cheaper, and sufficient to answer the POC questions. The evaluation set should comprise 30 to 50 (text, CEG) pairs annotated by two or three CEG experts spanning both domains: small enough to be feasible, large enough to detect meaningful signal.

8.3 POC resourcing

The POC should take 6 to 9 months of active work, roughly aligned with the first phase of the timeline shown in Figure 4. Because the researcher and PI time are already funded, and because the compute footprint at this scale is comfortably within Eddie's normal-priority allocation, the incremental cash requirement is genuinely small. Teacher-LLM API access for synthetic-data generation at the 5,000 to 20,000 pair scale costs approximately £500 to £2,000, using a mix of a frontier commercial teacher for the initial high-quality seed set and cheaper open-weight teachers running on Eddie for bulk generation. Expert-annotation honoraria for the 30 to 50 held-out pairs, at academic rates of £20 to £50 per hour across two or three annotators, total approximately £1,000 to £3,000. Modest allowance for compute overflow and unforeseen expenses adds another £500 to £1,000.

Total incremental cash requirement for the POC is therefore approximately £2,000 to £6,000, comfortably within a small internal grant, a school pump-priming award, or a discretionary PI budget. For comparison, an equivalent POC executed in the commercial setup of Section 7.4 (a two-engineer team for four months on commercial cloud infrastructure with paid specialist annotation) would cost approximately $250,000 to $400,000, which reinforces the same conclusion Table 1 draws for the full project: the research route is dramatically cheaper for the same scientific artefact, at the cost of pace and production polish.

Compute for the POC runs entirely on Eddie. LoRA fine-tuning of a 7B model with 20,000 examples runs in a few hundred A100-hours, and iteration across 15 to 25 experiments brings the total to roughly 3,000 to 8,000 A100-hours across the POC phase. This is well within Eddie's default allocation for a research group with reasonable priority access, and no external HPC application is required at this stage. The question of whether Eddie will remain sufficient through the full project is deferred to Section 7.5 and Figure 4; for the POC specifically, the answer is unambiguously yes.

8.4 What the POC deliberately does not attempt

To avoid scope creep, the POC explicitly excludes frontier-scale base models (no 30B or 70B fine-tunes; those are for the full project and would require HPC beyond Eddie), RLHF or RL-against-scoring-function training (supervised fine-tuning only), broad multi-domain coverage (two domains, not ten), a production API or user interface (command-line evaluation is sufficient), and publication-quality benchmark release (the expert-annotated set is internal until the full project matures the methodology). Each of these is important for the full project but including them in the POC turns a six-month exercise into a two-year one and defeats the purpose.

8.5 Decision criteria at the end of the POC

The POC should produce a clear go/no-go signal, and it is essential that the thresholds are set before the results come in rather than tuned to match them afterwards. A "go" signal would consist of at least 90% syntactic validity of outputs, at least 60% of held-out examples rated "acceptable or better" by at least two experts, and measurable (even if modest) cross-domain transfer: secondary-domain performance no worse than 60% of primary-domain performance. A "reconsider" signal is high syntactic validity but low semantic quality, which indicates the model is producing plausible-looking but wrong CEGs, a well-known failure mode of structured LLM output. A "no-go" signal is persistent failure to produce valid CEGs even after prompt and data iteration, or expert agreement that outputs are systematically worse than what an untrained analyst would produce manually. These thresholds are guidelines to be refined once the evaluation methodology is in place, not fixed contract terms.

8.6 From POC to full project

If the POC signals go, the transition to the full project is unusually smooth because the same researcher, PI, base infrastructure, and funding structure carry through. The synthetic-data generator scales from thousands to hundreds of thousands of examples with no new engineering. The expert-annotated evaluation set expands from 30–50 pairs into the full 200-pair benchmark and becomes a headline scientific contribution in its own right. The prompt structure, CEG serialisation, and fine-tuning recipe transfer directly.

The additional cash requirement between the POC and the full project (approximately £10,000 to £34,000 for incremental teacher-LLM access, expanded annotation, conference travel, publication fees, and the growing commercial-cloud contingency as Eddie's practical capacity is exhausted) is small enough that it does not require a major grant application to bridge. What the transition does require is a national-scale HPC allocation as the project moves into its final phase: an EIDF application should be timed to submit around the go decision, so that the allocation is in place by the time Eddie's shared capacity starts constraining the timeline. Failure to secure that allocation is one of the concrete risks flagged in Section 7.5 and is worth planning against explicitly rather than assuming it away.

Should the group choose to pursue a formal funding programme after the POC succeeds (an EPSRC Standard Grant, a Turing Institute engagement, an industry partnership, or a route toward the commercial setup of Section 7.4 with substantial external investment) the POC results provide the empirical grounding to make that case strongly. But this is optional rather than necessary, and this optionality is itself part of what makes the research route attractive for this particular project: the same POC that validates the scientific bet also unlocks a range of subsequent funding trajectories, from the very lean continuation described here to the commercial scaling described in Table 1, depending on how the group chooses to proceed.

9. The unsolved bits

The engineering is mostly known: LLM fine-tuning is a mature craft, and Section 7's cost estimates assume a well-charted method being applied to a new target. The scientific bets, however, are real. Several open problems sit between the plan as described and the artefact it aims to produce, and it is worth naming them explicitly rather than folding them into vague "future work" language.

LLMs do not natively reason about graphs. This is the most fundamental problem and the one most likely to determine whether the whole approach works. When an LLM produces a CEG, it is not executing graph algorithms or reasoning over adjacency structure; it is generating tokens whose surface form happens to describe a graph, driven by statistical patterns in its training data. The distinction matters because CEGs encode strict structural constraints (a directed acyclic topology, positions defined by stochastic equivalence of downstream subtrees, staging that induces conditional-independence claims), and these are exactly the sort of constraints that a token-prediction process is poor at enforcing. Wang et al.'s NLGraph benchmark [33] found that state-of-the-art LLMs show only preliminary graph-reasoning abilities, that the benefit of prompting strategies diminishes as complexity grows, and that models are surprisingly brittle in the face of spurious correlations in graph problems, all of which speak directly to what we are asking a fine-tuned LLM to do here. Constrained decoding [32] helps with syntactic validity (no cycles in a DAG, no malformed edges) but not with semantic correctness (whether the DAG actually represents the process the text describes). Whether a fine-tuned LLM at the 7B–13B scale can produce structurally-sound CEGs at all, or whether we will need to hybridise with a symbolic post-processor that repairs violations, is genuinely open. This is not a gap that scaling data or compute is guaranteed to close, and the POC's syntactic-validity threshold (Section 8.5) is specifically designed to test it early.

Evaluation. Even if the model produces valid graphs, telling whether they are good graphs is a separate problem. Without a benchmark where domain experts agree on target CEGs for a set of case descriptions, no model can be shown to work. Building this benchmark, and quantifying the inter-rater ceiling among human experts, should precede large-scale training runs. It is also, conveniently, one of the deliverables the POC produces as a natural byproduct of Section 8. The harder question underneath is what agreement metric to use when comparing CEGs: exact structural equality is too strict, tree-edit distance ignores staging, and log-likelihood on held-out event data assumes a downstream inference task that not every use case has.

Staging and distributional output. Section 6.2 flagged that text underspecifies staging: a narrative tells you what could happen but not which situations are stochastically equivalent, which is a conditional-independence claim usually justified by data. The realistic response is that the model should produce distributions over CEGs, or ranked candidate stagings, rather than a single deterministic answer. But there is no established fine-tuning recipe for producing structured distributions of this kind, and no evaluation methodology for scoring them. Both are open. A pragmatic first step is to have the model produce the event tree confidently and propose stagings as hypotheses, but that is a workaround, not a solution.

Serialisation. There is no standard machine-readable format for CEGs. Section 8.2 assumes a JSON schema for representing positions, stages, edges, and probabilities, but this is a working choice, not an established convention. The ceg R package [12] uses an internal representation optimised for the scoring routines it implements, which is not designed for LLM output. A useful side-contribution of this project may simply be a serialisation standard that the CEG methodology community can adopt regardless of whether the specific text-to-CEG model succeeds; without it, comparing outputs across systems is difficult and reproducibility is weaker than it should be.

Synthetic-data validity and the teacher-model problem. The plan in Section 7 leans heavily on synthetic (text, CEG) pairs generated by prompting a teacher LLM, and the cost estimates in Table 1 assume that open-weight teachers running on Eddie can carry much of that generation load with a smaller commercial-teacher seed set. Practical experience with structured-output tasks suggests this assumption is fragile. Producing (text, CEG) pairs that are genuinely diverse, semantically coherent, and non-degenerate seems to require near-frontier teacher models; open-weight teachers of the 7B–13B class tend to produce narratives that are superficially plausible but structurally repetitive, with a narrow effective distribution over event trees and staging patterns. If this holds for CEGs specifically, the cost model shifts: the frontier-teacher API budget grows from the lower end of the £3k–£15k range towards the upper end, and possibly beyond. There is also no established methodology for validating synthetic (text, CEG) pairs before they enter training. Whether a generated CEG is "reasonable enough" is a judgement usually made by a domain expert, and doing that at the scale of 50,000 to 500,000 pairs is not feasible. Cheap proxy validators (LLM-as-judge, structural sanity checks, downstream inference performance on real data) are all promising but none is established for this problem. Getting synthetic-data validation right is arguably a precondition for the project to work at all, and it is one of the specific outputs the POC should try to produce even at small scale.

Cross-domain transfer. Section 8.1 makes transfer across domains one of the three decision criteria for the POC, but whether it actually happens is genuinely an open scientific question rather than a matter of implementation. The problem is entangled with synthetic-data validity above: if the training pairs are generated by a teacher LLM, the fine-tuned model may learn the teacher's implicit distribution over process descriptions rather than anything about actual case reports or clinical narratives. Whether the model then transfers to a genuinely novel domain, or merely to a domain the teacher has already generalised over, is not something the POC can fully disambiguate. This is a limitation worth being upfront about, and one that only large-scale real-data evaluation in the full project can properly address.

None of these are reasons not to proceed. They are reasons to be honest about what the POC and the full project can and cannot conclude, and to build the evaluation and serialisation deliverables in a way that outlasts the specific model being trained.

References