Research, product and leadership

In plain words

Good AI research starts with a clinical question, not with a dataset or a technique. The question should say which patients, what the model will predict or detect, at what moment in care, and what decision it would change. If nobody would act differently with the answer, the model is not worth building.

Healthcare example (illustrative)

Weak: "Can machine learning predict complications after surgery?" Stronger: "In adults having elective abdominal surgery, can data available at the end of the operation predict which patients will need unplanned critical care admission within the next day or two, well enough to change where they recover?" The second version fixes the population, the prediction time, the outcome and the decision.

A little more technical

A useful check is to write down the intended use before touching any data: the target population, the predictors available at the prediction time, the outcome and how it is defined, and the action that follows. This protects against leakage (using information that would not yet be known) and against models that are accurate but useless. It also sets up the later reporting, because reporting guidelines ask for exactly these details.

Don't learn this yet

Skip model choice and coding at this stage. A sharp question, agreed with the clinicians who would use the answer, matters more than any technique.

Go deeper (optional)

• Health Research Fundamentals (noc26_hs82) #153
• Artificial Intelligence for Improved Patient Outcomes (book) #82

In plain words

The dataset decides what your model can learn. You need data that represents the patients you care about, recorded at the time the prediction would be made, with an outcome defined consistently. You also need permission: ethics approval and data-governance rules apply before you start.

Healthcare example (illustrative)

Imagine a research team wants to predict hypotension after induction of anaesthesia. Their records come from one teaching hospital, mostly younger elective patients, and blood pressure was recorded more often in sicker patients. A model trained here may learn the hospital's recording habits rather than the physiology, and may not work in an older, emergency population elsewhere.

A little more technical

Check the data before modelling: missing values and why they are missing, how the outcome was labelled and by whom, changes in practice or coding over time, and whether the same patient appears more than once. Split data for testing in a way that mirrors real use, for example by time or by site, not at random rows. Track 2 covers data quality, leakage and shift, and common data models; multi-site network studies (#137) are one route to broader data.

Don't learn this yet

Leave building data pipelines and learning a common data model in depth until your project needs it. First learn to describe your dataset honestly: who is in it, who is missing, and how the outcome was defined.

Go deeper (optional)

• OHDSI Education / network studies (see T2) #137
• OHDSI Education hub (incl. EHDEN Academy pointer) #108

In plain words

The model is the method that turns inputs into a prediction. In clinical research, a simple, well-understood model can perform as well as a complex one, and easier to explain and check. Choose the simplest approach that answers the question, and compare anything complex against it.

Healthcare example (illustrative)

Imagine a team predicting sepsis in the emergency department. They first fit a regression model on a handful of routine observations and blood results, then try a more complex machine-learning model. If the complex model is only slightly better and much harder to explain, the simpler one may be the better clinical choice.

A little more technical

Good practice includes: fixing the analysis plan before looking at test results; handling missing data deliberately; tuning the model only on training data; and reporting both discrimination (how well it ranks patients) and calibration (whether predicted risks match observed risks). Keep a record of every choice, because reporting guidelines and reviewers will ask. Track 5 covers the technical building if you plan to do it yourself.

Don't learn this yet

Skip chasing the newest model architecture. For most clinical prediction questions, rigour in data, validation and reporting matters more than the model type.

Go deeper (optional)

• AI in Health Care: From Strategies to Implementation #75
• Artificial Intelligence for Improved Patient Outcomes (book) #82

In plain words

Here, validation means testing scientifically whether the model's predictions hold up on patients it has not seen. The strength of evidence rises from internal validation (same data source), to external (a different hospital or population), to temporal (a later period), to prospective (used as care happens). A model that only has internal validation has not yet shown it works elsewhere.

Healthcare example (illustrative)

Imagine a deterioration model built on ward patients at one hospital performs well on held-out data from the same hospital. When a second hospital tests it, it flags far too many patients as high risk, because their patients and recording practices differ. Recalibration at the new site, and a prospective check, would be needed before anyone relied on it.

A little more technical

Report discrimination (for example ROC and PR-AUC), calibration and, ideally, decision curves that show whether the model would help decisions. Look at performance in subgroups, not just overall, to find bias. #135 argues that models should be validated locally and repeatedly, because performance can drift over time. Trials of AI interventions have their own reporting guidelines (for example CONSORT-AI, #25, covered in Track 3).

Don't learn this yet

Leave the advanced statistics of model updating until you need them. Know the validation hierarchy and why external performance is usually worse.

Go deeper (optional)

• All models are local (see T3) #135
• Health Research Fundamentals (noc26_hs82) #153
• Artificial Intelligence for Improved Patient Outcomes (book) #82

In plain words

When you write up a prediction-model study, a reporting guideline tells you what to include so readers can judge and reproduce your work. TRIPOD+AI (#22, covered in Track 3) is a 27-item checklist for prediction models built with regression or machine learning, and it supersedes TRIPOD 2015. Using it from the start, not just at submission, makes the study better as well as the paper.

Healthcare example (illustrative)

Imagine a critical care registrar writing up a model that predicts failed extubation. Before drafting, they go through the TRIPOD+AI checklist and realise they never recorded exactly how the outcome was defined, or how missing data were handled. Fixing those gaps before writing is much easier than after a reviewer asks.

A little more technical

Work through the checklist alongside your protocol, and keep a table mapping each item to the section of your manuscript where it is answered. Other guidelines cover other study types: TRIPOD-LLM (#27) is for studies of large language models and is separate in scope from TRIPOD+AI, and CONSORT-AI (#25) is for trials of AI interventions. PROBAST+AI (#23) assesses the quality, risk of bias and applicability of regression and AI prediction models, so reading your own study through it shows how others may appraise it.

Don't learn this yet

Don't memorise the checklist items. Open it when you plan a study and again when you write it up.

In plain words

Discovery is finding a real problem worth solving before building anything. It means talking to the people who would use or pay for the product, watching the work as it actually happens, and checking whether the problem is common, painful and not already solved. A product that solves a problem nobody prioritises is unlikely to be used, however good the technology.

Healthcare example (illustrative)

Imagine a radiologist who thinks an AI tool to measure lung nodules would sell. After interviewing colleagues, managers and IT staff, she finds the real bottleneck is chasing follow-up scans that never get booked. The better product idea turns out to be a follow-up tracking tool, where AI plays only a small part.

A little more technical

Useful discovery questions: who has the problem, how often, what it costs them in time, safety or money, what they do now, who decides on buying, and what would make them switch. Separate the user (often a clinician) from the buyer (often a manager or committee). #136 maps what regulated AI actually does, which helps spot crowded areas and gaps.

Don't learn this yet

Skip pitch decks, fundraising and company formation. Come back to them once you have evidence that the problem is real.

Go deeper (optional)

• How AI is used in FDA-authorized medical devices: a taxonomy across 1,016 authorizations #205
• Emerging Tech Venture Creation in Health Care: AI and Beyond #76

In plain words

An MVP (minimum viable product) is the smallest version of your product that lets you test your main assumption with real users. It is deliberately limited: enough to learn from, not a finished system. In healthcare it must still be safe, which usually means it supports rather than replaces clinical decisions, and uses no real patient data until approvals are in place.

Healthcare example (illustrative)

Imagine a founding team wants to reduce time spent on pre-operative assessment letters. Instead of building a full product, their MVP drafts letters from de-identified sample cases, and a few anaesthetists rate whether the drafts would save them time. If the drafts are not useful, the team has learned that cheaply.

A little more technical

Decide in advance what the MVP must prove and how you will measure it: time saved, error rates, user ratings or willingness to adopt. Keep a human reviewing every output. Check early whether the intended use would make the product a regulated medical device, because that changes what you can test and how (Track 4).

Don't learn this yet

Leave scaling, integration with record systems and sales until the MVP has shown something worth scaling.

Go deeper (optional)

• AI in Health Care: From Strategies to Implementation #75
• Emerging Tech Venture Creation in Health Care: AI and Beyond #76

In plain words

For a product, validation means showing two things: that it solves the real problem you found in discovery, and that it works in practice in the places it will be used. This is different from 7a, where validation means scientifically testing a model's predictions. A product can have an accurate model and still fail if it does not fit the workflow or change outcomes that matter.

Healthcare example (illustrative)

Imagine a triage tool for general practice messages that sorts them accurately in testing. In a pilot at a few practices, staff ignore it because it adds a step to their routine, and urgent messages are still handled the old way. Product validation would record this, and the team would redesign before claiming the tool works.

A little more technical

Measure accuracy at each site, but also usability, time taken, safety events, adoption over time and whether the intended benefit actually appears. Pilots should include sites unlike the ones you built at. #135 argues for repeated local validation, and Track 3 covers the evaluation methods.

Don't learn this yet

Skip designing large trials yourself at this stage. A well-run pilot with honest measures teaches more than a big claim built on weak evidence.

Go deeper (optional)

• All models are local (see T3) #135
• AI in Healthcare (Postgraduate Award) #80

In plain words

Procurement is how a hospital or health system decides to buy something. For AI, buyers usually want to know what the product does, what evidence supports it, its regulatory status, how patient data are protected, how it connects to existing systems, and who is responsible when it goes wrong. Procurement rules and processes vary a great deal by country and by organisation.

Healthcare example (illustrative)

Imagine an intensive care unit that wants a documentation tool. The clinical lead is keen, but the purchase also needs review by IT, information governance, clinical safety and finance. The supplier who has prepared clear answers on evidence, data protection and integration moves forward; the one with only a demonstration stalls.

A little more technical

Expect several reviewers with different concerns: clinical benefit and safety, data protection, cybersecurity, integration, support and ongoing cost. Buyers may also ask about local validation, monitoring after launch and a plan for switching the tool off. Check the rules for your own country and organisation, and see Track 4 for regulation and hospital governance.

Don't learn this yet

Skip contract law and tender procedures. Understand who is involved in a buying decision and what evidence each of them needs.

In plain words

Reimbursement is whether, and how, someone pays for the use of a product: a public health system, an insurer or another payer. It varies by country and by payer, and an AI tool may have no dedicated payment route at all. A product that is useful but unpaid for may still struggle to be adopted.

Healthcare example (illustrative)

Imagine a company with an AI tool that helps read retinal images in primary care. In one country the service may be funded as part of a screening programme; in another, no payer covers it, so practices would have to pay from their own budgets. The same tool can face very different business cases in different places.

A little more technical

Payers generally want evidence of benefit that matters to them, such as better outcomes, fewer admissions or lower costs, not just accuracy. Whether a tool is paid for as a separate item, bundled into existing payments, or bought directly by providers depends on the local system. Seek advice specific to your country and payer; this roadmap does not list codes, rules or amounts.

Don't learn this yet

Skip the details of any specific payment system until you know which market you are entering. First understand who would pay, and what evidence would convince them.

In plain words

For a clinical leader, AI is an organisational change, not just a technology purchase.

• Strategy: decide which problems AI should help with, in line with the organisation's priorities, rather than buying tools because they are available.
• ROI (return on investment): weigh the full costs (buying, integrating, training, local validation, monitoring) against realistic benefits.
• Governance committee: a group that approves, monitors and, if needed, retires AI tools.
• Workforce change: how roles, skills and workload shift when AI arrives.

Healthcare example (illustrative)

Imagine a hospital board is offered several AI tools at once: ambient note-taking, a deterioration alert and an imaging triage tool. The AI governance committee, including clinicians, nursing, IT, information governance, patient representatives and finance, asks for each: what problem it solves, what evidence exists, how it will be validated locally, who supervises it, and how success will be measured. They pilot one, review the results at a fixed point, and only then decide on the others.

A little more technical

ROI in healthcare AI is hard to measure, because benefits such as reduced burnout or safer care are real but indirect, and costs such as monitoring continue after purchase. A governance committee needs a clear process: intake, risk assessment, local validation (#135 argues this should be repeated, not one-off), approval with conditions, monitoring for drift and bias, and a route for staff to report problems. Workforce planning should cover training in supervising AI and resisting automation bias, changes to job roles, and honest communication with staff.

Don't learn this yet

Skip detailed financial modelling and building a data-science team. Start with one well-governed pilot, a clear owner, and a way to stop if it does not help.

Go deeper (optional)

• AI in Health Care: From Strategies to Implementation #75
• Multi-program journey: AI in Health Care + Leading Digital Transformation in Health Care #77
• All models are local (see T3) #135
• Artificial Intelligence (AI) for Management (noc26_cs10) #87
• Graduate Certificate in Healthcare Innovation (HCI 611–614 AI courses) #81