Milestone 2 UAT Script

Purpose

This script is for QA analysts and business analysts validating the Milestone 2 hosted build of the NCBI Word ingestion app.

The goal is to confirm that the app supports the intended Milestone 2 reviewer workflow:

• manage Template Profiles with Word base documents
• upload DOCX source files with the correct Template Profile selected
• parse structure, metadata, tables, figures/images, references, and citations
• inspect references, citations, and external validation outcomes
• review top-level Pass / Warning / Reject decisions
• confirm each document was evaluated against the intended template rules

Environment Under Test

• Environment: production UAT
• URL: http://138.197.33.233:8080
• Template Profiles page: http://138.197.33.233:8080/template-profiles
• Main upload page: http://138.197.33.233:8080/upload
• Review page: http://138.197.33.233:8080/review

Template Profiles Under Test

Before uploading UAT files, open http://138.197.33.233:8080/template-profiles and confirm these active profiles are present:

For each profile, confirm the Profile Base Document appears as a download/view link. Editing a profile should allow the reviewer/admin to replace the base document by uploading a new DOCX.

Profile-specific metadata expectation:

• NCBI Author Template Advanced should exercise NCBI-style metadata table detection when those tables are present.
• OAR Guidelines Template v4 does not require NCBI-style metadata or keywords. Missing metadata/keywords alone should not create a pre-flight issue for OAR guideline uploads.

Test Data

Use the following gold-standard OAR files from the shared Google Drive folder Gold Standard Word Templates.

Core OAR UAT files:

• 2 virologic-failure-adult-adolescent-arv.docx
• 3 antiretroviral-therapy-cost-considerations-adult-adolescent-arv.docx
• 4 management-infants-utero-intrapartum-breastfeeding-hiv-exposure-perinatal.docx
• 5 human-papillomavirus-disease-adult-adolescent-oi NLM.docx
• 6 bacterial-enteric-infections-adult-adolescent-oi NLM.docx

Special OAR baseline file:

• 1 Updated Clinical Guidelines Template_v4_for NLM.docx

NCBI Author Template Advanced UAT files:

• Acarbose.docx
• celecoxib.docx

Expected Current Outcomes

These are the expected current high-level outcomes for Milestone 2 scope:

• File 1: Reject or template-contract failure behavior is acceptable and should be interpreted as a baseline/template artifact, not as a manuscript failure
• File 2: Pass
• File 3: Warning
• File 4: Warning
• File 5: Pass
• File 6: Pass
• Acarbose.docx: parses with the NCBI Author Template Advanced snapshot
• celecoxib.docx: parses with the NCBI Author Template Advanced snapshot and citation/reference review data
• OAR guideline files should not be warned or rejected only because NCBI-style metadata tables or keywords are absent

UAT Scenarios

Scenario 1 - Template Profile Setup

Objective: Confirm that Template Profiles are visible, editable, and tied to base Word documents.

Steps:

• Open http://138.197.33.233:8080/template-profiles.
• Confirm NCBI Author Template Advanced and OAR Guidelines Template v4 are active.
• Confirm each profile shows a Profile Base Document link.
• Click an Edit button.
• Confirm the edit form shows Profile Name, Profile Base Document, and an optional replacement file upload.
• Return to the Template Profiles list.

Expected result:

• Profile Base Document links open or download DOCX files.
• Edit controls are visible for each active profile.
• The replacement upload is present but does not need to be used during normal UAT.

Scenario 2 - Upload With Correct Template Profile

Objective: Confirm that the upload flow evaluates each DOCX against the selected template-specific rules.

Steps:

• Open http://138.197.33.233:8080/upload.
• Confirm the page allows DOCX upload.
• Confirm the Template profile selector is visible.
• Select OAR Guidelines Template v4 before uploading File 2.
• Upload File 2.
• On the review page, confirm the document metadata shows OAR Guidelines Template v4.
• Repeat with NCBI Author Template Advanced for Acarbose.docx or celecoxib.docx.

Expected result:

• Upload succeeds.
• The user is redirected directly to the review page.
• The review metadata shows the selected Template Profile.
• The stored profile snapshot slug matches the profile used for upload.

Pass/fail notes:

• Fail if the Template Profile selector is missing or confusing.
• Fail if the wrong profile is shown on the review page after upload.
• Fail if the user is not redirected to a usable review page after upload.

Scenario 3 - Review Decision Summary

Objective: Confirm that the review page shows a clear top-level decision and summary counts.

Steps:

• On the review page for File 2, locate the decision banner.
• Confirm the page clearly reads Pass, Warning, or Reject.
• Confirm the summary shows counts for headings/blocks, references, citations, matched citations, blocking issues, and warning issues.

Expected result:

• The decision state is visible above the fold.
• Summary counts appear coherent and non-empty for a real manuscript.

Scenario 4 - Parsed Structure, Tables, Lists, And Figures

Objective: Confirm that the app renders parsed document structure clearly enough for reviewer inspection.

Steps:

• Upload a document that contains tables or figures/images, such as the NYSDOH AI PEP document or a representative OAR file with tables.
• Review the parsed structure section.
• Locate table blocks.
• Confirm tables appear as first-class table objects rather than flattened prose.
• Confirm figures/images outside tables appear as figure/media blocks.
• Confirm unordered and nested lists remain list-like instead of plain paragraphs when present.
• Confirm images inside tables remain part of the table context and do not incorrectly become standalone figures.

Expected result:

• Tables are rendered as tables.
• Figure/image handling is visible for non-table media.
• The reviewer can distinguish narrative content, lists, figures, and tables.

Scenario 5 - Reference Review

Objective: Confirm that extracted references and normalized reference metadata are visible.

Steps:

• In File 2, File 5, File 6, or celecoxib.docx, open the reference review section.
• Confirm reference order is shown.
• Confirm normalized reference text is shown.
• Confirm extracted metadata fields appear when available: authors, year, title, DOI, and PMID.

Expected result:

• References are reviewable without needing database-level knowledge.
• Missing extracted fields, when present, are visible as review signals rather than silent loss.

Scenario 6 - Reference-Centered Citation Review And External Validation

Objective: Confirm that a reviewer can inspect local citation matching and external validation outcomes.

Steps:

• In File 2, File 5, File 6, or celecoxib.docx, open the Reference list and validation section.
• Confirm each reference row shows normalized reference text and extracted metadata.
• Expand matched or possible in-text citation details under a reference row.
• Confirm each reference row shows DOI, PMID, and PubMed/DOI validation status when available.
• Confirm explicit external outcomes can appear, such as Validated, Probable, Ambiguous, Not found, Not applicable, and Pending.

Expected result:

• The distinction between local match and external validation is understandable.
• Ambiguous and Not found are visible to the reviewer.
• The page does not imply that those items can already be corrected in-app.

Scenario 7 - In-Text Citation Navigation And Highlighting

Objective: Confirm that parsed citations in the document body connect to the reference review section.

Steps:

• In File 2, File 5, File 6, or celecoxib.docx, inspect the parsed document body.
• Click one or more highlighted in-text citations.
• Confirm matched citations jump to their reference row and unmatched/problem citations jump to open citation issues.
• Confirm validated citations and review-needed/problem citations are visually distinguishable.

Expected result:

• Navigation works.
• Citation highlighting is visually consistent between the document body and reference review section.

Scenario 8 - Warning-Grade Review Case

Objective: Confirm that a warning-grade manuscript remains reviewable and understandable.

Steps:

• Upload File 3 with OAR Guidelines Template v4 selected.
• Confirm the result is Warning rather than Reject.
• Confirm there are no open citation issues for the earlier false date phrase.
• Review the warning list.
• Confirm the warnings are understandable as fix-in-Word items rather than fatal ingestion failures.

Expected result:

• File 3 is reviewable.
• The user is not blocked by the earlier false citation As of April 30, 2025.

Scenario 9 - Template Contract / Baseline Artifact

Objective: Confirm that the team understands the baseline template file is not a normal manuscript UAT case.

Steps:

• Upload File 1 with OAR Guidelines Template v4 selected.
• Observe the top-level outcome.
• Confirm with the BA/QA reviewer that the file is being interpreted as a template/baseline artifact, not as a normal chapter manuscript.

Expected result:

• The team records this as a baseline/template contract scenario.
• A rejection-like outcome is not treated as a Milestone 2 manuscript failure.

Questions QA And BA Should Answer

For each scenario, capture answers to these questions:

• Was the result understandable to a non-developer reviewer?
• Did the top-level decision feel justified by the evidence shown?
• Was the selected Template Profile visible and correct?
• Were citations, references, tables, figures, and lists easy enough to inspect?
• Were any labels or statuses confusing or too technical?
• Did the page make clear what still requires Milestone 3 editing/remediation?

Defect Logging Guidance

Classify findings into these buckets:

• Milestone 2 blocker
• upload flow broken
• Template Profile selection or snapshot is wrong
• review page unusable
• citations/references/tables/figures not inspectable
• wrong top-level decision due to an obvious parser failure
• Milestone 2 warning-grade issue
• confusing label/copy
• localized parsing imperfection that does not block review
• external validation nuance that is visible but not yet editable
• Milestone 3 backlog
• requests for editing/correction workflows
• requests for manual overrides
• requests for richer editorial history or remediation actions

Suggested UAT Sign-Off Rule

Milestone 2 can be considered UAT-ready if:

• upload and review succeed for the core manuscript files
• the correct Template Profile is selected, displayed, and snapshotted for each template family
• expected Pass / Warning outcomes are reproducible
• citations, references, tables, figures, and lists are inspectable
• external validation outcomes are visible to reviewers
• no new manuscript-grade blocking parser failures are found