Brief description of the organization
FirstPassResearch is a technology company that developed PRISM-AI, an intelligent editorial triage platform designed to streamline scientific and biomedical peer review. By delivering automated peer reviews of manuscript submissions, the platform evaluates methodological rigor, critical reasoning, and research-integrity risks (such as paper mills and citation manipulation). PRISM-AI functions as an assistant to journal editorial boards, reducing reviewer burden and speeding up workflows by providing customizable recommendations while ensuring human editors maintain full oversight and final decision-making authority.
Problem area
Scientific journals receive large numbers of research manuscripts that must be screened by editors before being sent to external experts for full peer review. PRISM-AI is an AI-assisted editorial screening system that generates an initial “Reject / Not Reject” recommendation and reviewer-style feedback. The central technical problem is how to make this assessment accurate, transparent, multimodal, secure, and adaptable across different journals. The current system primarily evaluates extracted manuscript text. The next stage will incorporate figures, graphs, tables, and other visual information, link AI critiques to the evidence supporting them, compare alternative AI models and architectures, and improve the system’s ability to recognize uncertainty and resist manipulation.
Main objectives
- Multimodal review: Integrate and evaluate vision-capable models that can assess manuscript text, figures, graphs, and tables together, including consistency between visual results and claims made in the text.
- Verifiable and source-linked outputs: Link AI-generated critiques to exact supporting text or visual evidence and identify critiques that cannot be adequately supported by the manuscript.
- Model development and evaluation: Expand and curate model-development datasets, compare and improve alternative models and architectures using standardized evaluation methods, develop uncertainty measures, and track model, prompt, and dataset versions and performance over time.
- Journal adaptation and editor feedback: Develop data-efficient methods for adapting PRISM-AI to journals with different editorial standards using journal-specific data and editor feedback, while maintaining secure separation between journals and safe model versioning and rollback.
- Security and reliability: Improve authentication, manuscript data protection and secure file processing, concurrent-user performance, automated testing, monitoring and failure recovery, and resistance to prompt injection or other adversarial manuscript content.
- Citation and research-integrity analysis: Develop reference checking, related-literature retrieval, citation and metadata analysis, and paper-mill/integrity screening, with findings presented as evidence-supported alerts for editor review.
Scope of work
Students will first review the existing PRISM-AI model pipeline, application architecture, and available de-identified datasets. Baseline performance of the current system will be established before new components are introduced.
The team will then design and evaluate a multimodal manuscript-review pipeline capable of incorporating both manuscript text and visual information. The team will select and benchmark suitable multimodal models and system approaches for PRISM-AI, including approaches such as fine-tuning, retrieval-augmented generation, or task-specific AI components where appropriate. Additional work will include evidence retrieval and linking, model evaluation and versioning, journal-specific adaptation, editor-feedback capture, citation and research-integrity analysis, adversarial testing, and production reliability improvements. Depending on team interests and progress, secondary work may include scientific image-integrity screening, reporting-guideline assessment, and task-specific AI reviewer components.
Deliverables
- Website
- Resources
- New protocols/processes
Team meeting frequency
Bi-weekly
Skills and training required
- Software engineering, Python and/or TypeScript, machine learning, LLMs or vision-language models, backend/API development, databases, information retrieval, and automated testing.
- Experience with computer vision, NLP, cybersecurity, cloud infrastructure, or MLOps would be useful but is not required.
- No biomedical or scientific-publishing background is required. The PRISM-AI team will provide the necessary domain expertise, de-identified data, and guidance on how scientific manuscripts are evaluated.
Resources required
- Existing PRISM-AI application and model pipeline
- De-identified manuscript and editorial datasets
- Cloud computing resources
- AI/model APIs and training resources
- Scholarly metadata and citation APIs
- Domain expertise from journal editors and the PRISM-AI team
- Reasonable approved computing, API, and model-training expenses required for the project will be supported by the project team.
NDA or a commercialization agreement for this project?
Yes