Open position
Data Science & Analytic Methodology SME
Micro1
Data Science & Analytic Methodology SME
Company: MSI (for Micro1)
Role Type: Contractor
Location: Remote
Payment: $60 - $100/hr
Experience Level: Senior
Professional Summary
micro1 is seeking Data Science & Analytic Methodology Subject Matter Experts (SMEs) to contribute expertise to a high-impact project focused on advanced analytic pipelines and Large Language Models (LLMs). In this role, you will help train next-generation AI systems by providing high-quality, real-world domain inputs to shape how models learn, reason, and perform. Prior AI experience is not required; deep domain expertise in data science and analytic methodology is primary.
Key Responsibilities
- Taxonomy Mapping: Map scaled-analysis and pipeline-integration scenarios to an established evaluation taxonomy.
- Analytic Case Authoring: Create detailed analytic cases reflecting authentic deployment patterns (e.g., automated triage, enrichment, analysis-at-scale) and real-world failure modes.
- Response Evaluation: Grade model outputs against defined criteria including source handling, reasoning, understanding of limitations, and practical utility.
- Over-Refusal Detection: Identify and flag instances of model over-refusal that hinder legitimate analytic workflows or disrupt pipeline continuity.
- Continuous Improvement: Document challenging edge cases and emerging methodology concerns, providing actionable recommendations for model refinement.
- Feedback & Collaboration: Provide clear, insightful written and verbal feedback to support iterative, high-quality project outcomes.
Technical Requirements (Must-Have)
- Strong domain expertise in Data Science, Analytic Methodology, or related quantitative disciplines.
- Track record of conducting analysis at scale and understanding data pipeline mechanics.
- Ability to author clear, concise, and structured technical documentation in English.
Desired Skills (Nice-to-Have)
- Experience integrating LLMs (including government GPT variants) into complex data or analytic pipelines.
- Experience in designing evaluation frameworks or benchmarking systems.
- Background in teaching, mentoring, or technical leadership within data science or analytics.
- Familiarity with defense, intelligence, or government analytic domains.
- Experience operating effectively in distributed, international project environments with minimal clearance constraints.