ZACHARY QUIGLEY
Arlington,TX
Summary
Detail-oriented AI training and data annotation specialist with experience creating SFT responses, ranking LLM outputs, auditing factual accuracy, and red teaming for safety. Background also includes search relevance, NER, audio transcription, and team-based QA support, with work shaped by clear guidelines, research, and high accuracy standards.
Overview
1
1
Certification
6
6
years of professional experience
Work History
Generalist & Data Annotation Specialist
Outlier AI / Scale AI
03.2023 - Current
- Remote Contractor - San Francisco, CA (Remote) | March 2023 - Present
- Created 600+ gold-standard SFT ideal responses for complex prompts in coding, math reasoning, and general knowledge including chain-of-thought explanations.
- Evaluated over 2,000 pairs of LLM responses for preference ranking using Helpfulness, Honesty, and Harmlessness criteria.
- Justified rankings with 250+ word explanations, achieving 98% approval rating on Outlier leaderboard.
- Audited 1,200+ model outputs for factuality and hallucinations, utilizing web research and authoritative sources, and corrected answers with citations, decreasing factual errors by 18%.
- Conducted red teaming and safety testing by developing adversarial prompts to evaluate model compliance with disallowed content policies, addressing hate and misinformation.
- Executed tasks following Outlier generalist guidelines, including coding helpfulness, factuality check, summarization, and instruction following.
Founder & QA Lead
Handshake, Remotasks, and Outlier-type tasks
Arlington
01.2021 - Current
- Founded and led team of 25+ remote taskers for US AI training projects, enhancing operational support for Handshake, Remotasks, and Outlier-type tasks.
- Created internal QA system and SOPs, increasing client acceptance rate from 88% to 98.5% and decreasing error rate from 12% to 3%.
- Developed workflows for remote tasks in Handshake, Remotasks, and Outlier projects. across Handshake marketplace, Remotasks platform, and Outlier projects.
- Managed AI data labeling, prompt evaluation, and content review assignments.
- Managed task queues by providing clear instructions and quality guidelines. using clear instructions and quality guidelines.
Team Lead
COLINS TASKING EMPIRE
REMOTE, USA
01.2025 - 01.2026
AI Data Rater
Appen & Invisible Technologies
REMOTE, USA
02.2024 - 02.2025
Data Annotation Specialist
TELUS Digital (TELUS International AI)
06.2022 - 02.2023
- Remote - US AI Community | June 2022 - February 2023
- Labeled 4,500+ search queries for intent classification as Informational, Transactional, or Navigational, achieving 97% QA score for search result relevance.
- Annotated 3,000+ texts for Named Entity Recognition and sentiment analysis, enhancing model accuracy for PERSON, ORGANIZATION, LOCATION, and sentiment categories.
- Transcribed and quality-checked 800 hours of US English audio data for Automatic Speech Recognition model, ensuring less than 2% Word Error Rate.
Remote Contractor
Kirkland, WA (Remote)
Kirkland
01.2021 - 05.2022
- Evaluated 2,500+ LLM outputs for policy compliance, ensuring adherence to client guidelines on hate speech, self-harm, and misinformation.
- Created concise summaries for long-form documents and rewrote AI responses, improving clarity and grammatical accuracy in US English.
- Labeled 5,000+ images with bounding boxes and polygon segmentation for object detection in autonomous vehicle dataset, enhancing data quality for training models using CVAT and SuperAnnotate.
Education
Associate of Science - Information Systems - Data Analytics Concentration
University of Texas at Arlington (UTA)
Arlington, TX05-2021
Some College (No Degree) - Computer Support and Data Management
Austin Community College
Fort Worth, TXSkills
- AI Training & Evaluation: RLHF, RLIAF, Supervised Fine-Tuning (SFT), LLM Evaluation, Prompt Engineering, Chain-of-Thought Reasoning, Preference Ranking, Hallucination Detection, HHH framework, Instruction Following, Red Teaming, AI Safety Alignment
- Data Annotation: Text Classification, Named Entity Recognition (NER), Sentiment Analysis, Intent Classification, Search Relevance Rating, Bounding Box, Image Segmentation, Audio Transcription, Content Moderation, Data quality assurance
- Tools & Platforms: Labelbox, Label Studio, CVAT, SuperAnnotate, Scale Rapid, Appen Crowd Platform, TELUS Digital Workspace, Jira, Notion, Google Workspace, Microsoft Excel, Python Basics
- Prompt evaluation
- Model auditing
- Data Quality Assurance
- Generalist Skills: Research skills, Technical Writing, Summarization, Creative Writing, STEM Reasoning, Critical Thinking, US English Grammar and Style
- Strengths: Fast Learner, High Attention to Detail, Meets Tight Deadlines, Written communication
- Data Quality Assurance
Personal Information
- Availability: 8 AM - 8 PM EST / CST, Weekends Available
- Work Permit: US Citizen - Authorized to Work in the United States
Certification
- Labelbox Academy - Advanced Data Annotation and QA - 2024
- Scale AI - Rapid Tasker Qualification - NER, Classification, RLHF - 2024
- TELUS Digital - US English Rater Certification and Search Relevance - 2023
- Outlier AI - Generalist Skills Test - Reasoning, Math, Coding Helpfulness - Passed 2024
- Google Data Analytics Professional Certificate - Coursera - 2023
- Learn Prompting - Prompt Engineering for AI Trainers - 2024
ADDITIONAL INFORMATION
- Work Authorization: US Citizen - Authorized to Work in the United States - Background Check Ready
- Work Type: Open to W2, 1099, Full-Time, Part-Time, Contract
- Equipment: Windows 11 Laptop, High-Speed Internet, Backup Power, Quiet Home Office
Timeline
Team Lead
COLINS TASKING EMPIRE
01.2025 - 01.2026
AI Data Rater
Appen & Invisible Technologies
02.2024 - 02.2025
Generalist & Data Annotation Specialist
Outlier AI / Scale AI
03.2023 - Current
Data Annotation Specialist
TELUS Digital (TELUS International AI)
06.2022 - 02.2023
Founder & QA Lead
Handshake, Remotasks, and Outlier-type tasks
01.2021 - Current
Remote Contractor
Kirkland, WA (Remote)
01.2021 - 05.2022
Associate of Science - Information Systems - Data Analytics Concentration
University of Texas at Arlington (UTA)
Some College (No Degree) - Computer Support and Data Management
Austin Community College