[Remote] Generative AI Data Analyst - California, USA (Remote)

Note: The job is a remote job and is open to candidates in USA. Welo Data is seeking a Generative AI Analyst to support a high-impact machine learning project. This role focuses on creating high-quality prompts and responses across diverse topics and leading labeling initiatives with internal and external partners.

Responsibilities

• Creatively writing prompts and responses to a variety of diverse topics
• Perform LLM annotation and evaluation tasks (ranking, scoring, labeling, tagging)
• Evaluate model outputs for accuracy, relevance, and instruction-following
• Identify and document issues such as hallucinations and inconsistencies
• Participating in and/or supporting labeling workflows, including hands-on annotation and collaboration with internal or external teams
• Training teams on best practices for creating Large Language Models/Data sets

Skills

• Hands-on experience performing data annotation or evaluation tasks (e.g., labeling, ranking, scoring, or tagging LLM outputs)
• Native or near-native English with excellent writing skills
• Strong attention to detail and ability to follow guidelines consistently
• Self-driven, motivated and enthusiastic to work on state-of-art machine learning tools
• 4 year Accredited College degree or equivalent experience
• Must have valid work authorization in the US (Welo Data does not sponsor VISAs at this time)
• College Degree or experience in Linguistics, English Literature, Creative Writing, Journalism, and domain knowledge (Law/Medical/Math/Coding/etc.)
• Experience working in annotation platforms or structured labeling environments is a plus
• Deep understanding of Large Language Models/RLHF
• Experience in labeling/tagging of frames/tasks/prompts to prepare for DNNQA/testing experience

Benefits

• Full-time W-2 employee with benefits

Company Overview