Research Engineer, Post-training Instruction Following

Company: OpenAI
Location: San Francisco
Posted on: November 18, 2024

Job Description:

Research Engineer, Post-training Instruction FollowingPost-training - San FranciscoAbout the TeamOur post-training team are the chefs behind GPT-4 and o1-preview, cooking up the raw ingredients of base models into something nutritious, tasty, and non-toxic for consumers.If you care about impact, this could be a good team for you. Your daily work will push the leading edge of AI and make a real difference to hundreds of millions of people across thousands of products.About the RoleWe are seeking a research engineer to help us post-train some of the world's most powerful, cutting-edge AI models, used by hundreds of millions of people. In particular, we're looking for an early, impactful hire on a subteam focused on training models to more reliably do what's asked of them. Lots of low hanging fruit to be picked, so lots of room for impact and growth.This role is in San Francisco, CA. We nominally expect at least 3 days in the office per week, not because we care about where you sit, but because we care about the value you produce and believe that you'll be best positioned to learn, teach, and succeed when sitting alongside collaborators. If you don't already live here, we'll assist you with relocation.In this role, you will:

• Train state-of-the-art language models using new techniques and new data
• Become fluent in OpenAI's deep learning infrastructure
• Create evaluations to measure success
• Rapidly iterate through experiments to find what works and what doesn't
• Prioritize approaches that (a) scale with compute and (b) endure as capabilities rise
• Collaborate with product teams to ensure your work actually translates to better experiences for people using GPTYou might thrive in this role if you:The only truly required qualification is that you're able to learn to do the job and adapt as it changes. However, we'll have more confidence in hiring you if you demonstrate a decent fraction of the following:
• Strong software engineering skills (e.g., good at the command line, good at shaping the right abstractions, good at debugging, good at anticipating future design needs)
• Strong Python skills (able to write high-quality readable code, and read others' code)
• Experience wrangling distributed systems
• Experience managing projects in complex technical environments
• Good intuitions of fundamental ML concepts (e.g., fluent in thinking about overfitting, generalization, reward hacking, etc.)
• Good intuitions of language models and their quirks (e.g., why is it hard to count the R's in strawberry, why chain of thought works)
• Eagerness to dig into data and play with trained models
• Curiosity about how to push the frontiers of AI performance
• [Bonus] Experience fine-tuning large language models
• [Bonus] Experience deploying large language models in a product, or using the OpenAI API
• [Bonus] Building front end interfaces for looking at data, sharing results, etc.This might be a bad role for you if:
• You want to work deeply on a single problem for a long time
• You want to publish your findings
• You want to write elegant code without interacting with downstream users
• You want to set new records on academic benchmarks
• You're more interested in model architecture than training / evaluation / dataAbout OpenAIOpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status.OpenAI Affirmative Action and Equal Employment Opportunity Policy StatementFor US Based Candidates: Pursuant to the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records.We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.OpenAI Global Applicant Privacy PolicyAt OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.Compensation$295K - $360K + Offers Equity
  #J-18808-Ljbffr

Keywords: OpenAI, Mountain View , Research Engineer, Post-training Instruction Following, Engineering , San Francisco, California