Pre-Training Data Engineer - Remote

Overview

As a Pre-Training Data Engineer, you will play a pivotal role in developing the data infrastructure that underpins Cohere’s advanced language models.

In Short

• Design and build scalable data pipelines to ingest, clean, filter, and optimize diverse datasets.
• Conduct data ablations to assess data quality and experiment with data mixtures.
• Develop robust data modeling techniques for optimal training efficiency.
• Research and implement innovative data curation methods.
• Collaborate with cross-functional teams to ensure data pipelines meet model demands.

Requirements

• Strong software engineering skills, with proficiency in Python.
• Familiarity with data processing frameworks like Apache Spark or Pandas.
• Experience with large-scale datasets.
• Knowledge of data quality assessment techniques.
• A passion for bridging research and engineering in AI.

Benefits

• An open and inclusive culture and work environment.
• Work closely with a team on cutting-edge AI research.
• Weekly lunch stipend and in-office lunches.
• Full health and dental benefits, including mental health support.
• Remote-flexible work with offices in major cities.
• 6 weeks of vacation.