Open Source Engineer - Remote

Posted 1 week ago

Software Development

Full Time

CA, USA

High-Performance Computing

Data Infrastructure

Overview

Join our world-class team as an experienced Open Source Engineer, where you'll drive the evolution of high-performance multimodal databases.

In Short

Drive OSS community effort to integrate Lance format into various Data Infrastructure systems.
Promote Lance format in Big Data conferences and meetups.
Design and maintain efficient distributed Lance dataset operations.
Design efficient indices for predicate push down in Spark, Ray, or Trino.
Work on table format, data encodings, and various aspects of the Lance format in Rust.
Operate on in-house data processing infrastructure.

Requirements

At least five years of experience building high-performance databases or big data systems.
Experience with Open Source Big Data or AI training systems.
Hands-on experience with high-performance computing in Java or Scala.
Ability to work with a small, high-caliber team with autonomy.

Benefits

Work with experts in data science and machine learning.
Contribute to cutting-edge open source projects.
Be part of a team creating tools used by millions worldwide.

LanceDB

LanceDB is a developer-friendly, open-source database designed for multimodal AI applications. It offers hyper-scalable vector search and advanced retrieval capabilities, making it an ideal foundation for AI applications. The company focuses on providing tools for streaming training data and interactive exploration of large-scale AI datasets, catering to some of the most innovative applications and complex requirements in the field today. Founded by experts with extensive experience in data science and machine learning, including co-authors of pandas and Apache PMC members, LanceDB aims to enhance the Data Infrastructure ecosystem through community-driven open-source projects.

Share This Job!

Save This Job!

LanceDB

Share This Job!

Save This Job!

Similar Jobs:

Resend