Remote Otter LogoRemoteOtter

Big Data Engineer - Remote

Posted 12 weeks ago

Overview

Orion Innovation is a premier, award-winning, global business and technology services firm. Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and engineering, with a unique combination of agility, scale, and maturity. We work with a wide range of clients across many industries including financial services, professional services, telecommunications and media, consumer products, automotive, industrial automation, professional sports and entertainment, life sciences, ecommerce, and education.

In Short

  • Requirement analysis, requirement gathering, design, business impact analysis, gap analysis, estimation, development, testing, coding, code review, unit testing and deployment of the application.
  • Design, build, install, configure, and support Hadoop (Big Data).
  • Design and implement big data pipelines to ingest and process data in real-time and monitor for data miss or error in progress to ensure that data reaches the end system.
  • Measure the performance of various APIs and implement optimization on slow services to enhance the responsiveness of the system.
  • Deliver functional enhancements to the existing big data applications.
  • Provide inputs on solution architecture based on evaluation/understanding of solution alternatives, frameworks, and products.
  • Develop Hadoop batch jobs for data extraction from multiple unstructured and structured data sources for populating into various repositories (Hadoop, Neo4J, MongoDB, Apache SOLR).
  • Interact with clients to elicit architectural and non-functional requirements such as performance, scalability, reliability, availability, maintainability.
  • Participate in designing the data model structured data in RDBMS into a graph database solution and NoSQL database solution.
  • Develop near real-time data processing solutions using Kafka and Spark Streaming.
  • Participate in designing Spark architecture with Databricks and Structured Streaming.
  • Set up Microsoft Azure with Databricks, and Databricks Workspace for business analytics.
  • Contribute to the automate build process by using Jenkins and Ansible to achieve continuous integration and continuous deployment.

Requirements

  • Master’s in Computer Science/Applications, Information Technology/Systems or Electronics/Electrical Engineering + minimum 1 year experience as Big Data Engineer, Data Engineer, Data Engineering Specialist, Data Warehousing Analyst or related occupation.
  • Must be willing to travel/relocate to anywhere in the US.

Benefits

  • Orion is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex, sexual orientation, gender identity or expression, pregnancy, age, national origin, citizenship status, disability status, genetic information, protected veteran status, or any other characteristic protected by law.

Similar Jobs:

Experian logo

Big Data Engineer - Remote

Experian

2 days ago

Join Experian as a Big Data Engineer, where you will work remotely to develop and implement innovative data solutions.

Spark
Scala
Python
AWS
USA
Full-time
Software Development
MGID logo

Big Data Engineer - Remote

MGID

2 weeks ago

Join MGID as a Big Data Engineer to work with large datasets and support the Data Science team in developing ML and AI solutions.

BIG Data
Data Engineering
Machine Learning
AI
Ukraine
Full-time
Data Analysis

Capco

Big Data Engineer - Remote

Capco

2 weeks ago

Join Capco as a Big Data Engineer to work on transformative projects in the financial services industry.

Google Cloud Platform
BigQuery
Python
SQL
Poland
Full-time
Data Analysis
Coherent Solutions logo

Big Data Engineer - Remote

Coherent Solutions

3 weeks ago

Join a leading digital advertising company as a Big Data Engineer, focusing on high-scale solutions and team mentorship.

BIG Data
Spark
Python
Java
Moldova
Full-time
Software Development
Inetum logo

Big Data Engineer - Remote

Inetum

8 weeks ago

Join Inetum as a Big Data Engineer to design and optimize data pipelines and collaborate with data scientists.

BIG Data
Hadoop
Spark
Hive
Spain
Full-time
Software Development
Competitive salary based on experience