Experienced Big Data Engineer – Hadoop, Spark & Real-Time Data Pipeline Development (Remote/Hybrid)

Posted 2026-05-06
Remote, USA Full-time Immediate Start

Join arenaflex as a Big Data Engineer

Are you ready to be part of a transformative technology journey? At arenaflex, we're reimagining how data powers businesses, and we're looking for a talented Big Data Engineer to help us build the next generation of data-driven solutions. This is an exciting opportunity to work with cutting-edge technologies, collaborate with brilliant minds, and make a meaningful impact on how millions of customers experience financial services.

At arenaflex, we believe that great technology comes from diverse perspectives, collaborative innovation, and a relentless focus on customer needs. Our engineering teams are at the heart of everything we do—from developing sophisticated fraud detection systems to creating seamless digital payment experiences. As a Big Data Engineer joining our team, you'll be instrumental in architecting and building the data infrastructure that powers these transformative solutions.

What You'll Do

As a key member of our data engineering team, you'll have the opportunity to work on complex problems that sit at the intersection of big data, machine learning, and real-time processing. Your technical expertise will be crucial in designing and implementing data pipelines that handle massive volumes of information with speed and reliability.

Software Development & Engineering


  • Design, develop, and maintain robust data pipelines using Hadoop, Spark (Core, SQL, and PySpark), and related big data technologies

  • Write clean, efficient, and well-documented code for data processing applications and APIs

  • Participate actively in agile ceremonies, contributing to sprint planning, daily stand-ups, and retrospectives

  • Conduct thorough code reviews and implement comprehensive testing strategies to ensure code quality

  • Troubleshoot and resolve complex issues in data processing systems, identifying root causes and implementing permanent solutions

  • Automate deployment processes and implement continuous integration/continuous deployment (CI/CD) pipelines

  • Scale applications dynamically based on demand projections and performance requirements

  • Collaborate with data scientists and analysts to build features that enable advanced analytics and machine learning initiatives

Leadership & Mentorship


  • Lead and mentor a small team of engineers, fostering their professional growth and helping them reach their full potential

  • Inspire innovation by encouraging team members to explore new approaches and challenge conventional thinking

  • Drive technical excellence by establishing best practices and coding standards within the team

  • Contribute to architectural decisions that impact our data platform's long-term success

Innovation & Continuous Improvement


  • Identify opportunities to adopt emerging technologies that can enhance our data capabilities

  • Propose and implement improvements to existing systems, always looking for ways to increase efficiency and reduce technical debt

  • Share knowledge with the broader engineering community through internal tech talks and documentation

  • Contribute to open-source projects and stay current with industry trends and best practices

What We're Looking For

We're seeking a passionate engineer who thrives in a collaborative environment and is excited about solving complex data challenges. The ideal candidate will have a strong foundation in software engineering combined with hands-on experience in big data technologies.

Required Qualifications


  • Education: Bachelor's degree in Computer Science, Information Systems, Engineering, or a related technical field. Master's degree in Computer Applications or equivalent experience is a plus

  • Experience: 3+ years of software development experience with at least 1-2 years in a leadership role, guiding teams of developers and facilitating scrum processes

  • Technical Expertise: Strong hands-on experience with Hadoop ecosystem technologies including MapReduce, Hive, and Spark

  • Programming Skills: Proficiency in at least one programming language such as Python, Scala, or Java, with the ability to write efficient, maintainable code

  • Data Engineering: In-depth knowledge of distributed computing environments and experience with large-scale data processing

  • SQL Proficiency: Strong ability to write and optimize complex SQL queries, including Hive and Spark DataFrame operations

  • Operating Systems: Experience with UNIX/Linux shell scripting for automation and data processing tasks

  • Data Pipeline Development: Demonstrated ability to design and develop advanced data pipelines for both batch and real-time processing

  • System Design: Experience in analyzing, designing, developing, testing, and implementing enterprise-level applications

  • Documentation: Ability to create comprehensive technical and functional specifications while analyzing software and system processing flows

Preferred Qualifications


  • Experience with cloud platforms such as AWS, GCP, or Azure, particularly in building microservices and scalable data solutions

  • 3+ years of experience designing and building solutions using Kafka streams or message queuing systems

  • Proficiency with GitHub or Bitbucket and CI/CD pipeline implementation

  • Familiarity with NoSQL databases such as HBase, Couchbase, or MongoDB

  • Understanding of financial reporting environments and regulatory compliance requirements

  • Experience with containerization technologies like Docker and Kubernetes

  • Knowledge of machine learning frameworks and their integration with data pipelines

Skills & Competencies for Success

Beyond technical qualifications, we value engineers who bring strong problem-solving abilities and excellent communication skills to the table. You should be comfortable working independently while also being an effective team player who enjoys collaborating across functional boundaries.


  • Analytical Thinking: Strong analytical and problem-solving skills to tackle complex data challenges

  • Communication: Excellent verbal and written communication skills to effectively articulate technical concepts to both technical and non-technical stakeholders

  • Project Management: Strong organizational skills with the ability to manage multiple priorities and meet deadlines

  • Adaptability: Comfortable working in a fast-paced, dynamic environment with evolving requirements

  • Innovation: A curious mindset that drives you to explore new technologies and approaches

  • Collaboration: Ability to work effectively with cross-functional teams and build positive relationships

Career Growth & Development

At arenaflex, your growth is our priority. We invest heavily in our employees' professional development and provide numerous opportunities for career advancement. As part of this role, you'll have access to:


  • Dedicated time and resources for professional development and skill enhancement

  • Mentorship from senior engineers and technical leaders

  • Opportunities to work on diverse projects across different business lines

  • Exposure to cutting-edge technologies and emerging trends in the data engineering space

  • Clear career pathways for technical leadership and principal engineering roles

  • Internal mobility opportunities to explore different roles and teams

  • Support for obtaining industry certifications and attending conferences

You'll also have the chance to contribute to the broader engineering community through knowledge sharing, open-source contributions, and participation in internal technical communities of practice.

Work Environment & Culture

arenaflex is committed to fostering an inclusive, innovative, and collaborative work environment. We believe that diverse perspectives drive innovation, and we welcome engineers from all backgrounds to join our team.

Our culture is built on trust, transparency, and a commitment to excellence. We encourage our employees to voice their ideas, challenge the status quo, and take ownership of their work. You'll find a supportive ecosystem where your contributions are valued, and your voice matters. Our teams embrace agile methodologies and continuous improvement, ensuring that we deliver high-quality solutions while maintaining a healthy work-life balance.

Whether you work remotely or prefer the collaboration of our hybrid office spaces, you'll have access to state-of-the-art tools and technologies that enable you to do your best work. We maintain a flexible approach to work arrangements, recognizing that our employees have different needs and preferences.

Compensation & Benefits

We offer competitive compensation packages that reflect your skills, experience, and the value you bring to arenaflex. In addition to attractive salary packages, we provide a comprehensive benefits suite that includes:


  • Health, dental, and vision insurance coverage

  • Retirement savings plans with company matching

  • Generous paid time off and holidays

  • Parental leave and family support programs

  • Wellness programs and resources

  • Employee assistance programs for mental health and well-being

  • Stock options or equity participation programs

  • Relocation assistance for eligible positions

The compensation for this role is competitive and commensurate with experience. We regularly review our pay structures to ensure we remain competitive in the talent market.

Ready to Make an Impact?

If you're passionate about big data, excited by the opportunity to work with cutting-edge technologies, and want to be part of a team that's transforming the financial services industry, we encourage you to apply for this position. At arenaflex, you'll find more than just a job—you'll find a career where you can grow, innovate, and make a real difference.

We believe in hiring the best talent and creating an environment where everyone can succeed. Join us and be part of a team that's shaping the future of data engineering. Your journey with arenaflex starts here.

Note: Final offers of employment are contingent upon successful completion of a background check, in accordance with applicable laws and regulations. We are an equal opportunity employer and value diversity at our company. We do not discriminate based on race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

Similar Jobs

Back to Job Board