Senior Hadoop / J2EE Engineer DriveCam
Senior Hadoop/J2EE engineer is a key role for cutting edge big data solution implementation at Lytx. He/she will be a part of core team designing and implementing Hadoop ecosystem for Data Science, Data Analytics, Data Warehouse and Business Intelligence by leveraging variety of data types.
Primary roles include designing and developing large scale of real-time and batch data ingestion and integrations using Hadoop Platform. The ideal candidate possesses solid 4 plus years of Hadoop ecosystem design and development experience, and 6 plus years of J2EE application development experience.
This position will be guiding and assisting software and data engineering teams in designing and developing appropriate big data solutions in Hadoop framework for different data needs and use cases.
Design and develop end-to-end Hadoop data ingestion process. Sufficiently decompose and transform the NoSQL, Binary, Sequential Flat file or XML data to Data warehouse through MapReduce, Pig, Hive, Avro and etc.
Design and develop Hadoop data integration in both real-time and scheduled mode leveraging appropriate Hadoop open integration framework (Storm, Flume, Scoop, Oozie, and etc.)
Manage Hadoop clusters and guide the operational best practices for Hadoop cluster configuration and job processing.
Build Libraries, user defined functions (UDF), and frameworks around Hadoop designed to be reusable and scalable for enterprise wide use.
Manage the performance and allocations in open distributed file system (HDFS) leveraging the Hadoop data lifecycle and retention policy.
Implement standards based monitoring and tuning practices for the 30+ node Hadoop ecosystem.
Translate the functional and non-functional requirements from business, product and engineering teams to appropriate Hadoop technology solution design.
Document the big data solution design and clearly communicate the target solution patterns to the technical audiences.
Must have an experience setting up, optimizing and sustaining Hadoop ecosystem at mid-to-large scale (30+ nodes, 100+ TB).
Solid 4 plus years of experience of designing and developing Hadoop solutions with strong knowledge of related framework including Pig, Hive, Sqoop, Flume, Storm, Hbase and HDFS.
Minimum 6 years of experience of Java application development experience; strong knowledge of object-oriented-programming (OOP) with J2EE architecture expertise.
Must have an experience in analysis, design, development and implementation of large scale Java/J2EE projects.
Must have an experience of setting up J2EE/Hadoop application build, deploy and promote configurations experience; Java library/source control management experience preferred.
MS SQL server database design/development experience with knowledge of OLAP and OLTP data modeling and implementation.
Greenplum/Pivotal database development experience for large scale of data warehouse.
Familiar with Talend Data Integration Studio and Eclipse for J2EE developers.
Lytx, Inc. is proud to be an equal opportunity/affirmative action employer. We are committed to attracting, retaining and maximizing the performance of a diverse and inclusive workforce.
San Diego, California, United States
||8911 Balboa Avenue |
San Diego, CA 92123