KellyMitchell matches the best IT and business talent with premier organizations nationwide. Our clients, ranging from Fortune 500 corporations to rapidly growing high-tech companies, are exceptionally served by our 1500+ IT and business consultants. Our industry is growing rapidly, and now is a great time to launch your career with the KellyMitchell team.
Title: Data Engineer
Job Description & Responsibilities
We are in need of a Data Integration Engineer that has experience or a strong interest in learning big data technology like Kafka & Nifi to provide reusable, extensible and standardized integration solutions within our Commerce Fulfillment platforms. This role will be a key player in supporting our Behavioral Insights Engine which is the SaaS engagement platform for our incentive programs, the heart of our business. Our environment includes various data feeds coming into our Employee Engagement platform (employee, sales, job, activity) that needs to be manipulated into usable formats that are pushed out to a Data Lake utilized for various internal teams for insights and analytics.
- Design and develop scalable and reliable real-time stream processing solutions using Hortonworks Data Flow HDF product suite (Apache Nifi/Kafka) under the direction of team of platform architects
- Provide expertise and hands on experience working with Kafka brokers and Kafka connectors
- Work directly with business partners to translate complex functional and technical requirements for streaming data ingestion solutions into detailed design & implementation plans
- Work closely with enterprise architects and cross-functional technical resources to devise and recommend solutions based on the understood requirements
- Work closely with Platform Engineering team to analyze complex distributed production deployments, and make recommendations to optimize performance
- Implement application development lifecycle management using industry standard frameworks
- Write and produce technical documentation and knowledge base for these integration processes
- Work with Production Support to assist with troubleshooting service stability, message topic or delivery issues, perform data related benchmarking, performance analysis and tuning.
- Perform design & code reviews
Skills, Experiences, Knowledge
- Experience in development and support of stream processing solutions in datawarehouse technologies, like Hadoop
- Experience with integration solutions development