Data Engineer Data Platforms-Google Full Time

About Company :

IBM’s greatest invention is the IBM. We believe that through the application of intelligence, reason and science, we can improve business, society and the human condition, bringing the power of an open hybrid cloud and AI strategy to life for our clients and partners around the world.

Restlessly reinventing since 1911, we are not only one of the largest corporate organizations in the world, we’re also one of the biggest technology and consulting employers, with many of the Fortune 50 companies relying on the IBM Cloud to run their business.

At IBM, we pride ourselves on being an early adopter of artificial intelligence, quantum computing and blockchain. Now it’s time for you to join us on our journey to being a responsible technology innovator and a force for good in the world.

Job Description :

As Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You’ll contribute to data gathering, storage, and both batch and real-time processing.
Collaborating closely with diverse teams, you’ll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you’ll tackle obstacles related to database integration and untangle complex, unstructured data sets.

Role & Responsibility :

• Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big data, incorporating a variety of statistical and machine learning techniques
• Designing and implementing various enterprise seach applications such as Elasticsearch and Splunk for client requirements
• Work in an Agile, collaborative environment, partnering with other scientists, engineers, consultants and database administrators of all backgrounds and disciplines to bring analytical rigor and statistical methods to the challenges of predicting behaviors.
• Build teams or writing programs to cleanse and integrate data in an efficient and reusable manner, developing predictive or prescriptive models, and evaluating modeling results

Your primary responsibilities include:

• Develop & maintain data pipelines for batch & stream processing using informatica power centre or cloud ETL/ELT tools.
• Liaise with business team and technical leads, gather requirements, identify data sources, identify data quality issues, design target data structures, develop pipelines and data processing routines, perform unit testing and support UAT.
• Work with data scientist and business analytics team to assist in data ingestion and data-related technical issues.

Required Technical and Professional Expertise

• Develop/Convert the database (Hadoop to GCP) of the specific objects (tables, views, procedures, functions, triggers, etc.) from one database to another database platform Implementation of a specific Data Replication mechanism (CDC, file data transfer, bulk data transfer, etc.).
• Expose data as API Participation in modernization roadmap journey Analyze discovery and analysis outcomes Lead discovery and analysis workshops/playbacks Identification of the applications dependencies, source, and target database incompatibilities.
• Analyze the non-functional requirements (security, HA, RTO/RPO, storage, compute, network, performance bench, etc.).
• Prepare the effort estimates, WBS, staffing plan, RACI, RAID etc. .
• Leads the team to adopt right tools for various migration and modernization method
• Experience with relational SQL, Big Data etc

Preferred Technical and Professional Expertise

• You thrive on teamwork and have excellent verbal and written communication skills.
• Ability to communicate with internal and external clients to understand and define business needs, providing analytical solutions
• Ability to communicate results to technical and non-technical audiences