Datastage/ Python developer

Location: Columbus, OH Salary: Depends on Experience Description:
Job Description:

• Develop Shell scripts for automation of the data flow process and ETL orchestration pipeline.
• Implement ETL transformations using DataStage, Hadoop Big data technologies like Spark-Scala/Spark-Java, Hive, or Pig based on the business use case.
• Develop One-time/On-demand data Extractions from the data marts in the database systems for the business users for analytical purposes.
• Direct Podium for end-to-end data management.
• Develop automation and ETL jobs to schedule data processing utilities using reusable ETL components and frameworks of the data pipelines.
• Implement Data profiling, Data quality, and Data standardization for all the workflows based on EDM standards (Enterprise Data Management).
• Build Hadoop Data Lake for analytics using Hadoop echo system.
• Migrate tables from traditional RDBMS into Data Lake using Sqoop.
• Transfer real-time data from different data sources into HDFS systems using Kafka producers, consumers, and Kafka brokers.
• Use cloud-native development, micro-services architecture, operations, and governance framework to gather required technical data for purposes of formulating recommended customization of AWS.
• Perform Unit and regression testing, prepare unit test case documents for comparing the results with business requirements.
• Ensure the technical implementations are in line with the requirements and specifications.
• Help Quality Analyst Team in preparing a comprehensive test plan to ensure proper functioning of the application for the zero-defect delivery for the continuous integration and deployment to the production.
• Prepare Technical Specification documents that explain the technical solutions and list the technologies to be used while implementing the solution.
• Prepare data models to be certain that the data objects are represented accurately by the ETL and Data engineering software and application tools.
• Participate in business requirement meetings, perform requirement, and impact analysis with Business Analysts.
• Coordinate with business users to understand the data sources/entities.
• Design the on-prem or cloud data warehousing models using ETL tools such as IBM Infosphere Data Stage, Oracle, Python, and cloud platforms such as Snowflake and AWS (Amazon Web Services)
• Create Technical design documents as part of project deliverables.
• Analyse Teradata databases to design extract strategies into Big data and cloud platforms.
• These duties are complex and require at least a bachelor's degree in computer science or a related field.

By providing your phone number, you consent to: (1) receive automated text messages and calls from the Judge Group, Inc. and its affiliates (collectively "Judge") to such phone number regarding job opportunities, your job application, and for other related purposes. Message & data rates apply and message frequency may vary. Consistent with Judge's Privacy Policy, information obtained from your consent will not be shared with third parties for marketing/promotional purposes. Reply STOP to opt out of receiving telephone calls and text messages from Judge and HELP for help.
Contact:
This job and many more are available through The Judge Group. Please apply with us today!