The Cloud Data Architect will design solutions that enable data scientists and analysts to gain insights into data using data-driven cloud-based services and infrastructures. This role will be subject matter experts and will be responsible for the stakeholder management and technical leadership for data ingestion and processing engagements. A good understanding of cloud platforms and prior experience working with big data tooling and frameworks is required.
- Building technical solutions required for optimal ingestion, transformation, and loading of data from a wide variety of data sources using open source, AWS, Azure or GCP ‘big data’ frameworks and services.
- Working with the product and software teams to provide feedback surrounding data-related technical issues and support for data infrastructure needs uncovered during customer engagements / testing.
- Understanding and formulating processing pipelines of large, complex data sets that meet functional / non-functional business requirements.
- Creating and maintaining optimal data pipeline architecture.
- Working alongside Cloud Data Engineers, Cloud System Developers and Cloud Enablement Manager to implement Data Engineering solutions.
- Redesigning and building a new data ecosystem on the cloud.
- Extending on-premise data supply chain and modernizing data supply chain on the cloud.
- Collaborating with the customer’s data scientists and data stewards during workshop sessions to uncover more detailed business requirements related to data engineering.
- Building business cases to support MDM efforts and adoption.
- Providing subject-matter-expertise in the assessment of solution proposals and concepts related to Master Data Management; Data Quality; Data Datalogging; and Metadata management and publication.
- Facilitating cross-functional discussions working groups.
- Identifying, defining, and communicating success factors.
Required Skills & Qualifications
- 8 years experience in building scalable end-to-end data ingestion and processing solutions
- 8 years EIM Principles: architecture, sourcing, ETL, data modeling, pipelines and connectors, integration hubs, data access (SOA, API, SQL), platforms (traditional servers, cloud, hybrid), database types (traditional, proprietary MPP)
- Excellent understanding of EIM principles, capabilities, and best practices with extensive experience and domain knowledge in the areas of Master Data Management, Reference Data, Data Quality, Metadata Management, and Data Governance capabilities
- Good understanding of data infrastructure and distributed computing principles
- Good understanding of data governance and how regulations can impact data storage and processing solutions such as HIPAA and FedRAMP
- Ability to identify and select the right tools for a given problem, such as knowing when to use a relational or non-relational database
- Working knowledge of non-relational and row/columnar based relational databases
- Confidently taking responsibility for the technical output of a project
- Ability to quickly pick up new skills and learn on the job
- Metadata Management experience including business definitions, business processing rules, data lineage
- Master Data Management experience, with a good understanding of MDM full-lifecycle concepts/techniques
- Understanding of database and analytical technologies in the industry including massive parallel processing and NoSQL databases, cloud data warehouse and data lake design, BI reporting and dashboard development
- Delivering production scale data engineering solutions leveraging one or more cloud services
- Experience with object-oriented and/or functional programming languages, such as Python, Java and Scala
- Experience with Machine Learning toolkits