Senior Data and Analytics Engineer

Job Description

The Senior Data and Analytics Engineer will be responsible for translating complex business and data requirements into scalable, secure, and high-performing analytics and data engineering solutions for the Operations (Supply Chain) organization. The role will partner closely with internal DATA (Data, AI, Transformation and Automation) teams, and cross functional Business Technology Solutions (BTS), business stakeholders, architects, and delivery teams to understand strategic priorities and design, build, and deliver modern data solutions that enables analytics, business intelligence, and AI use cases.

This role requires a strong combination of hands-on technical expertise and strategic thinking, understanding the North Star platform architecture to implement practical engineering solutions while ensuring alignment with enterprise architecture, development standards, data governance, security, and engineering best practices.

The individual will also provide technical leadership throughout the solution lifecycle, proactively manage technical risks and dependencies, and communicate effectively with both technical and business stakeholders.

Responsibilities

• Hands-on engineering and solution delivery: Design, develop, test, deploy, and support production-grade data pipelines and data products for structured, semi-structured, and unstructured data. Build for both batch and near-real-time/streaming patterns, with reusable components, parameterized frameworks, and automated recovery.
• Lakehouse and modern data warehouse architecture: Implement solutions using Amazon S3, Apache Iceberg/open table formats, AWS Glue Data Catalog, Amazon EMR/Spark, Amazon Redshift and Redshift Spectrum. Apply medallion patterns (Bronze, Silver, Gold), dimensional and domain-oriented modeling, workload isolation, partitioning, clustering/sort strategies, and performance optimization.
• Data product and data mesh enablement: Translate business domains into discoverable, interoperable, and governed data products. Define product contracts, ownership, metadata, quality expectations, service-level objectives, lineage, and approved consumption patterns for dashboards, self-service analytics, APIs, data science, and AI.
• Ingestion and orchestration: Engineer reliable ingestion from enterprise applications, databases, files, APIs, and event streams. Develop orchestration using tools such as Apache Airflow, dbt, AWS Step Functions or equivalent approved platforms, with dependency management, idempotency, observability, alerting, and backfill capability.
• Data quality, governance, and security: Embed automated data-quality controls, reconciliation, schema validation, lineage, cataloging, retention, encryption, least-privilege access, and auditability. Partner with data governance, security, privacy, architecture, and platform teams to meet AbbVie standards and regulated-industry expectations.
• DevSecOps and engineering excellence: Establish source-control, code-review, automated testing, CI/CD, infrastructure-as-code, environment promotion, release management, and operational monitoring practices. Improve reliability, cost, performance, and developer productivity through measurable engineering standards.
• Business partnership and technical leadership: Understands organization's vision, goals, and strategies and partner with DATA, BTS, Operations stakeholders to translate complex requirements into scalable designs. Provide technical direction, resolve design and delivery issues, communicate risks and trade-offs, and contribute to portfolio prioritization and roadmap decisions.
• Operational ownership: Own solutions through the full lifecycle, including production readiness, deployment, support transition, incident resolution, root-cause analysis, technical debt reduction, and continuous improvement. Work with minimal oversight while escalating material risks early.

Qualifications

Required:

• Bachelor’s Degree in Computer Science, Information Systems, Data Engineering, or a related field plus 7 years of relevant experience; Master's Degree plus 6 years of experience; PhD plus 2 years of experience.
• Respective years of hands-on experience designing and delivering enterprise modern data warehouse, Lakehouse, data engineering, and analytical solutions.
• Advanced proficiency in SQL and Python, with strong hands-on experience in PySpark or Scala and distributed data processing on Apache Spark/EMR or a comparable platform.
• Demonstrated experience building large Lakehouse and modern warehouse solutions using Amazon S3, Apache Iceberg or comparable open table formats, AWS Glue, Amazon EMR, Amazon Redshift, and Redshift Spectrum.
• Proven experience with medallion architecture, data modeling, schema design, incremental processing, change data capture, partitioning, performance tuning, and cost optimization.
• Experience developing production-grade batch and streaming data pipelines, including orchestration, retries, backfills, observability, alerting, and operational support.

…