Principal Software Development Engineer - Monitoring

Thousand Oaks, CA
Aug 03, 2021
Required Education
High School or equivalent
Position Type
Full time


If you feel like youre part of something bigger, its because you are. At Amgen, our shared missionto serve patientsdrives all that we do. It is key to our becoming one of the worlds leading biotechnology companies. We are global collaborators who achieve togetherresearching, manufacturing, and delivering ever-better products that reach over 10 million patients worldwide. Its time for a career you can be proud of. Join us.

Principal Software Development Engineer Monitoring

Would you like to take your full stack engineering skills and team leadership skills and help deliver the world's most sophisticated and life-changing biotech medicines by building creative and ground-breaking software frameworks? Amgen is using cloud technology, DevSecOps frameworks, and migration and continuous deployment pipelines to build new drug therapies for patients that address grievous illnesses. We are seeking a Sr Mgr Software Development Engineering Observability & Monitoring to fill a position in Amgen's Global Infrastructure Services Group. Our small product team is a high-energy, tight-knit group who builds our DevSecOps framework used globally by Amgen development teams.


What you will do

Lets do this. Lets change the world. In this vital role you will focus on innovating and developing our current full stack, open-source monitoring service as well as extending its value to all users of our internally developed DevSecOps framework. Our mission is to meet Amgen's software development needs as well as our DataOps needs for AI and ML data scientists. Our frameworks are built using open-source and AWS services. They manage custom code development, DataOps, as well as test and deploy Commercial-Off-The-Shelf (COTS) applications into Amgens cloud environment. We are currently extending our capabilities to integrate serverless and microservices CI/CD pipelines with AWS.

Role responsibilities:

Be the owner and lead engineer for our end-to-end observability and monitoring service. Develop, refactor, research and improve its capabilities across our cloud security architecture and our continuous CI/CD pipeline. Understand how the instrumentation trade-offs affect the value users get from the service.

  • Provide technical leadership across Amgen to enhance the culture of innovation, automation, and solving difficult scientific and business challenges. Technical leadership includes providing solutions based on DevSecOps, DevOps, and DataOps frameworks with full stack monitoring and observability imbedded into our frameworks.

  • Be constantly curious and feed your passion and interests in groundbreaking technology. Use your given time to look for new and innovative ways to do business differently, better, and more cost effectively.

  • Lead selected efforts to design, build, and develop rapid Proof-of-Concept (POC) solutions and services. Quickly and iteratively prove or disprove the concepts being considered.

  • Evaluate, decide, and implement new open-source software that will enhance our observability tools and scale to meet our global needs

  • Be an integral member of the core DevSecOps/DataOps development team and help develop, test, deploy, and operate the service 24x7x365 globally.


What we expect of you

We are all different, yet we all use our unique contributions to serve patients. The [vital attribute] professional we seek is a [type of person] with these qualifications.

Basic Qualifications:

Doctorate degree


Masters degree and 3 years of Information Systems experience


Bachelors degree and 5 years of Information Systems experience


Associates degree and 10 years of Information Systems experience


High school diploma / GED and 12 years of Information Systems experience

Minimum Qualifications:

  • Minimum of 8+ years overall IT industry experience
  • Minimum 3+ years in architecture role for modern monitoring/observability system
  • 10+ years of experience architecting, building, or operating cloud-based applications or infrastructures
  • 3-5 years experience designing, developing, deploying, and operating observability frameworks using open-source tools such as InfluxDB, Grafana, Telegraf, FluentD, Prometheus, Loki, Elasticsearch, etc.
  • Experience designing, implementing, and operating observability and monitoring services in public cloud environmentsAWS preferred
  • Strong automation/scripting or programming skills - proficiency in Python, NodeJS or Java is a plus.

Preferred Qualifications

  • Expert level experience in monitoring and logging technologies, both open source and closed source (e.g. Splunk, SumoLogic, ELK)
  • Experience in RBAC and user based security services such as ISE, Radius, LDAP, and AD.
  • Experience in instrumenting applications with user analytics frameworks like Matomo and Google Analytics
  • Working knowledge providing OpenTracing frameworks using tools such as Jaeger; Zipkin experience considered equivalent.
  • Fundamental knowledge of TCP/IP stack, application protocols (DHCP/DNS/HTTPs) and networking concepts (HSRP/NAT/VPN/VLANs/802.1x/Wireless/Clustering/High Availability/Load Balancing).
  • Fundamental knowledge of Configuration Management and Automation tools, with experience in Terraform, Ansible, Chef, Puppet, Jenkins.
  • Designing and implementing CI/CD pipelines, Infrastructure provisioning and management.
  • Extensive and practical knowledge of native cloud security controls and design patterns for AWS, GCP, or Azure
  • Experience in distilling complex technical challenges to actionable decisions for stakeholders and guiding project teams by building consensus and mediating compromises when necessary.
  • Contributed to Open Source - your public Git repos/contributions show good examples of giving back to the community.
  • Experience in highly regulated industry will be a plus
  • Experience of working in global distributed team will be a plus


What you can expect of us

As we work to develop treatments that take care of others, so we work to care for our teammates professional and personal growth and well-being.

  • Vast opportunities to learn and move up and across our global organization

  • Diverse and inclusive community of belonging, where teammates are empowered to bring ideas to the table and act

  • Generous Total Rewards Plan comprising health, finance and wealth, work/life balance, and career benefits

Apply now

for a career that defies imagination

In our quest to serve patients above all else, Amgen is the first to imagine, and the last to doubt. Join us.

Equal Opportunity Statement

Amgen is an Equal Opportunity employer and will consider you without regard to your race, color, religion, sex, sexual orientation, gender identity, national origin, protected veteran status, or disability status.

We will ensure that individuals with disabilities are provided a reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request an accommodation.

Join Us

If you're seeking a career where you can truly make a difference in the lives of others, a career where you can work at the absolute forefront of biotechnology with the top minds in the field, you'll find it at Amgen.

Amgen, a biotechnology pioneer, discovers, develops and delivers innovative human therapeutics. Our medicines have helped millions of patients in the fight against cancer, kidney disease, rheumatoid arthritis and other serious illnesses.

As an organization dedicated to improving the quality of life for people around the world, Amgen fosters an inclusive environment of diverse, ethical, committed and highly accomplished people who respect each other but compete intensely to win. Together, we live the Amgen values as we continue advancing science to serve patients.

Amgen is an Equal Opportunity employer and will consider all qualified applicants for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, protected veteran status, or disability status.