Pagerduty

Tommy + the technology of Pagerduty

Home Capabilities Technologies Pagerduty

Pagerduty Examples

Below are some of my projects involving Pagerduty, grouped by company. Click to read more about the relevant projects and chat with me to follow up on any topic you'd like to hear more about!
pull.systems logo

pull.systems

EV Observability + Analytics

Staff Engineer

2023 - 2024

Project: Pull Workbench v1

Upon joining, I came up to speed quickly on the stack of the early version of Pull Workbench, which was very buggy but demonstrated the initial ideas and had a solid set of the latest technologies and patterns established in the codebase, providing for a solid starting point.

I was entrusted to aid our CTO in hiring several additional employees, and so I joined and conducted interviews for the first several months while working with the existing team AI + Full Stack to deliver features and solidify the system, with the aim of keeping it fully working with each merge, after playing a little catch-up to fix the early bugs that worried our business partners, giving them confidence that our team could deliver.

From there, I developed full stack features solo or by pairing with team members, and ultimately led a squad of 5 team members alongside a second squad that together comprised our engineering team.

Much of my time went into authoring complex analytics sql queries using the impressive Kysely library, a fluent, typesafe query builder that we used for our postgres and redshift databases. Given the nature of the product, we needed to make decisions on which queries could be run in real time vs. which queries and subqueries would need to be computed offline as part of a network of airflow dags.

On the ML Ops side I advocated for traceability and reproducibility / determinism of all models and artifacts, and integrated with systems that implemented that, such as Airflow to coordinate DAGs of ML training jobs and Sagemaker's metadata API, which we controlled via model lifecycle automations that produced and stored models, artifacts and metadata that were in turn consumed at runtime or in batch by our analytics stack

On the frontend, I helped us deliver an initial version of the Pattern Editor, a UI and set of APIs that users could use to put together their own patterns of interest, such as looking for certain anomalous ranges of quantities that themselves may be derived from other user-defined patterns. This entailed not only a UI that was DAG-aware but also a layer that converted the json representation of these patterns from the frontend into typesafe kyesely queries to be executed against redshift.

Read more...

Key Results

  • Led 5-person squad delivering Pattern Editor enabling custom anomaly detection workflows
  • Processed 10M+ daily records with type-safe SQL queries using Kysely
  • Improved hiring velocity conducting 30+ technical interviews while building product

Full Details

Appen AI logo

Appen AI

Formerly Figure Eight
Project: DevOps as a Practice

Instead of splitting devops and infrastructure and tests completely separate from development teams, I moved the needle so that product development teams could own more of their own infrastructure and tests, creating less back-and-forth and empowering teams to deliver.

We used Devspace, which meant any dev or team could stand up a reproducible, isolated stack with multiple services and frontends running, in the cloud, as well as modify the definitions of the infrastructure and code themselves, directly, without permission or external team tickets.

This enabled product engineers to do more experimentation and testing thru declarative infrastructure and configuration management while still protecting our production environments, unlocking their shackles and potential as the experts in the software.

At the same time I worked to reduce the outsized role our amazing DevOps team was playing in the day to day management as well as enhancement of environments, which unfairly impeded expert developers by introducing red tape and inter-team processes that didn't add value.

Read more...
Project: ML Platform Enhancements

I ran Appen's ML Platform, which was used by FAANG and many other startups and enterprises to automate and scale their ML practices, including running both supervised and unsupervised workloads, as well as their global annotation workforce which enabled customers to leverage our crowdsourced professionals to elastically obtain labelling and quality checking services for text, voice, image, video and LIDAR annotation, training and validation use cases.

I reported to the CTO and directed multiple full stack teams each with their own tech leads and range of engineering skills to do both regular maintenance and product enhancements using technologies like Sagemaker, React, K8s (Kubernetes), Spark, Kafka, Airflow, Spring(Boot), Ruby, Python, Java, Typescript and SQL.

Maintenance included regular updates to infrastructure, bug fixes, and performance optimizations across the platform. We migrated more and more services to K8s (Kubernetes) and Ambassador as our API gateway, where we could consolidate cross-cutting logic like auth and versioning.

Enhancements included changes to simplify the UX, kill redundant or unused features, add measurement to inform our choices, and larger efforts like Enterprise OAuth.

Read more...
Project: Enterprise OAuth

There were 4 different websites in different technologies, acquired from different companies, and some APIs, that all needed to be unified in terms of sign up, sign in, and sign out, given their existing state of each having separate user stores, including 3rd party vendor users who logged in with vendors and then authed to us with a hidden token.

It was a stalled project, so I started with missing requirements, incomplete designs and misleading progress indicators and focused other leaders and teams on delivery thru tested working software, focusing on tested user stories and on-the-ground learnings as units of progress, instead of large, outdated PRDs waterfall style.

Contributed directly in React / Typescript, Nodejs / express, Ruby on Rails and custom gems, OAuth configuration, Java Spring with runtime loaded SPI implementations from across separate applications domains.

There was a complex architecture at play and teams that did not know each other and weren't working as a single unit, so the landscape was difficult and rife with demoralized team members.

Although my team was to play but one part in many on the project, I realized quickly that there was no single leader or coherent plan, and so there was lots of blame game and treading water.

With permission from our VP of Engineering, I took charge of the teams and worked with product to firm up requirements, and replace the initially conceived solution architecture, which would not have worked and was created in a bit of a vacuum, into one that would actually work, by digging in and running all the services and web apps myself and understanding the multiple data stores and existing auth mechanisms including auth via 3rd party vendors to some parts of the system.

I delivered the project within 5 months and for my efforts was rewarded not long after with a promotion.

Read more...

Key Results

  • Reduced deployment lead time by 75% enabling product teams to self-serve infrastructure
  • Ran ML platform to support 100K+ annotation jobs daily across FAANG clients
  • Unified authentication across 4 legacy systems reducing login friction by 85%

Full Details

Sourceability logo

Sourceability

Electronic Component Parts Distributor
Project: Sourceability Insights

My PM and the business wanted to illustrate to other teams that a fast-paced, fail-fast approach where we released daily (as opposed to 1-3 times per year) would serve us much better in that we could learn quickly, iterate and pivot, without huge costly investments into products that did not meet expectations or deadlines.

Before hiring my team, I set up a CICD pipeline and basic framework of a site that could sustain a heavy and intense crawl from google.

New hires all released to production on their first day of work - a principle I had brought to the table, that it should be so automated and simple that someone could set up and deploy a small feature within their first few hours of working at Sourceability.

Our parts and datasheets website, which also incorporated proprietary availability and quality scores, was used - within 3 months of inception - to successfully sell a 3 year Analytics API contract to an international multibillion dollar company, as well as driving organic traffic and learning how to scale to sustain google crawls of the hundreds of thousands of electronic component parts in our inventory while scaling down outside of the crawl / high-traffic moments.

  • Full Stack - React, NodeJS, Typescript, Kubernetes, Gitlab
  • Functional Reactive Programming - RxJS, highlandjs
  • Daily Production Deploys - Canary Deployment w/ K8s
  • Constant Collaboration - No “throwing over the wall”
  • CI/CD Automation Pipeline - Every user story gets an instant shareable environment
  • Coaching / Mentoring / Leading diverse team
Read more...

Key Results

  • Secured $3M analytics API contract within 3 months of product launch
  • Achieved 400% increase in organic search index uptake thru SEO optimization
  • Enabled team to deploy on day one reducing time-to-first-deploy from weeks to hours

Full Details

Tommy Sullivan - AI + Full Stack Software Builder + Leader