Cutting core-service load by 70%

Context

myITProcess is deeply integrated with the company's core service that owns the Organization and User hierarchy. Reports across the platform required a full sync of all organizations, replicated across every child product. The pipeline was scaled out across eight Kubernetes pods just to keep up.

The problem

The sync took ~24 hours and generated over 100,000 requests per day, most of them re-fetching data that had not changed. The shared core service was saturated by sync traffic, hurting stability for adjacent products that depended on it. Capacity planning was reactive — onboarding a new tenant meant adding more pods.

Approach

• Profiled the existing pipeline and confirmed most traffic was re-fetching unmodified data.
• Negotiated a contract change with the core service API team to expose a `modifiedSince` parameter, enabling true delta sync.
• Migrated the synchronization layer to AWS Lambda — serverless, event-driven, and cheap enough to stay under the 1M-invocation free tier.
• Re-sized the pod topology against the new workload profile.

Outcomes

Load on core service

-70%

Sync time

24 h → 15 min

Kubernetes pods

8 → 2

Stack