When Realtracs began offering our product in new markets, it meant more than servicing additional customers. It required being able to support a lot more listing data. Every new market brought listing data that our new customers expected to access on day one. At the same time, our existing platform already housed millions of listings.
The issue wasn’t just the number of listings, but the combination of high volume and the near-infinite ways users can search them that made scaling so complex.
The Confidence Problem
Each time we onboarded a new market, we faced the same uneasy question:
How will the system perform now?
Our listings architecture at the time ran on an expensive Microsoft SQL Server cluster. As the dataset grew, performance degraded. Query complexity increased. Indexes became heavier. Maintenance windows grew longer. Costs climbed. The infrastructure wasn’t failing, but it wasn’t designed for unlimited growth either. We needed more than incremental improvement. We needed architectural change.
Rethinking the Foundation
We moved to PostgreSQL. Postgres offered two critical advantages:
- Significantly lower infrastructure cost
- Native table partitioning for scalable data management
Partitioning allows large tables to be split into distinct, manageable segments. But partitioning only works well if there’s a predictable pattern in how data is accessed. That forced us to step back and ask: Can we limit the number of listings any single request needs to search?
Designing Around Real Usage Patterns
The breakthrough came from analyzing user behavior. Two insights stood out:
- Market data access is restricted to members of that market.
- Most searches focus on recent history.
This was pivotal.
The number of listings from the past two years is dramatically smaller than the total historical dataset. That meant we didn’t need every query to touch decades of data. If we could design the system to naturally operate within those boundaries, partitioning would become incredibly effective.
Encouraging the Right Constraints
Architecture alone wasn’t enough. We also made product-level changes to encourage better query behavior. As an example, when searching off-market listings, we began enforcing a date range instead of allowing open-ended historical queries. This forced users to take a direct action to query the much larger set of historical data.
By combining:
- Postgres partitioning
- Market-based data isolation
- Time-based access patterns
- Thoughtful product constraints
We transformed our listings platform.
The results:
- Lower infrastructure cost
- Dramatically improved and predictable performance
- High confidence when onboarding new markets
- A system designed for unlimited growth
Performance, Predictability, and Unlimited Scale
Instead of wondering how the system would behave after adding millions more listings, we now know because we didn’t just scale infrastructure; we architected for growth.