Inception: projects within projects

When we converted from PHP to Python I mentioned how deceptively large (lines of code) the site had grown with so many views and features. We've since built on literally the same code base (a decision I hoped to blog about some day, alas) and the line count continues upwards. The MVC paradigm is one of the most common out there, easy to understand, and built in to our frameworks so it was naturally a good choice. However, as the site continues to expand we've had to review what the best architecture for the code would be and the main point that stands out is that our entire site and its services are a monolithic blob of code. Both sites, all of our APIs, all of the PayPal code, the administrative tools, the developer control panel, the reviewer queues, the list goes on - it's all in one repository and changing one means potentially affecting all of them.

With big plans for 2013 and no signs of slowing our code expansion we've hit the point where it makes more sense to break the site up into smaller projects. We've already started with our payment processor and metrics processor and more are soon to follow. These projects will be tied together around a central API on the back end. We've had partial front end APIs for years (like search) which we will expand to make the entire site accessible via API. At that point our front end code will be completely separated and can be optimized to speak directly to JSON APIs instead of pulling HTML with every request (something which the developers have been demoing the past couple weeks at our show and tells). A rough diagram is below, subject to change, of course.

Marketplace Architecture Diagram

Some benefits of this approach include:

There are a couple of downsides: maintenance/configuration of each project and integration testing. The former is diminished a bit because not every developer works on every project so with some prudent mocking and a few virtual machines there should be an easy flow there. The second is something we currently neglect but will become critical with these changes so expect some solid developments there. With all the other work on our schedule I think we'll be touching parts of this project in every quarter this year. If you hear the codename INCEPTION it's referring to this project.

Post a comment

All comments are held for moderation; basic HTML formatting accepted.