IODA Architecture - naming actual problem with Microservices

Few days ago I've run into this article on InfoQ:
http://www.infoq.com/news/2015/05/ioda-architecture
I found it amusing enough to dive deeper here (link taken from the article):
http://geekswithblogs.net/theArchitectsNapkin/archive/2015/04/29/the-ioda-architecture.aspx

I won't keep you guessing any longer - my amusement isn't about the overall idea of IODA or this horizontal to vertical (or the other way) thingie - IMHO the best thing about IODA is that it very clearly addresses the problem of (micro)service integration & dependencies between (micro)services.

This particular topic is usually (in books, articles, presentations, even warstories) skipped or skimmed through very quickly. Sam Newman may have dedicated a full chapter (#4) of his great book to integration, but I find it more shallow (except of some tech specs which are less crucial) than the rest of the book (orchestration VS choreography being the best part).

Same applies to the microservice talks from conferences I've attended - even author of the best of them (Adrian Trenaman "Scaling microservices at Gilt" - http://www.ustream.tv/recorded/61442163) has pretty much skipped the issue of scaling interactions & dependencies (while number of microservices grows).

Integration-shmintegration ...

... but what's the actual question / issue?

  1. If you let microservices to communicate with each other - what do you do to prevent distributed big ball of mud?

  2. If you don't let microservices to communicate with each other, do you put (& duplicate) integration logic in your applications?

Those two may be less relevant in 100% asynchronous scenarios, but once you have request-response sync communication, it becomes a matter of life & death (well, sort of).

And Ralf Westphal seems like the first one that doesn't pretend that the problem doesn't exist - he makes a clear separation between bare (dependency-less) operations & integrations that are all about "putting all things together":

  • operations can refer to any other operations or integrations, they work on their dedicated portion of data & they encapsulate actual logic
  • integrations orchestrate operations & other integrations (actually author didn't build any assumptions about the hierarchy of integrations (tree? acyclic graph? whatever?), but all his diagrams say differently); they are supposed NOT to contain any logic (but I believe it's kinda oversimplification) & NOT to manipulate any data directly

The last (fourth) element (API) is completely irrelevant in the context of this blog post, so I'll skip it.

IODA style

I like the overall idea because ... it's very similar to something we've started trying about 3 years ago. There were few clear differences though:

  • we've limited the number of allowed levels of integrations & we've introduced a rule that each level can call only levels below
  • we've introduced domain-based module split for operations & integrations (tricky, I know) - aka horizontal siloing
  • some "infrastructure" operations were allowed to be called from other operations (for instance: security)

Both of these points were aimed to make sure that we retain control over system growth & the coupling within. Actually Westphal's idea of IODA's similarity to Sierpinski's triange freaks me out a bit: I've seen call hierarchies getting out of control (thoughtlessly) very quickly & with a very serious consequences (of absolute & utter unmanageability).

So, as we're doing something similar, an obvious question appears instantly: did it (does it) work? I'll answer in a roundabout way: yes, it helped a lot in keeping out functional logic tidy & organized, but the actual challenge & a real monster to bit was somewhere else:

  • asynchrony
  • sensible API management
  • tons, tons, tons, tons of automation - especially around deployment)

Pic: http://www.shapeways.com/product/LRU8ESVXZ/sierpinski-tetrahedron-level-5