Smart Endpoints but not so Dumb Pipes

Martin Fowler’s famous approach “Smart endpoints and dumb pipes” for integration middleware was accepted as a solution to eliminate single points of failure in the integration middleware, making them lean and singly focused on delivering the message. Trivial transformations, such as between formats (e.g., XML to JSON) or between protocols (e.g., SFTP to HTTPS) and routing them to the appropriate consumer were their only responsibilities. The rest of the VETRO pattern was not necessary to be implemented. This was also an answer to places where the integration middleware had gone beyond its responsibilities and taken over tasks of implementing business logic inside interfaces and services. While most organizations appreciate that “Smart endpoints and dumb pipes” is the way to go for application integration, it has not been easy to implement. The reason for this being the varied application landscape and organizations’ culture.

Application landscapes with custom applications are able to implement this approach relatively easily. They make their applications (endpoints) smart by handling all the validations, enrichment and business logic in them. The integration middleware is only used to distribute data. Modern applications built following Microservices patterns are a good example of this.

Application landscapes dominated by packaged and COTS applications find the approach difficult to implement as changing the application(endpoint) to make it “smart” is difficult. They tend to go back to the older approach of putting the business logic and validations in the integration middleware. They are in that sense, dumb endpoints. Legacy and some packaged applications are examples of this.

So, what is the solution in application landscapes that are full of applications that cannot be easily changed?

One approach is to build components that fill the gap in the application functionality for business logic and validation separate from the integration middleware. These components can either be built in the same technology as that of the integration middleware or in a different one but they need to be hosted and managed separately from the integration middleware. The important aspect is that the ownership of the components should be with the application teams. At least the business functionality, to start with, should be owned by the application teams. The integration team could host and manage the components until such time that the applications can be updated or migrated to include the functionality. This would serve like an intermediate step and ensure that the integration middleware would become lighter and “dumber” at some point in time and help in realizing the Smart endpoints and dumb pipes approach. If not, the pipe would continue to remain smart and the endpoints dumb.

Cross posted at https://www.linkedin.com/in/daneshzaki/