Application Integration

Network based architecture: Fielding, Fowler, and Haussmann

This post intends to draw a parallel between the challenges, decisions and heritage of Haussmann over Paris, and the network-based architecture style defined by HTTP and REST, and software architecture methods designated as microservices. Audience is by design double: either software experts interested in city architecture to illustrate the beauty of HTTP Rest and Continuous Delivery, or anybody wanting to understand the success of the Web, and get acquainted to key web linguos and concepts.

EPISODE I: Who was Haussmann, and his challenge for Paris?

haussmann-pic

Eugene Haussmann was a civil-servant chosen by Napoleon III to accelerate the modernization of Paris in 1853. He reigned over Paris architecture for 16 years. When he took office, Paris UX was awful, probably worth less than a star on the Play or Apple store:

 

 

 

deaths-cholera

Eugène Haussmann

  • High dropout rate (cholera struck every 5 years with 10,000 dead each time),
  • Servers (houses) were congested (up to 1 person per square meter, US translation: 100 people per townhouse floor),

 

19,000 deaths of Cholera in Paris, 1832

 

over-crowded

 

  •  and data centers would work only intermittently: no power (gas, water), no cooling (sewage). No cache (cellar), no proxy (concierge), no streaming (subway)… a UX nightmare.

 

 

 

narrow-street

 Congested servers, crowded apartments in Paris

Outside was even worse: ridden with cybercrime (you got it 🙂 ) in obscure streets, slow with narrow access lines (streets) and unexisting backbones (boulevards), without a shared protocol for polling traffic (sidewalk for pedestrians) or streaming (subway), and no garbage collection (gotcha). Worse, when a user would go from one page to another, TLP was terrible because of these  congested un-protocoled lines, but he would come home with viruses ( 🙂 ) by lack of continuous delivery of patches to the servers  (air circulation and sun down these narrow streets hidden by overly high buildings).

 

Pre-haussmannian street in Paris,

unsafe, ridden with viruses, no cooling

 

To top it off, service was fully interrupted and redesigned regularly (Revolutions in 1789, 1815, 3 Glorieuses in 1830, 1848), without backward compatibility. Although users would benefit from these changes on the long run, they definitely did not appreciate the long laliberteguidantlepeupleperiod of adaptation to the new UI, not to mention calls and escalations to nonexistent service desk (votes for   poor and women). Well, actually, these small access lines made it easy to build DDOS attacks (barricades), a feature the business people did not like (Napoleon III).

 

 

 

‘Liberty Leading the People’, July 28th, 1830 by Delacroix

 

EPISODE II: What properties did Haussmann select for his system?

Some elements of style that Haussmann ultimately selected were actually imposed upon him by his boss, Napoleon III, nephew of Napoleon. My intention here is definitely not to write an apology of Napoleon who spread war for years across Europe, just to point to one feature he introduced in the Process: the Code Civil. The Code Civil details the way people can interact together, things they can and can’t do. It came out of heated debates during the French Revolution. Its publication had an impact similar to the CERN initial paper on HTTP in 1990. Code Civil rules citizens life the same way HTTP is the protocol governing our exchanges on the Web.

code-civil

First Page of Code Civil, 1804

A fundamental principle in the Code Civil is the separation of concern: it defines what is a citizen (Client), and separates its  interests and interaction from companies, associations and the state (servers .com, .org or .gov). Any mixing of interest is unlawful (misuse of social assets), pretty much like HTTP is by design Client Server and Stateless. This also means that a server cannot treat two clients differently (equality), or two servers unequally (this links to the current debate on Net Neutrality):

  • Client-Server: the most fundamental concept allowing for a network-based architecture: the system is considered as a set of services provided by servers (shops, public infrastructure services, associations) to clients (citizens). Once roles and possible interactions are defined, each can evolve independently: a citizen can grow from student to shop-owners, a grocery store to a juice provider. People living in a building are not forced to buy their meat from the store in the same building, and a store-owner may charge whatever prices he would like. This principle of separation of concern allows for citizens freedom to choose, and for entrepreneurial spirit (ability to create, invest, adapt) to service citizens (with food, entertainment, social bonds through associations or culture).

 

  • Stateless: no session state, request from client to server must contain all necessary information for server to process. Whoever the citizen, the server must serve him without knowing more about him than the other citizen. No mixing of genres is allowed between client and server: all citizens are treated equally, and must be serviced the same way for a same request. Citizens cannot be bound to a single service by services creating dependencies or local monopoly over their services. This separation of concerns from one building to another, and one person to a company is a foundation of citizen life even today (usually).

 

palaisdejustice

Palais de Justice, Ile de la Cité. Here to guarantee that client and servers interact statelessly. 

Another feature Haussmann had to take into consideration was the addressing scheme of Paris, defined in 1805, similar to our DNS scheme used for HTTP requests:

  • The main backbone, la Seine, defines the start of every street (our root);
  • Each association (.org), state organization (.gov) and company (.com) can define its scheme within its own domain;

 

numbsyst

Paris numbering scheme, example on an haussmannian building

Wanting to build on Code Civil/HTTP, Napoleon III ego did not tolerate for his capital city be less civilized than London or emerging New York.

napo-haussmannIn terms of performance, his network-based architecture needed to do the job on:

  • Network (streets) performance: throughput (fast pedestrians and carriages), small overhead (no need for a citizen to walk the street with a policeman to protect him), bandwidth (wide street)
  • User-perceived performance: latency (ability to quickly reach ground floor of buildings), and completion (get business done)
  • Network-efficiency: best way to be efficient is to avoid using the street too much. Examples are homeworking (differential data) and news or music kiosques avoiding to hear music only in the Opera or get news from Le Monde headquarter (cache)

 

Napoleon III describing his mission to Haussmann, 1853

As did Fielding, he would also select his architectural style against the following metrics:

  1. Scalable: make it possible for Paris to grow
  2. Simple: citizens, civil servants and visitors would need to understand the way the city worked without a user manual,
  3. Modifiable: ability to evolve in the future through change
  4. Extensible: add new neighborhood without impacting the system
  5. Customizable: specialize a building without impacting others
  6. Configurable: easily modify a building (component) after construction (post-deployment)
  7. Reusable: a building hosting an accounting firm one day can serve as cremerie the next
  8. Visible: to provide best security and auditability of the system, interactions between components needed to be visible (people should see each other in the street)
  9. Portable: style should work well in other regions, with other materials and weather conditions
  10. Reliable: susceptible to failure (no single event could stop water, gas or circulation for citizens)

Looking into the challenges he wanted to address in Paris through his architectural style, Haussmann weighted each of these  properties for his evaluation criteria. Main objectives appeared to be:

  • Low entry-barrier: citizens are not forced to live in Paris, and Haussmann wanted to provide them best possible UX to increase adoption. A citizen needed to be able to simply find an address, and a builder to publish a new reference, allowing for the creation of an initial system very quickly.

montmartre

Low-entry barrier for citizens, Montmartre example of a popular neighborhood in Paris.

  • Extensibility: low-entry barrier would help create the modern community Haussmann wanted, and for  the long-term, Paris needed to be ready for changes in its style to adapt to new technologies.

metro

Paris architecture extended to new technologies, e.g. Metro streaming network

  • Distributed Hypermedia: Paris needed to provide citizens with life experience ranging from music (Opera and kiosque), films (actually theaters), e-commerce (food from les Halles), and health (parks). All these experiences were rich in content and would attract many citizens, so much so that they needed to be distributed across the city.

moulin-rouge

Paris supporting hypermedia, e.g. Pigalle, distributed in various neighborhoods

  • Anarchic scalability: once the first set of new neighborhoods would be in place, the city could grow in one direction or another, at a very large scale, without the need for a centralized control (anarchy) to ensure integrity of the entire system. This required for each building to ensure its own authentication, and be able to inspect incoming traffic through firewalls (double door, concierge).

anarchic-growth

Paris metropole, 10m+ citizens today, developed anarchically

Independant deployment: each server (building) or application (neighborhood) could be deployed independently from the others, without compromising the system. Legacy systems (older neighborhoods that could/should not be changed, eg Notre Dame de Paris) needed to be easily encapsulated to interact and be part of the entire system.

indep-deployment

Independent deployment, with great “legacy” of Notre-Dame de Paris

EPISODE III: Why Haussmann decided that Paris should be HTTP RESTful

After he evaluated architecture-styles (medieval like Guérande, renaissance like Florence, centralized neoclassic like Washington DC) against his performance criteria and each of these 10 properties, Haussmann selected the LCODC$SS&UI style, aka HTTP REST. In his world, LCODC$SS&UI meant:

Client-Server & Stateless: inherited from the Code Civil/HTTP as already mentioned.

Layered: the Paris system would include several networks interacting with each other through common and standard interfaces: the water/sewage network, gas, roofs, and streets/boulevards.

roof-network

Roof network, Paris

sewage system

Sewage and gas network, Paris

streets-network

Streets and Boulevards network, Paris

System allowed for creation of additional networks in the future, for example the Paris Metro in 1900. Each layer could communicate with its neighbor in its layer, and with other layers via standard interfaces (standards to deliver gas and water into buildings, all doors on the street). This allows for one network to keep its integrity by communicating with standard interfaces, while servicing another network, so that final service rendered to a citizen are a combination of service provided via several networks, without the complexity of managing interface with each.

cutview-of-networks

Cut of Paris infrastructures networks : buildings, metro, sewage, water pipes and gas.

Cache: duplication of the result of one user request so that it can be used for other citizens. As servers audience would grow, some servers could decide to go from custom craftsmanship to industrial manufacturing, and the Paris system should allow for manufacturers to conduct business in the city, and distribute their products to citizens. This meant that cellars could be used to store goods (storage), that stores could present a storefront for products (Grands Magasins),

bon-marche

Le Bon Marché, Left Bank Grand Magasin/Mall

and cultural service providers could build venues to service crowds of concurrent citizens (Opera for music, kiosque in parks), again without compromising the integrity of the system.

opera

Opera, a server with cache.

Code On Demand: some communities of users would have the ability to provide service to others, with actions delivered to them dynamically by servers. For example, the food community would take its hourly/daily instructions from Les Halles, and then be able to run its own menu in restaurants for citizens.

les-halles

Les Halles, where all fresh goods arrived in Paris

Uniform Interface: applies the principle of generality to its interface. Each resource has a unique identifier (building address and floors), that users can simply identify in their requests. Whether it be habitation on the 4th floor, student room on the 6th or the store at ground level, each level is expected to provide a certain service and be presented the same way. This feature requires some work to adapt to the new standard, and discipline to stick to it (regardless of constraints), but this uniformity is what makes Paris beautiful, and REST APIs so easy to use and scale.

blueprint

Paris Haussmann uniform façade, blueprint

Whether the service you provide is medicine (doctor’s office), education (school) or finance (bank), you can be hosted in a building following the Haussmann architectural style.  The same street can host all these services without the result being ugly. Homogeneity brings harmony.

hauss-street

A Haussmannian street, uniformity brings harmony

And for those of us who like patterns, this homogeneity also brings beauty from atop.

paris-pattern

Paris Streets and Boulevards patterns

EPISODE IV: Paris System Transformation & Microservices

Defining the best architectural style for his constraints and mission would not bring citizens any value in itself. Haussmann had to execute the plan. In terms of execution, he could not shut down the city, destroy existing buildings in one go, and then recall millions of citizens. He had to transform forcefully, with limited resources, while keeping the system alive and complying with the law, and the values that citizens lived for (Liberté, Egalité, Fraternité). Here we can draw a parallel with a software architect working at a bank today, with thousands of mission-critical applications supporting lots of high-value transactions with strong compliance and security constraints. You cannot start fresh a new system, but need to make sure your system delivers as good UX to your customers, as that of newcomers perfectly atuned to the Web, such as Amazon, Paypal, Apple or Google, or even new players experts in finance models as well as web technologies, eg Lending Club or Mint.com.

If you wish to adopt best practices from the Web, and believe REST is the best integration method, what should you do ? In a similar situation, what did Haussmann do ?

Continuous Delivery

Haussmann made certain that the adoption of his basic building block could scale. This meant that buildings once built with the standard footprint needed to work, evolve and grow continuously to satisfy citizens and operators.  That is why Haussmann tackled all issues that could interrupt delivery: water/sewage for health, gas to avoid crimes at night by lighting street, and boulevards to connect neighborhoods. With such features, each building could be autonomous (self-contained) and grow at its own pace, while always satisfying users. Microservices are the same: small, autonomous, and independently deployable pieces of software. All interactions between microservices need to be based on network call: you need to get out to the street to enter a new building. APIs are the storefront of each building, decoupled from any other service the store may use to provide its service: customers are serviced without any view on where the meat or water come from to provide the service.

storefront

A street in Paris, APIs as storesfront only reachable via the street (network calls)

 Each store can run independently thanks to basic resources such as hosting, cooling (water) and security (lit streets) provided to all in the system. In the process, entire neighborhoods had to be reshaped. For example, Ile de La Cité was a medieval small town, with its own security and poor hygiene in narrow streets, overcrowded buildings and lots of unofficial traffic and activities, a city in the city. Something like a set of applications glued by a middleware that made lots of sense 10 years ago. Haussmann creation of larger streets meant the Cité was almost completely rebuilt.

cite_pre-hauss

Ile de La Cité pre-Haussmann: no sewage, dark streets, buildings protected against carriages, no boardwalk

cite_today

Ile de La Cité today, sacred buildings (legacy mainframes) enshrined in RESTful neighborhoods

Zoning

Haussmann wanted the city to grow and evolve autonomously, but also recognized that some services would more easily develop if gathered and designed together. Haussmann prepared entire neighborhoods to live for a function (Les Halles for food distribution, Grands Magasins for luxury shopping), or to host monuments and the circulation around it (Opera, Bastille, Chatelet, Parks). Specifities of these functions could then be accounted for, without impacting the entire system. You could not slaughter a cow in the middle of the street, you would have to build a slaughterhouse in Les Halles for that.

zoning

Example of zoning: quartier Panthéon

Lead by example

For his style to diffuse within the city, Haussmann started by creating template buildings around the city, financed on public debt, exhibiting all the advantages of his style not only for citizens inhabiting them, but also stores using them, or financier entrepreneurs looking at the ROI. This created a second, much bigger, wave of investments from private investors also borrowing money from booming banks to service more people and corporations. While Haussmann acted for 15 years on over a hundred buildings, all of Paris adopted the style as the benefits of it became obvious for all land owners and citizens alike. This is the power of example once continuous delivery is in place.

opera-zoning

Opera neighborhood, zoning and boulevards giving incentives for privates investors to align their building to the new style, to their own benefit too

Cut corners

A combination of exemplar and visible implementation started a transformation process which led Haussmann to begin focusing on corner buildings. To facilitate circulation of air between streets, Haussmann wanted for all corner buildings to be cut. This was quite painful for land owners being left with weird rooms in these corners, but allowed for the City to impose decisions only on the toughest pieces, while creating buildings visible from two streets at the same time. With corner building being renovated and exploiting latest technology then available (water, sewage and gas then, containers and cloud today), all owners in the neighborhood now wanted to benefit from the same technology they could rent at a higher price than their older buildings, triggering a snowball effect in the neighborhood and over the City.

corner

Corner Building : exemplar, hard to implement, but driving change.

Polyglot

Within a building, each owner could select its own architecture and colors, in accordance to its needs and skills. Pretty much like microservice inhabitants who can select the database technology and language they wish, even though architects must ensure resilience of each service with recommended standards. This tolerance for inner worlds even allowed even legacy XVIII Hotel Particuliers to be hidden behind Haussmannian façades, a good idea for Cobol mainframes.

hauss-block

A Haussmann block with Uniform Interface, with Hotel Particulier in the middle

EPISODE V: Haussmann Web, 160 years later

What a pleasure to assess sustainability with real and accessible data! A future architectural fan (probably a droid) may look into the Web the same way in 2150 AD.

Performance. In terms of UX performance, Haussmann did succeed. Epidemics ceased, traffic circulation improved. In just 15 years, Paris went from under-civilized, to a model city.  For example, thanks to the lights installed in the streets, Paris became the City of Lights. Paris today attracts 20 million visitors every year.

Scalable. Paris population doubled to 2.2 millions today. What else.

Reproductible. Boulevards and Parks were emulated in Brussels, Rome, Vienna, Stockholm, Madrid, Berlin, Cologne and Barcelona, and into the design of Central Park or central Chicago.

chicago

Chicago after the Burnham Plan, designed in 1909

Sustainable. If we define sustainability as the ability to face time, there is no question Haussmann succeeded, even compared to “modern” concrete projects hastily built in the 1960s who did not last 30 years, with poor quality of life for citizens. In building durable buildings in a resilient architecture, Haussmann did well for investors at that time, and for the planet in the long run (best way to help our planet is to avoid making, subject for a future blog).

destroyed-building

Lifespan of alternative architectural style. La Courneuve, 1986, Project destroyed after 22 years of existence. It had taken 10 years to build it.

Evolvable. Many years after Haussman left office in 1869, Paris accepted new technologies such as the subway (1900). Subway provided scalability and efficiency to the City, very much like data streaming today, gathering updates for many users at the same time. Paris also hosted iconic new constructions such as Tour Eiffel (1889), Grand Palais (1897) or Alexandre III Bridge (1900). I should not draw much opposition in saying they integrate pretty well in the City.

eiffel-construction

Tour Eiffel under construction, evolution built years after Haussmann had left office

EPISODE VI: Day-to-day life in the city//dev

This section regroups anecdotes contributed by the community on hacker’s life // Paris. It is work in progress, by definition.

Initial comment: same study would be interested over London and New York.

Got confirmed that Paris structure had been selected as a sustainable architecture: need to find the sources. Angle/metrics is consumption per inhabitant in electricity.

Hopefully also covers gas, electricity, waste…

Carbon footprint: 6.5m tons for 2.

Denis BAUPIN, http://denisbaupin.fr/

On Error Handling (@rit 2015-06-05)

Even when the protocol is well written, and APIs clean, you can still wreck your dev if you do not handle error properly. Ignoring them or not handling them properly can throw your stream through an API.

Montparnasse train station, 1895

montparnasse-train

 

EPISODE VII: Paris-Web dictionary & Further Readings

Code CivilParisBoulevardsStreetsStreet NumbersLight in the streets

Concierge

Cellar

Carriage

Subway/Metro

Homeworking

Pierre de Carrière

Compagnon du Devoir

Haussmann

Beaux Arts

Façade

Haussmann Façade

Utilities (gas, water)

Annonces Parisiennes

Grands Magasins

Opera

Theaters

Prefecture

Bibliotheque Nat.

Senat

Assemblee Nationale

Louvre

Bois de Boulogne

Bois de Vincennes

Bon Marché

Eiffel Tower

Latin Quarter

Left bank spirit

Right bank spirit

HTTPWebInternet BackboneAccess LinesURIAuthentication

Proxy

Cache

Request/Response

Streaming

Differential update over JSON Patch

JSON

API Developer

Architect

Apple, Android and JS Developers

API

RESTful API

Cloud/PaaS

Criteo

Amazon

Deezer

Netflix

NSA

Wiki

W3C

IETF

Pinterest

MineCraft

Facebook

Ebay

Twitter

Udacity, Coursera MOOCs

Android

Apple iPhone

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

apps-paris

Paris Web in 2015, apps icons on locations of main Paris buildings

Do go deeper (not exhaustive, please leave comments on the blog with additional resources)

  • Paris Architecture & Urbanism: Musée Carnavalet, Cité Chaillot
  • Haussmann: Haussmann à Paris : Architecture et urbanisme Seconde moitié du XIXe siècle, Book, 2012
  • HTTP: Tim Berners-Lee & Team, Cern, 1990
  • REST: Network-Based Architecture Style, Thesis, Fielding 2000
  • Continuous Delivery : Farley & Humbl, 2010
  • Microservices : Microservices in a Nutshell, Fowler & Lewis, Thoughtworks, 2014
  • Web Corp Organization: How Google Works, Eric Schmidt, 2014
  • Design & UX : Jonathan Ive, Apple by The New Yorker, 2015
  • Microservices : Building Microservices, Sam Newman, O’Reilly Book, 2015
  • Code Civil : on github by Steeve

And to stay tuned on best practices (again, not exhaustive, please add up) :

  • UX/API articulation at Netflix : Dan Jacobson blog
  • API best practices: Kin Lane APIEvangelist, Jason Harmon, ProgrammableWeb, Mike Amundsen, Steve Klabnik, Mark O’Neill,  Runscope inc. Neill & Darrel , Guillaume Laforge, and many more
**Original source: streamdata.io blog