How we solved authentication and authorization

At AppsFlyer, we provide our customers (app advertisers) an analytics dashboard to measure the performance of their advertising campaign.

Originally this system was created as a monolithic Python web application, however, as time passed and our system evolved, we started experiencing pain caused by the constraints that the monolithic app imposed on us.

To name a few:

• The app had to be deployed as a monolith
• Every endpoint had to conform to the web framework architecture (Pyramid in our case)
• Authorization had to be declared on each endpoint

From the beginning we worked with a service oriented (event driven) architecture on our backend. We wanted to maintain the service orientation benefits on our customers facing web app as well.

The first implementation outside the monolithic app walls was the our retention report. It was developed in Clojure and really left us wanting to release ourselves from the monolithic app shackles.

The new challenges

During the development of the retention reports, we obviously had to implement authentication and authorization. However, to make it feel as part of the whole, it had to be compatible with the current authentication mechanism of the Pyramid web app.

In order to achieve that, we reviewed the Pyramid source code and implemented the flow in Clojure. We also had to implement automatic logout and all the behaviors of the pyramid web app. This kind of authentication mechanism needs to be embedded in each and every service that we’ll create. Including it as a library would solve the problem but it means that every update to that library might require upgrading and re-deploying all services.

In addition, we had to configure nginx to route properly between our pyramid web app and the new retention reports service.

Get the latest marketing news and expert insights delivered to your inbox

The solution

In order to simplify our lives we decided to create a central service called Bouncer. As its name suggests, it stands at the entrance of our system and decides who can access and where he/she can access.

Bouncer can be described as an authenticating (and authorizing) reverse proxy. It draws inspiration from the Gogeta project and extends its concept with authentication and authorization.

For the sake of saving the need to access a central user data service from every backend service, the Bouncer adds a user data header to the HTTP request which is sent to the backend service. This helps reduce the complexity and external dependencies of the backend service.

We created a client library for the Bouncer which each backend service can use to provides an easy to use and consistent API for communicating with the Bouncer in order to:

• Register the backend service location
• Let the Bouncer know about the endpoints exposed by the backend service
• Parse and add the user data header to the request object

Design concepts

Input streams

In order to perform its job, the Bouncer is dependent on external data such as the user and app database which holds the domain data about the users who can log to the system and their relations with the apps.

Instead of going for the traditional architecture of request/reply possibly with caching, we went for an architecture of continuously listening for updates to the this database and manipulating an in-memory representation of this data that fits our use case. I will elaborate on that later (hint: rule engine). As we are working with CouchDB to store our domain data, we get the benefit of using its changes feed as an input stream.

Another stream of incoming data is the routes configuration. We use Consul to communicate this data between the backend services and the Bouncer and use Consul’s blocking queries in order to emulate a real-time input stream.

Convention-based authorization

In order to offload as much burden as possible from the backend service, we decided to make the authorization convention based. The routing uses Clout syntax.

We defined reserved keywords that when used in routes defined by backend services will restrict access to that route on the bouncer level. For example, one of these keywords is :app-id. When used in a route it will proxy the call to the backend service only if the logged user can access that app, otherwise a 401 unauthorized HTTP response is sent back. This we believe will cover most of the authorization cases (80/20 principle).

For backend endpoints that require fine-grained authorization control, the Bouncer adds a header that holds the record of the user data.

Declarative access control

As our domain model evolves and gets more complex with greater access control requirements such as providing sub users account access only to several apps belonging to the account or limiting their data access only to a certain date range, we wanted to keep authorization rules as maintainable as possible.

To achieve that we used a rule engine named Clara Rules, which allows us to keep these rules succinct.

System structure

The diagram above illustrates the structure of the Bouncer and its building blocks. Let’s go through the components and describe their responsibilities in a few words.

Event stream

Opens an HTTP stream to the master users and apps database (the main entities in our domain). It translates the HTTP stream into a core.async channel that is used in process.

Domain

Holds an internal representation of the domain model as used by the Bouncer as a system. The data is stored in an embedded relational database (H2 Database Engine) for rich querying. It also exposes a real-time stream of changes to the internal state of changes in the domain model. The incoming stream of the domain component is of the new state of an entity in the system and its output stream is of domain specific events to the model.

For example, a new input event might look like this:

{“type”: “user”,

“user-id”: “xyz123”,

“is-admin?”: false,

…}

The domain will compare it with the current state stored in the domain model. Let’s assume it is:

{“type”: “user”,

“user-id”: “xyz123”,

“is-admin?”: true,

…}

By comparing the two states it will infer that the the user is not longer an admin, and will send a UserAdminPrivilegesRevoked event to its output stream. The subscribers of the output stream will be able to respond accordingly.

Bouncer

This is the heart of the authorization system. It is subscribed to the Domain’s output stream mentioned above and updates the facts that its internal rule engine uses. The external API it exposes is a simple function that receives a user record and a URL it would like to access and return whether the user is authorized or not.

Proxy

This is the HTTP reverse proxy.

It checks whether a user is:

1. Authenticated
2. Authorized to access the target URL

If these conditions are fulfilled, the request will be directed to the backend service as it is registered by the Service Spec component.

In case the user is not authenticated, it will be displayed with a login page provided by the auth app component.

Service spec

Constantly listens to updates from the Consul for backend services location and configured endpoint URLs.

Auth app

A simple web application that provides a login form handler and a logout handler so that the unauthenticated user will be able to login.

Summary

So what does the future hold for the Bouncer?

Our next mission to provide a user interface for account admins in which they will be able to provide users and groups of users fine grained access to certain resources.

I hope to follow up this post  with another  to continue the story of the Bouncer so stay tuned.