Requests and how to handle them

During the past month I have been looking into implementing a simple statistics page to visualise the website traffic of this site. There are a couple of ready-made python packages that does this for us and the package flask-statistics suited my taste. After researching the package some more I found that I wanted to make some slight adjustments and that the author had stopped the development of the package. So I decided to try and implement the source code into the website myself!

The basis of this type of endeavour is to tear down the incoming requests and store the data of the request in a database. Luckily, the flask API implements three methods which we can use to call functions before, after and during a request. See my implementation of these three functions below:

from flask import g, request, Response

    def before_request(self) -> None:
        """Function called before handling any request."""
        g.request_date = datetime.datetime.utcnow()

    def after_request(self, response: Response) -> Response:
        """Function called after handling any request."""

        return response

    def teardown_request(self, exception: Exception = None) -> None:
        """Function called on every request."""
        try:
            obj: dict = {}

            obj["date"] = g.request_date
            obj["path"] = request.path
            obj["remote_address"] = request.environ.get(
                "HTTP_X_REAL_IP", request.remote_addr)
            obj["referrer"] = request.referrer

            if "static" not in obj["path"]:
                self.db.session.add(self.model(**obj))
                self.db.session.commit()

        except Exception as e:
            self.app.logger.warning(f"Error tearing down a request: {e}")

The before_request method temporarily stores the date and time of the incoming request in the application context. Flask implements this through the g variable which is a very useful feature for storing temporary data for a certain action; in this case while handling the request.

The teardown_request method is used for just that, tearing down the request and permanently store its related data in our database. In my case I'm only interested in the time of the request, the website path which was requested, the IP-address the request came from and from where the requester was referred from. Since I have an animated favicon every user makes several requests to the "static"-folder which is of little importance and should not be stored in the database.

Lastly, the after_request method is called after the request has been teared down. In my case this is a dummy function but could be used attach cookies to each request as mentioned in this stackoverflow answer.

As I have an app factory in my Flask app, as briefly mentioned in the previous blogpost, I decided to implement a init_app method which takes the app, the database and the model from the factory and attaches the above methods to the app.

class Statistics:
    def init_app(self, app: Flask, db: SQLAlchemy, model: Model) -> None:
        self.app = app
        self.db = db
        self.model = model

        self.app.before_request(self.before_request)
        self.app.after_request(self.after_request)
        self.app.teardown_request(self.teardown_request)

With this implemented I now can see the some basic statistics of the requests sent to the website such as the amount of requests per path over a time period. Hopefully this was insightful and thanks to HealYouDown for laying the groundwork for me to learn from.

Cheers,
Theo