fix/enhancement - Fix slow auth tokens. (#164)

Current auth tokens are slow because they use the user's password (hashed) as a uniquifier (the
user id isn't really enough since it might be reused). This requires checking the (hashed) password against
what is in the token on EVERY request - however hashing is (on purpose) slow. So this can add almost a whole second
to every request!

This PR introduces a new UserModel field - fs_uniquifier - that if present in the UserModel will be populated and used rather than the password. This results in 50x reduction in time when authenticating via token.

Furthermore, the actual token verification has been moved from request_loader into the UserMixin - so
that it could be overridden (creating the auth_token already was in the UserMixin).

Note that this does require a DB migration to add the field. The fsqla model has been updated, and docs
describing at least one way to migrate the DB have been added.

2 new backwards compatibility configurations have been added that can revert some new default behavior.

First - in the past- the auth token was included always in JSON responses to login, reset and change -
even if the caller was a browser. This is
really not great since auth tokens may have very long expire times (or none) and it shouldn't even be sent if not needed.
Now, by default, the auth token is NEVER returned - the caller may request is during login, reset, or change by adding the
'include_auth_token' query param.

Second, since auth tokens used to be checked against the hashed password - changing a user's password meant that any
outstanding auth tokens would be invalidated. That seems like strange behavior - so by default, tokens that are verified
with the new fs_uniquifier won't be invalidated just because the user's password changes. The
BACKWARDS_COMPAT_AUTH_TOKEN_INVALID config variable will cause the fs_uniquifier to be changed whenever the user's password changes, thus restoring the older behavior.

closes: #156

CHANGES.rst

@@ -8,11 +8,15 @@ Version 3.3.0
 
 Released TBD
 
+**There are several default behavior changes that might break existing applications.
+Most have configuration variables that restore prior behavior**.
+
 - (:pr:`120`) Native support for Permissions as part of Roles. Endpoints can be
   protected via permissions that are evaluated based on role(s) that the user has.
 - (:issue:`126`, :issue:`93`, :issue:`96`) Revamp entire CSRF handling. This adds support for Single Page Applications
   and having CSRF protection for browser(session) authentication but ignored for
   token based authentication. Add extensive documentation about all the options.
+- (:issue:`156`) Token authentication is slow. Please see below for details on how to enable a new, fast implementation.
 - (:issue:`130`) Enable applications to provide their own :meth:`.render_json` method so that they can create
   unified API responses.
 - (:issue:`121`) Unauthorization callback not quite right. Split into 2 different callbacks - one for
@@ -35,7 +39,8 @@ Released TBD
 - (:issue:`159`) The ``/register`` endpoint returned the Authentication Token even though
   confirmation was required. This was a huge security hole - it has been fixed.
 
-Possible compatibility issues:
+Possible compatibility issues
++++++++++++++++++++++++++++++
 
 - (:pr:`120`) :class:`.RoleMixin` now has a method :meth:`.get_permissions` which is called as part
   each request to add Permissions to the authenticated user. It checks if the RoleModel
@@ -55,8 +60,39 @@ Possible compatibility issues:
   on Flask-Security's blueprint by sending it as `json_encoder_cls` as part of initialization. Be aware that your
   JsonEncoder needs to handle LazyStrings (see speaklater).
 
+- (:issue:`156`) Faster Authentication Token introduced 2 non-backwards compatible behavior changes - each can
+  be reverted using a configuration variable.
+
+    * In prior releases, the Authentication Token was returned as part of the JSON response to each
+      successful call to `/login`, `/change`, or `/reset/{token}` API call. This is not a great idea since
+      for browser-based UIs that used JSON request/response, and used session based authentication - they would
+      be sent this token - even though it was likely ignored. Since these tokens by default have no expiration time
+      this exposed a needless security hole. The new default behavior is to ONLY return the Authentication Token from those APIs
+      if the query param ``include_auth_token`` is added to the request. Prior behavior can be restored by setting
+      the `BACKWARDS_COMPAT_AUTH_TOKEN` configuration variable.
+    * Since the old Authentication Token algorithm used the (hashed) user's password, those tokens would be invalidated
+      whenever the user changed their password. This is not likely to be what most users expect. Since the new
+      Authentication Token algorithm doesn't refer to the user's password, changing the user's password won't invalidate
+      outstanding Authentication Tokens. There is a new method and an administrator could use to force changing
+      of a user's ``fs_uniquifier`` - but nothing the user themselves can do to invalidate their Authentication Tokens.
+      Setting the `BACKWARDS_COMPAT_AUTH_TOKEN_INVALIDATE` configuration variable will cause the user's ``fs_uniquifier`` to
+      be changed when they change their password.
+
+
+New fast authentication token implementation
+++++++++++++++++++++++++++++++++++++++++++++
+Current auth tokens are slow because they use the user's password (hashed) as a uniquifier (the
+user id isn't really enough since it might be reused). This requires checking the (hashed) password against
+what is in the token on EVERY request - however hashing is (on purpose) slow. So this can add almost a whole second
+to every request.
+
+To solve this a new attribute in the User model was added - ``fs_uniquifier``. If this is present in your
+User model, then it will be used instead of the password for ensuring the token corresponds to the correct user.
+This is very fast. If that attribute is NOT present - then the behavior falls back to existing (slow) method.
+
 
 DB Migration
+~~~~~~~~~~~~
 
 To use the new UserModel mixins or to add the column ``user.fs_uniquifier`` to speed up token
 authentication, a schema AND data migration needs to happen. If you are using Alembic the schema migration is

docs/customizing.rst

@@ -272,23 +272,23 @@ JSON Response
 +++++++++++++
 Applications that support a JSON based API need to be able to have a uniform
 API response. Flask-Security has a default way to render its API responses - which can
-be easily overridden by either providing a callback function via :meth:`.Security.render_json`.
-As documents in :meth:`Security.render_json`, be aware that Flask-Security registers
+be easily overridden by providing a callback function via :meth:`.Security.render_json`.
+As documented in :meth:`Security.render_json`, be aware that Flask-Security registers
 its own JsonEncoder on its blueprint.
 
 401, 403, Oh My
 +++++++++++++++
 For a very long read and discussion; look at `this`_. Out of the box, Flask-Security in
 tandem with Flask-Login, behaves as follows:
 
-    * If authentication fails as the result of a @login_required, @auth_required,
-      @http_auth_required, or @token_auth_required then if the request 'wants' a JSON
+    * If authentication fails as the result of a `@login_required`, `@auth_required`,
+      `@http_auth_required`, or `@token_auth_required` then if the request 'wants' a JSON
       response, :meth:`.Security.render_json` is called with a 401 status code. If not
       then flask_login.LoginManager.unauthorized() is called. By default THAT will redirect to
       a login view.
 
-    * If authorization fails as the result of @roles_required, @roles_accepted,
-      @permissions_required, or @permissions_accepted, then if the request 'wants' a JSON
+    * If authorization fails as the result of `@roles_required`, `@roles_accepted`,
+      `@permissions_required`, or `@permissions_accepted`, then if the request 'wants' a JSON
       response, :meth:`.Security.render_json` is called with a 403 status code. If not,
       then if ``SECURITY_UNAUTHORIZED_VIEW`` is defined, the response will redirected.
       If ``SECURITY_UNAUTHORIZED_VIEW`` is not defined, then ``abort(403)`` is called.

docs/features.rst

@@ -59,16 +59,23 @@ Token Authentication
 --------------------
 
 Token based authentication is enabled by retrieving the user auth token by
-performing an HTTP POST with the authentication details as JSON data against the
+performing an HTTP POST with a query param of ``include_auth_token`` with the authentication details
+as JSON data against the
 authentication endpoint. A successful call to this endpoint will return the
 user's ID and their authentication token. This token can be used in subsequent
 requests to protected resources. The auth token is supplied in the request
 through an HTTP header or query string parameter. By default the HTTP header
 name is `Authentication-Token` and the default query string parameter name is
-`auth_token`. Authentication tokens are generated using the user's password.
+`auth_token`. Authentication tokens are generated using a uniquifier field in the
+user's UserModel. If that field is changed (via :meth:`.UserDatastore.set_uniqifier`)
+then any existing authentication tokens will no longer be valid. Changing
+the user's password will not affect tokens.
+
+Note that prior to release 3.3.0 or if the Usermodel doesn't contain the ``fs_uniquifier``
+attribute the authentication tokens are generated using the user's password.
 Thus if the user changes his or her password their existing authentication token
 will become invalid. A new token will need to be retrieved using the user's new
-password.
+password. Verifying tokens created in this way is very slow.
 
 Two-factor Authentication (experimental)
 ----------------------------------------

docs/models.rst

@@ -24,6 +24,7 @@ your `User` and `Role` model should include the following fields:
 * ``email``
 * ``password``
 * ``active``
+* ``fs_uniquifier``
 
 
 **Role**

docs/openapi.yaml

@@ -29,6 +29,8 @@ paths:
   /login:
     get:
       summary: Retrieve login form and/or user information
+      parameters:
+        - $ref: "#/components/parameters/include_auth_token"
       responses:
         200:
           description: >
@@ -60,6 +62,7 @@ paths:
               URL to redirect to on successful registration. Ignored for json request.
           schema:
             type: string
+        - $ref: "#/components/parameters/include_auth_token"
       requestBody:
         required: true
         content:
@@ -278,6 +281,7 @@ paths:
           in: header
           schema:
             $ref: "#/components/headers/X-CSRF-Token"
+        - $ref: '#/components/parameters/include_auth_token'
       requestBody:
         required: true
         content:
@@ -406,6 +410,8 @@ paths:
                   value: redirect(cv('FORGOT_PASSWORD'))
     post:
       summary: Reset password
+      parameters:
+        - $ref: '#/components/parameters/include_auth_token'
       requestBody:
         required: true
         content:
@@ -557,15 +563,18 @@ components:
           type: object
           required: [id]
           description: >
-            By default just 'id', and 'authentication_token' are returned. However by overriding _User::get_security_payload()_ any attributes of the User model can be returned.
+            By default just 'id'is returned. However by overriding _User::get_security_payload()_ any attributes of the User model can be returned.
           properties:
             id:
               type: integer
               example: 42
               description: Unique user id (primary key)
             authentication_token:
               type: string
-              description: Token to be used in future token-based API calls.
+              description: >
+                Token to be used in future token-based API calls.
+                Note this only returned from those APIs that accept a
+                'include_auth_token' query param.
         csrf_token:
           type: string
           description: Session CSRF token
@@ -660,6 +669,14 @@ components:
             Email address to send link email to.
   headers:
     X-CSRF-Token:
+      description: CSRF token
+      schema:
+        type: string
+  parameters:
+    include_auth_token:
+      name: include_auth_token
+      description: If set/sent, will return an Authentication Token for user
+      in: query
       schema:
         type: string
 

flask_security/changeable.py

@@ -39,6 +39,8 @@ def change_user_password(user, password):
     :param password: The unhashed new password
     """
     user.password = hash_password(password)
+    if config_value("BACKWARDS_COMPAT_AUTH_TOKEN_INVALID"):
+        _datastore.set_uniquifier(user)
     _datastore.put(user)
     send_password_changed_notice(user)
     password_changed.send(current_app._get_current_object(), user=user)

flask_security/core.py

@@ -205,6 +205,8 @@
     "CSRF_HEADER": "X-XSRF-Token",
     "CSRF_COOKIE_REFRESH_EACH_REQUEST": False,
     "BACKWARDS_COMPAT_UNAUTHN": False,
+    "BACKWARDS_COMPAT_AUTH_TOKEN": False,
+    "BACKWARDS_COMPAT_AUTH_TOKEN_INVALIDATE": False,
 }
 
 #: Default Flask-Security messages
@@ -364,11 +366,11 @@ def _request_loader(request):
             local_cache.verify_hash_cache = cache
         if cache.has_verify_hash_cache(user):
             return user
-        if verify_hash(data[1], user.password):
+        if user.verify_auth_token(data):
             cache.set_cache(user)
             return user
     else:
-        if verify_hash(data[1], user.password):
+        if user.verify_auth_token(data):
             return user
 
     return _security.login_manager.anonymous_user()
@@ -567,10 +569,37 @@ def is_active(self):
         return self.active
 
     def get_auth_token(self):
-        """Returns the user's authentication token."""
+        """Constructs the user's authentication token.
+
+        This data MUST be securely signed using the ``remember_token_serializer``
+        """
         data = [str(self.id), hash_data(self.password)]
+        if hasattr(self, "fs_uniquifier"):
+            data.append(self.fs_uniquifier)
         return _security.remember_token_serializer.dumps(data)
 
+    def verify_auth_token(self, data):
+        """
+        Perform additional verification of contents of auth token.
+        Prior to this being called the token has been validated (via signing)
+        and has not expired.
+
+        :param data: the data as formulated by :meth:`get_auth_token`
+
+        .. versionadded:: 3.3.0
+        """
+        if len(data) > 2 and hasattr(self, "fs_uniquifier"):
+            # has uniquifier - use that
+            if data[2] == self.fs_uniquifier:
+                return True
+            # Don't even try old way - if they have defined a uniquifier
+            # we want that to be able to invalidate tokens if changed.
+            return False
+        # Fall back to old and very expensive check
+        if verify_hash(data[1], self.password):
+            return True
+        return False
+
     def has_role(self, role):
         """Returns `True` if the user identifies with the specified role.
 
@@ -961,15 +990,19 @@ def want_json(self, fn):
 
             :request: Werkzueg/Flask request
 
+        The default implementation returns True if either the Content-Type is
+        "application/json" or the best Accept header value is "application/json".
+
         .. versionadded:: 3.3.0
         """
         self._state._want_json = fn
 
     def unauthz_handler(self, cb):
         """
         Callback for failed authorization.
-        This is called by the various decorators if a role or permission
-        is missing.
+        This is called by the :func:`roles_required`, :func:`roles_accepted`,
+        :func:`permissions_required`, or :func:`permissions_accepted`
+        if a role or permission is missing.
 
         :param cb: Callback function with signature (func, params)
 
@@ -990,7 +1023,8 @@ def unauthz_handler(self, cb):
     def unauthn_handler(self, cb):
         """
         Callback for failed authentication.
-        This is called by the various decorators if authentication fails.
+        This is called by :func:`auth_required`, :func:`auth_token_required`
+        or :func:`http_auth_required` if authentication fails.
 
         :param cb: Callback function with signature (mechanisms, headers=None)
 

flask_security/datastore.py

@@ -232,6 +232,25 @@ def activate_user(self, user):
             return True
         return False
 
+    def set_uniquifier(self, user, uniquifier=None):
+        """ Set user's authentication token uniquifier.
+        This will immediately render outstanding auth tokens invalid.
+
+        :param user: User to modify
+        :param uniquifier: Unique value - if none then uuid.uuid4().hex is used
+
+        This method is a no-op if the user model doesn't contain the attribute
+        ``fs_uniquifier``
+
+        .. versionadded:: 3.3.0
+        """
+        if not hasattr(user, "fs_uniquifier"):
+            return
+        if not uniquifier:
+            uniquifier = uuid.uuid4().hex
+        user.fs_uniquifier = uniquifier
+        self.put(user)
+
     def create_role(self, **kwargs):
         """
         Creates and returns a new role from the given parameters.

flask_security/models/fsqla.py

@@ -4,6 +4,9 @@
 
 
 Complete models for all features when using Flask-SqlAlchemy
+
+BE AWARE: Once any version of this is shipped no changes can be made - instead
+a new version needs to be created.
 """
 
 import datetime

flask_security/recoverable.py

@@ -100,6 +100,8 @@ def update_password(user, password):
     :param password: The unhashed new password
     """
     user.password = hash_password(password)
+    if config_value("BACKWARDS_COMPAT_AUTH_TOKEN_INVALID"):
+        _datastore.set_uniquifier(user)
     _datastore.put(user)
     send_password_reset_notice(user)
     password_reset.send(app._get_current_object(), user=user)

flask_security/utils.py

@@ -620,6 +620,8 @@ def default_want_json(req):
 class FsJsonEncoder(JSONEncoder):
     """  Flask-Security JSON encoder.
     Extends Flask's JSONencoder to handle lazy-text.
+
+    .. versionadded:: 3.3.0
     """
 
     def default(self, obj):

flask_security/views.py

@@ -107,8 +107,13 @@ def _base_render_json(
                 payload["user"] = user.get_security_payload()
 
             if include_auth_token:
-                token = user.get_auth_token()
-                payload["user"]["authentication_token"] = token
+                # view wants to return auth_token - check behavior config
+                if (
+                    config_value("BACKWARDS_COMPAT_AUTH_TOKEN")
+                    or "include_auth_token" in request.args
+                ):
+                    token = user.get_auth_token()
+                    payload["user"]["authentication_token"] = token
 
         # Return csrf_token on each JSON response - just as every form
         # has it rendered.
@@ -212,7 +217,7 @@ def logout():
         logout_user()
 
     # No body is required - so if a POST and json - return OK
-    if request.method == "POST" and request.is_json:
+    if request.method == "POST" and _security._want_json(request):
         return _security._render_json({}, 200, headers=None, user=None)
 
     return redirect(get_post_logout_redirect())