Jwt for secure inbound and outbound communication

[marcosgopen]

addresses #314

When JWT is enabled:

• Inbound auth: JwtAuthFilter validates Bearer tokens on coordinator endpoints using RSA public key verification with issuer/audience checks, returning 401 on failure.
• Token capture: JwtTokenRequestFilter extracts the inbound token and stores it in LRABearerToken (thread-local), making it available for outbound calls to participants.
• Outbound propagation: JwtTokenClientRequestFilter re-attaches the captured token to outgoing REST client requests so coordinator-to-participant calls are authenticated.
• Recovery support: ServiceTokenProvider supplies tokens for async recovery threads (no inbound request context) via file, classpath, or HTTP sources with configurable refresh.

cc @arjantijms

jbosstm/jboss-as#102

[marcosgopen]

Note: jwt dependencies needs to be added to the wildfly modules

[arjantijms]

Looking good! I'll play with it a bit tonight or tomorrow, and if possible see if there's any feedback to provide.

[arjantijms]

I'm looking at the PR a little and I wonder what the use of JwtAuthFilter is.

It parses the incoming JWT token using SmallRye JWT. Isn't this however more of a task for the existing MP JWT implementation to do? (which happens to be SmallRye JWT again for WildFly and Quarkus).

Would it not be better to omit JwtAuthFilter, and annotate or programmatically protect the coordinator methods with roles? Or am I missing something?

[arjantijms]

Token capture: JwtTokenRequestFilter extracts the inbound token and stores it in LRABearerToken (thread-local), making it available for outbound calls to participants.

Likewise, MP JWT already captures the inbound token.

For a similar project I just did something like:

public class MyFilter implements ClientRequestFilter {

    @Inject
    JsonWebToken jwtToken;

     public void filter(ClientRequestContext context) {
         String rawToken = jwtToken.getRawToken();
         // [check empty ...]

        context.getHeaders()
                    .putSingle("Authorization", "Bearer " + token)l
     }
}

[marcosgopen]

I'm looking at the PR a little and I wonder what the use of JwtAuthFilter is.

It parses the incoming JWT token using SmallRye JWT. Isn't this however more of a task for the existing MP JWT implementation to do? (which happens to be SmallRye JWT again for WildFly and Quarkus).

Would it not be better to omit JwtAuthFilter, and annotate or programmatically protect the coordinator methods with roles? Or am I missing something?

Good point, JwtAuthFilter was initially intended for a standalone jar without a MP JWT runtime but I see what you mean. I will use the roles as you suggested and remove the JwtAuthFilter. Thanks

[marcosgopen]

Token capture: JwtTokenRequestFilter extracts the inbound token and stores it in LRABearerToken (thread-local), making it available for outbound calls to participants.

Likewise, MP JWT already captures the inbound token.

For a similar project I just did something like:

public class MyFilter implements ClientRequestFilter {

    @Inject
    JsonWebToken jwtToken;

     public void filter(ClientRequestContext context) {
         String rawToken = jwtToken.getRawToken();
         // [check empty ...]

        context.getHeaders()
                    .putSingle("Authorization", "Bearer " + token)l
     }
}

Yes, I wanted to Inject the token but I thought that for registered filters would have been harder to make it work. Let me have another try and see if I can make it work as in your example.

[marcosgopen]

I think the shift in the PR would be to implement the JWT secure inbound connection relying to the container's (WildFly, Quarkus or else) mp jwt implementation. I believe this is the right approach as it avoids duplication and gives more flexibility (the container chooses the mp jwt implementation) but it is less consistent with the outbound connection which is managed by lra coordinator instead (I need to document it). I will add integration tests with arquillian/WildFly to test the secured inbound connections.

[marcosgopen]

@arjantijms I have addressed your comments and refined the PR. Could I ask you another review?
(mainly the first commit, as the second one is for integration testing with WildFly)

[arjantijms]

I have addressed your comments and refined the PR. Could I ask you another review?

Thanks! I'll check it out tonight! (I'm currently busy with a somewhat related task; JWT authentication in Jakarta Security ;))

[marcosgopen]

Note:
a follow-up task will be to integrate the http-client-providers, service-token-location and service-token-refresh-seconds into the WildFly lra coordinator subsystem with a schema bump

[arjantijms]

Sorry for the long delay in getting back to this. I finally was able to play with the code a bit, and test it in a small application.

I found a few things:

1. There is no clear documentation for the applying the JwtTokenClientRequestFilter for the participans. They typically start the LRA, so would be the first caller to supply the JWT token to the coordinator.
2. Following from 1), the io.narayana.lra.coordinator.security.JwtTokenClientRequestFilter can't be used for the participants, since putting the jar file from the coordinator on the classpath of the participant would run the risk of clashing with the Application instance provided by the coordinator jar.
3. The JwtTokenClientRequestFilter uses @Inject Instance<JsonWebToken> jwtTokenInstance; for a filter documented to be added via the lra.http-client.providers property. In RestClientConfig this does a Class.forName and a builder.register. On Quarkus at least (and it may be a bug in Quarkus), this does not seem to do injection of the Instance field. In Quarkus, Instance<JsonWebToken> jwtTokenInstance = CDI.current().select(JsonWebToken.class); in the filter method body does work.
4. RestClientConfig as mentioned above does a Class.forName(className);, which, on Quarkus again, uses a Quarkus root classloader that does not find classes in the application. Class.forName(className), true, Thread.currentThread().getContextClassLoader()); does find those. As there is currently no filter provided for partipants, I guess end-users would try to add one from the application. The global alternative, not using lra.http-client.providers, but registering a filter using @Provider does work though.

Given the above, it may be good to

1. Provide a filter for the client jar as well, so end-users can configure e.g. lra.http-client.providers=io.narayana.client.JwtTokenClientRequestFilter
2. Instead of getting the token from the current request or from the configured location, let Coordinator.startLRA (additionally) store the token with the LRA data itself (in the ObjectStore). That way all subsequent operations involving that LRA ID can use the JWT.
3. Maybe, instead of asking the participant to configure lra.http-client.providers, introduce an annotation to set next @LRA on the resource method of the participant, particularly intended for those starting a new LRA. Maybe we can eventually standardise this as a new attribute for the existing MP @LRA annotation.

[marcosgopen]

Sorry for the long delay in getting back to this. I finally was able to play with the code a bit, and test it in a small application.

I found a few things:

1. There is no clear documentation for the applying the JwtTokenClientRequestFilter for the participans. They typically start the LRA, so would be the first caller to supply the JWT token to the coordinator.

2. Following from 1), the `io.narayana.lra.coordinator.security.JwtTokenClientRequestFilter` can't be used for the participants, since putting the jar file from the coordinator on the classpath of the participant would run the risk of clashing with the Application instance provided by the coordinator jar.

3. The JwtTokenClientRequestFilter uses `@Inject Instance<JsonWebToken> jwtTokenInstance;` for a filter documented to be added via the `lra.http-client.providers` property. In RestClientConfig this does a `Class.forName` and a builder.register. On Quarkus at least (and it may be a bug in Quarkus), this does not seem to do injection of the Instance field. In Quarkus, `Instance<JsonWebToken> jwtTokenInstance = CDI.current().select(JsonWebToken.class);` in the filter method body does work.

4. RestClientConfig as mentioned above does a `Class.forName(className);`, which, on Quarkus again, uses a Quarkus root classloader that does not find classes in the application. `Class.forName(className), true, Thread.currentThread().getContextClassLoader());` does find those. As there is currently no filter provided for partipants, I guess end-users would try to add one from the application. The global alternative, not using `lra.http-client.providers`, but registering a filter using `@Provider` does work though.

Given the above, it may be good to

1. Provide a filter for the client jar as well, so end-users can configure e.g. `lra.http-client.providers=io.narayana.client.JwtTokenClientRequestFilter`

2. Instead of getting the token from the current request or from the configured location, let `Coordinator.startLRA` (additionally) store the token with the LRA data itself (in the ObjectStore). That way all subsequent operations involving that LRA ID can use the JWT.

3. Maybe, instead of asking the participant to configure lra.http-client.providers, introduce an annotation to set next `@LRA` on the resource method of the participant, particularly intended for those starting a new LRA. Maybe we can eventually standardise this as a new attribute for the existing MP `@LRA` annotation.

Thank you for your useful observations! I will surely update the PR to include your suggestions. The only point I am not sure about is:
Instead of getting the token from the current request or from the configured location, let Coordinator.startLRA (additionally) store the token with the LRA data itself (in the ObjectStore). That way all subsequent operations involving that LRA ID can use the JWT.
That can be a nice optimization for a short-lived LRA but if the token has already expired than the thread should also re-request the token and for the recovery thread that can be quite a burden.

[arjantijms]

That can be a nice optimization for a short-lived LRA but if the token has already expired than the thread should also re-request the token and for the recovery thread that can be quite a burden.

Yes, true.

The expiration should be taken into account. Maybe we should not go for a re-request (with the refresh token). The documentation should simply say that the lifetime of the LRA and the token should match in this case. The @LRA already has a timelimit (org.eclipse.microprofile.lra.annotation.ws.rs.LRA.timeLimit()), which we could use to validate.

Additionally, storing the token on disk (the ObjectStore) may be an additional risk compared to just storing it in memory. Of course a process memory space can be read too by an attacker who gets access to the system, but it's a thing to be aware of.

Probably the idea of storing the token with the LRA is better left out of this PR and we could consider it as an opt-in (with system token fallback) for a future PR.

[marcosgopen]

This annotation is something that can be considered, in the future, to be proposed to be integrated in the LRA annotation defined in the MP LRA spec, e.g. @PropagateToken → @LRA(propagateToken=true)

[mmusgrov]

I am by no means an expert in JWT but the changes look reasonable and I defer to Arjan's review for the expert analysis.

It would be nice to see the coordinator markdown doc included in the LRA documentation - asciidoc has an include directive that can be used to achieve that.

I did not fully understand section 6.4 (Recovery Thread and Service Token) so it would be useful if we can follow up on the recovery tests (ServiceTokenProviderTest) with quickstart that demonstrates how to do it from the participants point of view.

Thanks @arjantijms for providing your review.

[mmusgrov]

This would be useful to have in the docs. It can be included in the top level doc by adding the following to docs/src/main/asciidoc/index.adoc:

include::../../../../coordinator/docs/security.md[]

A more brittle solution would be to create a symlink in the docs dir:

ln -s ../../../../coordinator/docs/security.md security.md

Remark: the resulting markdown tables do not render cleanly in the final asciidoc.

[marcosgopen]

Good point Mike, I am thinking of creating a doc folder to store these kind of documentation files. I will update the PR with the current location of that doc.

[marcosgopen]

I am by no means an expert in JWT but the changes look reasonable and I defer to Arjan's review for the expert analysis.

It would be nice to see the coordinator markdown doc included in the LRA documentation - asciidoc has an include directive that can be used to achieve that.

I did not fully understand section 6.4 (Recovery Thread and Service Token) so it would be useful if we can follow up on the recovery tests (ServiceTokenProviderTest) with quickstart that demonstrates how to do it from the participants point of view.

Thanks @arjantijms for providing your review.

Thanks Mike, a quickstart is surely useful. The token can be passed by the client and forwarded, but for the Reaper thread the token must be generated by the coordinator itself (as the reaper thread runs periodically without a client request).

[mmusgrov]

Thanks Mike, a quickstart is surely useful. The token can be passed by the client and forwarded, but for the Reaper thread the token must be generated by the coordinator itself (as the reaper thread runs periodically without a client request).

Like you say the coordinator needs to call the participants during recovery but I did not fully understand section 6.4 and how it obtain the clients' token?

[marcosgopen]

Thanks Mike, a quickstart is surely useful. The token can be passed by the client and forwarded, but for the Reaper thread the token must be generated by the coordinator itself (as the reaper thread runs periodically without a client request).

Like you say the coordinator needs to call the participants during recovery but I did not fully understand section 6.4 and how it obtain the clients' token?

The client Token is obtained from the ClientRequestContext: see https://github.com/jbosstm/lra/pull/321/changes#diff-f1593b1349b91adc0202797accc074b3d3a592c9902a13db261ef4dcd5f108a9R39
It is done in the filter method of the JwtTokenCallbackRequestFilter overriding the ClientRequestFilter method.

It is quite similar to what the ServerLRAFilter does here:

lra/jaxrs/src/main/java/io/narayana/lra/filter/ServerLRAFilter.java

Line 250 in 49e519d

headers.putSingle(LRA_HTTP_PARENT_CONTEXT_HEADER, Current.getFirstParent(incomingLRA));