WIP: Beta distribution support for PPO

[araffin]

Description

closes #2142

Note: done with zed + claude

Motivation and Context

• I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)

Checklist

• I've read the CONTRIBUTION guide (required)
• I have updated the changelog accordingly (docs/misc/changelog.md) (required).
• My change requires a change to the documentation.
• I have updated the tests accordingly (required for a bug fix or a new feature).
• I have updated the documentation accordingly.
• I have opened an associated PR on the SB3-Contrib repository (if necessary)
• I have opened an associated PR on the RL-Zoo3 repository (if necessary)
• I have reformatted the code using make format (required)
• I have checked the codestyle using make check-codestyle and make lint (required)
• I have ensured make pytest and make type both pass. (required)
• I have checked that the documentation builds using make doc (required)

Note: You can run most of the checks using make commit-checks.

Note: we are using a maximum length of 127 characters per line

[lukaskiss222]

I would recommend for beta distributions to use mean instead of mode for deterministic action by overriding get_auctions (to follow the Petrazzini & Antonelo (2021) paper:

    
    def mean(self) -> th.Tensor:
        # Mean of Beta(alpha, beta) = alpha / (alpha + beta)
        alpha = self.distribution.concentration1
        beta = self.distribution.concentration0
        return alpha / (alpha + beta)
    
    def get_actions(self, deterministic: bool = False) -> th.Tensor:
        if deterministic:
            # paper used the mean as deterministic action
            return self.mean()
        return self.sample()