Activations Plus

Activations Plus is a Python package designed to provide a collection of activation functions which are not implemented in PyTorch.

Features

• Entmax: Sparse activation function for probabilistic models.
• Sparsemax: Sparse alternative to softmax.

some experimental features:

Installation

To install the package, use pip:

pip install activations-plus

Usage

Import and use any activation function in your PyTorch models:

import torch
from activations_plus.sparsemax import Sparsemax
from activations_plus.entmax import Entmax

# Example with Sparsemax
sparsemax = Sparsemax()
x = torch.tensor([[1.0, 2.0, 3.0], [1.0, 2.0, -1.0]])
output_sparsemax = sparsemax(x)
print("Sparsemax Output:", output_sparsemax)

# Example with Entmax
entmax = Entmax(alpha=1.5)
output_entmax = entmax(x)
print("Entmax Output:", output_entmax)

These examples demonstrate how to use Sparsemax and Entmax activation functions in PyTorch models.

Documentation

Comprehensive documentation is available documentation.

Supported Advanced Activation Functions

1. Entmax: Sparse activation function for probabilistic models. Reference Paper
2. Sparsemax: Sparse alternative to softmax for probabilistic outputs. Reference Paper

Contributing

Contributions are welcome! Please read the CONTRIBUTING.md file for guidelines.

Testing

To run the tests, use the following command:

pytest tests/

License

This project is licensed under the MIT License. See the LICENSE file for details.

Acknowledgments

Special thanks to the contributors and the open-source community for their support.