ncxlib
  • ⚡Welcome
  • Getting Started
    • Quickstart
    • API Documentation
      • Overview
        • Neural Network
          • _compile
          • add_layer
          • forward_propagate_all
          • forward_propagate_all_no_save
          • back_propagation
          • train
          • predict
          • evaluate
          • save_model
          • load_model
        • Activation
          • ReLU
          • LeakyReLU
          • Sigmoid
          • Softmax
          • Tanh
        • Layer
          • InputLayer
          • FullyConnectedLayer
          • OutputLayer
        • LossFunction
          • MeanSquaredError
          • BinaryCrossEntropy
          • CategoricalCrossEntropy
        • Optimizer
          • SGD
          • SGDMomentum
          • RMSProp
          • Adam
        • Initializer
          • HeNormal
          • Zero
        • PreProcessor
          • OneHotEncoder
          • MinMaxScaler
          • Scaler
          • ImageRescaler
          • ImageGrayscaler
        • DataLoader
          • CSVDataLoader
          • ImageDataLoader
        • Generators
          • random_array
          • integer_array
          • generate_training_data
        • Utils
          • train_test_split
          • k_fold_cross_validation
Powered by GitBook
On this page
  1. Getting Started
  2. API Documentation
  3. Overview
  4. Activation

Softmax

class Softmax(Activation):
    def apply(self, x: np.ndarray) -> np.ndarray:
        """
        Applies the Softmax function to the input array.

        Args:
          x (np.ndarray): Input array.

        Returns:
          np.ndarray: Softmax output.
        """
        e_x = np.exp(x - np.max(x, axis=-1, keepdims=True))
        self.activated = e_x / np.sum(e_x, axis=-1, keepdims=True)
        return self.activated

    def derivative(self, x: np.ndarray) -> np.ndarray:
        """
        Calculates the derivative of the Softmax function.

        Args:
          x (np.ndarray): Input array (not used directly, 
                          but kept for consistency with other activations).

        Returns:
          np.ndarray: Derivative of Softmax (Jacobian matrix).
        """
        s = self.activated.reshape(-1, 1)
        return np.diagflat(s) - np.dot(s, s.T)
PreviousSigmoidNextTanh

Last updated 7 months ago