GCSAM: Gradient-Centralized Sharpness-Aware Minimization

This repository contains the official implementation of Gradient-Centralized Sharpness-Aware Minimization (GCSAM) for Deep Neural Networks (DNNs) and Vision Transformers.
Paper: GCSAM: Gradient-Centralized Sharpness-Aware Minimization

Abstract

The generalization performance of deep neural networks (DNNs) is a critical factor in achieving robust model behavior on unseen data. Recent studies have highlighted the importance of sharpness-based measures in promoting generalization by encouraging convergence to flatter minima. Among these approaches, Sharpness-Aware Minimization (SAM) has emerged as an effective optimization technique for reducing the sharpness of the loss landscape, thereby improving generalization. However, SAM's computational overhead and sensitivity to noisy gradients limit its scalability and efficiency. To address these challenges, we propose Gradient-Centralized Sharpness-Aware Minimization (GCSAM), which incorporates Gradient Centralization (GC) to stabilize gradients and accelerate convergence. GCSAM normalizes gradients before the ascent step, reducing noise and variance, and improving stability during training. Our evaluations indicate that GCSAM consistently outperforms SAM and the Adam optimizer in terms of generalization and computational efficiency. These findings demonstrate GCSAM's effectiveness across diverse domains, including general and medical imaging tasks.

BUS Dataset Preprocessing

We evaluate GCSAM on the Breast Ultrasound (BUS) dataset.
For reproducibility, we provide the preprocessing pipeline below.

• Resize all images to 256×256 pixels (RGB).
• Normalize pixel values to [0,1].
• Convert to (C, H, W) format for PyTorch.
• Assign labels from BIRADS&FOLD.xlsx (0 = benign, 1 = malignant).

PyTorch Example

import os
import pandas as pd
from PIL import Image
from torch.utils.data import Dataset, DataLoader
from torchvision import transforms

class BUSDataset(Dataset):
    def __init__(self, img_dir, label_file, transform=None):
        self.img_dir = img_dir
        self.labels = pd.read_excel(label_file)
        self.transform = transform if transform else transforms.Compose([
            transforms.Resize((256, 256)),
            transforms.ToTensor()
        ])

    def __len__(self):
        return len(self.labels)

    def __getitem__(self, idx):
        img_name = os.path.join(self.img_dir, self.labels.iloc[idx, 0])
        image = Image.open(img_name).convert("RGB")
        label = 1 if "malignant" in self.labels.iloc[idx, 0] else 0
        return self.transform(image), label

# Example usage
bus_dataset = BUSDataset("path/to/BUS/images", "BIRADS&FOLD.xlsx")
bus_loader = DataLoader(bus_dataset, batch_size=32, shuffle=True)

Citation

@article{hassan2025gcsam, author = {Mohamed Hassan and Aleksandar Vakanski and Boyu Zhang and Min Xian}, title = {GCSAM: Gradient-Centralized Sharpness-Aware Minimization}, journal = {arXiv preprint arXiv:2501.11584}, year = {2025} }

References

[1] Foret et al. 2021
[2] Yong et al. 2020