PyTorch reproduction of TensorFlow paper underperforms by 4 pp on DermaMNIST , what cross-framework issues should I check? [R]
Our take
When a paper reports 77.01% test accuracy and your careful reproduction lands at 73.8%, the gap feels personal. It is not personal. It is a reminder that cross-framework reproduction is harder than most published results make it look, and the reasons run deeper than mismatched random seeds. The user here is working with a fixed Gabor filter bank front-end, an SE block, a residual block, and three fully connected layers on DermaMNIST. That is a narrow, well-defined architecture. The fact that 4 percentage points remain stubbornly out of reach after adjusting sigma factors, trying multiple seeds, and varying low-pass and high-pass kernel parameters points to something more structural than a single hyperparameter quirk.
The most likely culprits live in the spaces between frameworks that rarely get documented. How Gabor kernels are initialized, normalized, and applied — especially around padding logic and zero-mean handling — can diverge meaningfully between TensorFlow's convolution semantics and PyTorch's. The user already normalized to unit L2 norm, which is the right instinct, but it is worth confirming whether the paper's implementation applies normalization per-channel or globally, and whether the magnitude computation uses the same epsilon value for numerical stability. Beyond the front-end, data loading pipelines differ in ways that compound over training. Augmentation strategies, class weighting, and even how train-val-test splits are generated can shift reported metrics by a few points. If the paper used stratified splits with a specific version of the dataset that has since been updated, the ground truth itself may have shifted. Related posts in our community highlight exactly how these silent divergences accumulate — from Having issues printing a document where a formatting mismatch obscured a simple configuration error, to Only show Yes percentages where a visualization filter created the illusion of missing data. The pattern is the same: the tool behaves correctly, but the assumptions behind the output are misaligned.
The observation that test accuracy runs lower than validation accuracy per epoch is not necessarily a bug. It often reflects the absence of test-time augmentation or the fact that validation is evaluated on a single fixed split while the test set behaves differently under random shuffling. What matters more is whether the gap persists across seeds and whether the model is actually converging or plateauing early. With roughly 340k parameters and three FC layers, the architecture is small enough that overfitting should be visible if training loss keeps dropping while validation stalls. If it is not, the issue may be upstream — in how the Gabor features are being fed into the CNN. A residual block expects an input shape it can match, and any mismatch between the front-end output channel count and the backbone input can silently degrade gradient flow without producing obvious errors.
This kind of reproducibility friction is becoming the default friction point in applied deep learning. As more researchers adopt PyTorch for new work while legacy papers remain locked in TensorFlow, the toolchain gap widens even as the mathematics stays identical. The practical question worth watching is whether community-driven reproducibility checklists will start treating cross-framework translation as a first-class verification step, the way statistical reporting standards now treat pre-registration. For now, the most productive move for this user is to audit the Gabor front-end against the paper's kernel construction line by line, confirm the data split, and test whether removing the front-end entirely and replacing it with raw inputs brings the CNN closer to the reported numbers. If it does, the filter bank is the variable. If it does not, the bottleneck is elsewhere, and that narrows the search considerably.
I'm reproducing a published paper's hybrid Gabor + CNN architecture in PyTorch. The original implementation is in TensorFlow. My reproduction consistently lands ~4 pp below the paper's reported test accuracy on DermaMNIST (73-74% vs paper's 77.01%). I'd like to know which cross-framework differences are most likely to cause this gap.
Ahmed et al., "A Lightweight Hybrid Gabor Deep Learning Approach", IJCV 2026 (DOI: 10.1007/s11263-025-02658-2). The architecture is a fixed Gabor filter bank front-end followed by a small CNN with one SE block, one residual block, and three FC layers. ~340k parameters total. I've already tried Different sigma_factor values (1.0 vs 1.2) and Multiple random seeds (42, 0, 123) and tried diffrent sigma valyes of the lpf and hpf channels but its didnt close the gap.
please any idea on how to at least get a 76% to match the paper because i wanted to add improvements to see the diffrence, i would really appreciate it on how to fix this problem or any advice on what to do.
also here is just example of one epoch i have noticed that the test accuracy is lower than the validation accuracy: im i doing something wrong
[ 47/100] Train: 75.70% Val: 76.07% Best: 76.97% Loss: 0.6827 [paper] test acc = 0.7382 Code example:
python
class FixedGaborFrontEnd(nn.Module): def __init__(self, scales=(0.10, 0.20, 0.40), orientations=(4, 4, 4), sigma_factor=1.0, input_size=224, output_size=56): super().__init__() # Build Gabor parameters (fixed buffers, not learnable) sigmas, thetas, freqs, kernel_sizes = [], [], [], [] for f, o in zip(scales, orientations): sigma = sigma_factor / (math.pi * f) N = 2 * int(math.floor(3 * sigma)) + 1 for k in range(o): sigmas.append(sigma) thetas.append(math.pi * k / o) freqs.append(f) kernel_sizes.append(N) # ... build real/imag kernels with zero-mean + L2 normalization ... def forward(self, x): # Convert RGB to grayscale if x.shape[1] != 1: x = 0.299 * x[:, 0:1] + 0.587 * x[:, 1:2] + 0.114 * x[:, 2:3] real = F.conv2d(x, self.real_kernels, padding=self.max_kernel_size // 2) imag = F.conv2d(x, self.imag_kernels, padding=self.max_kernel_size // 2) magnitude = torch.sqrt(real ** 2 + imag ** 2 + 1e-8) lpf = F.conv2d(x, self.lpf_kernel, padding=self.lpf_pad) hpf = F.conv2d(x, self.hpf_kernel, padding=self.hpf_pad) feats = torch.cat([magnitude, lpf, hpf], dim=1) feats = F.avg_pool2d(feats, 4, 4) # 224 → 56 return feats # Standard backbone follows: SE → Conv-BN-ReLU → MaxPool → ResBlock → Dropout → GAP → FC × 3 optimizer = torch.optim.Adam(model.parameters(), lr=1e-3) scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer, mode='min', factor=0.5 [link] [comments]
Read on the original site
Open the publisher's page for the full experience