Citation

Zaman, Zahid and Khan, Yousaf and Ali, Farman and Armghan, Ammar and Shereen, Muhammad Kamran and Aldkeelalah, Sultan S. and Roslee, Mardeni (2026) Autoencoder-Pelican optimization for nonlinear impairment mitigation in long-haul optical fiber systems. Physical Communication, 75. p. 102972. ISSN 1874-4907

Text 1-s2.0-S1874490725003751-main.pdf - Published Version Restricted to Repository staff only Download (4MB)

Abstract

Long-haul optical transmission (LHOT) systems are affected by nonlinear impairments (NIs), including self-phase modulation (SPM), cross-phase modulation (XPM), four-wave mixing (FWM), amplified spontaneous emission (ASE) noise, and Kerr nonlinearities, which limit achievable data rates and system reach. Conventional methods, such as digital back-propagation (DBP), optical phase conjugation (OPC), and DSP-assisted receivers, have demonstrated mitigation capabilities but suffer from high computational complexity, latency, and power consumption, making them impractical for large-scale networks. Machine learning (ML) approaches, including label propagation and transformer-based schemes, reduce some processing overhead yet do not perform dimensionality reduction for feature compression and lack a mechanism to jointly handle multiple nonlinear effects across LHOT. Furthermore, most reported works do not align with optical communication standards, such as ITU-T G.652.D or OS1/OS2 fibers, which limits their practical implementation in standardized infrastructures. This work proposes an autoencoder-based pelican optimization algorithm (APOA) for NIs mitigation in LHOT systems. The autoencoder compresses high-dimensional signal distortions into a latent space that preserves nonlinear mappings, reducing computational load while maintaining representation accuracy. The POA performs parameter tuning to optimize signal recovery in the presence of nonlinear effects and noise. The transmission channel is modeled using the nonlinear Schrŏdinger equation (NLSE), with propagation distortions characterized by ITU-T G.652.D single-mode fiber (SMF) parameters: attenuation of 0.20 dB/km, chromatic dispersion of  ∼ 17 ps/nm/km at 1550 nm, effective area of 80 µm2, and nonlinear coefficient γ ≈ 1.3 W km . Simulations are conducted using parameter settings aligned with OS1/OS2 fiber specifications (9 µm core diameter) and representative optical communication terminal (OCT) configurations, to reflect realistic long-haul transmission environments. Performance evaluation across multiple OSNR levels, fiber lengths, and modulation formats uses FEC thresholds and operating ranges that are consistent with IEEE 802.3 Ethernet and ITU-T G.709 OTN reference values, showing that APOA achieves BER values below the adopted FEC thresholds, increases spectral efficiency, and extends transmission reach.

Item Type:	Article
Uncontrolled Keywords:	Long-haul optical transmissions, Autoencoder-based pelican, optimization, algorithm, Nonlinear impairments, Mutli channel transmission, Optical communication standards
Subjects:	Q Science > QC Physics > QC350-467 Optics. Light
Divisions:	Faculty of Engineering (FOE)
Depositing User:	Ms Suzilawati Abu Samah
Date Deposited:	10 Feb 2026 01:14
Last Modified:	10 Feb 2026 01:14
URII:	http://shdl.mmu.edu.my/id/eprint/15255

Downloads

Edit (login required)