1. Modulation Evolution Beyond Higher-Order QAM

Advances in fundamental physical-layer technologies are essential for improving the spectral and power efficiency of cellular systems. Channel coding provides a clear example: turbo codes were introduced in 3G and remained central to data transmission in 4G LTE, while 5G NR adopted low-density parity-check (LDPC) codes for shared data channels and polar codes for broadcast and control information.

Regarding data modulation, evolution has largely focused on increasing the order of quadrature amplitude modulation (QAM). Successive generations have supported larger constellations while retaining regularly spaced signal points and statistically equal transmission probabilities. Although increasing the modulation order raises the number of coded bits carried by each symbol, it does not eliminate the efficiency loss associated with a uniform signal distribution.

Constellation shaping addresses this limitation by optimizing either the locations or the transmission probabilities of constellation points. During the 6G study phase, it became a prominent topic in 3GPP RAN1, with multiple companies proposing geometric and probabilistic shaping schemes and evaluating their performance and implementation requirements [1, 2]. Although constellation shaping was not selected for the initial 6G design, these studies examined a fundamental modulation enhancement beyond increasing the constellation size.

This article introduces constellation shaping, with a focus on probabilistic shaping and its integration into cellular modems. It then presents Systematic Probabilistic Shaping (S-PS), a design that combines selective probability shaping with the existing 5G NR coding and modulation architecture.

2. Constellation Shaping: Principles and Implementation Requirements

For a memoryless additive white Gaussian noise (AWGN) channel subject to an average transmit-power constraint, a Gaussian input distribution achieves Shannon capacity. This follows from the maximum-entropy property of the Gaussian distribution: at a fixed variance, the Gaussian output has the largest differential entropy, while the conditional output entropy is determined by the additive noise. A Gaussian input therefore maximizes the mutual information between the transmitted and received signals [3].

Practical cellular modems use finite constellations to support efficient mapping and detection. In bit-interleaved coded modulation (BICM), binary channel coding is followed by bit interleaving and mapping to constellation points; the receiver then generates bit-wise soft information for the channel decoder. Conventional square QAM uses a regular grid with equal symbol probabilities. Even with an effective channel code, this input distribution leaves a shaping gap relative to the Gaussian-input capacity.

Geometric shaping (GS) and probabilistic shaping (PS) reduce this gap through different design variables. GS optimizes the constellation coordinates, usually while maintaining equal symbol probabilities. One-dimensional GS adjusts the amplitude levels of the in-phase and quadrature components separately, whereas two-dimensional GS jointly optimizes their coordinates. The resulting Euclidean distances and bit labeling determine the achievable rate and demapping requirements.

PS retains the constellation geometry and changes the symbol probabilities. For AWGN-oriented shaping, lower-energy points are generally transmitted more frequently. A common reference is the Maxwell–Boltzmann distribution, in which symbol probability decreases exponentially with symbol energy [4]. The probability distribution and power normalization must be selected for the target operating point: stronger shaping reduces average unnormalized energy but also reduces symbol entropy. The objective is to improve the achievable information rate at a given signal-to-noise ratio (SNR), or to reduce the required SNR at a given net information rate.

Preserving the square-QAM geometry offers significant implementation advantages. Existing receivers can reuse processing algorithms that exploit equally spaced amplitude levels and the Cartesian structure of the constellation. This is particularly important for downlink MIMO reception, where practical nonlinear detectors may use QAM-specific slicing, candidate generation, and tree-search procedures to reduce complexity. GS requires changes to these operations, depending on the constellation design. In contrast, PS retains the underlying geometry, although detection and demapping metrics must account for the nonuniform symbol probabilities.

Generating these probabilities requires additional processing. A distribution matcher maps approximately equiprobable input bits into sequences with a target nonuniform distribution through a reversible mapping [5]. At the receiver, distribution dematching recovers the original bits after successful channel decoding. Representative distribution matching approaches include constant-composition distribution matching (CCDM) [6], enumerative sphere shaping (ESS) [7], and polar-code-based distribution matching [8]. CCDM fixes the composition of each output block; ESS selects sequences from an energy-bounded set; and polar-code-based methods use coding operations to generate biased binary sequences.

These methods introduce trade-offs among finite-length rate loss, computation, memory, and latency [6, 7]. The challenge is particularly important for high-order modulation and multiple spatial layers, where the distribution matcher and dematcher must sustain high aggregate data rates. Sequential processing or limited parallelism can make these functions a throughput bottleneck. Independently, finite-length rate loss reduces the net information rate supported by the shaped sequence. Practical shaping must therefore provide a net gain after accounting for both matching efficiency and implementation constraints.

Ensuring compatibility with the existing 5G architecture is another critical requirement. The distribution generated before channel coding must survive the subsequent encoding, rate matching, interleaving, and symbol mapping. Parity generation generally does not preserve a nonuniform input distribution, so the placement of shaped and unshaped bits must be coordinated with the code structure and modulation labeling. Designs that require substantial changes to this processing chain can diminish hardware reuse and introduce separate processing paths for conventional and shaped transmission.

S-PS addresses these requirements by aligning selectively shaped bits with the transmitted systematic bits of the NR LDPC codeword. Within the applicable coding and rate-matching configurations, its main features include:

     - Binary shaping of only the second-most-significant bit (2nd MSB) in each in-phase and quadrature component.
     - Reuse of NR LDPC coding, rate matching, bit interleaving, and square-QAM mapping.
     - Localized distribution matching and dematching, with probability-aware receiver metrics and appropriate power normalization.
     - Support for different binary shaping algorithms and parallel processing of independent segments.
     - A shared processing architecture for shaped transmission and conventional uniform QAM.

The following sections explain the selective shaping principle and its integration into the NR BICM chain.

3. Systematic Probabilistic Shaping: Design and Performance

S-PS introduces selective probability shaping into the NR LDPC-based BICM chain [1, 2]. Its design combines three fundamental elements:

     ·   Shaping only the 2nd MSB of each in-phase (I) and quadrature (Q) component.
     ·   Aligning the shaped bits with the transmitted systematic bits through the existing NR processing chain.
     ·   Applying distribution matching and dematching only to the selected information subblock.

Under the NR square-QAM labeling, the most-significant bit (MSB) of each real component determines its sign, while the remaining bits determine its amplitude. The 2nd MSB distinguishes the inner and outer amplitude regions [9]. Increasing the probability of the value associated with the inner region therefore increases the occurrence of lower-energy constellation points.

Figure 1 illustrates this operation for 64-QAM. Shaping the second MSBs of both I and Q components produces different symbol probabilities according to whether each component belongs to the inner or outer region. The central region receives the highest probability, while points with both components in the outer regions receive the lowest. The constellation coordinates and bit labeling remain unchanged, apart from the common scaling required for power normalization.

This approach shapes two bit levels per complex symbol, regardless of the QAM order. While this provides less distributional flexibility than shaping all amplitude bit levels, it limits the amount of data processed by the matcher. The shaping probability is selected together with the modulation order and the target information rate.

The systematic structure of the LDPC code provides the connection between probability shaping and the NR processing chain. A systematic encoder retains the input bits within its codeword and generates additional parity bits. S-PS places the distribution-matched sequence at specific information-bit positions whose systematic copies reach the selected modulation bit levels after rate matching and bit interleaving.

The existing NR rectangular bit interleaver maps specific segments of the rate-matched sequence to specific modulation bit levels [10]. S-PS leverages this mapping to preserve the intended second-MSB probabilities without introducing an additional permutation after encoding or changing the QAM labeling. Its applicability depends on whether the selected modulation positions can be populated by transmitted systematic bits. This condition is associated primarily with higher code rates, although the exact operating region depends on the LDPC coding and rate-matching configuration.

Figure 2 shows the resulting transmitter and receiver architecture. At the transmitter, the information is partitioned into a subblock associated with the selected second-MSB positions and subblocks that bypass shaping. A binary distribution matcher converts the selected subblock into a longer sequence with the desired probability bias. The subblocks are then combined and passed through the conventional LDPC encoding, rate matching, bit interleaving, and QAM mapping procedures. The transmitted symbols are normalized to the required average power.

At the receiver, soft-information generation accounts for the nonuniform probabilities. A low-complexity implementation retains the conventional uniform-QAM demapper and adds a prior-dependent offset to the log-likelihood ratios (LLRs) of the shaped bit levels. This serves as an effiective approximation to full symbol-prior-aware demapping [2]. For joint MIMO detection, symbol priors can be incorporated directly into candidate metrics while retaining the QAM geometry and underlying distance calculations.

Bit deinterleaving, rate recovery, and LDPC decoding follow the conventional NR procedures. After successful decoding, distribution dematching is applied only to the recovered shaped subblock. The remaining information bypasses dematching, and the subblocks are recombined to recover the original input.

Restricting shaping to two bit levels reduces the matching workload relative to schemes that shape more amplitude bits. The selected subblock can also be divided into independent segments for parallel processing. Segment length controls the trade-off between matching efficiency and latency: longer segments generally improve matching efficiency, whereas shorter segments allow greater parallelism and lower processing latency at the cost of increased finite-length rate loss.

S-PS does not prescribe a particular binary shaping algorithm. CCDM, ESS, or polar-code-based methods can be selected according to throughput, latency, memory, and matching-efficiency requirements. The sequence expansion introduced by matching must also be included in code-block processing and net-rate calculations.

The resulting architecture supports shared processing between shaped and conventional transmission. Uniform-QAM operation can bypass the matcher and dematcher and use uniform priors with the corresponding power normalization. This preserves common coding and modulation blocks while confining shaping-specific processing to selected stages. Nevertheless, shaped operation still requires coordinated transmitter and receiver configurations.

Figure 3 illustrates the performance evaluation in an AWGN channel. The left panel compares the spectral efficiency achieved by S-PS and uniform QAM with Shannon capacity. The evaluated S-PS configurations cover 64-QAM, 256-QAM, and 1024-QAM. Across the illustrated operating points, selective shaping increases the achieved spectral efficiency at a given SNR and reduces the gap to the theoretical bound.

The right panel expresses the same result as a reduction in required SNR at equal spectral efficiency. These gains are obtained from the curves in the left panel and reach approximately 1.5 dB in selected configurations. The magnitude varies with modulation order, coding rate, and shaping probability.

The AWGN evaluation demonstrates that selective second-MSB shaping provides a measurable gain within the NR LDPC-based coding and modulation chain. The result connects the probability-shaping principle to coded transmission performance, with the additional implementation requirements concentrated in matching, dematching, power normalization, and prior-aware receiver processing.

4. Conclusion

Constellation shaping extends modulation design beyond merely increasing the QAM order by optimizing the transmitted signal distribution. The proposed S-PS scheme implements this principle through selective second-MSB shaping aligned with the systematic structure of NR LDPC codes. The AWGN results demonstrate a significant improvement in spectral efficiency while retaining the core NR coding and modulation chain. Selective processing and support for parallel distribution matching provide flexibility in balancing shaping gain, processing throughput, and implementation cost.

These properties are highly relevant to applications requiring sustained high data rates within limited spectrum, including Fixed Wireless Access (FWA), wireless backhaul, and emerging immersive communication services. Where channel conditions support high-order modulation, shaping can improve spectral efficiency or reduce the SNR required for a given information rate. Its practical value depends on achieving these benefits within the modem’s processing and latency constraints. S-PS offers an architecture that addresses both signal efficiency and implementation continuity, establishing a technically meaningful approach to modulation enhancement.