This paper identifies convolutional codes (CCs) used in conjunction with a CC-specific cyclic redundancy check (CRC) code as a promising paradigm for short blocklength codes. The resulting CRC-CC concatenated code naturally permits the use of the ser
ial list Viterbi decoding (SLVD) to achieve maximum-likelihood decoding. The CC of interest is of rate-$1/omega$ and is either zero-terminated (ZT) or tail-biting (TB). For CRC-CC concatenated code designs, we show how to find the optimal CRC polynomial for a given ZTCC or TBCC. Our complexity analysis reveals that SLVD decoding complexity is a function of the terminating list rank, which converges to one at high SNR. This behavior allows the performance gains of SLVD to be achieved with a small increase in average complexity at the SNR operating point of interest. With a sufficiently large CC constraint length, the performance of CRC-CC concatenated code under SLVD approaches the random-coding union (RCU) bound as the CRC size is increased while average decoding complexity does not increase significantly. TB encoding further reduces the backoff from the RCU bound by avoiding the termination overhead. As a result, several CRC-TBCC codes outperform the RCU bound at moderate SNR values while permitting decoding with relatively low complexity.
Federated learning (FL) is a fast-developing technique that allows multiple workers to train a global model based on a distributed dataset. Conventional FL employs gradient descent algorithm, which may not be efficient enough. It is well known that N
esterov Accelerated Gradient (NAG) is more advantageous in centralized training environment, but it is not clear how to quantify the benefits of NAG in FL so far. In this work, we focus on a version of FL based on NAG (FedNAG) and provide a detailed convergence analysis. The result is compared with conventional FL based on gradient descent. One interesting conclusion is that as long as the learning step size is sufficiently small, FedNAG outperforms FedAvg. Extensive experiments based on real-world datasets are conducted, verifying our conclusions and confirming the better convergence performance of FedNAG.
Building on the work of Horstein, Shayevitz and Feder, and Naghshvar emph{et al.}, this paper presents algorithms for low-complexity sequential transmission of a $k$-bit message over the binary symmetric channel (BSC) with full, noiseless feedback. T
o lower complexity, this paper shows that the initial $k$ binary transmissions can be sent before any feedback is required and groups messages with equal posteriors to reduce the number of posterior updates from exponential in $k$ to linear in $k$. Simulation results demonstrate that achievable rates for this full, noiseless feedback system approach capacity rapidly as a function of average blocklength, faster than known finite-blocklength lower bounds on achievable rate with noiseless active feedback and significantly faster than finite-blocklength lower bounds for a stop feedback system.
Cyclic redundancy check (CRC) codes combined with convolutional codes yield a powerful concatenated code that can be efficiently decoded using list decoding. To help design such systems, this paper presents an efficient algorithm for identifying the
distance-spectrum-optimal (DSO) CRC polynomial for a given tail-biting convolutional code (TBCC) when the target undetected error rate (UER) is small. Lou et al. found that the DSO CRC design for a given zero-terminated convolutional code under low UER is equivalent to maximizing the undetected minimum distance (the minimum distance of the concatenated code). This paper applies the same principle to design the DSO CRC for a given TBCC under low target UER. Our algorithm is based on partitioning the tail-biting trellis into several disjoint sets of tail-biting paths that are closed under cyclic shifts. This paper shows that the tail-biting path in each set can be constructed by concatenating the irreducible error events (IEEs) and circularly shifting the resultant path. This motivates an efficient collection algorithm that aims at gathering IEEs, and a search algorithm that reconstructs the full list of error events with bounded distance of interest, which can be used to find the DSO CRC. Simulation results show that DSO CRCs can significantly outperform suboptimal CRCs in the low UER regime.
In this paper, we consider the problem of sequential transmission over the binary symmetric channel (BSC) with full, noiseless feedback. Naghshvar et al. proposed a one-phase encoding scheme, for which we refer to as the small-enough difference (SED)
encoder, which can achieve capacity and Burnashevs optimal error exponent for symmetric binary-input channels. They also provided a non-asymptotic upper bound on the average blocklength, which implies an achievability bound on rates. However, their achievability bound is loose compared to the simulated performance of SED encoder, and even lies beneath Polyanskiys achievability bound of a system limited to stop feedback. This paper significantly tightens the achievability bound by using a Markovian analysis that leverages both the submartingale and Markov properties of the transmitted message. Our new non-asymptotic lower bound on achievable rate lies above Polyanskiys bound and is close to the actual performance of the SED encoder over the BSC.
This paper studies the joint design of optimal convolutional codes (CCs) and CRC codes when serial list Viterbi algorithm (S-LVA) is employed in order to achieve the target frame error rate (FER). We first analyze the S-LVA performance with respect t
o SNR and list size, repsectively, and prove the convergence of the expected number of decoding attempts when SNR goes to the extreme. We then propose the coded channel capacity as the criterion to jointly design optimal CC-CRC pair and optimal list size and show that the optimal list size of S-LVA is always the cardinality of all possible CCs. With the maximum list size, we choose the design metric of optimal CC-CRC pair as the SNR gap to random coding union (RCU) bound and the optimal CC-CRC pair is the one that achieves a target SNR gap with the least complexity. Finally, we show that a weaker CC with a strong optimal CRC code could be as powerful as a strong CC with no CRC code.
Let $X^n$ be a uniformly distributed $n$-dimensional binary vector, and $Y^n$ be the result of passing $X^n$ through a binary symmetric channel (BSC) with crossover probability $alpha$. A recent conjecture postulated by Courtade and Kumar states that
for any Boolean function $f:{0,1}^nto{0,1}$, $I(f(X^n);Y^n)le 1-H(alpha)$. Although the conjecture has been proved to be true in the dimension-free high noise regime by Samorodnitsky, here we present a calculus-based approach to show a dimension-dependent result by examining the second derivative of $H(alpha)-H(f(X^n)|Y^n)$ at $alpha=1/2$. Along the way, we show that the dictator function is the most informative function in the high noise regime.
A Viterbi-like decoding algorithm is proposed in this paper for generalized convolutional network error correction coding. Different from classical Viterbi algorithm, our decoding algorithm is based on minimum error weight rather than the shortest Ha
mming distance between received and sent sequences. Network errors may disperse or neutralize due to network transmission and convolutional network coding. Therefore, classical decoding algorithm cannot be employed any more. Source decoding was proposed by multiplying the inverse of network transmission matrix, where the inverse is hard to compute. Starting from the Maximum A Posteriori (MAP) decoding criterion, we find that it is equivalent to the minimum error weight under our model. Inspired by Viterbi algorithm, we propose a Viterbi-like decoding algorithm based on minimum error weight of combined error vectors, which can be carried out directly at sink nodes and can correct any network errors within the capability of convolutional network error correction codes (CNECC). Under certain situations, the proposed algorithm can realize the distributed decoding of CNECC.
