Information Theory
New submissions
[ showing up to 1000 entries per page: fewer | more ]
New submissions for Tue, 21 May 24
- [1] arXiv:2405.11072 [pdf, other]
-
Title: Next-slot OFDM-CSI Prediction: Multi-head Self-attention or State Space Model?Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
The ongoing fifth-generation (5G) standardization is exploring the use of deep learning (DL) methods to enhance the new radio (NR) interface. Both in academia and industry, researchers are investigating the performance and complexity of multiple DL architecture candidates for specific one-sided and two-sided use cases such as channel state estimation (CSI) feedback, CSI prediction, beam management, and positioning. In this paper, we set focus on the CSI prediction task and study the performance and generalization of the two main DL layers that are being extensively benchmarked within the DL community, namely, multi-head self-attention (MSA) and state-space model (SSM). We train and evaluate MSA and SSM layers to predict the next slot for uplink and downlink communication scenarios over urban microcell (UMi) and urban macrocell (UMa) OFDM 5G channel models. Our numerical results demonstrate that SSMs exhibit better prediction and generalization capabilities than MSAs only for SISO cases. For MIMO scenarios, however, the MSA layer outperforms the SSM one. While both layers represent potential DL architectures for future DL-enabled 5G use cases, the overall investigation of this paper favors MSAs over SSMs.
- [2] arXiv:2405.11089 [pdf, ps, other]
-
Title: Optimal Update Policy for the Monitoring of Distributed SourcesComments: Accepted at ISIT 2024Subjects: Information Theory (cs.IT); Distributed, Parallel, and Cluster Computing (cs.DC); Systems and Control (eess.SY)
When making decisions in a network, it is important to have up-to-date knowledge of the current state of the system. Obtaining this information, however, comes at a cost. In this paper, we determine the optimal finite-time update policy for monitoring the binary states of remote sources with a reporting rate constraint. We first prove an upper and lower bound of the minimal probability of error before solving the problem analytically. The error probability is defined as the probability that the system performs differently than it would with full system knowledge. More specifically, an error occurs when the destination node incorrectly determines which top-K priority sources are in the ``free'' state. We find that the optimal policy follows a specific ordered 3-stage update pattern. We then provide the optimal transition points for each stage for each source.
- [3] arXiv:2405.11405 [pdf, ps, other]
-
Title: On the Rate-Distortion Function for Sampled Cyclostationary Gaussian Processes with Memory: Extended Version with ProofsComments: 11 pages, 0 figures, accepted by the 2024 IEEE International Symposium on Information Theory (ISIT 2024)Subjects: Information Theory (cs.IT)
In this work we study the rate-distortion function (RDF) for lossy compression of asynchronously-sampled continuous-time (CT) wide-sense cyclostationary (WSCS) Gaussian processes with memory. As the case of synchronous sampling, i.e., when the sampling interval is commensurate with the period of the cyclostationary statistics, has already been studied, we focus on discrete-time (DT) processes obtained by asynchronous sampling, i.e., when the sampling interval is incommensurate with the period of the cyclostationary statistics of the CT WSCS source process. It is further assumed that the sampling interval is smaller than the maximal autocorrelation length of the CT source process, which implies that the DT process possesses memory. Thus, the sampled process is a DT wide-sense almost cyclostationary (WSACS) processes with memory. This problem is motivated by the fact that man-made communications signals are modelled as CT WSCS processes; hence, applications of such sampling include, e.g., compress-and-forward relaying and recording systems. The main challenge follows because, with asynchronous sampling, the DT sampled process is not information-stable, and hence the characterization of its RDF should be carried out within the information-spectrum framework instead of using conventional information-theoretic arguments. This work expands upon our previous work which addressed the special case in which the DT process is independent across time. The existence of dependence between the samples requires new tools to obtain the characterization of the RDF.
- [4] arXiv:2405.11520 [pdf, other]
-
Title: On Performance of FAS-aided Wireless Powered NOMA Communication SystemsComments: This manuscript has been submitted to the 20th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob)Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
This paper studies the performance of a wireless powered communication network (WPCN) under the non-orthogonal multiple access (NOMA) scheme, where users take advantage of an emerging fluid antenna system (FAS). More precisely, we consider a scenario where a transmitter is powered by a remote power beacon (PB) to send information to the planar NOMA FAS-equipped users through Rayleigh fading channels. After introducing the distribution of the equivalent channel coefficients to the users, we derive compact analytical expressions for the outage probability (OP) in order to evaluate the system performance. Additionally, we present asymptotic OP in the high signal-to-noise ratio (SNR) regime. Eventually, results reveal that deploying the FAS with only one activated port in NOMA users can significantly enhance the WPCN performance compared with using traditional antenna systems (TAS).
- [5] arXiv:2405.11541 [pdf, other]
-
Title: R-NeRF: Neural Radiance Fields for Modeling RIS-enabled Wireless EnvironmentsSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Recently, ray tracing has gained renewed interest with the advent of Reflective Intelligent Surfaces (RIS) technology, a key enabler of 6G wireless communications due to its capability of intelligent manipulation of electromagnetic waves. However, accurately modeling RIS-enabled wireless environments poses significant challenges due to the complex variations caused by various environmental factors and the mobility of RISs. In this paper, we propose a novel modeling approach using Neural Radiance Fields (NeRF) to characterize the dynamics of electromagnetic fields in such environments. Our method utilizes NeRF-based ray tracing to intuitively capture and visualize the complex dynamics of signal propagation, effectively modeling the complete signal pathways from the transmitter to the RIS, and from the RIS to the receiver. This two-stage process accurately characterizes multiple complex transmission paths, enhancing our understanding of signal behavior in real-world scenarios. Our approach predicts the signal field for any specified RIS placement and receiver location, facilitating efficient RIS deployment. Experimental evaluations using both simulated and real-world data validate the significant benefits of our methodology.
- [6] arXiv:2405.11563 [pdf, other]
-
Title: User-Centric Association and Feedback Bit Allocation for FDD Cell-Free Massive MIMOSubjects: Information Theory (cs.IT)
In this paper, we introduce a novel approach to user-centric association and feedback bit allocation for the downlink of a cell-free massive MIMO (CF-mMIMO) system, operating under limited feedback constraints. In CF-mMIMO systems employing frequency division duplexing, each access point (AP) relies on channel information provided by its associated user equipments (UEs) for beamforming design. Since the uplink control channel is typically shared among UEs, we take account of each AP's total feedback budget, which is distributed among its associated UEs. By employing the Saleh-Valenzuela multi-resolvable path channel model with different average path gains, we first identify necessary feedback information for each UE, along with an appropriate codebook structure. This structure facilitates adaptive quantization of multiple paths based on their dominance. We then formulate a joint optimization problem addressing user-centric UE-AP association and feedback bit allocation. To address this challenge, we analyze the impact of feedback bit allocation and derive our proposed scheme from the solution of an alternative optimization problem aimed at devising long-term policies, explicitly considering the effects of feedback bit allocation. Numerical results show that our proposed scheme effectively enhances the performance of conventional approaches in CF-mMIMO systems.
- [7] arXiv:2405.11714 [pdf, ps, other]
-
Title: Generalized regenerating codes and node repair on graphsComments: 29ppSubjects: Information Theory (cs.IT)
We consider regenerating codes in distributed storage systems where connections between the nodes are constrained by a graph. In this problem, the failed node downloads the information stored at a subset of vertices of the graph for the purpose of recovering the lost data. Compared to the standard setting, regenerating codes on graphs address two additional features. The repair information is moved across the network, and the cost of node repair is determined by the graphical distance from the helper nodes to the failed node. Accordingly, the helpers far away from the failed node may be expected to contribute less data for repair than the nodes in the neighborhood of that node. We analyze regenerating codes with nonuniform download for repair on graphs. Moreover, in the process of repair, the information moved from the helpers to the failed node may be combined through intermediate processing, reducing the repair bandwidth. We derive lower bounds for communication complexity of node repair on graphs, including repair schemes with nonuniform download and intermediate processing, and construct codes that attain these bounds.
Additionally, some of the nodes may act as adversaries, introducing errors into the data moved in the network. For repair on graphs in the presence of adversarial nodes, we construct codes that support node repair and error correction in systematic nodes. - [8] arXiv:2405.11734 [pdf, other]
-
Title: Finite Field Multiple Access for Sourced Massive Random Access with Finite BlocklengthSubjects: Information Theory (cs.IT)
For binary source transmission, this paper proposes an element-pair (EP) coding scheme for supporting sourced massive random access, which is used to solve the finite blocklength (FBL) of multiuser reliability transmission problem. In this paper, we first give the definition of an EP, which is used as a virtual resource. If the Cartesian product of $J$ distinct EPs satisfies the unique sum-pattern mapping (USPM) structural property, the $J$ distinct EPs can form an uniquely-decodable EP (UD-EP) code. Then, we introduce a type of orthogonal EP code $\Psi_{\rm o, B}$ constructed over an extension field GF($2^m$). Based on the proposed EP code, we present finite-field multiple-access (FFMA) systems, including both the sparse-form-based and diagonal-form-based forms. Simulation results show that, for the massive random access scenario, the error performance of the proposed FFMA systems over a Gaussian multiple-access channel can provide much better error performance than that of a slotted ALOHA system.
- [9] arXiv:2405.11818 [pdf, other]
-
Title: A Rate-Distortion Analysis for Composite Sources Under Subsource-Dependent Fidelity CriteriaComments: 16 pages, 8 figures, submitted to IEEE Journal on Selected Areas in CommunicationsSubjects: Information Theory (cs.IT)
A composite source, consisting of multiple subsources and a memoryless switch, outputs one symbol at a time from the subsource selected by the switch. If some data should be encoded more accurately than other data from an information source, the composite source model is suitable because in this model different distortion constraints can be put on the subsources. In this context, we propose subsource-dependent fidelity criteria for composite sources and use them to formulate a rate-distortion problem. We solve the problem and obtain a single-letter expression for the rate-distortion function. Further rate-distortion analysis characterizes the performance of classify-then-compress (CTC) coding, which is frequently used in practice when subsource-dependent fidelity criteria are considered. Our analysis shows that CTC coding generally has performance loss relative to optimal coding, even if the classification is perfect. We also identify the cause of the performance loss, that is, class labels have to be reproduced in CTC coding. Last but not least, we show that the performance loss is negligible for asymptotically small distortion if CTC coding is appropriately designed and some mild conditions are satisfied.
- [10] arXiv:2405.11883 [pdf, other]
-
Title: Asynchronous MIMO-OFDM Massive Unsourced Random Access with Codeword CollisionsAuthors: Tianya Li, Yongpeng Wu, Junyuan Gao, Wenjun Zhang, Xiang-Gen Xia, Derrick Wing Kwan Ng, Chengshan XiaoComments: 13 pages, 12 figures, submitted to the IEEE for possible publicationSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
This paper investigates asynchronous MIMO massive unsourced random access in an orthogonal frequency division multiplexing (OFDM) system over frequency-selective fading channels, with the presence of both timing and carrier frequency offsets (TO and CFO) and non-negligible codeword collisions. The proposed coding framework segregates the data into two components, namely, preamble and coding parts, with the former being tree-coded and the latter LDPC-coded. By leveraging the dual sparsity of the equivalent channel across both codeword and delay domains (CD and DD), we develop a message passing-based sparse Bayesian learning algorithm, combined with belief propagation and mean field, to iteratively estimate DD channel responses, TO, and delay profiles. Furthermore, we establish a novel graph-based algorithm to iteratively separate the superimposed channels and compensate for the phase rotations. Additionally, the proposed algorithm is applied to the flat fading scenario to estimate both TO and CFO, where the channel and offset estimation is enhanced by leveraging the geometric characteristics of the signal constellation. Simulations reveal that the proposed algorithm achieves superior performance and substantial complexity reduction in both channel and offset estimation compared to the codebook enlarging-based counterparts, and enhanced data recovery performances compared to state-of-the-art URA schemes.
- [11] arXiv:2405.11996 [pdf, other]
-
Title: Max-Min Fairness and PHY-Layer Design of Uplink MIMO Rate-Splitting Multiple Access with Finite BlocklengthSubjects: Information Theory (cs.IT)
Rate-Splitting Multiple Access (RSMA) has emerged as a potent and reliable multiple access and interference management technique in wireless communications. While downlink Multiple-Input Multiple-Ouput (MIMO) RSMA has been widely investigated, uplink MIMO RSMA has not been fully explored. In this paper, we investigate the performance of uplink RSMA in short-packet communications with perfect Channel State Information at Transmitter (CSIT) and Channel State Information at Receiver (CSIR). We propose an uplink MIMO RSMA framework and optimize both precoders and combiners with Max-Min Fairness (MMF) metric and Finite Blocklength (FBL) constraints. Due to the high coupling between precoders and combiners, we apply the Alternating Optimization (AO) to decompose the optimization problem into two subproblems. To tackle these subproblems, we propose a Successive Convex Approximation (SCA)-based approach. Additionally, we introduce a low-complexity scheme to design the decoding order at the receiver. Subsequently, the Physical (PHY)-layer of the uplink MIMO RSMA architecture is designed and evaluated using multi-user Link-Level Simulations (LLS), accounting for finite constellation modulation, finite length polar codes, message splitting, adaptive modulation and coding, and Successive Interference Cancellation (SIC) at the receiver. Numerical results demonstrate that applying RSMA in uplink MIMO with FBL constraints not only achieves MMF gains over conventional transmission schemes such as Space Division Multiple Access (SDMA) and Non-orthogonal Multiple Access (NOMA) but also exhibits robustness to network loads. The benefits of splitting messages from multiple users are also illustrated. LLS results confirm the improved max-min throughput benefits of RSMA over SDMA and NOMA.
- [12] arXiv:2405.12026 [pdf, other]
-
Title: Enzymatic cycle-based receivers with high input impedance for approximate maximum a posteriori demodulation of concentration modulated signalsAuthors: Chun Tung ChouSubjects: Information Theory (cs.IT)
Molecular communication is a bio-inspired communication paradigm where molecules are used as the information carrier. This paper considers a molecular communication network where the transmitter uses concentration modulated signals for communication. Our focus is to design receivers that can demodulate these signals. We impose three features on our receivers. We want the receivers to use enzymatic cycles as their building blocks, have high input impedance and can work approximately as a maximum a posteriori (MAP) demodulator. No receivers with all these three features exist in the current molecular communication literature. We consider enzymatic cycles because they are a very common class of chemical reactions that are found in living cells. Since a receiver is to be placed in the communication environment, it should ideally have a high input impedance so that it has minimal impact on the environment and on other receivers. Lastly, a MAP receiver has good statistical performance. In this paper, we show how we can use time-scale separation to make an enzymatic cycle to have high input impedance and how the parameters of the enzymatic cycles can be chosen so that the receiver can approximately implement a MAP demodulator. We use simulation to study the performance of this receiver. In particular, we consider an environment with multiple receivers and show that a receiver has little impact on the bit error ratio of a nearby receiver because they have high input impedance.
- [13] arXiv:2405.12073 [pdf, ps, other]
-
Title: Goal-Oriented Communication for Networked Control Assisted by Reconfigurable Meta-SurfacesSubjects: Information Theory (cs.IT); Optimization and Control (math.OC)
In this paper, we develop a theoretical framework for goal-oriented communication assisted by reconfigurable meta-surfaces in the context of networked control systems. The relation to goal-oriented communication stems from the fact that optimization of the phase shifts of the meta-surfaces is guided by the performance of networked control systems tasks. To that end, we consider a networked control system in which a set of sensors observe the states of a set of physical processes, and communicate this information over an unreliable wireless channel assisted by a reconfigurable intelligent surface with multiple reflecting elements to a set of controllers that correct the behaviors of the physical processes based on the received information. Our objective is to find the optimal control policy for the controllers and the optimal phase policy for the reconfigurable intelligent surface that jointly minimize a regulation cost function associated with the networked control system. We characterize these policies, and also propose an approximate solution based on a semi-definite relaxation technique.
- [14] arXiv:2405.12155 [pdf, other]
-
Title: Embracing Radiance Field Rendering in 6G: Over-the-Air Training and Inference with 3D ContentsComments: 15 pages,7 figuresSubjects: Information Theory (cs.IT)
The efficient representation, transmission, and reconstruction of three-dimensional (3D) contents are becoming increasingly important for sixth-generation (6G) networks that aim to merge virtual and physical worlds for offering immersive communication experiences. Neural radiance field (NeRF) and 3D Gaussian splatting (3D-GS) have recently emerged as two promising 3D representation techniques based on radiance field rendering, which are able to provide photorealistic rendering results for complex scenes. Therefore, embracing NeRF and 3D-GS in 6G networks is envisioned to be a prominent solution to support emerging 3D applications with enhanced quality of experience. This paper provides a comprehensive overview on the integration of NeRF and 3D-GS in 6G. First, we review the basics of the radiance field rendering techniques, and highlight their applications and implementation challenges over wireless networks. Next, we consider the over-the-air training of NeRF and 3D-GS models over wireless networks by presenting various learning techniques. We particularly focus on the federated learning design over a hierarchical device-edge-cloud architecture. Then, we discuss three practical rendering architectures of NeRF and 3D-GS models at wireless network edge. We provide model compression approaches to facilitate the transmission of radiance field models, and present rendering acceleration approaches and joint computation and communication designs to enhance the rendering efficiency. In particular, we propose a new semantic communication enabled 3D content transmission design, in which the radiance field models are exploited as the semantic knowledge base to reduce the communication overhead for distributed inference. Furthermore, we present the utilization of radiance field rendering in wireless applications like radio mapping and radio imaging.
- [15] arXiv:2405.12168 [pdf, other]
-
Title: WiDRa -- Enabling Millimeter-Level Differential Ranging Accuracy in Wi-Fi Using Carrier PhaseAuthors: Vishnu V. Ratnam, Bilal Sadiq, Hao Chen, Wei Sun, Shunyao Wu, Boon L. Ng, Jianzhong (Charlie) ZhangComments: Accepted to IEEE JSAC special issue on Positioning and Sensing Over Wireless Networks, 2024Subjects: Information Theory (cs.IT)
Although Wi-Fi is an ideal technology for many ranging applications, the performance of current methods is limited by the system bandwidth, leading to low accuracy of $\sim 1$ m. For many applications, measuring differential range, viz., the change in the range between adjacent measurements, is sufficient. Correspondingly, this work proposes WiDRa - a Wi-Fi based Differential Ranging solution that provides differential range estimates by using the sum-carrier-phase information. The proposed method is not limited by system bandwidth and can track range changes even smaller than the carrier wavelength. The proposed method is first theoretically justified, while taking into consideration the various hardware impairments affecting Wi-Fi chips. In the process, methods to isolate the sum-carrier phase from the hardware impairments are proposed. Extensive simulation results show that WiDRa can achieve a differential range estimation root-mean-square-error (RMSE) of $\approx 1$ mm in channels with a Rician-factor $\geq 7$ (a $100 \times$ improvement to existing methods). The proposed methods are also validated on off-the-shelf Wi-Fi hardware to demonstrate feasibility, where they achieve an RMSE of $< 1$ mm in the differential range. Finally, limitations of current investigation and future directions of exploration are suggested, to further tap into the potential of WiDRa.
- [16] arXiv:2405.12203 [pdf, other]
-
Title: Accelerating Relative Entropy Coding with Space PartitioningComments: 28 pages, 9 figuresSubjects: Information Theory (cs.IT); Machine Learning (cs.LG)
Relative entropy coding (REC) algorithms encode a random sample following a target distribution $Q$, using a coding distribution $P$ shared between the sender and receiver. Sadly, general REC algorithms suffer from prohibitive encoding times, at least on the order of $2^{D_{\text{KL}}[Q||P]}$, and faster algorithms are limited to very specific settings. This work addresses this issue by introducing a REC scheme utilizing space partitioning to reduce runtime in practical scenarios. We provide theoretical analyses of our method and demonstrate its effectiveness with both toy examples and practical applications. Notably, our method successfully handles REC tasks with $D_{\text{KL}}[Q||P]$ about three times what previous methods can manage and reduces the compression rate by approximately 5-15\% in VAE-based lossless compression on MNIST and INR-based lossy compression on CIFAR-10 compared to previous methods, significantly improving the practicality of REC for neural compression.
Cross-lists for Tue, 21 May 24
- [17] arXiv:2405.11195 (cross-list from cs.LG) [pdf, other]
-
Title: Trustworthy Actionable PerturbationsComments: Accepted at the 41st International Conference on Machine Learning (ICML) 2024Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Theory (cs.IT)
Counterfactuals, or modified inputs that lead to a different outcome, are an important tool for understanding the logic used by machine learning classifiers and how to change an undesirable classification. Even if a counterfactual changes a classifier's decision, however, it may not affect the true underlying class probabilities, i.e. the counterfactual may act like an adversarial attack and ``fool'' the classifier. We propose a new framework for creating modified inputs that change the true underlying probabilities in a beneficial way which we call Trustworthy Actionable Perturbations (TAP). This includes a novel verification procedure to ensure that TAP change the true class probabilities instead of acting adversarially. Our framework also includes new cost, reward, and goal definitions that are better suited to effectuating change in the real world. We present PAC-learnability results for our verification procedure and theoretically analyze our new method for measuring reward. We also develop a methodology for creating TAP and compare our results to those achieved by previous counterfactual methods.
- [18] arXiv:2405.11493 (cross-list from cs.CV) [pdf, other]
-
Title: Point Cloud Compression with Implicit Neural Representations: A Unified FrameworkComments: 6 Pages, 6 Figures, submitted to IEEE ICCCSubjects: Computer Vision and Pattern Recognition (cs.CV); Information Theory (cs.IT); Signal Processing (eess.SP)
Point clouds have become increasingly vital across various applications thanks to their ability to realistically depict 3D objects and scenes. Nevertheless, effectively compressing unstructured, high-precision point cloud data remains a significant challenge. In this paper, we present a pioneering point cloud compression framework capable of handling both geometry and attribute components. Unlike traditional approaches and existing learning-based methods, our framework utilizes two coordinate-based neural networks to implicitly represent a voxelized point cloud. The first network generates the occupancy status of a voxel, while the second network determines the attributes of an occupied voxel. To tackle an immense number of voxels within the volumetric space, we partition the space into smaller cubes and focus solely on voxels within non-empty cubes. By feeding the coordinates of these voxels into the respective networks, we reconstruct the geometry and attribute components of the original point cloud. The neural network parameters are further quantized and compressed. Experimental results underscore the superior performance of our proposed method compared to the octree-based approach employed in the latest G-PCC standards. Moreover, our method exhibits high universality when contrasted with existing learning-based techniques.
- [19] arXiv:2405.11684 (cross-list from cs.LG) [pdf, other]
-
Title: Learning Regularities from Data using Spiking Functions: A TheorySubjects: Machine Learning (cs.LG); Information Theory (cs.IT)
Deep neural networks trained in an end-to-end manner are proven to be efficient in a wide range of machine learning tasks. However, there is one drawback of end-to-end learning: The learned features and information are implicitly represented in neural network parameters, which cannot be used as regularities, concepts or knowledge to explicitly represent the data probability distribution. To resolve this issue, we propose in this paper a new machine learning theory, which defines in mathematics what are regularities. Briefly, regularities are concise representations of the non-random features, or 'non-randomness' in the data probability distribution. Combining with information theory, we claim that regularities can also be regarded as a small amount of information encoding a large amount of information. Our theory is based on spiking functions. That is, if a function can react to, or spike on specific data samples more frequently than random noise inputs, we say that such a function discovers non-randomness from the data distribution, and encodes the non-randomness into regularities. Our theory also discusses applying multiple spiking functions to the same data distribution. In this process, we claim that the 'best' regularities, or the optimal spiking functions, are those who can capture the largest amount of information from the data distribution, and then encode the captured information in the most concise way. Theorems and hypotheses are provided to describe in mathematics what are 'best' regularities and optimal spiking functions. Finally, we propose a machine learning approach, which can potentially obtain the optimal spiking functions regarding the given dataset in practice.
- [20] arXiv:2405.12011 (cross-list from math.CO) [pdf, ps, other]
-
Title: Higher weight spectra of ternary codes associated to the quadratic Veronese $3$-foldSubjects: Combinatorics (math.CO); Information Theory (cs.IT)
The problem studied in this work is to determine the higher weight spectra of the Projective Reed-Muller codes associated to the Veronese $3$-fold $\mathcal V$ in $PG(9,q)$, which is the image of the quadratic Veronese embedding of $PG(3,q)$ in $PG(9,q)$. We reduce the problem to the following combinatorial problem in finite geometry: For each subset $S$ of $\mathcal V$, determine the dimension of the linear subspace of $PG(9,q)$ generated by $S$. We develop a systematic method to solve the latter problem. We implement the method for $q=3$, and use it to obtain the higher weight spectra of the associated code. The case of a general finite field $\mathbb F_q$ will be treated in a future work.
- [21] arXiv:2405.12046 (cross-list from cs.LG) [pdf, other]
-
Title: Energy-Efficient Federated Edge Learning with Streaming Data: A Lyapunov Optimization ApproachComments: Submitted to IEEE journals for possible publicationSubjects: Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC); Information Theory (cs.IT); Signal Processing (eess.SP)
Federated learning (FL) has received significant attention in recent years for its advantages in efficient training of machine learning models across distributed clients without disclosing user-sensitive data. Specifically, in federated edge learning (FEEL) systems, the time-varying nature of wireless channels introduces inevitable system dynamics in the communication process, thereby affecting training latency and energy consumption. In this work, we further consider a streaming data scenario where new training data samples are randomly generated over time at edge devices. Our goal is to develop a dynamic scheduling and resource allocation algorithm to address the inherent randomness in data arrivals and resource availability under long-term energy constraints. To achieve this, we formulate a stochastic network optimization problem and use the Lyapunov drift-plus-penalty framework to obtain a dynamic resource management design. Our proposed algorithm makes adaptive decisions on device scheduling, computational capacity adjustment, and allocation of bandwidth and transmit power in every round. We provide convergence analysis for the considered setting with heterogeneous data and time-varying objective functions, which supports the rationale behind our proposed scheduling design. The effectiveness of our scheme is verified through simulation results, demonstrating improved learning performance and energy efficiency as compared to baseline schemes.
- [22] arXiv:2405.12109 (cross-list from cs.CL) [pdf, other]
-
Title: Linguistic Structure from a Bottleneck on Sequential Information ProcessingSubjects: Computation and Language (cs.CL); Information Theory (cs.IT)
Human language is a unique form of communication in the natural world, distinguished by its structured nature. Most fundamentally, it is systematic, meaning that signals can be broken down into component parts that are individually meaningful -- roughly, words -- which are combined in a regular way to form sentences. Furthermore, the way in which these parts are combined maintains a kind of locality: words are usually concatenated together, and they form contiguous phrases, keeping related parts of sentences close to each other. We address the challenge of understanding how these basic properties of language arise from broader principles of efficient communication under information processing constraints. Here we show that natural-language-like systematicity arises from minimization of excess entropy, a measure of statistical complexity that represents the minimum amount of information necessary for predicting the future of a sequence based on its past. In simulations, we show that codes that minimize excess entropy factorize their source distributions into approximately independent components, and then express those components systematically and locally. Next, in a series of massively cross-linguistic corpus studies, we show that human languages are structured to have low excess entropy at the level of phonology, morphology, syntax, and semantics. Our result suggests that human language performs a sequential generalization of Independent Components Analysis on the statistical distribution over meanings that need to be expressed. It establishes a link between the statistical and algebraic structure of human language, and reinforces the idea that the structure of human language may have evolved to minimize cognitive load while maximizing communicative expressiveness.
Replacements for Tue, 21 May 24
- [23] arXiv:2010.06873 (replaced) [pdf, ps, other]
-
Title: Computability of the Zero-Error Capacity of Noisy ChannelsSubjects: Information Theory (cs.IT)
- [24] arXiv:2101.09754 (replaced) [pdf, ps, other]
-
Title: Computability of the Channel Reliability Function and Related BoundsSubjects: Information Theory (cs.IT)
- [25] arXiv:2211.03584 (replaced) [pdf, ps, other]
-
Title: MARS: Message Passing for Antenna and RF Chain Selection for Hybrid Beamforming in MIMO Communication SystemsComments: Accepted by IEEE TCOMSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
- [26] arXiv:2401.02701 (replaced) [pdf, ps, other]
-
Title: Joint User Association and Power Control for Cell-Free Massive MIMOAuthors: Chongzheng Hao, Tung Thanh Vu, Hien Quoc Ngo, Minh N.Dao, Xiaoyu Dang, Chenghua Wang, Michail MatthaiouComments: minor revision of the previous versionSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
- [27] arXiv:2402.00845 (replaced) [pdf, ps, other]
-
Title: When to Preempt in a Status Update System?Subjects: Information Theory (cs.IT); Computer Science and Game Theory (cs.GT); Networking and Internet Architecture (cs.NI); Signal Processing (eess.SP); Systems and Control (eess.SY)
- [28] arXiv:2402.04888 (replaced) [pdf, other]
-
Title: RSCNet: Dynamic CSI Compression for Cloud-based WiFi SensingComments: The paper has been accepted by IEEE International Conference on Communications (ICC) 2024Subjects: Information Theory (cs.IT); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG); Signal Processing (eess.SP)
- [29] arXiv:2403.16477 (replaced) [pdf, other]
-
Title: Safeguarding Next Generation Multiple Access Using Physical Layer Security Techniques: A TutorialAuthors: Lu Lv, Dongyang Xu, Rose Qingyang Hu, Yinghui Ye, Long Yang, Xianfu Lei, Xianbin Wang, Dong In Kim, Arumugam NallanathanComments: Invited paper by Proceedings of the IEEESubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
- [30] arXiv:2405.02904 (replaced) [pdf, ps, other]
-
Title: Distributed Structured Matrix MultiplicationAuthors: Derya MalakComments: Proc., IEEE ISIT 2024Subjects: Information Theory (cs.IT)
- [31] arXiv:2405.07803 (replaced) [pdf, other]
-
Title: Decoding Geometric Properties in Non-Random Data from First Information-Theoretic PrinciplesComments: arXiv:2303.16045 is based on this paper. arXiv admin note: substantial text overlap with arXiv:2303.16045Subjects: Information Theory (cs.IT); Computation and Language (cs.CL); Cryptography and Security (cs.CR); Information Retrieval (cs.IR); Statistics Theory (math.ST)
- [32] arXiv:2405.10007 (replaced) [pdf, ps, other]
-
Title: Sampling Theorem and interpolation formula for non-vanishing signalsAuthors: Nikolai DokuchaevComments: arXiv admin note: substantial text overlap with arXiv:2405.05566Subjects: Information Theory (cs.IT)
- [33] arXiv:2405.10496 (replaced) [pdf, other]
-
Title: Electromagnetic Information Theory for Holographic MIMO CommunicationsAuthors: Li Wei, Tierui Gong, Chongwen Huang, Zhaoyang Zhang, Wei E. I. Sha, Zhi Ning Chen, Linglong Dai, Merouane Debbah, Chau YuenSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
- [34] arXiv:2204.12723 (replaced) [pdf, ps, other]
-
Title: On the limitations of data-based price discriminationSubjects: Computer Science and Game Theory (cs.GT); Information Theory (cs.IT); Machine Learning (cs.LG); Theoretical Economics (econ.TH)
- [35] arXiv:2307.04191 (replaced) [pdf, ps, other]
-
Title: On the sample complexity of parameter estimation in logistic regression with normal designSubjects: Statistics Theory (math.ST); Information Theory (cs.IT); Machine Learning (cs.LG); Machine Learning (stat.ML)
- [36] arXiv:2310.06604 (replaced) [pdf, other]
-
Title: Near and Far Field Model Mismatch: Implications on 6G Communications, Localization, and SensingComments: This work has been accepted by IEEE Internet of Things Magazine (IoTM). Copyright may be transferred without notice, after which this version may no longer be accessibleSubjects: Signal Processing (eess.SP); Information Theory (cs.IT)
- [37] arXiv:2310.07972 (replaced) [pdf, other]
-
Title: Interpretable Diffusion via Information DecompositionComments: 32 pages, 18 figuresSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Theory (cs.IT)
- [38] arXiv:2402.08274 (replaced) [pdf, ps, other]
-
Title: Nearly Orthogonal Sets over Finite FieldsComments: 19 pagesSubjects: Computational Geometry (cs.CG); Discrete Mathematics (cs.DM); Information Theory (cs.IT); Combinatorics (math.CO)
- [39] arXiv:2402.15603 (replaced) [pdf, ps, other]
-
Title: Differentially Private Fair Binary ClassificationsSubjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Information Theory (cs.IT); Machine Learning (stat.ML)
[ showing up to 1000 entries per page: fewer | more ]
Disable MathJax (What is MathJax?)
Links to: arXiv, form interface, find, cs, recent, 2405, contact, help (Access key information)