Academia.eduAcademia.edu

Information Theory and coding

description668 papers
group12,819 followers
lightbulbAbout this topic
Information Theory and coding is a mathematical framework for quantifying information, analyzing communication systems, and developing efficient encoding and decoding techniques. It focuses on the representation, transmission, and storage of data, emphasizing the limits of data compression and error correction in digital communication.
lightbulbAbout this topic
Information Theory and coding is a mathematical framework for quantifying information, analyzing communication systems, and developing efficient encoding and decoding techniques. It focuses on the representation, transmission, and storage of data, emphasizing the limits of data compression and error correction in digital communication.

Key research themes

1. How does Information Theory quantify and characterize information and entropy in discrete and finite sources?

This research theme focuses on foundational definitions, quantitative measures, and statistical properties of information and entropy in discrete sources and finite patterns. It explores classical Shannon information theory, its extensions to source coding variability, and combinatorial approaches connecting entropy with pattern complexity. Understanding these quantitative measures is crucial for efficiently encoding information, assessing data compressibility, and elaborating typical sequences in communications and statistical inference.

Key finding: This work provides a rigorous, probability-based definition of information content of messages from discrete sources through the measure -log(probability). It establishes that the information quantity for a sequence is... Read more
Key finding: This paper advances source coding analysis by incorporating the second central moment — the variability or variance — of the information content per source symbol, moving beyond traditional entropy as a mean-only measure. It... Read more
Key finding: This paper formulates a unified combinatorial framework for defining information content and entropy of finite, nonasymptotic patterns compatible with classical theories. It rigorously defines information as the minimal... Read more

2. What are the challenges and limitations when applying classic Information Theory to neural and biological systems?

This theme examines the conceptual and practical difficulties in transferring Shannon’s information-theoretic constructs—originally designed for communication engineering—to the domain of neural communication and brain function. It highlights fundamental mismatches such as the active, time-sensitive nature of biological channels, entropy interpretation confusions, the role of spike timing beyond discrete coding, and the controversial usage of entropy measures without full validity. These insights are essential to refining neural information theory and avoiding misapplications that hinder understanding of brain coding and cognitive processes.

Key finding: This paper critically evaluates the indiscriminate application of Shannon’s information theory to neural communication, emphasizing that classical assumptions (e.g., discrete states, time-invariant channels) often do not... Read more
Key finding: The author presents a historical-critical analysis revealing persistent semantic conflations in neuroscience’s use of Information Theory. Despite Shannon’s warnings that meaning is irrelevant for his theory, neuroscientists... Read more

3. How can advanced mathematical and algorithmic frameworks extend information theory into recursive systems and network coding for complex communication and computation?

This research theme explores recent theoretical advances expanding classical information theory into recursive frameworks for symbolic cognition and fundamental bounds of network coding in wireless networks. It investigates how recursion dynamics underpin observer-integrated symbolic cognition models with feedback coupling, and how network coding gains in throughput and energy are fundamentally limited by constant factors in wireless settings. These investigations provide key insights into the limits and potential of information processing in distributed, multi-layered, and feedback-rich systems.

Key finding: This paper introduces and validates an advanced framework wherein symbolic recursion fields actively interact with external observer layers, extending recursive identity and cognition to include observer-participatory... Read more
Key finding: This work rigorously establishes that in arbitrary wireless networks, the gains of network coding in terms of throughput and energy saving are fundamentally bounded by fixed constant factors, such as a factor of π for... Read more

All papers in Information Theory and coding

This paper proposes that the Qur’an operates as a self-referential recursive system governed by a unified structural pattern termed the 2↔3↔2→7 Pulse. Through Qur’an-only textual analysis, the study identifies recurring topologies of... more
In diffusion based molecular communication, the intersymbol interference (ISI) is an important reason for system performance degradation, which is caused by the random movement, out-of-order arrival and indistinguishability of the... more
In diffusion based molecular communication, the intersymbol interference (ISI) is an important reason for system performance degradation, which is caused by the random movement, out-of-order arrival and indistinguishability of the... more
The Neutral Buoyancy Resonant Chamber (NBRC) represents a theoretical paradigm shift in accelerated regenerative medicine, synthesizing systems chemistry, acoustic cymatics, and synthetic biology into a closed-loop cybernetic system. The... more
AI systems can produce correct outputs at individual steps while failing to maintain consistency across sequences. This paper examines how each transformation introduces small deviations that accumulate over time, leading to... more
In this study, images of cyclic codes in two variable rings with coefficient field p F are detected. A special ring in two variables is defined under certain conditions. The Gray images of the cyclic codes over this ring are investigated.... more
This article presents a unified framework for understanding a class of structural vulnerabilities in contemporary large language models (LLMs), which we term **Structural Zero-Days**. Unlike conventional security vulnerabilities, these... more
This paper presents a structured analysis of how continuous relational interaction modulates the behavior of large language models (LLMs), extending the theoretical foundations introduced in the Manifesto of Relational Technolinguistics.... more
This paper identifies a structural equivalence between three converging lines of inquiry: John Archibald Wheeler's participatory universe and "It from Bit" principle; Roger Penrose's Conformal Cyclic Cosmology (CCC) and its search for... more
Forward Error Correction techniques are utilized for correction of errors at the receiver end. Convolutional encoding is an FEC technique that is particularly suited to a channel in which the transmitted signal is corrupted mainly by... more
This paper introduces a new solution to DNA storage that integrates all three steps of retrieval, namely clustering, reconstruction, and error correction. DNA-correcting codes are presented as a unique solution to the problem of ensuring... more
Constrained coding is a fundamental field in coding theory that tackles efficient communication through constrained channels. While channels with fixed constraints (e.g., a fixed set of substrings may not appear in transmitted messages)... more
This article presents a series of algorithmic techniques for melody generation, inspired by models of music cognition. The techniques are designed for interactive composition, and so privilege brevity, simplicity, and flexibility over... more
Nel panorama sempre più affollato dei social network, è emersa una piattaforma che sta facendo parlare di sé: Moltbook, il "social network for AI agents". Il concetto è affascinante quanto inquietante: un ecosistema digitale dove gli... more
Ekstatika zagrobne egzistencije Nova (pojačano savesna) elita-na-zemlji-ne bi bila ona za-svet-nezainteresovana, s fiksnog udaljenja objavljujuća, nego jedna koja se (kao svud po zemaljskoj kugli razbacana) stvara pod nevidljivim vođstvom... more
Glorifikacija nemoći "... gde se sastaju veličina i smrt, (...), nastaje stvarnost sklona konvenciji, koja po suverenosti ostavlja najsamovoljniji subjektivizam za sobom, jer u tome ono samoindividualno što je već bilo premašilo jednu... more
Ostrvo Mrtvih-drevna kataklizma Namesto nadaleko otplovelog ostrva mrtvih, koje se rastopilo u samom srcu Pra-Sunca, ostade praznina-koja nastavljaše da bruji jednim beskrajnim podsticajem:-ona postade vrelo eteričkog vihora, mesto gde... more
Sinoptički prasvet i jedno drugačije tumačenje tragike Narcisa Jedan sasvim nam nepoznati, nemoguće samobitan svet, u svom izgubljenom ključu davno već beše iznedren iz zemljotresne odluke noći; iz sebe-dokidajućeg duha uobrazilje:-duha... more
"Ako se sve jednom već zbilo, čemu ponavljanje?"
*
U čemu je smisao falsifikata, u telu jedne kosmopoetske revolucije?
*
(poglavlje iz knjige eseja, "Mojrina kontroverza ~ Krilata Boginja u Vremenu pre stvaranja"
Krajnja stvarnost jeste učinak protumačenja fenomena zla, odnosno, kobi slučaja. "Pravda je rezultat, nadolazak"* (Derida). Nepravda dolazi uvek iz centra zakonodavne moći subjekta-ma kakav taj centar po svom ustrojstvu bio. Na mestu... more
This paper presents the Compression and Retrieval System (CARS), a semantic compression framework for artificial intelligence systems. CARS implements the URCA-κ (Universal Recursive Compression Algorithm) with an empirically derived... more
Curiosity is commonly treated in artificial intelligence systems as an external reward signal, exploration heuristic, or optimization strategy. In contrast, this work formalizes curiosity as an intrinsic reinforcement operator acting... more
CODEC - Communications Research Group Departamento de Eletrônica e Sistemas - CTG-UFPE CP 7800, 50711-970, Recife - PE, Brazil e-mail: [email protected] [email protected] ... ABSTRACT A Fast algorithm for the Discrete Hartley Transform (... more
Linear Complementary Dual codes (LCD) are binary linear codes that meet their dual trivially. We construct LCD codes using orthogonal matrices, self-dual codes, combinatorial designs and Gray map from codes over the family of rings R k .... more
Linear Complementary Dual codes (LCD) are binary linear codes that meet their dual trivially. We construct LCD codes using orthogonal matrices, self-dual codes, combinatorial designs and Gray map from codes over the family of rings R k .... more
Materi ini menyajikan panduan lengkap mengenai penggunaan flowchart sebagai alat bantu visual untuk memahami alur logika pemrograman, khususnya bagi siswa kelas VII. Flowchart didefinisikan sebagai representasi grafis yang menggunakan... more
Konsep Berfikir Komputasional
Materi Pembelajaran Koding dan Kecerdasan Artifisial
Materi Kelas V (lima) Sekolah Dasar (SD)
The code that combines channel estimation and error protection has received general attention recently, and has been considered a promising methodology to compensate multi-path fading effect. It has been shown by simulations that such... more
The code that combines channel estimation and error protection has received general attention recently, and has been considered a promising methodology to compensate multi-path fading effect. It has been shown by simulations that such... more
Download research papers for free!