{"id":316,"date":"2021-02-10T23:45:55","date_gmt":"2021-02-10T23:45:55","guid":{"rendered":"https:\/\/staging.cdcharalambous.com\/?page_id=316"},"modified":"2021-02-17T13:34:54","modified_gmt":"2021-02-17T13:34:54","slug":"journal-publications","status":"publish","type":"page","link":"https:\/\/staging.cdcharalambous.com\/index.php\/journal-publications\/","title":{"rendered":"Journal Publications"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t

Articles in Journals (In reverse chronological order):<\/i><\/u><\/span><\/h4><\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

C.D. Charalambous and C. Kourtellaris, \u201cTime-Invariant Feedback Strategies Do not Increase Capacity of AGN Channels Driven by Stable and Unstable Autoregressive,\u201dIEEE Transactions on Information Theory, pages-54, 27 July 2019 (revised 26 July 2020)\u00a0External Link<\/a><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

The capacity of additive Gaussian noise (AGN) channels with feedback, when the noise is described by, stable and unstable, autoregressive models, is characterized, under the condition that, channel input feedback strategies are time-invariant and induce asymptotic stationarity, and ergodicity of the channel output process. This condition is more relaxed than imposing stationarity or asymptotic stationarity of the joint input and output process, as done in [1]. Based on this characterization, new closed form capacity formulas, and lower bounds are derived, for the stable and unstable AR(c) noise, Vt=cVt\u22121+Wt,V0=v0,t=1,\u2026,n, where c\u2208(\u2212\u221e,\u221e), Wt,t=1,\u2026,n, is a zero mean, variance KW, independent Gaussian sequence, independent of V0. Our capacity formulas are fundamentally different from existing formulas found in the literature. The new formulas illustrate multiple regimes of capacity, as a function of the parameters (c,KW,\u03ba), where \u03ba is the total average power allocated to the transmitter. In particular, 1) feedback increases capacity for the regime, c2\u2208(1,\u221e), for \u03ba>KW(c2\u22121)2, 2) feedback does not increase capacity for the regime c2\u2208(1,\u221e), for \u03ba\u2264KW(c2\u22121)2, and 3) feedback does not increase capacity for the regime c\u2208[\u22121,1], for \u03ba\u2208[0,\u221e). We show that our disagreement with [1] is mainly attributed to the use of necessary and sufficient conditions for convergence of generalized difference Riccati equations (DREs) to analogous generalized algebraic Riccati equations (AREs), of Gaussian estimation problems, known as detectability and stabilizability conditions, to ensure asymptotic stationarity and ergodicity of the channel output process.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
C.D. Charalambous, T. Charalambous, C. Kourtellaris, and Jan H. van Schuppen, \u201cComplete Characterization of Gorbunov and Pinsker Nonanticipatory Epsilon Entropy of Multivariate Gaussian Sources: Structural Properties,\u201d IEEE Transactions on Information Theory, pages-48, 26 June, 2020\u00a0<\/span>External Link<\/a><\/div>
\u00a0<\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

The complete characterization of the Gorbunov and Pinsker [1], [2] nonanticipatory epsilon entropy of multivariate Gauss-Markov sources with square-error fidelity is derived, which remained an open problem since 1974. Specifically, it is shown that the optimal matrices of the stochastic realization of the optimal test channel or reproduction distribution, admit spectral representations with respect to the same unitary matrices, and that the optimal reproduction process is generated, subject to pre-processing and post-processing by memoryless parallel additive Gaussian noise channels. The derivations and analyses are new and bring out several properties of such optimization problems over the space of conditional distributions and their realizations.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n

T. Charalambous and C.D. Charalambous, \u201cCharacterizations of Nonanticipative Rate Distortion Function for Multivariate Nonstationary Gaussian Finite-Memory Autoregressive Processes,\u201d IEEE Journal on Selected Areas in Information Theory, Special Issue on Estimation and Inference pages-34, May 15, 2020.<\/p>


<\/p>\n

\n<\/p>

\n

 <\/span>
<\/p>\n

\n<\/p><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/section>\n

\n
\n
\n
\n
\n
\n
\n

<\/div>\n

<\/div>
C.D. Charalambous and C. Kourtellaris and I. Tzortzis, \u201cOptimal Control and Signalling Strategies of Control-Coding Capacity of General Decision Models: Applications to Gaussian Models and Decentralized Strategies,\u201d SIAM Journal on Control and Optimization, pages-28, 3 February 2020.<\/span>
<\/span><\/div>\n
\n
 <\/div>\n
\n<\/div>\n
 <\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/section>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/section>\n<\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

C.D. Charalambous and C.Kourtellaris and I. Tziortzis, \u201cErgodic Control Coding Capacity of Stochastic Control Systems:\u201d Information Signalling and Hierarchical Optimality of Gaussian Systems,\u201d SIAM Journal on Control And Optimization, vol.58, no.1, pages 104-135, January 2020\u00a0External Link<\/a><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

The control-coding (CC) capacity of dynamical decision models (DMs) is defined as the maximum amount of information transfer per unit time from its inputs to its outputs, called CC rate $R$ in bits\/second, which is operational with the aid of a controller-encoder and a decoder, as in Shannon\u2019s mathematical theory of communication over noisy channels, with the encoder replaced by a controller-encoder [C. Kourtellaris and C. D. Charalambous, IEEE Trans. Inform. Theory, 64 (2018), pp. 4962\u20134992]. In the first part of the paper, data processing inequalities and information structures of optimal controllers-encoders are derived for controlled and uncontrolled information processes. Further, the ergodic theory of a Markov decision is applied to establish direct and converse CC theorems. In the second part of the paper, the problem of signalling information via a Gaussian decision model (G-DM) subject to a quadratic cost constraint to another Gaussian control system (G-CS) with a quadratic pay-off is investigated. A hierarchical decomposition and decentralized optimality of linear quadruple of strategies, \\controller-encoder, decoder, controller\\, is shown by construction as follows: (i) the controller-encoder simultaneously controls the output of the G-DM and encodes the state of the G-CS, and operates at the CC capacity of the G-DM; (ii) the decoder is optimal with respect to a mean square error (MSE) criterion; and (iii) the controller of the G-CS which acts on the decoder\u2019s information is optimal.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

I.Tziortzis, C.D. Charalambous and T.Charalambous, \u201cInfinite Horizon Average Dynamic Programming Subject to Total Variation Ambiguity\u201d SIAM Journal on Control and Optimization, Vol.57, No4, pp.2843-2872, 2019\u00a0External Link<\/a><\/p>

\u00a0<\/div><\/div><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

This paper addresses the optimality of stochastic control strategies based on the infinite horizon average cost criterion, subject to total variation distance ambiguity on the conditional distribution of the controlled process. This stochastic optimal control problem is formulated using minimax theory, in which the minimization is over the control strategies and the maximization is over the conditional distributions. Under the assumption that, for every stationary Markov control law the maximizing conditional distribution of the controlled process is irreducible, we derive a new dynamic programming recursion which minimizes the future ambiguity, and we propose a new policy iteration algorithm. The new dynamic programming recursion includes, in addition to the standard terms, the oscillator semi-norm of the cost-to-go. The maximizing conditional distribution is found via a water-filling algorithm. The implications of our results are demonstrated through an example.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

P. Stavrou and J. Ostergaard and C.D. Charalambous, \u201cZero-Delay Rate Distortion via Filtering for Vector-Valued Gaussian Sources,\u201d IEEE Journal on Selected Topics on Signal Processing, vol.12, no.5, pp.841-856, 2018\u00a0External Link<\/a>\u00a0<\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

We deal with zero-delay source coding of a vector-valued Gauss-Markov source subject to a mean-squared error (MSE) fidelity criterion characterized by the operational zero-delay vector-valued Gaussian rate distortion function (RDF). We address this problem by considering the nonanticipative RDF (NRDF), which is a lower bound to the causal optimal performance theoretically attainable function (or simply causal RDF) and operational zero-delay RDF. We recall the realization that corresponds to the optimal \u201ctest-channel\u201d of the Gaussian NRDF, when considering a vector Gauss-Markov source subject to a MSE distortion in the finite time horizon. Then, we introduce sufficient conditions to show existence of solution for this problem in the infinite time horizon (or asymptotic regime). For the asymptotic regime, we use the asymptotic characterization of the Gaussian NRDF to provide a new equivalent realization scheme with feedback, which is characterized by a resource allocation (reverse-waterfilling) problem across the dimension of the vector source. We leverage the new realization to derive a predictive coding scheme via lattice quantization with subtractive dither and joint memoryless entropy coding. This coding scheme offers an upper bound to the operational zero-delay vector-valued Gaussian RDF. When we use scalar quantization, then for r active dimensions of the vector Gauss-Markov source the gap between the obtained lower and theoretical upper bounds is less than or equal to 0.254r + 1 bits\/vector. However, we further show that it is possible when we use vector quantization, and assume infinite dimensional Gauss-Markov sources to make the previous gap to be negligible, i.e., Gaussian NRDF approximates the operational zero-delay Gaussian RDF. We also extend our results to vector-valued Gaussian sources of any finite memory under mild conditions. Our theoretical framework is demonstrated with illustrative numerical experiments.<\/span>
<\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Photios Stavrou and T. Charalambous, C.D. Charalambous, and S. Loyka, \u201cOptimal Estimation via Nonanticipative Rate Distortion Function and Applications for Time-Varying Gauss-Markov Processes,\u201d SIAM Journal on Control and Optimization, vol.56, no.5, pp.3731-3765, 2018\u00a0External Link<\/a><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

In this paper, we develop finite-time horizon causal filters for general processes taking values in Polish spaces using the nonanticipative rate distortion function ($NRDF$). Subsequently, we apply the $NRDF$ to design optimal filters for time-varying vector-valued Gauss\u2013Markov processes, subject to a mean-squared error ($MSE$) distortion. Unlike the classical Kalman filter design, the developed filters based on the $NRDF$ are characterized parametrically by a dynamic reverse-waterfilling optimization problem obtained via Karush\u2013Kuhn\u2013Tucker conditions. We develop algorithms that provide, in general, tight upper bounds to the optimal solution to the dynamic reverse-waterfilling optimization problem subject to a total and per-letter $MSE$ distortion constraint. Under certain conditions, these algorithms produce the optimal solutions. Further, we establish a universal lower bound on the total and per-letter $MSE$ of any estimator of a Gaussian random process. Our theoretical framework is demonstrated via simple examples.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

P. Stavrou and T. Charalambous and C.D. Charalambous, \u201cFinite-Time Nonanticipative Rate Distortion Function for Time-Varying Scalar-Valued Gauss-Markov Sources,\u201d IEEE Control Systems Letters, vol.2, issue 1, pages 175-180, January 2018\u00a0External Link<\/a><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

We derive the finite-time horizon nonanticipative rate distortion function (NRDF) of timevarying scalar Gauss-Markov sources under an average mean squared-error (MSE) distortion fidelity. Further, we show that a conditionally Gaussian reproduction process realizes the optimal reproduction distribution, and this is determined from the solution of a dynamic reversewaterfilling optimization problem. We provide an iterative algorithm that approximates the solution of the dynamic reverse-waterfilling problem. From the above results, we also obtain, as a special case, the NRDF under a per-letter orpointwise MSE distortion fidelity, and we draw connections to the classical RDF of Gaussian processes. Our results are corroborated with illustrative examples.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

C.D. Charalambous and C. Kourtellaris and S. Loyka, \u201cCapacity Achieving Distributions and Separation Principle for Feedback Channels with Memory: The LQG Theory of Directed Information,\u201d IEEE Transactions on Information Theory, vol.64, no.9, pages 6384-6418, September 2018\u00a0External Link<\/a><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

A method is developed to realize optimal channel input conditional distributions, which maximize the finite transmission feedback information (FTFI) capacity, often called n -block length feedback capacity, by information lossless randomized strategies. The method is applied to compute closed form expressions for the FTFI capacity and feedback capacity, of nonstationary, nonergodic, unstable, multiple input multiple output Gaussian channels with memory on past channel outputs, subject to average transmission cost constraints of quadratic form in the channel inputs and outputs. It is shown that randomized strategies decompose into two orthogonal parts-an deterministic part, which controls the channel output process, and an innovation part, which transmits new information over the channel. Then a separation principle is shown between the computation of the optimal deterministic part and the random part of the optimal randomized strategies. Finally, the ergodic theory of linear-quadratic-Gaussian stochastic optimal control theory, is applied to identify sufficient conditions, expressed in terms of solutions to matrix difference and algebraic Riccati equations, so that the optimal control part of randomized strategies induces asymptotic stationarity and ergodicity, and feedback capacity is characterized by the per unit time limit of the FTFI capacity. The method reveals an interaction of the control and the information transmission parts of the optimal randomized strategies, and that whether feedback increases capacity, is directly related to the channel parameters and the transmission cost function, through the solutions of the matrix Riccati equations. For unstable channels, it is shown that feedback capacity exists and it is strictly positive, provided the power exceeds a critical threshold.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

C. Kourtellaris and C.D. Charalambous, \u201cInformation Structures of Capacity Achieving Distributions for Feedback Channels with Memory and Transmission Cost: Stochastic Optimal Control & Variational Equalities,\u201d IEEE Transactions on Information Theory, Vol.64, No.7, pp.4962-4992, July, 2018\u00a0External Link<\/a><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

The information structures of the optimal channel input distributions P [0,n] = \u0394 {P Ai|Ai-1, Bi-1 : i = 0,1,\u2026, n}, which correspond to the extremum problem of feedback capacity C An\u2192Bn FB = \u0394 sup P [0,n] \u03a3 i=0 n I(A i ;B i |B i-1 ) are identified, for any class of channel distributions {P Bi|Bi-1,Ai : i = 0,1,\u2026,n} and {P Bi|B i-M i-1 ,Ai : i = 0,1,\u2026,n}, where B n = \u0394 {B j : j = 0,1,\u2026,n} are the channel output RVs, A n = \u0394 {A j : j = 0,1,\u2026,n} are the channel inputs RVs, and M is a finite nonnegative integer. The methodology utilizes stochastic optimal control theory, to identify the control process, the controlled process, and a variational equality of directed information, to derive upper bounds on I(A n \u2192 B n )= \u0394 \u03a3 i=0 n I(A i ;B i |B i-1 ), which are achievable over specific subsets of P [0,n] , which satisfy conditional independence. The main theorem states, that for any channel with memory M, the optimal channel input conditional distribution occur in the subset P [0,n] = \u0394 {P Ai|B i-M i-1 : i = 1,\u2026,n} \u2282 P [0,n] , and the corresponding extremum problem simplifies to the following characterization C An\u2192Bn FB,M = \u0394 sup P [0,n] \u03a3 i=0 n I(A i ;B i |B i-M i-1 ).<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

C.D. Charalambous and N.U. Ahmed, \u201cCentralized Versus Decentralized Team Optimality Conditions of Distributed Stochastic Differential Decision Systems with Different Information Structures-Part II: Applications,\u201d IEEE Transactions on Automatic Control, vol.63, no.7, pp.1913-1928, 2018\u00a0External Link<\/a><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

In this second part of the two-part paper, a stochastic maximum principle, conditional Hamiltonians, and the coupled backward-forward stochastic differential equations of the first part [1] are employed to derive decentralized team optimal strategies for distributed decision systems with different information structures. Examples of such team problems are presented in nonlinear and linear quadratic forms. In many cases, the expressions of the optimal decentralized strategies are obtained. An interesting feature of any one optimal decentralized strategy is the dependence on the conditional estimates of the other optimal responses. For team problems of linear quadratic form and independent nonanticipative information structures, any optimal strategy depends linearly on the mean values of the other optimal responses. This property makes their computation feasible, even for large-scale distributed systems with many decision makers (DMs). It is also related to mean field stochastic optimal control problems, with finite number of DMs. The examples presented illustrate the effect of information signaling among the DMs in reducing the computational complexity of optimal decentralized strategies.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

C.D. Charalambous and C. Kourtellaris and I. Tzortzis, \u201cInformation Transfer of Control Strategies: Dualities of Stochastic Optimal Control Theory and Feedback Capacity of Information Theory,\u201d IEEE Transactions on Automatic Control, vol.62, no.10, pp.5010-5025, November 2017\u00a0External Link<\/a><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

The control-coding capacity of stochastic control systems is introduced, and its operational meaning is established using randomized control strategies, which simultaneously control output processes encode information, and communicate information from control processes to output processes. The control-coding capacity is the analog Shannon\u2019s coding-capacity of noisy channels. Furthermore, duality relations to stochastic optimal control problems with deterministic and randomized control strategies are identified including the following. First, extremum problems of stochastic optimal control with directed information payoff are equivalent to feedback capacity problems of information theory, in which the control system act as a communication channel. Second, for Gaussian linear decision models with average quadratic constraints, it is shown that optimal randomized strategies are Gaussian, and decompose into a deterministic part and a random part. The deterministic part is precisely the optimal strategy of the linear quadratic Gaussian stochastic optimal control problem, whereas the random part is the solution of an water-filling information transmission problem that encodes information, which is estimated by a decoder.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Photios Stavrou and C.D. Charalambous and C. Kourtellaris, \u201cSequential Necessary and Sufficient Conditions for Capacity Achieving Distributions of Channels with Memory and Feedback,\u201d IEEE Transactions on Information Theory, vol.63, no.11, pp.7095-7115, July 2017\u00a0External Link<\/a><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

We derive sequential necessary and sufficient conditions for any channel input conditional distribution P 0,n =\u0394 {P Xt|Xt-1,Yt-1 : t = 0, \u2026, n} to maximize the finite-time horizon directed information defined by C Xn\u2192Yn FB =\u0394 sup P0,n I(X n \u2192 Y n ), where I(X n \u2192 Y n ) = \u03a3 t=0 n I(X t ; Y t |Y t-1 ), for channel distributions {P Yt |Y t-1 ,Xt: t = 0, \u2026, n} and {P Yt |Y t-M t-1 ,X t : t = 0, \u2026, n}, where Y t =\u0394 {Y -1 , Y 0 , \u2026, Y t } and X t =\u0394 {X 0 , \u2026, X t } are the channel input and output random processes, and M is a finite non-negative integer. We apply the necessary and sufficient conditions to application examples of time-varying channels with memory to derive recursive closed form expressions of the optimal distributions, which maximize the finite-time horizon directed information. Furthermore, we derive the feedback capacity from the asymptotic properties of the optimal distributions by investigating the limit C X\u221e\u2192Y\u221e FB =\u0394 lim n\u2192\u221e (1\/(n + 1))C Xn\u2192Yn FB without any \u00e1 priori assumptions, such as stationarity, ergodicity, or irreducibility of the channel distribution. The framework based on sequential necessary and sufficient conditions can be easily applied to a variety of channels with memory, beyond the ones considered in this paper.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

C.D. Charalambous and N.U. Ahmed, \u201cTeam Games Optimality Conditions of Distributed Stochastic Differential Decision Systems with Decentralized Noisy Information Structures,\u201d IEEE Transactions on Automatic Control, vo.62, no.2, pp.708-723, June 2017\u00a0External Link<\/a><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

We derive team and Person-by-Person (PbP) optimality conditions using the stochastic Pontryagin\u2019s maximum principle, for distributed stochastic differential decision systems with decentralized noisy information structures. The optimality conditions are given by a Hamiltonian system consisting of forward and backward stochastic differential equations, and conditional Hamiltonians. We also show that, under global convexity conditions, PbP optimality implies team optimality. Finally, we apply the stochastic maximum principle to examples from communications, filtering and control.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

C.D. Charalambous and N.U. Ahmed, \u201cCentralized Versus Decentralized Team Optimality of Distributed Stochastic Differential Decision Systems with Different Information Structures-Part I: General Theory,\u201d IEEE Transactions on Automatic Control, vol.62, issue, 3, pages 1194-1209, March 2017\u00a0External Link<\/a>\u00a0<\/span><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

Decentralized optimization of distributed stochastic dynamical systems with two or more controls of the decision makers (DMs) has been an active area of research for over half a century. Although, such decentralized optimization problems are often formulated utilizing static team and person-by-person (PbP) optimality criteria, the corresponding static team theory results have not been extended to dynamical systems. In this first part of the two-part paper, we derive team and PbP optimality conditions for distributed stochastic differential systems, when the controls of the DMs generate actions based on different information structures. The necessary conditions are given by a Hamiltonian System described by coupled backward and forward stochastic differential equations (SDEs) and a conditional Hamiltonian, conditioned on the information structures available to the controls of the DMs. The sufficient conditions state that PbP optimality implies team optimality, if the Hamiltonian is convex in the state and\/or actions spaces of the controls of the DMs. We show existence of relaxed team optimal strategies, when the information structures are not affected by the controls of the DMs. Throughout the paper we discuss similarities to analogous optimality conditions of centralized decision or control of stochastic systems, and we note a connections to mean field stochastic optimal control problems.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

I. Tziortzis, C.D. Charalambous, T. Charalambous, C. Hadjicostis, M. Johansson, \u201cApproximation of Markov Processes by Lower Dimensional Processes via Total Variation Metrics,\u201d IEEE Transactions on Automatic Control, vol.62, no.3, pp.1030-1045, March 2017\u00a0External Link<\/a><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

The aim of this paper is to approximate a Finite-State Markov (FSM) process by another process defined on a lower dimensional state space, called the approximating process, with respect to a total variation distance fidelity criterion. The approximation problem is formulated as an optimization problem using two different approaches. The first approach is based on approximating the transition probability matrix of the FSM process by a lower-dimensional transition probability matrix, resulting in an approximating process which is a Finite-State Hidden Markov (FSHM) process. The second approach is based on approximating the invariant probability vector of the original FSM process by another invariant probability vector defined on a lower-dimensional state space. Going a step further, a method is proposed based on optimizing a Kullback-Leibler divergence to approximate the FSHM processes by FSM processes. The solutions of these optimization problems are described by optimal partition functions which aggregate the states of the FSM process via a corresponding water-filling solution, resulting in lower-dimensional approximating processes which are FSHM or FSM processes. Throughout the paper, the theoretical results are justified by illustrative examples that demonstrate our proposed methodology.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

S. Loyka and C.D. Charalambous, \u201cOptimal Signaling for Secure Communications Over Gaussian MIMO Wiretap Channels,\u201d IEEE Transactions on Information Theory, vol.62, no.12, pp. 7207-7215, December 2016\u00a0External Link<\/a><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

Optimal signaling over the Gaussian multiple-input multiple-output wire-tap channel is studied under the total transmit power constraint. A closed-form solution for an optimal transmit covariance matrix is obtained when the channel is strictly degraded. In combination with the rank-1 solution, this provides the complete characterization of the optimal covariance for the case of two transmit antennas. The cases of weak eavesdropper and high SNR are considered. It is shown that the optimal covariance does not converge to a scaled identity in the high-SNR regime. Necessary optimality conditions and a tight upper bound on the rank of an optimal covariance matrix are established for the general case, along with a lower bound to the secrecy capacity, which is tight in a number of scenarios.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

C.D. Charalambous and Photios A. Stavrou, \u201cDirected Information on Abstract Spaces: Properties and Extremum Problems,\u201d IEEE Transactions on Information Theory, vol.62, no.11, pages 6019-6052, November 2016\u00a0External Link<\/a><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

Directed information or its variants are utilized extensively in the characterization of the capacity of channels with memory and feedback, nonanticipative lossy data compression, and their generalizations to networks. In this paper, we derive several functional and topological properties of directed information, defined on general abstract alphabets (complete separable metric spaces), using the topology of weak convergence of probability measures. These include the convexity of the set of consistent distributions, which uniquely define causally conditioned distributions, convexity, and concavity of directed information with respect to the sets of consistent distributions, weak compactness of such sets of distributions, their joint distributions, and their marginals. Furthermore, we show lower semicontinuity of directed information, and under certain conditions, we also establish continuity. Finally, we derive variational equalities for directed information, including sequential versions. These may be viewed as the analog of the variational equalities of mutual information (utilized in Blahut-Arimoto algorithms). In summary, we extend the basic functional and topological properties of mutual information to directed information. These properties are discussed throughout this paper, in the context of extremum problems of directed information.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

S. Loyka and C.D. Charalambous, \u201cA General Formula for Compound Channel Capacity,\u201d IEEE Transactions on Information Theory, vol.62, no.7, pp.3971-3991, July 2016\u00a0External Link<\/a><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

A general formula, for the capacity of arbitrary compound channels with the receiver channel state information, is obtained using the information density approach. No assumptions of ergodicity, stationarity, or information stability are made and the channel state set is arbitrary. A direct (constructive) proof is given. To prove achievability, we generalize Feinstein Lemma to the compound channel setting, and to prove converse, we generalize Verdu-Han Lemma to the same compound setting. A notion of a uniform compound channel is introduced and the general formula is shown to reduce to the familiar sup-inf expression for such channels. As a by-product, the arbitrary varying channel capacity is established under maximum error probability and deterministic coding. Conditions are established under which the worst-case and compound channel capacities are equal, so that the full channel state information at the transmitter brings in no advantage. The compound inf-information rate plays a prominent role in the general formula. Its properties are studied and a link between the information-unstable and information-stable regimes of a compound channel is established. The results are extended to include \u03b5-capacity of compound channels. Sufficient and necessary conditions for the strong converse to hold are given.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

S. Loyka and C.D. Charalambous, \u201cRank Deficient Solutions for Optimal Signaling Over Wiretap MIMO Channels,\u201d IEEE Transactions on Communications, vol.64, no.6, pp.2400-2411, May, 2016\u00a0External Link<\/a><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

Capacity-achieving signaling strategies for the Gaussian wiretap MIMO channel are investigated without the degradedness assumption. In addition to known solutions, a number of new rank-deficient solutions for the optimal transmit covariance matrix are obtained. The case of weak eavesdropper is considered in details and the optimal covariance is established in an explicit, closed-form with no extra assumptions. The conditions for optimality of zero-forcing signaling are established, and the standard water-filling is shown to be optimal under those conditions. No wiretap codes are needed in this case. The case of identical right singular vectors for the required and eavesdropper channels is studied and the optimal covariance is established in an explicit closed form. As a by-product of this analysis, we establish a generalization of celebrated Hadamard determinantal inequality using information-theoretic tools.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

C.D. Charalambous, \u201cDecentralized Optimality Conditions of Stochastic Differential Decision Problems via Girsanov\u2019s Measure Transformation,\u201d Mathematics of Control, Signals, and Systems, pp.1-55, 2016\u00a0External Link<\/a><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

In this paper, we apply two methods to derive necessary and sufficient decentralized optimality conditions for stochastic differential decision problems with multiple Decision Makers (DMs), which aim at optimizing a common pay-off, based on the notions of decentralized global optimality and decentralized person-by-person (PbP) optimality. Method 1: We utilize the stochastic maximum principle to derive necessary and sufficient conditions which consist of forward and backward Stochastic Differential Equations (SDEs), and conditional variational Hamiltonians, conditioned on the information structures of the DMs. The sufficient conditions for decentralized PbP optimality are local conditions, closely related to the necessary conditions for decentralized PbP optimality. However, under certain convexity condition on the Hamiltonian, and a global version of the sufficient conditions for decentralized PbP optimality, we show decentralized global optimality. Method 2: We utilize the value processes of decentralized PbP optimal policies, we relate them to solutions of backward SDEs, we identify sufficient conditions for decentralized PbP optimality, and we show these are precisely those derived via the maximum principle. For both methods, as usual, we utilize Girsanov\u2019s theorem to transform the initial decentralized stochastic optimal decision problems, to equivalent decentralized stochastic optimal decision problems on a reference probability space, in which the controlled process and the information processes which generate part of the information structures of the DMs, are independent of any of the decisions.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

S. Loyka and C.D. Charalambous, \u201cNovel Matrix Singular Value Inequalities and their Applications to Uncertain MIMO Channels,\u201d IEEE Transactions on Information Theory, Vol.61, No.12. pp.6623-6634, December 2015\u00a0External Link<\/a><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

Novel matrix singular value inequalities are established for a sum\/product of three matrices. Their application to the uncertain (compound) multiple-input multiple-output (MIMO) channel subject to normed additive uncertainty establishes the saddle-point property for a wide range of performance metrics monotonic in the channel singular values, including, among others, the mutual information, MMSE, error exponent, and pairwise error probability. This, in turn, implies that the transmission on the eigenmodes of the nominal (or worst case) channel is also optimal for the whole set of channels under a general power constraint and hence achieves the compound channel capacity. The worst case channel turns out to be antiparallel of the nominal one for all these performance metrics. An application of these results to beamforming over compound MIMO channels is discussed. An optimal robust precoder for the uncertain MIMO channel is obtained in a closed-form under the sum-MSE criterion and the total power constraint. The saddle-point property is shown to hold and the optimal strategy is to diagonalize the nominal (or worst case) channel.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

S. Loyka and C.D. Charalambous, \u201cAn Algorithm for Global Maximization of Secrecy Rates Maximization in Gaussian MIMO Wiretap Channel,\u201d IEEE Transactions on Communications, Vol.63, No.6, pp.2288-2299, 2015\u00a0External Link<\/a><\/p><\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

Optimal signaling for secrecy rate maximization in Gaussian MIMO wiretap channels is considered. While this channel has attracted a significant attention recently and a number of results have been obtained, including the proof of the optimality of Gaussian signalling, an optimal transmit covariance matrix is known for some special cases only and the general case remains an open problem. An iterative custom-made algorithm to find a globally-optimal transmit covariance matrix in the general case is developed in this paper, with guaranteed convergence to a global optimum. While the original optimization problem is not convex and hence difficult to solve, its minimax reformulation can be solved via the convex optimization tools, which is exploited here. The proposed algorithm is based on the barrier method extended to deal with a minimax problem at hand. Its convergence to a global optimum is proved for the general case (degraded or not) and a bound for the optimality gap is given for each step of the barrier method. The performance of the algorithm is demonstrated via numerical examples. In particular, 20 to 40 Newton steps are already sufficient to solve the sufficient optimality conditions with very high precision (up to the machine precision level), even for large systems. Even fewer steps are required if the secrecy capacity is the only quantity of interest. The algorithm can be significantly simplified for the degraded channel case and can also be adopted to include the per-antenna power constraints (instead or in addition to the total power constraint). It also solves the dual problem of minimizing the total power subject to the secrecy rate constraint.<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
\n

I.Tziortzis, C.D. Charalambous and T.Charalambous, \u201cDynamic Programming Subject to Total Variation Distance Ambiguity\u201d, SIAM Journal on Control and Optimization, Vol.53, No4, pp.2040-2075, 2015 External Link<\/a><\/p>\n<\/div><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t<\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tAbstract<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t

The aim of this paper is to address optimality of stochastic control strategies via dynamic programming subject to total variation distance ambiguity on the conditional distribution of the controlled process. We formulate the stochastic control problem using minimax theory, in which the control minimizes the payoff while the conditional distribution, from the total variation distance set, maximizes it. First, we investigate the maximization of a linear functional on the space of probability measures on abstract spaces, among those probability measures which are within a total variation distance from a nominal probability measure, and then we give the maximizing probability measure in closed form. Second, we utilize the solution of the maximization to solve minimax stochastic control with deterministic control strategies, under a Markovian and a non-Markovian assumption, on the conditional distributions of the controlled process. The results of this part include (1) minimax optimization subject to total variation distance ambiguity constraint; (2) new dynamic programming recursions, which involve the oscillator seminorm of the value function, in addition to the standard terms; and (3) a new infinite horizon discounted dynamic programming equation, the associated contractive property, and a new policy iteration algorithm. Finally, we provide illustrative examples for both the finite and infinite horizon cases. For the infinite horizon case, we invoke the new policy iteration algorithm to compute the optimal strategies. Read More: https:\/\/epubs.siam.org\/doi\/abs\/10.1137\/140955707<\/span><\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

Articles in Journals (In reverse chronological order): C.D. Charalambous and C. Kourtellaris, \u201cTime-Invariant Feedback Strategies Do not Increase Capacity of AGN Channels Driven by Stable and Unstable Autoregressive,\u201dIEEE Transactions on Information Theory, pages-54, 27 July 2019 (revised 26 July 2020)\u00a0External Link Abstract The capacity of additive Gaussian noise (AGN) channels with feedback, when the noise …<\/p>\n

Journal Publications<\/span> Read More »<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-316","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/staging.cdcharalambous.com\/index.php\/wp-json\/wp\/v2\/pages\/316","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/staging.cdcharalambous.com\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/staging.cdcharalambous.com\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/staging.cdcharalambous.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/staging.cdcharalambous.com\/index.php\/wp-json\/wp\/v2\/comments?post=316"}],"version-history":[{"count":92,"href":"https:\/\/staging.cdcharalambous.com\/index.php\/wp-json\/wp\/v2\/pages\/316\/revisions"}],"predecessor-version":[{"id":712,"href":"https:\/\/staging.cdcharalambous.com\/index.php\/wp-json\/wp\/v2\/pages\/316\/revisions\/712"}],"wp:attachment":[{"href":"https:\/\/staging.cdcharalambous.com\/index.php\/wp-json\/wp\/v2\/media?parent=316"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}