Go to:Table of Contents

Entropy of Kind t

To obtain an entropy of kind t, Arimoto (1971) [3] used a different approach. Let $f(u)$ be a real valued scalar function defined and nonnegative on (0,1] with a continuous derivative on (0,1] and $f(1)=0$. For all$P \in \ \Delta_n$, let us define

$$H_f(P)=H_f(p_1,p_2,...,p_n)= \mathrel{\mathop{inf}\limits_{Q}}\,\sum_{i=1}^n{p_if(q_i)},$$ (3.19)

where the inf. is taken over all probability distributions such that $Q=(q_1,...,q_n)$, $\sum_{i=1}^n{q_i}=1,\ q_i>0,\\forall\ i=1, 2, \cdots, n$.

The function $H_f(P)$ given by (3.19) satisfy some interesting properties (ref. Arimoto, 1971) [3]. These are summarized in the following property.

Property 3.24. We have

(i) $H_f(p_1,p_2,...,p_n)$ is a continuous and symmetric function with respect to its arguments $p_1,$ $p_2,...,p_n$.

(ii) $H_f(p_1,p_2,...,p_n)=H_f(p_1,p_2,...,p_n,0)$.

 

(iii) $H_f(P)$ is a concave function with respect to $P$ in $\Delta_n$.

 

(iv) $0\leq H_f(p_1,p_2,...,p_n)\leq f({1\over n})$.

 

(v) If $f(u)$ is convex, then

$$H_f(p_1,p_2,...,p_n)\leq H_f({1\over n},...,{1\over n})\leqf({1\over n})$$

(vi) In general

$$H_f(p_1,p_2,...,p_n)\leq \sum_{i=1}^n{p_if(p_i).}$$

(vii) If $f'(u)<0$ on (0,1), then (a) $H_f(P)\geq H_f\big(H_o(P),1-H_o(P)\big).$ (b) $H_f(P)\geq f({1\over 2})H_o(P).$

(viii) If $f(u)$ is convex with $f'(u)<0$ on (0,1], then

$$H_f(P)\geq f\Big(1-{H_o(P)\over 2}\Big),$$ where $H_o(P)=1-max\{p_1,p_2,...,p_n\}$.

Let $$f^t(u)=\left\{\begin{array}{ll}(2^{t-1}-1)^{-1}(u^{1-t}-1), & t\neq 1,\ t\geq 0 \\ -\log_2 u, &t=1\end{array}\right.$$ then from (3.19), one has

$$H_t(P)=\left\{\begin{array}{ll}1- max\{p_1,p_2,...,p_n\}, & t=0......&t\neq 1, \ t>0 \\ -\sum_{i=1}^n{p_i\,\log p_i}, & t=1\end{array}\right.$$

The function $H_t(P)$ is the entropy of kind $t$. It arises as a particular case of (3.19). Some other examples of (3.19) can be seen in Arimoto (1971) [3]. Boeeke et al. (1980) [16] studied extensively the entropy of kind $t$ by naming $R-$norm, considering $t={1\over R}$. Some properties due to Behara and Chawla (1974) [9] are also worth emphasizing. Following Rényi's approach, the measure $H_f(P)$ can be characterized. It is not specified here, because the next section cover it as a particular case.