Synonyms for kullback_leibler_divergence or Related words with kullback_leibler_divergence

shannon_entropy              covariance_matrix              maximum_likelihood_estimator              conditional_expectation              maximum_likelihood_estimate              covariance_matrices              covariance              differential_entropy              pearson_correlation_coefficient              conditional_probability              bivariate              multivariate_normal_distribution              dirac_delta_function              skewness              multivariate_normal              excess_kurtosis              mean_squared_error              covariances              conditional_probabilities              chi_squared              sufficient_statistic              mahalanobis_distance              radon_nikodym_derivative              kurtosis              kolmogorov_complexity              binomial_distribution              unnormalized              multinomial_distribution              regression_coefficient              kl_divergence              quantile_function              boundedness              scalar_curvature              heaviside_step_function              entropies              rayleigh_quotient              quantile              correlation_coefficient              frobenius_norm              hypergeometric_distribution              lagrange_multiplier              euclidean_norm              univariate              chebyshev_inequality              rényi_entropy              orthogonality              probit_model              hessian_matrix              poisson_distribution              von_neumann_entropy             

Examples of "kullback_leibler_divergence"
is the KullbackLeibler divergence in nats. When the sample space formula_9 is a finite set, the KullbackLeibler divergence is given by
where formula_47 is the Kullback-Leibler divergence.
Consider the KullbackLeibler divergence between the two distributions
which is the Kullback-Leibler divergence or relative entropy
Two simple divergence functions studied by Lee and Seung are the squared error (or Frobenius norm) and an extension of the KullbackLeibler divergence to positive matrices (the original KullbackLeibler divergence is defined on probability distributions).
where formula_2 is the KullbackLeibler divergence from "q" to "p". Viewing the KullbackLeibler divergence as a measure of distance, the I-projection formula_3 is the "closest" distribution to "q" of all the distributions in "P".
For the classical KullbackLeibler divergence, it can be shown that
The relative entropy, or KullbackLeibler divergence, is always non-negative. A few numerical examples follow:
The KullbackLeibler divergence is named after Kullback and Richard Leibler.
The total variation distance is related to the KullbackLeibler divergence by Pinsker's inequality.
This fundamental inequality states that the KullbackLeibler divergence is non-negative.
is the Kullback-Leibler divergence and it is used that formula_84.
The KullbackLeibler divergence from formula_50 to formula_51, for non-singular matrices Σ and Σ, is:
The canonical divergence is given by the Kullback-Leibler divergence formula_328
with equality when "g"("x") = "f"("x") following from the properties of KullbackLeibler divergence.
the KullbackLeibler divergence from "Q" to "P" is defined to be
The directed KullbackLeibler divergence of formula_13 ('approximating' distribution) from formula_14 ('true' distribution) is given by
which is proportional to KullbackLeibler divergence (which is always non-negative), where
the KullbackLeibler divergence of the prior from the posterior distribution.
where formula_8 is the KullbackLeibler divergence between formula_1 and formula_4.