2020/01/04學習筆記（《META-NAS》論文閱讀的預備知識……）

一、 maximum log-likelihood estimation最大log似然估計的概念

具體參考博文：https://towardsdatascience.com/probability-concepts-explained-maximum-likelihood-estimation-c7b4342fdbb1

要點：

1、爲什麼叫最大似然估計而不是最大似然概率？——答：理解下面這張圖，因爲雖然「the probability density of the data given the parameters【右式】」等價於「the likelihood of the parameters given the data【左式】」，但是左式要求的是參數，右式要求的是數據，此處我們要求參數，因此叫likelihood.

2、爲什麼要引入log？——答：因爲引入log之後，對乘或者除的求導，可以轉化成對加和減的求導，求導更加方便。

3、什麼是參數？——答：parameters define a blueprint for the model. It is only when specific values are chosen for the parameters that we get an instantiation for the model that describes a given phenomenon.

4、Intuitive explanation of maximum likelihood estimation？——答：Maximum likelihood estimation is a method that determines values for the parameters of a model. The parameter values are found such that they maximise the likelihood that the process described by the model produced the data that were actually observed.

二、KL-divergence KL散度的概念

參考博文：https://towardsdatascience.com/light-on-math-machine-learning-intuitive-guide-to-understanding-kl-divergence-2b382ca2b2a8

定義：

三、Jensen不等式

https://blog.csdn.net/baidu_38172402/article/details/89090383

四、變分貝葉斯

https://www.jianshu.com/p/86c5d1e1ef93