{"id":1819,"date":"2020-03-22T23:01:00","date_gmt":"2020-03-22T16:01:00","guid":{"rendered":"https:\/\/www.indowhiz.com\/articles\/?p=1819"},"modified":"2022-05-27T15:37:14","modified_gmt":"2022-05-27T08:37:14","slug":"the-simple-concept-of-expectation-maximization-em-algorithm","status":"publish","type":"post","link":"https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/","title":{"rendered":"The Simple Concept of Expectation \u2013 maximization (EM) Algorithm"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Learn an algorithm in autodidact is time-consuming; especially for people who are not interested in mathematics. Because, sometimes we don&#8217;t know that &#8220;we don&#8217;t know&#8221;, many things. Including when studying the Expectation \u2013 maximization (EM) Algorithm. Therefore, I could improve your understanding of the EM algorithm in this article, and I hope you can easily understand it.<\/p>\n\n\n\n<!--more-->\n\n\n\n<p class=\"wp-block-paragraph\">EM algorithm often used in machine learning as an algorithm for data clustering.<span id=\"c33f952b-9d71-4d1b-a600-ab9c0c68a530\" data-items=\"[&quot;1714597785&quot;,&quot;3387592305&quot;]\" class=\"abt-citation\" contenteditable=\"false\">\u200b[1], [2]\u200b<\/span> Sometimes, one of the clustering problems is when the data doesn&#8217;t have any labels. For example, if we know A in the data, then we can calculate B from the data; or vice versa, if we know B, we can calculate A. However, sometimes we have mixed data, and the label of A and B are unknown. In this case, the EM algorithm could help to solve the problem.<span id=\"ef87b378-3693-4f76-9484-6140ee041fc7\" data-items=\"[&quot;4284757828&quot;]\" class=\"abt-citation\" contenteditable=\"false\">\u200b[3]\u200b<\/span><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For a summary, the EM algorithm is an iterative method, involves expectation (E-step) and maximization (M-step); to find the local maximum likelihood from the data. Commonly, EM used on several distributions or statistical models, where there are one or more unknown variables. Therefore, it called missing or &#8220;<em>latent<\/em>&#8221; variables. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In other words, we can use the EM algorithm if: <\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>We have incomplete datasets (for example, it doesn&#8217;t have groups or labels in the data), but we need to predict its group or label.<\/li><li>The group or label has never been observed or recorded but is very important in explaining the data. It called a &#8220;<em>latent<\/em>&#8221; variable, meaning  missing, hidden, or invisible. <\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Note: Some theories that you may need to learn are: the probability of binomial random variable and conditional probability. <\/p>\n\n\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Case example for EM Algorithm<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">To easily understand the EM Algorithm, we can use an example of the coin tosses distribution.<span id=\"aa5907ae-d153-40e0-987d-7325f983bc8b\" data-items=\"[&quot;2889665632&quot;]\" class=\"abt-citation\" contenteditable=\"false\">\u200b[4]\u200b<\/span><\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"799\" height=\"518\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/uploads\/2020\/02\/IDR_1000_Koin.jpg\" alt=\"Figure 1. The IDR 1000 coins\" class=\"wp-image-1722\"\/><figcaption>Fig 1. <a rel=\"noreferrer noopener\" aria-label=\"Koin 1000 Rupiah (opens in a new tab)\" href=\"https:\/\/commons.wikimedia.org\/wiki\/File:IDR_1000_Koin.JPG\" target=\"_blank\">The IDR 1000 coins<\/a><\/figcaption><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">For example, I have 2 coins; Coin A and Coin B; where both have a different head-up probability. I will randomly choose a coin 5 times, whether coin A or B. Then, each coin selection is followed by tossing it 10 times. Therefore, we have the following outcomes: <\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Set 1: <code>H T T T H  H T H T H<\/code> (5H 5T)<\/li><li>Set 2: <code>H H H H T  H H H H H<\/code> (9H 1T)<\/li><li>Set 3: <code>H T H H H  H H T H H<\/code> (8H 2T)<\/li><li>Set 4: <code>H T H T T  T H H T T<\/code> (4H 6T)<\/li><li>Set 5: <code>T H H H T  H H H T H<\/code> (7H 3T)<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">From the fig. 1 above, the &#8220;<em>image<\/em>&#8221; side denoted by &#8220;<strong>H<\/strong>&#8221; or Head-up, and the &#8220;<em>number<\/em>&#8221; side denoted by &#8220;<strong>T<\/strong>&#8221; or Tail-up. Then, the probability of coin will land with head-up for each of these coins, could be denoted as theta or &#8220;<img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a8432efe7005e7528db837fc1555cb8a_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"9\" style=\"vertical-align: 0px;\"\/>&#8220;. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The question is, how do you estimate <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a8432efe7005e7528db837fc1555cb8a_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"9\" style=\"vertical-align: 0px;\"\/> for each coin?<span id=\"86f50168-4497-44c7-8c57-22588e83263c\" data-items=\"[&quot;610328443&quot;]\" class=\"abt-citation\" contenteditable=\"false\">\u200b[5]\u200b<\/span><\/p>\n\n\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Completed data<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">In different cases, with the same outcomes as above; I give you more information. I told you which coins I used for each set; means the coin identity. Then, the data you have will be like this: <\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Set 1, <span style=\"color:#008000\" class=\"color\">Coin B: <code>H T T T H  H T H T H<\/code><\/span> (5H 5T) <\/li><li>Set 2, <span style=\"color:#ff0000\" class=\"color\">Coin A: <code>H H H H T  H H H H H<\/code><\/span> (9H 1T) <\/li><li>Set 3, <span style=\"color:#ff0000\" class=\"color\">Coin A: <code>H T H H H  H H T H H<\/code><\/span> (8H 2T) <\/li><li>Set 4, <span style=\"color:#008000\" class=\"color\">Coin B: <code>H T H T T  T H H T T<\/code><\/span> (4H 6T) <\/li><li>Set 5, <span style=\"color:#ff0000\" class=\"color\">Coin A: <code>T H H H T  H H H T H<\/code><\/span> (7H 3T) <\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Because you already have all the data you need, you can easily calculate the probability of getting head-up on each coin.<mark class=\"annotation-text annotation-text-yoast\" id=\"annotation-text-f3936e90-dc73-46b1-9705-717dfd9baa57\"><\/mark> Therefore, the calculation of <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a8432efe7005e7528db837fc1555cb8a_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"9\" style=\"vertical-align: 0px;\"\/> will be:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">By knowing the <code>result of coin A = 24H 6T<\/code>, in 3 sets with a total of 30 tosses, then <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-2e17227ea61d4b1e398ddf4473b45ec4_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#65;&#125;&#32;&#61;&#32;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#50;&#52;&#125;&#123;&#51;&#48;&#125;&#32;&#61;&#32;&#48;&#46;&#56;&#48;\" title=\"Rendered by QuickLaTeX.com\" height=\"36\" width=\"120\" style=\"vertical-align: -12px;\"\/>. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Similarly to the <code>result of coin B = 9H 11T<\/code>, in 2 sets with a total of 20 tosses, then <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-c40f30333ce6bafcee9cb8a29f19cf45_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#66;&#125;&#32;&#61;&#32;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#57;&#125;&#123;&#50;&#48;&#125;&#32;&#61;&#32;&#48;&#46;&#52;&#53;\" title=\"Rendered by QuickLaTeX.com\" height=\"36\" width=\"120\" style=\"vertical-align: -12px;\"\/>.  <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This simple equation is known as Maximum Likelihood Estimation (MLE),  Where <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-f54d229b688ff7593a92fac9516fba50_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#88;&#125;&#32;&#32;&#61;&#32;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#92;&#116;&#101;&#120;&#116;&#105;&#116;&#123;&#110;&#117;&#109;&#98;&#101;&#114;&#32;&#111;&#102;&#32;&#104;&#101;&#97;&#100;&#115;&#45;&#117;&#112;&#32;&#117;&#115;&#105;&#110;&#103;&#32;&#99;&#111;&#105;&#110;&#32;&#88;&#125;&#125;&#123;&#92;&#116;&#101;&#120;&#116;&#105;&#116;&#123;&#116;&#111;&#116;&#97;&#108;&#32;&#116;&#111;&#115;&#115;&#101;&#115;&#32;&#119;&#105;&#116;&#104;&#32;&#99;&#111;&#105;&#110;&#32;&#88;&#125;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"37\" width=\"304\" style=\"vertical-align: -12px;\"\/>.<\/p>\n\n\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">The overview of EM algorithm<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Overall flowchart<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Before returning to the original case, let me explain briefly the stages of the EM algorithm. So you have an overall picture of the EM algorithm flow, which helps you reading until the end. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"661\" height=\"423\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/uploads\/2020\/03\/EM-FlowChart-en.jpg\" alt=\"Figure 2. Flowchart of EM algorithm\" class=\"wp-image-1829\"\/><figcaption> Fig 2. Flowchart of EM algorithm<\/figcaption><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">There are several steps in the EM algorithm, which are:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>Defining latent variables<\/li><li>Initial guessing<\/li><li>E-Step<\/li><li>M-Step<\/li><li>Stopping condition and the final result <\/li><\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Actually, the main point of EM is the iteration between E-step and M-step, which could be seen in Fig. 2 above. The E-step will estimate your hidden variables, and the M-step will re-update the parameters, based on the estimation of the hidden variables. In other words, this iteration aims to re-improve the estimation of current parameters.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Inside the EM algorithm<\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"819\" height=\"451\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/uploads\/2020\/03\/EM-coin-toss-calculation.png\" alt=\"Figure 3. Parameter calculation behind the E-step and M-step\" class=\"wp-image-1879\"\/><figcaption> Fig 3. Parameter calculation behind the E-step and M-step<\/figcaption><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Take a look at Fig 3 above, to see what EM really does in your case; where you don&#8217;t know the coin identity. The current approach is to distribute the number of heads for each set; proportionally based on the <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a8432efe7005e7528db837fc1555cb8a_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"9\" style=\"vertical-align: 0px;\"\/> parameter for each coin. Therefore, first, you need to guess the initial <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a8432efe7005e7528db837fc1555cb8a_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"9\" style=\"vertical-align: 0px;\"\/> parameter [Fig 3(1)] for each coin. Then, (in the beginning) the current <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a8432efe7005e7528db837fc1555cb8a_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"9\" style=\"vertical-align: 0px;\"\/> parameters [Fig 3(2)] are your guessing. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Next, in <strong>the E-step<\/strong>, you could calculate each coin&#8217;s probability to get the outcomes of each set [Fig 3(3)]; based on the current <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a8432efe7005e7528db837fc1555cb8a_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"9\" style=\"vertical-align: 0px;\"\/> parameters. For example, the probability of coin A to get 8H 2T in 10 tosses (set 3).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Then, compare the probabilities both coin A and coin B; based on their total probability [Fig 3(4)]. For example, the result for a set with 8H 2T, I have a 73% probability using coin A, compared to 27% using coin B. This is how you estimate the probability of coin identity used in each set. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Then, the estimated identity is used to estimate the number of heads (and tail) for each coin, for each set [Fig 3(5)]. For example, those 8 Heads divided proportionally 73% to coin A and 27% for coin B.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Finally, the estimation of total heads of each coin is used to recalculate the value of the <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a8432efe7005e7528db837fc1555cb8a_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"9\" style=\"vertical-align: 0px;\"\/> parameters at <strong>the M-Step<\/strong>. This re-calculation will improve the current <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a8432efe7005e7528db837fc1555cb8a_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"9\" style=\"vertical-align: 0px;\"\/> [Fig 3(2)]. In other words, this re-calculation is based on the estimated results of hidden variables and observable data. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This iteration will continue until reaches the &#8220;maximum iteration&#8221; limit or convergence threshold reached. When the iteration stopped, the final results obtained are the closest estimation to the local optimum solutions. <\/p>\n\n\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">1. Defining latent variables <\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Now, back to the original case, where the outcomes you have are:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Set 1: <code>H T T T H  H T H T H<\/code> (5H 5T)<\/li><li>Set 2: <code>H H H H T  H H H H H<\/code> (9H 1T)<\/li><li>Set 3: <code>H T H H H  H H T H H<\/code> (8H 2T)<\/li><li>Set 4: <code>H T H T T  T H H T T<\/code> (4H 6T)<\/li><li>Set 5: <code>T H H H T  H H H T H<\/code> (7H 3T) <\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">First, you must define what variables are required; but are not observed in the data. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The goal is to estimate the probability of getting heads-up for each coin. However, it cannot be calculated directly; if you don&#8217;t know the identity of the coin used in each set. Therefore, it is necessary to know which coin is used in each set. In other words, this coin identity is your latent variable. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Right now, besides the 5 sets of outcomes above, you only know that I used two coins; which are coin A and coin B. <\/p>\n\n\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">2. Initial guessing<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">As stated before, the probability of getting head-up for each of these coins denoted as theta or <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a8432efe7005e7528db837fc1555cb8a_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"9\" style=\"vertical-align: 0px;\"\/>. Currently, there are only coin A and coin B; with unknown parameter values of <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-f2e21ed0ec0708aa60f9e370e6eebdac_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#65;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"18\" style=\"vertical-align: -3px;\"\/> and <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a01ff4ced34bb684cdddfd2f593de428_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#66;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"19\" style=\"vertical-align: -3px;\"\/>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Then, before estimating with EM iteration, you need to provide an initial estimate to both parameters [Fig 3(1)]. Therefore, you may choose randomly for each <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a8432efe7005e7528db837fc1555cb8a_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"9\" style=\"vertical-align: 0px;\"\/>. For example, you may set <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-f2e21ed0ec0708aa60f9e370e6eebdac_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#65;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"18\" style=\"vertical-align: -3px;\"\/> is <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-be39b8d410d9915e24193e5feab70296_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#48;&#46;&#53;&#44;&#32;&#48;&#46;&#54;&#44;&#32;&#48;&#46;&#55;\" title=\"Rendered by QuickLaTeX.com\" height=\"17\" width=\"84\" style=\"vertical-align: -4px;\"\/>, or else, it&#8217;s up to you; as well as <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a01ff4ced34bb684cdddfd2f593de428_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#66;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"19\" style=\"vertical-align: -3px;\"\/>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">By the way, please noted that there is no relationship between parameters guessing; both <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-f2e21ed0ec0708aa60f9e370e6eebdac_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#65;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"18\" style=\"vertical-align: -3px;\"\/> and <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a01ff4ced34bb684cdddfd2f593de428_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#66;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"19\" style=\"vertical-align: -3px;\"\/>. For example, if you think that the sum of <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-f2e21ed0ec0708aa60f9e370e6eebdac_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#65;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"18\" style=\"vertical-align: -3px;\"\/> and <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a01ff4ced34bb684cdddfd2f593de428_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#66;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"19\" style=\"vertical-align: -3px;\"\/> must be = 1, NO!. This probability represents the individual value of getting heads-up on each coin. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-f498b0ad266799a5dc4bca262fdc01b5_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#65;&#125;&#61;&#48;&#46;&#54;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"66\" style=\"vertical-align: -3px;\"\/>, meaning that there is 60% probability of getting head-up, compared to 40% probability of getting tail-up using coin A. It has nothing to do with guessing the value of <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a01ff4ced34bb684cdddfd2f593de428_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#66;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"19\" style=\"vertical-align: -3px;\"\/> or for coin B. So even though you already set the initial of <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-f498b0ad266799a5dc4bca262fdc01b5_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#65;&#125;&#61;&#48;&#46;&#54;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"66\" style=\"vertical-align: -3px;\"\/>, you may set <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a01ff4ced34bb684cdddfd2f593de428_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#66;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"19\" style=\"vertical-align: -3px;\"\/> with <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-dd6b1f52f6a8106c6aa1d4bb7ae4b380_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#48;&#46;&#50;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"22\" style=\"vertical-align: 0px;\"\/>, <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-ad6fc9e6314c98de2dd093a5a58daab5_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#48;&#46;&#52;\" title=\"Rendered by QuickLaTeX.com\" height=\"13\" width=\"23\" style=\"vertical-align: -1px;\"\/>, <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-918746fc871dbe69de829c9d72f7bafd_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#48;&#46;&#53;\" title=\"Rendered by QuickLaTeX.com\" height=\"13\" width=\"22\" style=\"vertical-align: 0px;\"\/>, <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-6a25c1267aaad00d1a0a8ad871866d5e_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#48;&#46;&#55;\" title=\"Rendered by QuickLaTeX.com\" height=\"13\" width=\"23\" style=\"vertical-align: 0px;\"\/> or any number you want between 0 and 1. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Okay enough. Now, let&#8217;s say you have randomly set both initial values, which are: <\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a56a19e44787f39a3a72d17145f4e048_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#65;&#125;&#32;&#61;&#32;&#48;&#46;&#54;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"66\" style=\"vertical-align: -3px;\"\/> and <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-07a2eb81aad89e394403119bfc31db25_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#66;&#125;&#32;&#61;&#32;&#48;&#46;&#53;\" title=\"Rendered by QuickLaTeX.com\" height=\"16\" width=\"66\" style=\"vertical-align: -3px;\"\/> <\/p>\n\n\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">3. E-Step<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Now you have the required variables. So, you can start estimating the identity of the coin used in each set. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Each set, which contains the outcomes of heads and tails, can be denoted by  <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-e93700b2e5b0ca6b1fb78b43156e5af8_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#69;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"14\" style=\"vertical-align: 0px;\"\/> notation. Then, the probability of &#8220;using coin A&#8221; denoted as <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-e20a37b49a9ba97a6174883d4821b9bd_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#90;&#95;&#123;&#65;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"21\" style=\"vertical-align: -3px;\"\/>, and the probability of &#8220;using coin B&#8221; denoted as <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-181ac028fe94e6ff632dca987f46cede_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#90;&#95;&#123;&#66;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"22\" style=\"vertical-align: -3px;\"\/>. <\/p>\n\n\n\n<h3 class=\"wp-block-heading\">The probability a coin giving <em>E<\/em><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">At the beginning of E-Step, you need to know the probability of a set using coins A or B. Take an example from set 3, where <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-e93700b2e5b0ca6b1fb78b43156e5af8_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#69;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"14\" style=\"vertical-align: 0px;\"\/> = HTHHHHHTHH (8H 2T). If <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a56a19e44787f39a3a72d17145f4e048_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#65;&#125;&#32;&#61;&#32;&#48;&#46;&#54;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"66\" style=\"vertical-align: -3px;\"\/>, it means the probability of getting head is <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-edef76a83e479406ce2e85203a1ce7c2_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#48;&#46;&#54;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"23\" style=\"vertical-align: 0px;\"\/> (and tail <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-ad6fc9e6314c98de2dd093a5a58daab5_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#48;&#46;&#52;\" title=\"Rendered by QuickLaTeX.com\" height=\"13\" width=\"23\" style=\"vertical-align: -1px;\"\/>) uses coin A. Then, how much probability of coin A will give <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-ca6196abacf7f65681779ac60e93fdfe_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#56;&#72;&#32;&#92;&#59;&#32;&#50;&#84;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"52\" style=\"vertical-align: 0px;\"\/> in 10 tosses (a set)? This is what we need to estimate first [Fig 3(3)]. Then, this probability can be denoted by <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-c41f185f5d676419d72066c5841e2216_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#80;&#40;&#69;&#124;&#90;&#95;&#123;&#65;&#125;&#41;\" title=\"Rendered by QuickLaTeX.com\" height=\"18\" width=\"69\" style=\"vertical-align: -4px;\"\/>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Similarly, the probability of coin B giving <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-cd6196eac5802c8fe0360a51f442f5f5_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#69;&#32;&#61;&#32;&#56;&#72;&#32;&#92;&#59;&#32;&#50;&#84;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"90\" style=\"vertical-align: 0px;\"\/>, could be denoted by <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-dc68a87ae64459260c0a68eecd508690_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#80;&#40;&#69;&#124;&#90;&#95;&#123;&#66;&#125;&#41;\" title=\"Rendered by QuickLaTeX.com\" height=\"18\" width=\"70\" style=\"vertical-align: -4px;\"\/>. This actually relates to the probability distribution of a binomial random variable, which equation is:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-0f82133676fcdb8ab3b1457ee026341e_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#80;&#40;&#69;&#124;&#90;&#95;&#123;&#120;&#125;&#41;&#61;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#110;&#33;&#125;&#123;&#104;&#33;&#92;&#59;&#40;&#110;&#45;&#104;&#41;&#33;&#125;&#92;&#59;&#123;&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#120;&#125;&#125;&#94;&#123;&#104;&#125;&#92;&#59;&#40;&#49;&#45;&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#120;&#125;&#41;&#94;&#123;&#110;&#45;&#104;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"40\" width=\"300\" style=\"vertical-align: -16px;\"\/><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">where:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-7d334b2fb89f03d8361afee07ec4921d_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#80;&#40;&#69;&#124;&#90;&#95;&#123;&#120;&#125;&#41;\" title=\"Rendered by QuickLaTeX.com\" height=\"18\" width=\"66\" style=\"vertical-align: -4px;\"\/> = the probability of coins x gives <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-e93700b2e5b0ca6b1fb78b43156e5af8_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#69;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"14\" style=\"vertical-align: 0px;\"\/>.<br><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-9e1b4fae75a688379b6f1575b4df8e28_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#110;\" title=\"Rendered by QuickLaTeX.com\" height=\"8\" width=\"11\" style=\"vertical-align: 0px;\"\/> = total coin tosses in a set <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-e93700b2e5b0ca6b1fb78b43156e5af8_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#69;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"14\" style=\"vertical-align: 0px;\"\/>.<br><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-64d64de6ce455f1eec8bd32b08b6e4b7_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#104;\" title=\"Rendered by QuickLaTeX.com\" height=\"13\" width=\"10\" style=\"vertical-align: 0px;\"\/> = total number of heads in a set of <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-e93700b2e5b0ca6b1fb78b43156e5af8_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#69;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"14\" style=\"vertical-align: 0px;\"\/>.<br><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-4cfed9c528b647be436a366f43412289_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#120;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"16\" style=\"vertical-align: -3px;\"\/> = the probability of getting head-up using coin x.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Then, we can calculate the probability of getting <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-e93700b2e5b0ca6b1fb78b43156e5af8_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#69;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"14\" style=\"vertical-align: 0px;\"\/> using coins A and B as follows: <\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-66117536e7b0bcc9c6a5c4da3558cac0_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#80;&#40;&#69;&#95;&#123;&#56;&#72;&#50;&#84;&#125;&#124;&#90;&#95;&#123;&#65;&#125;&#41;&#61;&#32;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#49;&#48;&#33;&#125;&#123;&#56;&#33;&#92;&#59;&#50;&#33;&#125;&#92;&#59;&#123;&#48;&#46;&#54;&#125;&#94;&#123;&#56;&#125;&#92;&#59;&#48;&#46;&#52;&#94;&#123;&#50;&#125;&#61;&#48;&#46;&#49;&#50;&#49;&#32;&#92;&#92;&#091;&#52;&#112;&#116;&#093;\" title=\"Rendered by QuickLaTeX.com\" height=\"36\" width=\"297\" style=\"vertical-align: 0px;\"\/><br><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-8a24450e6d3f61772c87c09d8d361e72_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#80;&#40;&#69;&#95;&#123;&#56;&#72;&#50;&#84;&#125;&#124;&#90;&#95;&#123;&#66;&#125;&#41;&#61;&#32;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#49;&#48;&#33;&#125;&#123;&#56;&#33;&#92;&#59;&#50;&#33;&#125;&#92;&#59;&#123;&#48;&#46;&#53;&#125;&#94;&#123;&#56;&#125;&#92;&#59;&#48;&#46;&#53;&#94;&#123;&#50;&#125;&#61;&#48;&#46;&#48;&#52;&#52;\" title=\"Rendered by QuickLaTeX.com\" height=\"36\" width=\"302\" style=\"vertical-align: -12px;\"\/>  <\/p>\n\n\n\n<h3 class=\"wp-block-heading\">The ratio of coins A and B gives <em>E<\/em><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Next, you can compare the probabilities both of coin A and coin B give <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-e93700b2e5b0ca6b1fb78b43156e5af8_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#69;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"14\" style=\"vertical-align: 0px;\"\/> [Fig 3(4)]. So, according to Bayes&#8217; theorem and the law of total probability, we can determine the ratio of probability using the following equation:<span id=\"ac01e3cb-e909-439d-b28a-de4030608658\" data-items=\"[&quot;905564401&quot;]\" class=\"abt-citation\" contenteditable=\"false\">\u200b[6]\u200b<\/span><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-1e862bf63073b196bcf608687fca3aa7_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#80;&#40;&#90;&#95;&#123;&#120;&#125;&#124;&#69;&#41;&#61;&#32;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#92;&#116;&#101;&#120;&#116;&#105;&#116;&#123;&#116;&#104;&#101;&#32;&#112;&#114;&#111;&#98;&#97;&#98;&#105;&#108;&#105;&#116;&#121;&#32;&#111;&#102;&#32;&#99;&#111;&#105;&#110;&#115;&#32;&#120;&#32;&#103;&#105;&#118;&#105;&#110;&#103;&#32;&#69;&#32;&#117;&#115;&#105;&#110;&#103;&#125;&#125;&#123;&#92;&#116;&#101;&#120;&#116;&#105;&#116;&#123;&#116;&#111;&#116;&#97;&#108;&#32;&#112;&#114;&#111;&#98;&#97;&#98;&#105;&#108;&#105;&#116;&#121;&#32;&#111;&#102;&#32;&#99;&#111;&#105;&#110;&#115;&#32;&#120;&#32;&#97;&#110;&#100;&#32;&#121;&#32;&#105;&#110;&#32;&#103;&#105;&#118;&#105;&#110;&#103;&#32;&#69;&#125;&#125;&#32;&#32;&#92;&#92;&#091;&#49;&#50;&#112;&#116;&#093;&#32;\" title=\"Rendered by QuickLaTeX.com\" height=\"41\" width=\"433\" style=\"vertical-align: 0px;\"\/>  <\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-bce552ce1a85d4df2e62b31fffa7da28_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#80;&#40;&#90;&#95;&#123;&#120;&#125;&#124;&#69;&#41;&#61;&#32;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#80;&#40;&#69;&#124;&#90;&#95;&#123;&#120;&#125;&#41;&#80;&#40;&#90;&#95;&#123;&#120;&#125;&#41;&#125;&#123;&#80;&#40;&#69;&#124;&#90;&#95;&#123;&#120;&#125;&#41;&#80;&#40;&#90;&#95;&#123;&#120;&#125;&#41;&#43;&#80;&#40;&#69;&#124;&#90;&#95;&#123;&#121;&#125;&#41;&#80;&#40;&#90;&#95;&#123;&#121;&#125;&#41;&#125;&#32;&#92;&#92;&#091;&#56;&#112;&#116;&#093;&#32;\" title=\"Rendered by QuickLaTeX.com\" height=\"44\" width=\"342\" style=\"vertical-align: 0px;\"\/> <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">where:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-2968a6d0b35af87f1f50d94538431212_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#80;&#40;&#90;&#95;&#123;&#120;&#125;&#124;&#69;&#41;\" title=\"Rendered by QuickLaTeX.com\" height=\"18\" width=\"66\" style=\"vertical-align: -4px;\"\/> = the probability of coin <em>x<\/em> giving <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-e93700b2e5b0ca6b1fb78b43156e5af8_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#69;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"14\" style=\"vertical-align: 0px;\"\/> (compared to coin <em>y<\/em>).<br><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-ee9d639b96440b3f088a75e32a282c7c_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#80;&#40;&#90;&#95;&#123;&#120;&#125;&#41;\" title=\"Rendered by QuickLaTeX.com\" height=\"18\" width=\"47\" style=\"vertical-align: -4px;\"\/> = the probability of choosing coin <em>x<\/em>.<br><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-ee9d639b96440b3f088a75e32a282c7c_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#80;&#40;&#90;&#95;&#123;&#120;&#125;&#41;\" title=\"Rendered by QuickLaTeX.com\" height=\"18\" width=\"47\" style=\"vertical-align: -4px;\"\/> = the probability of choosing coin <em>y<\/em>. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">However, because our case is only using 2 coins, coins A and B, then the probability that I will choose one of them is <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-5992f1a801fd79b2af1dd2093c37e2cf_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#53;&#48;&#58;&#53;&#48;\" title=\"Rendered by QuickLaTeX.com\" height=\"13\" width=\"51\" style=\"vertical-align: 0px;\"\/>. Then <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-67ff139cdf4fd5d8f853978c496e9c89_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#80;&#40;&#90;&#95;&#123;&#65;&#125;&#41;&#32;&#61;&#32;&#80;&#40;&#90;&#95;&#123;&#66;&#125;&#41;&#32;&#61;&#32;&#48;&#46;&#53;\" title=\"Rendered by QuickLaTeX.com\" height=\"18\" width=\"172\" style=\"vertical-align: -4px;\"\/>. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Accordingly, for coin A, we can simplify the equation above into: <\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-54c5a1f58afd9deb993e9c109afb5038_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#98;&#101;&#103;&#105;&#110;&#123;&#97;&#108;&#105;&#103;&#110;&#101;&#100;&#125;&#80;&#40;&#90;&#95;&#123;&#65;&#125;&#124;&#69;&#41;&#32;&#38;&#61;&#32;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#80;&#40;&#69;&#124;&#90;&#95;&#123;&#65;&#125;&#41;&#92;&#99;&#97;&#110;&#99;&#101;&#108;&#123;&#80;&#40;&#90;&#95;&#123;&#65;&#125;&#41;&#125;&#125;&#123;&#80;&#40;&#69;&#124;&#90;&#95;&#123;&#65;&#125;&#41;&#92;&#99;&#97;&#110;&#99;&#101;&#108;&#123;&#80;&#40;&#90;&#95;&#123;&#65;&#125;&#41;&#125;&#43;&#80;&#40;&#69;&#124;&#90;&#95;&#123;&#66;&#125;&#41;&#92;&#99;&#97;&#110;&#99;&#101;&#108;&#123;&#80;&#40;&#90;&#95;&#123;&#66;&#125;&#41;&#125;&#125;&#32;&#92;&#92;&#091;&#49;&#50;&#112;&#116;&#093;&#32;&#32;&#32;&#38;&#61;&#32;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#80;&#40;&#69;&#124;&#90;&#95;&#123;&#65;&#125;&#41;&#125;&#123;&#80;&#40;&#69;&#124;&#90;&#95;&#123;&#65;&#125;&#41;&#43;&#80;&#40;&#69;&#124;&#90;&#95;&#123;&#66;&#125;&#41;&#125;&#32;&#92;&#92;&#091;&#49;&#50;&#112;&#116;&#093;&#32;&#92;&#101;&#110;&#100;&#123;&#97;&#108;&#105;&#103;&#110;&#101;&#100;&#125;&#32;\" title=\"Rendered by QuickLaTeX.com\" height=\"110\" width=\"367\" style=\"vertical-align: 0px;\"\/><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-855d9ea630e406925cd3297c6eb38bb2_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#98;&#101;&#103;&#105;&#110;&#123;&#97;&#108;&#105;&#103;&#110;&#101;&#100;&#125;&#32;&#80;&#40;&#90;&#95;&#123;&#65;&#125;&#124;&#69;&#95;&#123;&#56;&#72;&#50;&#84;&#125;&#41;&#32;&#38;&#61;&#32;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#92;&#99;&#97;&#110;&#99;&#101;&#108;&#123;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#49;&#48;&#33;&#125;&#123;&#56;&#33;&#92;&#59;&#50;&#33;&#125;&#125;&#92;&#59;&#123;&#48;&#46;&#54;&#125;&#94;&#123;&#56;&#125;&#92;&#59;&#48;&#46;&#52;&#94;&#123;&#50;&#125;&#125;&#123;&#92;&#99;&#97;&#110;&#99;&#101;&#108;&#123;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#49;&#48;&#33;&#125;&#123;&#56;&#33;&#92;&#59;&#50;&#33;&#125;&#125;&#92;&#59;&#123;&#48;&#46;&#54;&#125;&#94;&#123;&#56;&#125;&#92;&#59;&#48;&#46;&#52;&#94;&#123;&#50;&#125;&#43;&#92;&#99;&#97;&#110;&#99;&#101;&#108;&#123;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#49;&#48;&#33;&#125;&#123;&#56;&#33;&#92;&#59;&#50;&#33;&#125;&#125;&#92;&#59;&#123;&#48;&#46;&#53;&#125;&#94;&#123;&#56;&#125;&#92;&#59;&#48;&#46;&#53;&#94;&#123;&#50;&#125;&#125;&#32;&#92;&#92;&#091;&#50;&#52;&#112;&#116;&#093;&#32;&#32;&#32;&#38;&#61;&#32;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#123;&#48;&#46;&#54;&#125;&#94;&#123;&#56;&#125;&#92;&#59;&#48;&#46;&#52;&#94;&#123;&#50;&#125;&#125;&#123;&#123;&#48;&#46;&#54;&#125;&#94;&#123;&#56;&#125;&#92;&#59;&#48;&#46;&#52;&#94;&#123;&#50;&#125;&#43;&#123;&#48;&#46;&#53;&#125;&#94;&#123;&#56;&#125;&#92;&#59;&#48;&#46;&#53;&#94;&#123;&#50;&#125;&#125;&#32;&#92;&#92;&#091;&#49;&#50;&#112;&#116;&#093;&#32;&#32;&#38;&#61;&#32;&#48;&#46;&#55;&#51;&#32;&#92;&#92;&#091;&#49;&#50;&#112;&#116;&#093;&#32;&#92;&#101;&#110;&#100;&#123;&#97;&#108;&#105;&#103;&#110;&#101;&#100;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"210\" width=\"374\" style=\"vertical-align: 0px;\"\/>  <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Similarly, for coin B, become: <\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-e5903f648cb1dd3c03ee4b080b46879f_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#98;&#101;&#103;&#105;&#110;&#123;&#97;&#108;&#105;&#103;&#110;&#101;&#100;&#125;&#32;&#80;&#40;&#90;&#95;&#123;&#66;&#125;&#124;&#69;&#95;&#123;&#56;&#72;&#50;&#84;&#125;&#41;&#32;&#38;&#61;&#32;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#92;&#99;&#97;&#110;&#99;&#101;&#108;&#123;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#49;&#48;&#33;&#125;&#123;&#56;&#33;&#92;&#59;&#50;&#33;&#125;&#125;&#92;&#59;&#123;&#48;&#46;&#53;&#125;&#94;&#123;&#56;&#125;&#92;&#59;&#48;&#46;&#53;&#94;&#123;&#50;&#125;&#125;&#123;&#92;&#99;&#97;&#110;&#99;&#101;&#108;&#123;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#49;&#48;&#33;&#125;&#123;&#56;&#33;&#92;&#59;&#50;&#33;&#125;&#125;&#92;&#59;&#123;&#48;&#46;&#54;&#125;&#94;&#123;&#56;&#125;&#92;&#59;&#48;&#46;&#52;&#94;&#123;&#50;&#125;&#43;&#92;&#99;&#97;&#110;&#99;&#101;&#108;&#123;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#49;&#48;&#33;&#125;&#123;&#56;&#33;&#92;&#59;&#50;&#33;&#125;&#125;&#92;&#59;&#123;&#48;&#46;&#53;&#125;&#94;&#123;&#56;&#125;&#92;&#59;&#48;&#46;&#53;&#94;&#123;&#50;&#125;&#125;&#32;&#92;&#92;&#091;&#50;&#52;&#112;&#116;&#093;&#32;&#32;&#32;&#38;&#61;&#32;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#123;&#48;&#46;&#53;&#125;&#94;&#123;&#56;&#125;&#92;&#59;&#48;&#46;&#53;&#94;&#123;&#50;&#125;&#125;&#123;&#123;&#48;&#46;&#54;&#125;&#94;&#123;&#56;&#125;&#92;&#59;&#48;&#46;&#52;&#94;&#123;&#50;&#125;&#43;&#123;&#48;&#46;&#53;&#125;&#94;&#123;&#56;&#125;&#92;&#59;&#48;&#46;&#53;&#94;&#123;&#50;&#125;&#125;&#32;&#92;&#92;&#091;&#49;&#50;&#112;&#116;&#093;&#32;&#32;&#38;&#61;&#32;&#48;&#46;&#50;&#55;&#32;&#92;&#92;&#091;&#49;&#50;&#112;&#116;&#093;&#32;&#92;&#101;&#110;&#100;&#123;&#97;&#108;&#105;&#103;&#110;&#101;&#100;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"210\" width=\"375\" style=\"vertical-align: 0px;\"\/>   <\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Estimates the number of heads for each coin<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Previously, the ratio of coins A and B in giving each <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-e93700b2e5b0ca6b1fb78b43156e5af8_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#69;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"14\" style=\"vertical-align: 0px;\"\/> needs to be calculated. In this case, it is calculated from set 1 to 5. Then the results could be seen in the following table. <\/p>\n\n\n\n<table id=\"tablepress-14\" class=\"tablepress tablepress-id-14\">\n<thead>\n<tr class=\"row-1\">\n\t<th class=\"column-1\">Coin Tosses<\/th><th class=\"column-2\">E<\/th><th class=\"column-3\">Coin A Probability<\/th><th class=\"column-4\">Coin B Probability<\/th>\n<\/tr>\n<\/thead>\n<tbody class=\"row-striping row-hover\">\n<tr class=\"row-2\">\n\t<td class=\"column-1\">HTTTHHTHTH<\/td><td class=\"column-2\">5H 5T<\/td><td class=\"column-3\">0.45<\/td><td class=\"column-4\">0.55<\/td>\n<\/tr>\n<tr class=\"row-3\">\n\t<td class=\"column-1\">HHHHTHHHHH<\/td><td class=\"column-2\">9H 1T<\/td><td class=\"column-3\">0.80<\/td><td class=\"column-4\">0.20<\/td>\n<\/tr>\n<tr class=\"row-4\">\n\t<td class=\"column-1\">HTHHHHHTHH<\/td><td class=\"column-2\">8H 2T<\/td><td class=\"column-3\">0.73<\/td><td class=\"column-4\">0.27<\/td>\n<\/tr>\n<tr class=\"row-5\">\n\t<td class=\"column-1\">HTHTTTHHTT<\/td><td class=\"column-2\">4H 6T<\/td><td class=\"column-3\">0.35<\/td><td class=\"column-4\">0.27<\/td>\n<\/tr>\n<tr class=\"row-6\">\n\t<td class=\"column-1\">THHHTHHHTH<\/td><td class=\"column-2\">7H 3T<\/td><td class=\"column-3\">0.65<\/td><td class=\"column-4\">0.35<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<!-- #tablepress-14 from cache -->\n\n\n\n<p class=\"wp-block-paragraph\">After that, you need to estimate the total number of H for each coin [Fig 3(5)].  It is calculated based on the coin ratio above. To calculate &#8220;total heads and tails&#8221; for coin x, it is similar to the &#8220;complete data&#8221;. The method is quite easy, you just need to multiply the ratio of each coin to the number of heads in each <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-e93700b2e5b0ca6b1fb78b43156e5af8_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#69;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"14\" style=\"vertical-align: 0px;\"\/>, the results are shown in the following table.<\/p>\n\n\n\n<table id=\"tablepress-15\" class=\"tablepress tablepress-id-15\">\n<thead>\n<tr class=\"row-1\">\n\t<th class=\"column-1\">Coin Tosses<\/th><th class=\"column-2\">E<\/th><th class=\"column-3\">Estimated H for Coin A<\/th><th class=\"column-4\">Estimated H for Coin B<\/th>\n<\/tr>\n<\/thead>\n<tbody class=\"row-striping row-hover\">\n<tr class=\"row-2\">\n\t<td class=\"column-1\">HTTTHHTHTH<\/td><td class=\"column-2\">5H 5T<\/td><td class=\"column-3\">5 * 0.45 = 2.25<\/td><td class=\"column-4\">5 * 0.55 = 2.75<\/td>\n<\/tr>\n<tr class=\"row-3\">\n\t<td class=\"column-1\">HHHHTHHHHH<\/td><td class=\"column-2\">9H 1T<\/td><td class=\"column-3\">9 * 0.80 = 7.2<\/td><td class=\"column-4\">9 * 0.20 = 1.8<\/td>\n<\/tr>\n<tr class=\"row-4\">\n\t<td class=\"column-1\">HTHHHHHTHH<\/td><td class=\"column-2\">8H 2T<\/td><td class=\"column-3\">8 * 0.73 = 5.84<\/td><td class=\"column-4\">8 * 0.27 = 2.16<\/td>\n<\/tr>\n<tr class=\"row-5\">\n\t<td class=\"column-1\">HTHTTTHHTT<\/td><td class=\"column-2\">4H 6T<\/td><td class=\"column-3\">4 * 0.35 = 1.4<\/td><td class=\"column-4\">4 * 0.65 = 2.6<\/td>\n<\/tr>\n<tr class=\"row-6\">\n\t<td class=\"column-1\">THHHTHHHTH<\/td><td class=\"column-2\">7H 3T<\/td><td class=\"column-3\">7 * 0.65 = 4.55<\/td><td class=\"column-4\">7 * 0.35 = 2.45<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<!-- #tablepress-15 from cache -->\n\n\n\n<p class=\"wp-block-paragraph\">If you want, you can also calculate the tails for each coin as in the table above. But that is not necessary, because you can use another straightforward approach. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Until this step, you already have the E-Step calculations. Just a little bit more effort to finish your calculation in the M-Step.<\/p>\n\n\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">4. M-Step<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The results from the E-step can be used to improve the <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a8432efe7005e7528db837fc1555cb8a_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"9\" style=\"vertical-align: 0px;\"\/> parameter. Here, we can use the maximum likelihood estimation (MLE) equation similar to the &#8220;completed data&#8221;. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">As I said before, that it&#8217;s not necessary to calculate the tails for each coin; then you sum the heads and tails for each coin. Because each set contains 10 tosses, you just need to multiply the coin ratio with 10. That way, you could get the total estimated tosses from. Therefore: <\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-788b800c9da618fe1d2810c323e46e1f_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#98;&#101;&#103;&#105;&#110;&#123;&#97;&#108;&#105;&#103;&#110;&#101;&#100;&#125;&#32;&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#65;&#125;&#39;&#32;&#38;&#61;&#32;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#50;&#46;&#50;&#53;&#32;&#43;&#32;&#55;&#46;&#50;&#32;&#43;&#32;&#53;&#46;&#56;&#52;&#32;&#43;&#32;&#49;&#46;&#52;&#32;&#43;&#32;&#52;&#46;&#53;&#53;&#125;&#123;&#49;&#48;&#32;&#42;&#32;&#40;&#32;&#48;&#46;&#52;&#53;&#32;&#43;&#32;&#48;&#46;&#56;&#32;&#43;&#32;&#48;&#46;&#55;&#51;&#32;&#43;&#32;&#48;&#46;&#51;&#53;&#32;&#43;&#32;&#48;&#46;&#54;&#53;&#32;&#41;&#125;&#32;&#92;&#92;&#091;&#49;&#50;&#112;&#116;&#093;&#32;&#32;&#38;&#61;&#32;&#48;&#46;&#55;&#49;&#51;&#32;&#32;&#92;&#92;&#091;&#49;&#50;&#112;&#116;&#093;&#32;&#92;&#101;&#110;&#100;&#123;&#97;&#108;&#105;&#103;&#110;&#101;&#100;&#125;&#32;\" title=\"Rendered by QuickLaTeX.com\" height=\"84\" width=\"329\" style=\"vertical-align: 0px;\"\/> <\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-445985f36d1438e9b4193683e301d0e9_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#98;&#101;&#103;&#105;&#110;&#123;&#97;&#108;&#105;&#103;&#110;&#101;&#100;&#125;&#32;&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#66;&#125;&#39;&#32;&#38;&#61;&#32;&#92;&#100;&#102;&#114;&#97;&#99;&#123;&#50;&#46;&#55;&#53;&#32;&#43;&#32;&#49;&#46;&#56;&#32;&#43;&#32;&#50;&#46;&#49;&#54;&#32;&#43;&#32;&#50;&#46;&#54;&#32;&#43;&#32;&#50;&#46;&#52;&#53;&#125;&#123;&#49;&#48;&#32;&#42;&#32;&#40;&#32;&#48;&#46;&#53;&#53;&#32;&#43;&#32;&#48;&#46;&#50;&#32;&#43;&#32;&#48;&#46;&#50;&#55;&#32;&#43;&#32;&#48;&#46;&#54;&#53;&#32;&#43;&#32;&#48;&#46;&#51;&#53;&#32;&#41;&#125;&#32;&#92;&#92;&#091;&#49;&#50;&#112;&#116;&#093;&#32;&#32;&#38;&#61;&#32;&#48;&#46;&#53;&#56;&#49;&#32;&#92;&#101;&#110;&#100;&#123;&#97;&#108;&#105;&#103;&#110;&#101;&#100;&#125;&#32;\" title=\"Rendered by QuickLaTeX.com\" height=\"84\" width=\"330\" style=\"vertical-align: 0px;\"\/>  <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Finally, the both parameter of <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-f2e21ed0ec0708aa60f9e370e6eebdac_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#65;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"18\" style=\"vertical-align: -3px;\"\/> and <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-a01ff4ced34bb684cdddfd2f593de428_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#66;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"19\" style=\"vertical-align: -3px;\"\/> for the first iteration have been improved [Fig 3(2)]. For the next iteration, the E-Step will use this new parameter value; and re-improved at the next M-step. This iteration will always repeat the E-step and M-step, until it reaches any stop condition. <\/p>\n\n\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">5. Stopping condition and the final result<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The iteration of the E-Step and M-Step, will be repeated until they meet the stopping condition. Commonly, the EM algorithm has two options of stopping condition, which are: <\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><strong>Maximum iteration<\/strong>: means that, the EM Algorithm will stop if a certain number of iterations has been reached. For example, the maximum iteration is set to 10 iterations, then the EM Algorithm will not be more than 10 iterations. Or, <\/li><li><strong>Convergence threshold<\/strong>: means that, the M-step gives no significant parameter improvement; compared to the improvement in the previous iteration. The changes are very small below our threshold. <\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">The final result of EM algorithm <\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">In the EM case example above, the parameter improvement in each iteration can be seen in the following table: <\/p>\n\n\n\n<table id=\"tablepress-16\" class=\"tablepress tablepress-id-16\">\n<thead>\n<tr class=\"row-1\">\n\t<th class=\"column-1\">Iteration<\/th><th class=\"column-2\"><em>\u03b8<sub>A<\/sub><\/em><\/th><th class=\"column-3\"><em>\u03b8<sub>B<\/sub><\/em><\/th><th class=\"column-4\">Differences<\/th>\n<\/tr>\n<\/thead>\n<tbody class=\"row-striping row-hover\">\n<tr class=\"row-2\">\n\t<td class=\"column-1\">0<\/td><td class=\"column-2\">0.6<\/td><td class=\"column-3\">0.5<\/td><td class=\"column-4\">0.7810<\/td>\n<\/tr>\n<tr class=\"row-3\">\n\t<td class=\"column-1\">1<\/td><td class=\"column-2\">0.713<\/td><td class=\"column-3\">0.581<\/td><td class=\"column-4\">0.1390<\/td>\n<\/tr>\n<tr class=\"row-4\">\n\t<td class=\"column-1\">2<\/td><td class=\"column-2\">0.745<\/td><td class=\"column-3\">0.569<\/td><td class=\"column-4\">0.0342<\/td>\n<\/tr>\n<tr class=\"row-5\">\n\t<td class=\"column-1\">3<\/td><td class=\"column-2\">0.768<\/td><td class=\"column-3\">0.550<\/td><td class=\"column-4\">0.0298<\/td>\n<\/tr>\n<tr class=\"row-6\">\n\t<td class=\"column-1\">4<\/td><td class=\"column-2\">0.783<\/td><td class=\"column-3\">0.535<\/td><td class=\"column-4\">0.0212<\/td>\n<\/tr>\n<tr class=\"row-7\">\n\t<td class=\"column-1\">5<\/td><td class=\"column-2\">0.791<\/td><td class=\"column-3\">0.526<\/td><td class=\"column-4\">0.0120<\/td>\n<\/tr>\n<tr class=\"row-8\">\n\t<td class=\"column-1\">6<\/td><td class=\"column-2\">0.795<\/td><td class=\"column-3\">0.522<\/td><td class=\"column-4\">0.0057<\/td>\n<\/tr>\n<tr class=\"row-9\">\n\t<td class=\"column-1\">7<\/td><td class=\"column-2\">0.796<\/td><td class=\"column-3\">0.521<\/td><td class=\"column-4\">0.0014<\/td>\n<\/tr>\n<tr class=\"row-10\">\n\t<td class=\"column-1\">8<\/td><td class=\"column-2\">0.796<\/td><td class=\"column-3\">0.520<\/td><td class=\"column-4\">0.0010<br \/>\n<\/td>\n<\/tr>\n<tr class=\"row-11\">\n\t<td class=\"column-1\">9<\/td><td class=\"column-2\">0.796<\/td><td class=\"column-3\">0.520<\/td><td class=\"column-4\">0.0000<br \/>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<!-- #tablepress-16 from cache -->\n\n\n\n<p class=\"wp-block-paragraph\">To calculate the differences or improvements in each iteration, you can use the Euclidean Distance equation, which is: <\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-1204c880cfb0e425eba1cba3203f4f37_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#100;&#40;&#113;&#44;&#112;&#41;&#32;&#61;&#32;&#92;&#115;&#113;&#114;&#116;&#123;&#40;&#113;&#95;&#123;&#49;&#125;&#45;&#112;&#95;&#123;&#49;&#125;&#41;&#94;&#50;&#32;&#43;&#32;&#40;&#113;&#95;&#123;&#50;&#125;&#45;&#112;&#95;&#123;&#50;&#125;&#41;&#94;&#50;&#32;&#43;&#32;&#8230;&#32;&#43;&#32;&#40;&#113;&#95;&#123;&#110;&#125;&#45;&#112;&#95;&#123;&#110;&#125;&#41;&#94;&#50;&#125;&#32;&#32;&#92;&#92;&#091;&#49;&#50;&#112;&#116;&#093;&#32;\" title=\"Rendered by QuickLaTeX.com\" height=\"22\" width=\"386\" style=\"vertical-align: 0px;\"\/><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In your case, an examples from iteration 1 to iteration 2 is: <\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-48a1562771c0906536e775b66588f61f_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#98;&#101;&#103;&#105;&#110;&#123;&#97;&#108;&#105;&#103;&#110;&#101;&#100;&#125;&#32;&#100;&#40;&#92;&#116;&#104;&#101;&#116;&#97;&#44;&#92;&#116;&#104;&#101;&#116;&#97;&#39;&#41;&#32;&#38;&#61;&#32;&#92;&#115;&#113;&#114;&#116;&#123;&#40;&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#65;&#125;&#45;&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#65;&#125;&#39;&#41;&#94;&#50;&#32;&#43;&#32;&#40;&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#66;&#125;&#45;&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#66;&#125;&#39;&#41;&#94;&#50;&#125;&#32;&#32;&#92;&#92;&#091;&#49;&#50;&#112;&#116;&#093;&#32;&#100;&#40;&#105;&#116;&#101;&#114;&#49;&#44;&#105;&#116;&#101;&#114;&#50;&#41;&#32;&#38;&#61;&#32;&#92;&#115;&#113;&#114;&#116;&#123;&#40;&#48;&#46;&#55;&#49;&#51;&#45;&#48;&#46;&#55;&#52;&#53;&#41;&#94;&#50;&#32;&#43;&#32;&#40;&#48;&#46;&#53;&#56;&#49;&#45;&#48;&#46;&#53;&#54;&#57;&#125;&#32;&#32;&#92;&#92;&#091;&#49;&#50;&#112;&#116;&#093;&#32;&#38;&#61;&#32;&#48;&#46;&#48;&#51;&#52;&#50;&#32;&#92;&#101;&#110;&#100;&#123;&#97;&#108;&#105;&#103;&#110;&#101;&#100;&#125;&#32;\" title=\"Rendered by QuickLaTeX.com\" height=\"123\" width=\"404\" style=\"vertical-align: 0px;\"\/><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The final results will be taken, if one of the stop condition has been reached. It doesn&#8217;t matter which one is the first reached. For example, let&#8217;s just say that the maximum iteration limit is set to 10 iterations. It has been seen that the iteration above has been done 10 iterations (starting iterations from 0 to 9). However, if the Convergence Threshold set to a minimum of 0.001, then the iteration will stop at the 8<sup>th<\/sup> iteration. It because the convergence limit has been reached first, before the maximum iteration. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">So in our case, in 8<sup>th<\/sup> iteration, the final result obtained for coin A is <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-67d3c946c63fcf0492f66a1c1b17d935_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#65;&#125;&#61;&#48;&#46;&#55;&#57;&#54;\" title=\"Rendered by QuickLaTeX.com\" height=\"16\" width=\"84\" style=\"vertical-align: -3px;\"\/>, and for coin B is <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-3c34b88704cfd66779bb3feefac5d58b_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#66;&#125;&#61;&#48;&#46;&#53;&#50;&#48;\" title=\"Rendered by QuickLaTeX.com\" height=\"16\" width=\"85\" style=\"vertical-align: -3px;\"\/>.<\/p>\n\n\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Weaknesses of the EM algorithm<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Every algorithm has weaknesses, without any doubt; as well as the EM algorithm. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Every iteration in the EM algorithm, in general, will always improve the parameter closer to the local maximum likelihood. In other words, the EM algorithm will guarantee convergence, but will not guarantee to give a global maximum likelihood. And also there is no guarantee that you will get the maximum likelihood estimation (MLE). <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In some conditions, the EM algorithm may give unexpected results. for example, if we set the initial parameter with the same number, <img decoding=\"async\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/ql-cache\/quicklatex.com-981ce97ecef2aae39f74726a7865d242_l3.svg\" class=\"ql-img-inline-formula \" alt=\"&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#65;&#125;&#32;&#61;&#32;&#92;&#116;&#104;&#101;&#116;&#97;&#95;&#123;&#98;&#125;&#32;&#61;&#32;&#48;&#46;&#51;\" title=\"Rendered by QuickLaTeX.com\" height=\"15\" width=\"105\" style=\"vertical-align: -3px;\"\/>.<span id=\"fe100bf8-8b1a-4aab-9c25-580920192a2f\" data-items=\"[&quot;1032878754&quot;]\" class=\"abt-citation\" contenteditable=\"false\">\u200b[7]\u200b<\/span> Then the results are: <\/p>\n\n\n\n<table id=\"tablepress-17\" class=\"tablepress tablepress-id-17\">\n<thead>\n<tr class=\"row-1\">\n\t<th class=\"column-1\">Iteration<\/th><th class=\"column-2\"><em>\u03b8<sub>A<\/sub><\/em><\/th><th class=\"column-3\"><em>\u03b8<sub>B<\/sub><\/em><\/th><th class=\"column-4\">Diff<\/th>\n<\/tr>\n<\/thead>\n<tbody class=\"row-striping row-hover\">\n<tr class=\"row-2\">\n\t<td class=\"column-1\">0<\/td><td class=\"column-2\">0.3<\/td><td class=\"column-3\">0.3<\/td><td class=\"column-4\">0.7071<br \/>\n<\/td>\n<\/tr>\n<tr class=\"row-3\">\n\t<td class=\"column-1\">1<\/td><td class=\"column-2\">0.666<\/td><td class=\"column-3\">0.666<\/td><td class=\"column-4\">0.2263<br \/>\n<\/td>\n<\/tr>\n<tr class=\"row-4\">\n\t<td class=\"column-1\">2<\/td><td class=\"column-2\">0.666<\/td><td class=\"column-3\">0.666<\/td><td class=\"column-4\">0<\/td>\n<\/tr>\n<tr class=\"row-5\">\n\t<td class=\"column-1\">3<\/td><td class=\"column-2\">0.666<\/td><td class=\"column-3\">0.666<\/td><td class=\"column-4\">0<\/td>\n<\/tr>\n<tr class=\"row-6\">\n\t<td class=\"column-1\">4<\/td><td class=\"column-2\">0.666<\/td><td class=\"column-3\">0.666<\/td><td class=\"column-4\">0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<!-- #tablepress-17 from cache -->\n\n\n\n<p class=\"wp-block-paragraph\">It appears that there are no parameter improvements. For more details, try looking at Fig 4. &#8220;<em>Case 1<\/em>&#8221; is the normal case in the previous chapters, and &#8220;<em>Case 2<\/em>&#8221; is the case where we use the same initial parameter values.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"617\" height=\"516\" src=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/uploads\/2020\/03\/EM-plot-coin.jpg\" alt=\"Fig 4. The plot of EM algorithm convergence\" class=\"wp-image-1797\"\/><figcaption>Fig 4. The plot of EM algorithm convergence<\/figcaption><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">To avoid this, it&#8217;s best to do some experiments with several different initial parameters. In general, most of these values will point to the same local optimum, and maybe one of them can achieve the global optimum. <\/p>\n\n\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">The improvements of EM algorithm<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Well, usually every algorithm can be refined with other improvements. It may improve accuracy when solving your case.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">There are many things you can do, to improve this EM algorithm. For example:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>Use another method to determine the initial parameter values, rather than random values. Usually, this can reduce the number of EM iterations.<\/li><li>You can also use other methods to calculate the probabilities in E-steps according to the type of data distribution you have. <\/li><\/ol>\n\n\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">References<\/h2>\n\n\n\n<section aria-label=\"Bibliography\" class=\"wp-block-abt-bibliography abt-bibliography\" role=\"region\"><ol class=\"abt-bibliography__body\" data-maxoffset=\"3\" data-linespacing=\"1\" data-second-field-align=\"flush\"><li id=\"1714597785\">  <div class=\"csl-entry\">\n    <div class=\"csl-left-margin\">[1]<\/div><div class=\"csl-right-inline\">D. R. Kishor and N. B. Venkateswarlu, \u201cA Novel Hybridization of Expectation-Maximization and K-Means Algorithms for Better Clustering Performance,\u201d <i>International Journal of Ambient Computing and Intelligence<\/i>, pp. 47\u201374, Jul. 2016, doi: <a rel=\"noreferrer noopener\" target=\"_blank\" href=\"https:\/\/doi.org\/10.4018\/ijaci.2016070103\">10.4018\/ijaci.2016070103<\/a>.<\/div>\n  <\/div>\n<\/li><li id=\"3387592305\">  <div class=\"csl-entry\">\n    <div class=\"csl-left-margin\">[2]<\/div><div class=\"csl-right-inline\">J. Garriga, J. R. B. Palmer, A. Oltra, and F. Bartumeus, \u201cExpectation-Maximization Binary Clustering for Behavioural Annotation,\u201d <i>PLoS ONE<\/i>, p. e0151984, Mar. 2016, doi: <a rel=\"noreferrer noopener\" target=\"_blank\" href=\"https:\/\/doi.org\/10.1371\/journal.pone.0151984\">10.1371\/journal.pone.0151984<\/a>.<\/div>\n  <\/div>\n<\/li><li id=\"4284757828\">  <div class=\"csl-entry\">\n    <div class=\"csl-left-margin\">[3]<\/div><div class=\"csl-right-inline\">J. Hui, \u201cMachine Learning \u2014 Expectation-Maximization Algorithm (EM),\u201d <i>Medium<\/i>, Aug. 14, 2019. <a rel=\"noreferrer noopener\" target=\"_blank\" href=\"https:\/\/medium.com\/@jonathan_hui\/machine-learning-expectation-maximization-algorithm-em-2e954cb76959\">https:\/\/medium.com\/@jonathan_hui\/machine-learning-expectation-maximization-algorithm-em-2e954cb76959<\/a> (accessed Mar. 04, 2020).<\/div>\n  <\/div>\n<\/li><li id=\"2889665632\">  <div class=\"csl-entry\">\n    <div class=\"csl-left-margin\">[4]<\/div><div class=\"csl-right-inline\">C. B. Do and S. Batzoglou, \u201cWhat is the expectation maximization algorithm?,\u201d <i>Nat Biotechnol<\/i>, pp. 897\u2013899, Aug. 2008, doi: <a rel=\"noreferrer noopener\" target=\"_blank\" href=\"https:\/\/doi.org\/10.1038\/nbt1406\">10.1038\/nbt1406<\/a>.<\/div>\n  <\/div>\n<\/li><li id=\"610328443\">  <div class=\"csl-entry\">\n    <div class=\"csl-left-margin\">[5]<\/div><div class=\"csl-right-inline\">K. Rosaen, \u201cExpectation Maximization with Coin Flips,\u201d <i>Karl Rosaen: ML Study<\/i>, Dec. 22, 2016. <a rel=\"noreferrer noopener\" target=\"_blank\" href=\"http:\/\/karlrosaen.com\/ml\/notebooks\/em-coin-flips\/\">http:\/\/karlrosaen.com\/ml\/notebooks\/em-coin-flips\/<\/a> (accessed Jan. 14, 2019).<\/div>\n  <\/div>\n<\/li><li id=\"905564401\">  <div class=\"csl-entry\">\n    <div class=\"csl-left-margin\">[6]<\/div><div class=\"csl-right-inline\">stackexchange user, \u201cAnswer on &#8220;When to use Total Probability Rule and Bayes\u2019 Theorem&#8221;,\u201d <i>Mathematics Stack Exchange<\/i>, Sep. 28, 2014. <a rel=\"noreferrer noopener\" target=\"_blank\" href=\"https:\/\/math.stackexchange.com\/questions\/948888\/when-to-use-total-probability-rule-and-bayes-theorem\">https:\/\/math.stackexchange.com\/questions\/948888\/when-to-use-total-probability-rule-and-bayes-theorem<\/a> (accessed Mar. 02, 2020).<\/div>\n  <\/div>\n<\/li><li id=\"1032878754\">  <div class=\"csl-entry\">\n    <div class=\"csl-left-margin\">[7]<\/div><div class=\"csl-right-inline\">A. Abid, \u201cGently Building Up The EM Algorithm,\u201d <i>GitHub: abidlabs<\/i>, Mar. 01, 2018. <a rel=\"noreferrer noopener\" target=\"_blank\" href=\"https:\/\/abidlabs.github.io\/EM-Algorithm\/\">https:\/\/abidlabs.github.io\/EM-Algorithm\/<\/a> (accessed Feb. 28, 2020).<\/div>\n  <\/div>\n<\/li><\/ol><\/section>\n\n\n\n<p class=\"wp-block-paragraph\">Fig. 1 taken by Dennis Kwaria in <a rel=\"noreferrer noopener\" aria-label=\"wikimedia commons (opens in a new tab)\" href=\"https:\/\/commons.wikimedia.org\/wiki\/File:IDR_1000_Koin.JPG\" target=\"_blank\">wikimedia commons<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Learn an algorithm in autodidact is time-consuming; especially for people who are not interested in mathematics. Because, sometimes we don&#8217;t know that &#8220;we don&#8217;t know&#8221;, many things. Including when studying the Expectation \u2013 maximization (EM) Algorithm. Therefore, I could improve your understanding of the EM algorithm in this article, and I hope you can easily [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":1802,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_lmt_disableupdate":"no","_lmt_disable":"","site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[491],"tags":[377,381,379,383,372,370,75],"class_list":["post-1819","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-concept","tag-algorithm","tag-coin-toss","tag-concept","tag-data-clustering","tag-em","tag-expectation-maximization","tag-machine-learning"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.9 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>The Simple Concept of Expectation \u2013 maximization (EM) Algorithm &#8211; Indowhiz<\/title>\n<meta name=\"description\" content=\"If we know A, then we can calculate B; or vice versa. But how if the data is mixed where A and B unknown? Then EM algorithm may helps you. Read more...\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The Simple Concept of Expectation \u2013 maximization (EM) Algorithm &#8211; Indowhiz\" \/>\n<meta property=\"og:description\" content=\"If we know A, then we can calculate B; or vice versa. But how if the data is mixed where A and B unknown? Then EM algorithm may helps you. Read more...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/\" \/>\n<meta property=\"og:site_name\" content=\"Indowhiz\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/indowhiz\/\" \/>\n<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/4faster\" \/>\n<meta property=\"article:published_time\" content=\"2020-03-22T16:01:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-05-27T08:37:14+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.indowhiz.com\/articles\/wp-content\/uploads\/2020\/03\/EM-Algorithm-feature-image.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1000\" \/>\n\t<meta property=\"og:image:height\" content=\"600\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Philip F. E. Adipraja\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@4Faster\" \/>\n<meta name=\"twitter:site\" content=\"@4Faster\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Philip F. E. Adipraja\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"18 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/en\\\/the-simple-concept-of-expectation-maximization-em-algorithm\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/en\\\/the-simple-concept-of-expectation-maximization-em-algorithm\\\/\"},\"author\":{\"name\":\"Philip F. E. Adipraja\",\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/#\\\/schema\\\/person\\\/7b9a73764f92b8b7b6040eb1f83f8957\"},\"headline\":\"The Simple Concept of Expectation \u2013 maximization (EM) Algorithm\",\"datePublished\":\"2020-03-22T16:01:00+00:00\",\"dateModified\":\"2022-05-27T08:37:14+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/en\\\/the-simple-concept-of-expectation-maximization-em-algorithm\\\/\"},\"wordCount\":3068,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/en\\\/the-simple-concept-of-expectation-maximization-em-algorithm\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/wp-content\\\/uploads\\\/2020\\\/03\\\/EM-Algorithm-feature-image.png\",\"keywords\":[\"algorithm\",\"coin toss\",\"concept\",\"data clustering\",\"EM\",\"expectation\u2013maximization\",\"machine learning\"],\"articleSection\":[\"Concept\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/en\\\/the-simple-concept-of-expectation-maximization-em-algorithm\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/en\\\/the-simple-concept-of-expectation-maximization-em-algorithm\\\/\",\"url\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/en\\\/the-simple-concept-of-expectation-maximization-em-algorithm\\\/\",\"name\":\"The Simple Concept of Expectation \u2013 maximization (EM) Algorithm &#8211; Indowhiz\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/en\\\/the-simple-concept-of-expectation-maximization-em-algorithm\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/en\\\/the-simple-concept-of-expectation-maximization-em-algorithm\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/wp-content\\\/uploads\\\/2020\\\/03\\\/EM-Algorithm-feature-image.png\",\"datePublished\":\"2020-03-22T16:01:00+00:00\",\"dateModified\":\"2022-05-27T08:37:14+00:00\",\"description\":\"If we know A, then we can calculate B; or vice versa. But how if the data is mixed where A and B unknown? Then EM algorithm may helps you. Read more...\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/en\\\/the-simple-concept-of-expectation-maximization-em-algorithm\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/en\\\/the-simple-concept-of-expectation-maximization-em-algorithm\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/en\\\/the-simple-concept-of-expectation-maximization-em-algorithm\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/wp-content\\\/uploads\\\/2020\\\/03\\\/EM-Algorithm-feature-image.png\",\"contentUrl\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/wp-content\\\/uploads\\\/2020\\\/03\\\/EM-Algorithm-feature-image.png\",\"width\":1000,\"height\":600,\"caption\":\"EM Algorithm feature image\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/en\\\/the-simple-concept-of-expectation-maximization-em-algorithm\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Articles\",\"item\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/en\\\/home\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Science and Technology\",\"item\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/en\\\/category\\\/science-technology\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Concept\",\"item\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/en\\\/category\\\/science-technology\\\/concept\\\/\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"The Simple Concept of Expectation \u2013 maximization (EM) Algorithm\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/#website\",\"url\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/\",\"name\":\"Indowhiz\",\"description\":\"The reliable information provider\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/#organization\",\"name\":\"Indowhiz\",\"url\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/wp-content\\\/uploads\\\/2020\\\/07\\\/logo-indowhiz-v3.png\",\"contentUrl\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/wp-content\\\/uploads\\\/2020\\\/07\\\/logo-indowhiz-v3.png\",\"width\":280,\"height\":56,\"caption\":\"Indowhiz\"},\"image\":{\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/indowhiz\\\/\",\"https:\\\/\\\/x.com\\\/4Faster\",\"https:\\\/\\\/www.instagram.com\\\/indowhiz\\\/\",\"https:\\\/\\\/www.pinterest.com\\\/indowhiz\\\/\",\"https:\\\/\\\/www.youtube.com\\\/indowhiz\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/#\\\/schema\\\/person\\\/7b9a73764f92b8b7b6040eb1f83f8957\",\"name\":\"Philip F. E. Adipraja\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/0428d2ce65cedc300cc77fbea17fc8fdeee4e7d92b137a1f2cd3efc80bb88069?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/0428d2ce65cedc300cc77fbea17fc8fdeee4e7d92b137a1f2cd3efc80bb88069?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/0428d2ce65cedc300cc77fbea17fc8fdeee4e7d92b137a1f2cd3efc80bb88069?s=96&d=mm&r=g\",\"caption\":\"Philip F. E. Adipraja\"},\"description\":\"Philip is a Computer Science and Information Engineering student at National Central University, Taiwan. His fields of interest are software engineering, modeling &amp; simulation, intelligent transport system, and smart manufacturing.\",\"sameAs\":[\"https:\\\/\\\/www.indowhiz.com\",\"https:\\\/\\\/www.facebook.com\\\/4faster\"],\"url\":\"https:\\\/\\\/www.indowhiz.com\\\/articles\\\/author\\\/philipfaster\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"The Simple Concept of Expectation \u2013 maximization (EM) Algorithm &#8211; Indowhiz","description":"If we know A, then we can calculate B; or vice versa. But how if the data is mixed where A and B unknown? Then EM algorithm may helps you. Read more...","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/","og_locale":"en_US","og_type":"article","og_title":"The Simple Concept of Expectation \u2013 maximization (EM) Algorithm &#8211; Indowhiz","og_description":"If we know A, then we can calculate B; or vice versa. But how if the data is mixed where A and B unknown? Then EM algorithm may helps you. Read more...","og_url":"https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/","og_site_name":"Indowhiz","article_publisher":"https:\/\/www.facebook.com\/indowhiz\/","article_author":"https:\/\/www.facebook.com\/4faster","article_published_time":"2020-03-22T16:01:00+00:00","article_modified_time":"2022-05-27T08:37:14+00:00","og_image":[{"width":1000,"height":600,"url":"https:\/\/www.indowhiz.com\/articles\/wp-content\/uploads\/2020\/03\/EM-Algorithm-feature-image.png","type":"image\/png"}],"author":"Philip F. E. Adipraja","twitter_card":"summary_large_image","twitter_creator":"@4Faster","twitter_site":"@4Faster","twitter_misc":{"Written by":"Philip F. E. Adipraja","Est. reading time":"18 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/#article","isPartOf":{"@id":"https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/"},"author":{"name":"Philip F. E. Adipraja","@id":"https:\/\/www.indowhiz.com\/articles\/#\/schema\/person\/7b9a73764f92b8b7b6040eb1f83f8957"},"headline":"The Simple Concept of Expectation \u2013 maximization (EM) Algorithm","datePublished":"2020-03-22T16:01:00+00:00","dateModified":"2022-05-27T08:37:14+00:00","mainEntityOfPage":{"@id":"https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/"},"wordCount":3068,"commentCount":0,"publisher":{"@id":"https:\/\/www.indowhiz.com\/articles\/#organization"},"image":{"@id":"https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/#primaryimage"},"thumbnailUrl":"https:\/\/www.indowhiz.com\/articles\/wp-content\/uploads\/2020\/03\/EM-Algorithm-feature-image.png","keywords":["algorithm","coin toss","concept","data clustering","EM","expectation\u2013maximization","machine learning"],"articleSection":["Concept"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/","url":"https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/","name":"The Simple Concept of Expectation \u2013 maximization (EM) Algorithm &#8211; Indowhiz","isPartOf":{"@id":"https:\/\/www.indowhiz.com\/articles\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/#primaryimage"},"image":{"@id":"https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/#primaryimage"},"thumbnailUrl":"https:\/\/www.indowhiz.com\/articles\/wp-content\/uploads\/2020\/03\/EM-Algorithm-feature-image.png","datePublished":"2020-03-22T16:01:00+00:00","dateModified":"2022-05-27T08:37:14+00:00","description":"If we know A, then we can calculate B; or vice versa. But how if the data is mixed where A and B unknown? Then EM algorithm may helps you. Read more...","breadcrumb":{"@id":"https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/#primaryimage","url":"https:\/\/www.indowhiz.com\/articles\/wp-content\/uploads\/2020\/03\/EM-Algorithm-feature-image.png","contentUrl":"https:\/\/www.indowhiz.com\/articles\/wp-content\/uploads\/2020\/03\/EM-Algorithm-feature-image.png","width":1000,"height":600,"caption":"EM Algorithm feature image"},{"@type":"BreadcrumbList","@id":"https:\/\/www.indowhiz.com\/articles\/en\/the-simple-concept-of-expectation-maximization-em-algorithm\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Articles","item":"https:\/\/www.indowhiz.com\/articles\/en\/home\/"},{"@type":"ListItem","position":2,"name":"Science and Technology","item":"https:\/\/www.indowhiz.com\/articles\/en\/category\/science-technology\/"},{"@type":"ListItem","position":3,"name":"Concept","item":"https:\/\/www.indowhiz.com\/articles\/en\/category\/science-technology\/concept\/"},{"@type":"ListItem","position":4,"name":"The Simple Concept of Expectation \u2013 maximization (EM) Algorithm"}]},{"@type":"WebSite","@id":"https:\/\/www.indowhiz.com\/articles\/#website","url":"https:\/\/www.indowhiz.com\/articles\/","name":"Indowhiz","description":"The reliable information provider","publisher":{"@id":"https:\/\/www.indowhiz.com\/articles\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.indowhiz.com\/articles\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.indowhiz.com\/articles\/#organization","name":"Indowhiz","url":"https:\/\/www.indowhiz.com\/articles\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.indowhiz.com\/articles\/#\/schema\/logo\/image\/","url":"https:\/\/www.indowhiz.com\/articles\/wp-content\/uploads\/2020\/07\/logo-indowhiz-v3.png","contentUrl":"https:\/\/www.indowhiz.com\/articles\/wp-content\/uploads\/2020\/07\/logo-indowhiz-v3.png","width":280,"height":56,"caption":"Indowhiz"},"image":{"@id":"https:\/\/www.indowhiz.com\/articles\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/indowhiz\/","https:\/\/x.com\/4Faster","https:\/\/www.instagram.com\/indowhiz\/","https:\/\/www.pinterest.com\/indowhiz\/","https:\/\/www.youtube.com\/indowhiz"]},{"@type":"Person","@id":"https:\/\/www.indowhiz.com\/articles\/#\/schema\/person\/7b9a73764f92b8b7b6040eb1f83f8957","name":"Philip F. E. Adipraja","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/0428d2ce65cedc300cc77fbea17fc8fdeee4e7d92b137a1f2cd3efc80bb88069?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/0428d2ce65cedc300cc77fbea17fc8fdeee4e7d92b137a1f2cd3efc80bb88069?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/0428d2ce65cedc300cc77fbea17fc8fdeee4e7d92b137a1f2cd3efc80bb88069?s=96&d=mm&r=g","caption":"Philip F. E. Adipraja"},"description":"Philip is a Computer Science and Information Engineering student at National Central University, Taiwan. His fields of interest are software engineering, modeling &amp; simulation, intelligent transport system, and smart manufacturing.","sameAs":["https:\/\/www.indowhiz.com","https:\/\/www.facebook.com\/4faster"],"url":"https:\/\/www.indowhiz.com\/articles\/author\/philipfaster\/"}]}},"modified_by":"Philip F. E. Adipraja","_links":{"self":[{"href":"https:\/\/www.indowhiz.com\/articles\/wp-json\/wp\/v2\/posts\/1819","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.indowhiz.com\/articles\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.indowhiz.com\/articles\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.indowhiz.com\/articles\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.indowhiz.com\/articles\/wp-json\/wp\/v2\/comments?post=1819"}],"version-history":[{"count":2,"href":"https:\/\/www.indowhiz.com\/articles\/wp-json\/wp\/v2\/posts\/1819\/revisions"}],"predecessor-version":[{"id":5678,"href":"https:\/\/www.indowhiz.com\/articles\/wp-json\/wp\/v2\/posts\/1819\/revisions\/5678"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.indowhiz.com\/articles\/wp-json\/wp\/v2\/media\/1802"}],"wp:attachment":[{"href":"https:\/\/www.indowhiz.com\/articles\/wp-json\/wp\/v2\/media?parent=1819"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.indowhiz.com\/articles\/wp-json\/wp\/v2\/categories?post=1819"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.indowhiz.com\/articles\/wp-json\/wp\/v2\/tags?post=1819"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}