mirror-github-bspeice/bspeice.github.io
1
0
Fork 0
mirror of https://github.com/bspeice/bspeice.github.io synced 2026-06-09 14:52:07 -04:00
Code Issues Releases Wiki Activity

New earnings release post

Browse Source
This commit is contained in:
Bradlee Speice
2016-06-09 19:27:36 -04:00
parent 0ba3723cd2
commit f9e8c08491
26 changed files with 17371 additions and 1256 deletions
Download Patch File Download Diff File Expand all files Collapse all files
archives.html
+2
View File
@@ -82,6 +82,8 @@
<div class="container content archive"> <div class="container content archive">
<h2><a href="https://bspeice.github.io/archives.html"></a></h2> <h2><a href="https://bspeice.github.io/archives.html"></a></h2>
<dl class="dl-horizontal"> <dl class="dl-horizontal">
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
<dt>Sun 15 May 2016</dt> <dt>Sun 15 May 2016</dt>
<dd><a href="https://bspeice.github.io/the-unfair-casino.html">The Unfair Casino</a></dd> <dd><a href="https://bspeice.github.io/the-unfair-casino.html">The Unfair Casino</a></dd>
<dt>Wed 06 April 2016</dt> <dt>Wed 06 April 2016</dt>
author/bradlee-speice.html
+2
View File
@@ -82,6 +82,8 @@
<div class="container content archive"> <div class="container content archive">
<h2><a href="https://bspeice.github.io/author/bradlee-speice.html"></a></h2> <h2><a href="https://bspeice.github.io/author/bradlee-speice.html"></a></h2>
<dl class="dl-horizontal"> <dl class="dl-horizontal">
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
<dt>Sun 15 May 2016</dt> <dt>Sun 15 May 2016</dt>
<dd><a href="https://bspeice.github.io/the-unfair-casino.html">The Unfair Casino</a></dd> <dd><a href="https://bspeice.github.io/the-unfair-casino.html">The Unfair Casino</a></dd>
<dt>Wed 06 April 2016</dt> <dt>Wed 06 April 2016</dt>
author/bradlee-speice2.html
+2
View File
@@ -82,6 +82,8 @@
<div class="container content archive"> <div class="container content archive">
<h2><a href="https://bspeice.github.io/author/bradlee-speice.html"></a></h2> <h2><a href="https://bspeice.github.io/author/bradlee-speice.html"></a></h2>
<dl class="dl-horizontal"> <dl class="dl-horizontal">
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
<dt>Sun 15 May 2016</dt> <dt>Sun 15 May 2016</dt>
<dd><a href="https://bspeice.github.io/the-unfair-casino.html">The Unfair Casino</a></dd> <dd><a href="https://bspeice.github.io/the-unfair-casino.html">The Unfair Casino</a></dd>
<dt>Wed 06 April 2016</dt> <dt>Wed 06 April 2016</dt>
authors.html
+2
View File
@@ -82,6 +82,8 @@
<div class="container content archive"> <div class="container content archive">
<h2><a href="https://bspeice.github.io/author/bradlee-speice.html">Bradlee Speice</a></h2> <h2><a href="https://bspeice.github.io/author/bradlee-speice.html">Bradlee Speice</a></h2>
<dl class="dl-horizontal"> <dl class="dl-horizontal">
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
<dt>Sun 15 May 2016</dt> <dt>Sun 15 May 2016</dt>
<dd><a href="https://bspeice.github.io/the-unfair-casino.html">The Unfair Casino</a></dd> <dd><a href="https://bspeice.github.io/the-unfair-casino.html">The Unfair Casino</a></dd>
<dt>Wed 06 April 2016</dt> <dt>Wed 06 April 2016</dt>
autocallable-bonds.html
+32 -32
View File
@@ -95,7 +95,7 @@ Because these are all very similar, we decided to demonstrate all 3 products at
<div class="prompt input_prompt">In&nbsp;[1]:</div> <div class="prompt input_prompt">In&nbsp;[1]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-julia"><pre><span></span><span class="k">using</span> <span class="n">Gadfly</span> <div class=" highlight hl-julia"><pre><span class="k">using</span> <span class="n">Gadfly</span>
</pre></div> </pre></div>
</div> </div>
@@ -131,10 +131,10 @@ Because these are all very similar, we decided to demonstrate all 3 products at
<div class="prompt input_prompt">In&nbsp;[2]:</div> <div class="prompt input_prompt">In&nbsp;[2]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-julia"><pre><span></span><span class="n">S0</span> <span class="o">=</span> <span class="mf">102.2</span> <div class=" highlight hl-julia"><pre><span class="n">S0</span> <span class="o">=</span> <span class="mf">102.2</span>
<span class="n">nominal</span> <span class="o">=</span> <span class="mi">100</span> <span class="n">nominal</span> <span class="o">=</span> <span class="mi">100</span>
<span class="n">q</span> <span class="o">=</span> <span class="mf">2.84</span> <span class="o">/</span> <span class="mi">100</span> <span class="n">q</span> <span class="o">=</span> <span class="mf">2.84</span> <span class="o">/</span> <span class="mi">100</span>
<span class="n">σ</span> <span class="o">=</span> <span class="mf">15.37</span> <span class="o">/</span> <span class="mi">100</span> <span class="err">σ</span> <span class="o">=</span> <span class="mf">15.37</span> <span class="o">/</span> <span class="mi">100</span>
<span class="n">term</span> <span class="o">=</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="o">.</span><span class="mi">49</span><span class="p">,</span> <span class="o">.</span><span class="mi">9</span><span class="p">,</span> <span class="mf">1.21</span><span class="p">,</span> <span class="mf">1.45</span><span class="p">,</span> <span class="mf">1.69</span><span class="p">]</span> <span class="o">/</span> <span class="mi">100</span> <span class="o">+</span> <span class="mi">1</span> <span class="n">term</span> <span class="o">=</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="o">.</span><span class="mi">49</span><span class="p">,</span> <span class="o">.</span><span class="mi">9</span><span class="p">,</span> <span class="mf">1.21</span><span class="p">,</span> <span class="mf">1.45</span><span class="p">,</span> <span class="mf">1.69</span><span class="p">]</span> <span class="o">/</span> <span class="mi">100</span> <span class="o">+</span> <span class="mi">1</span>
<span class="c">###</span> <span class="c">###</span>
@@ -187,7 +187,7 @@ Because these are all very similar, we decided to demonstrate all 3 products at
<div class="prompt input_prompt">In&nbsp;[3]:</div> <div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-julia"><pre><span></span><span class="n">simulate_gbm</span> <span class="o">=</span> <span class="n">function</span><span class="p">(</span><span class="n">S0</span><span class="p">,</span> <span class="n">μ</span><span class="p">,</span> <span class="n">σ</span><span class="p">,</span> <span class="n">T</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span> <div class=" highlight hl-julia"><pre><span class="n">simulate_gbm</span> <span class="o">=</span> <span class="n">function</span><span class="p">(</span><span class="n">S0</span><span class="p">,</span> <span class="err">μ</span><span class="p">,</span> <span class="err">σ</span><span class="p">,</span> <span class="n">T</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
<span class="c"># Set the initial state</span> <span class="c"># Set the initial state</span>
<span class="n">m</span> <span class="o">=</span> <span class="n">length</span><span class="p">(</span><span class="n">S0</span><span class="p">)</span> <span class="n">m</span> <span class="o">=</span> <span class="n">length</span><span class="p">(</span><span class="n">S0</span><span class="p">)</span>
<span class="n">t</span> <span class="o">=</span> <span class="n">T</span> <span class="o">/</span> <span class="n">n</span> <span class="n">t</span> <span class="o">=</span> <span class="n">T</span> <span class="o">/</span> <span class="n">n</span>
@@ -196,7 +196,7 @@ Because these are all very similar, we decided to demonstrate all 3 products at
<span class="c"># Build out all states</span> <span class="c"># Build out all states</span>
<span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:(</span><span class="n">n</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:(</span><span class="n">n</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>
<span class="n">motion</span><span class="p">[:,</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="n">motion</span><span class="p">[:,</span><span class="n">i</span><span class="p">]</span> <span class="o">.*</span> <span class="n">exp</span><span class="p">((</span><span class="n">μ</span> <span class="o">-</span> <span class="n">σ</span><span class="o">^</span><span class="mi">2</span><span class="o">/</span><span class="mi">2</span><span class="p">)</span><span class="o">*</span><span class="n">t</span><span class="p">)</span> <span class="o">.*</span> <span class="n">exp</span><span class="p">(</span><span class="n">sqrt</span><span class="p">(</span><span class="n">t</span><span class="p">)</span> <span class="o">*</span> <span class="n">σ</span> <span class="o">.*</span> <span class="n">randn</span><span class="p">(</span><span class="n">m</span><span class="p">))</span> <span class="n">motion</span><span class="p">[:,</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="n">motion</span><span class="p">[:,</span><span class="n">i</span><span class="p">]</span> <span class="o">.*</span> <span class="n">exp</span><span class="p">((</span><span class="err">μ</span> <span class="o">-</span> <span class="err">σ</span><span class="o">^</span><span class="mi">2</span><span class="o">/</span><span class="mi">2</span><span class="p">)</span><span class="o">*</span><span class="n">t</span><span class="p">)</span> <span class="o">.*</span> <span class="n">exp</span><span class="p">(</span><span class="n">sqrt</span><span class="p">(</span><span class="n">t</span><span class="p">)</span> <span class="o">*</span> <span class="err">σ</span> <span class="o">.*</span> <span class="n">randn</span><span class="p">(</span><span class="n">m</span><span class="p">))</span>
<span class="k">end</span> <span class="k">end</span>
<span class="k">return</span> <span class="n">motion</span> <span class="k">return</span> <span class="n">motion</span>
@@ -258,9 +258,9 @@ Because these are all very similar, we decided to demonstrate all 3 products at
<div class="prompt input_prompt">In&nbsp;[4]:</div> <div class="prompt input_prompt">In&nbsp;[4]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-julia"><pre><span></span><span class="n">initial</span> <span class="o">=</span> <span class="n">ones</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span> <span class="o">*</span> <span class="n">S0</span> <div class=" highlight hl-julia"><pre><span class="n">initial</span> <span class="o">=</span> <span class="n">ones</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span> <span class="o">*</span> <span class="n">S0</span>
<span class="c"># Using μ=0, T=.25 for now, we&#39;ll use the proper values later</span> <span class="c"># Using μ=0, T=.25 for now, we&#39;ll use the proper values later</span>
<span class="n">motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">initial</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="n">σ</span><span class="p">,</span> <span class="o">.</span><span class="mi">25</span><span class="p">,</span> <span class="mi">200</span><span class="p">)</span> <span class="n">motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">initial</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="err">σ</span><span class="p">,</span> <span class="o">.</span><span class="mi">25</span><span class="p">,</span> <span class="mi">200</span><span class="p">)</span>
<span class="n">display_motion</span><span class="p">(</span><span class="n">motion</span><span class="p">,</span> <span class="o">.</span><span class="mi">25</span><span class="p">)</span> <span class="n">display_motion</span><span class="p">(</span><span class="n">motion</span><span class="p">,</span> <span class="o">.</span><span class="mi">25</span><span class="p">)</span>
</pre></div> </pre></div>
@@ -1866,7 +1866,7 @@ fig.select("#fig-3a6dd25ad25c4037a166889ee51bb151-element-20")
<div class="prompt input_prompt">In&nbsp;[5]:</div> <div class="prompt input_prompt">In&nbsp;[5]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-julia"><pre><span></span><span class="n">forward_term</span> <span class="o">=</span> <span class="n">function</span><span class="p">(</span><span class="n">yearly_term</span><span class="p">)</span> <div class=" highlight hl-julia"><pre><span class="n">forward_term</span> <span class="o">=</span> <span class="n">function</span><span class="p">(</span><span class="n">yearly_term</span><span class="p">)</span>
<span class="c"># It is assumed that we have a yearly term structure passed in, and starts at year 0</span> <span class="c"># It is assumed that we have a yearly term structure passed in, and starts at year 0</span>
<span class="c"># This implies a nominal rate above 0 for the first year!</span> <span class="c"># This implies a nominal rate above 0 for the first year!</span>
<span class="n">years</span> <span class="o">=</span> <span class="n">length</span><span class="p">(</span><span class="n">term</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span> <span class="c"># because we start at 0</span> <span class="n">years</span> <span class="o">=</span> <span class="n">length</span><span class="p">(</span><span class="n">term</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span> <span class="c"># because we start at 0</span>
@@ -1896,14 +1896,14 @@ fig.select("#fig-3a6dd25ad25c4037a166889ee51bb151-element-20")
<div class="prompt input_prompt">In&nbsp;[6]:</div> <div class="prompt input_prompt">In&nbsp;[6]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-julia"><pre><span></span><span class="c"># Example term structure taken from:</span> <div class=" highlight hl-julia"><pre><span class="c"># Example term structure taken from:</span>
<span class="c"># http://www.treasury.gov/resource-center/data-chart-center/interest-rates/Pages/TextView.aspx?data=yield</span> <span class="c"># http://www.treasury.gov/resource-center/data-chart-center/interest-rates/Pages/TextView.aspx?data=yield</span>
<span class="c"># Linear interpolation used years in-between periods, assuming real-dollar</span> <span class="c"># Linear interpolation used years in-between periods, assuming real-dollar</span>
<span class="c"># interest rates</span> <span class="c"># interest rates</span>
<span class="n">forward_yield</span> <span class="o">=</span> <span class="n">forward_term</span><span class="p">(</span><span class="n">term</span><span class="p">)</span> <span class="n">forward_yield</span> <span class="o">=</span> <span class="n">forward_term</span><span class="p">(</span><span class="n">term</span><span class="p">)</span>
<span class="n">calculated_term2</span> <span class="o">=</span> <span class="n">term</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">*</span> <span class="n">forward_yield</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="n">calculated_term2</span> <span class="o">=</span> <span class="n">term</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">*</span> <span class="n">forward_yield</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="n">println</span><span class="p">(</span><span class="s">&quot;Actual term[2]: </span><span class="si">$(term[2])</span><span class="s">; Calculated term[2]: </span><span class="si">$(calculated_term2)</span><span class="s">&quot;</span><span class="p">)</span> <span class="n">println</span><span class="p">(</span><span class="s">&quot;Actual term[2]: </span><span class="si">$</span><span class="s">(term[2]); Calculated term[2]: </span><span class="si">$(calculated_term2)</span><span class="s">&quot;</span><span class="p">)</span>
</pre></div> </pre></div>
</div> </div>
@@ -1940,11 +1940,11 @@ fig.select("#fig-3a6dd25ad25c4037a166889ee51bb151-element-20")
<div class="prompt input_prompt">In&nbsp;[7]:</div> <div class="prompt input_prompt">In&nbsp;[7]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-julia"><pre><span></span><span class="n">full_motion</span> <span class="o">=</span> <span class="n">ones</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span> <span class="o">*</span> <span class="n">S0</span> <div class=" highlight hl-julia"><pre><span class="n">full_motion</span> <span class="o">=</span> <span class="n">ones</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span> <span class="o">*</span> <span class="n">S0</span>
<span class="n">full_term</span> <span class="o">=</span> <span class="n">vcat</span><span class="p">(</span><span class="n">term</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">forward_yield</span><span class="p">)</span> <span class="n">full_term</span> <span class="o">=</span> <span class="n">vcat</span><span class="p">(</span><span class="n">term</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">forward_yield</span><span class="p">)</span>
<span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">T</span> <span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">T</span>
<span class="n">μ</span> <span class="o">=</span> <span class="p">(</span><span class="n">full_term</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span> <span class="o">-</span> <span class="n">q</span><span class="p">)</span> <span class="err">μ</span> <span class="o">=</span> <span class="p">(</span><span class="n">full_term</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span> <span class="o">-</span> <span class="n">q</span><span class="p">)</span>
<span class="n">year_motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">full_motion</span><span class="p">[:,</span><span class="k">end</span><span class="p">],</span> <span class="n">μ</span><span class="p">,</span> <span class="n">σ</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span> <span class="n">year_motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">full_motion</span><span class="p">[:,</span><span class="k">end</span><span class="p">],</span> <span class="err">μ</span><span class="p">,</span> <span class="err">σ</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
<span class="n">full_motion</span> <span class="o">=</span> <span class="n">hcat</span><span class="p">(</span><span class="n">full_motion</span><span class="p">,</span> <span class="n">year_motion</span><span class="p">)</span> <span class="n">full_motion</span> <span class="o">=</span> <span class="n">hcat</span><span class="p">(</span><span class="n">full_motion</span><span class="p">,</span> <span class="n">year_motion</span><span class="p">)</span>
<span class="k">end</span> <span class="k">end</span>
@@ -3501,14 +3501,14 @@ fig.select("#fig-0378e04b897742b597befd2e8e1c169e-element-20")
<div class="prompt input_prompt">In&nbsp;[8]:</div> <div class="prompt input_prompt">In&nbsp;[8]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-julia"><pre><span></span><span class="n">full_simulation</span> <span class="o">=</span> <span class="n">function</span><span class="p">(</span><span class="n">S0</span><span class="p">,</span> <span class="n">T</span><span class="p">,</span> <span class="n">n</span><span class="p">,</span> <span class="n">m</span><span class="p">,</span> <span class="n">term</span><span class="p">)</span> <div class=" highlight hl-julia"><pre><span class="n">full_simulation</span> <span class="o">=</span> <span class="n">function</span><span class="p">(</span><span class="n">S0</span><span class="p">,</span> <span class="n">T</span><span class="p">,</span> <span class="n">n</span><span class="p">,</span> <span class="n">m</span><span class="p">,</span> <span class="n">term</span><span class="p">)</span>
<span class="n">forward</span> <span class="o">=</span> <span class="n">vcat</span><span class="p">(</span><span class="n">term</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">forward_term</span><span class="p">(</span><span class="n">term</span><span class="p">))</span> <span class="n">forward</span> <span class="o">=</span> <span class="n">vcat</span><span class="p">(</span><span class="n">term</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">forward_term</span><span class="p">(</span><span class="n">term</span><span class="p">))</span>
<span class="c"># And an S0 to kick things off.</span> <span class="c"># And an S0 to kick things off.</span>
<span class="n">final_motion</span> <span class="o">=</span> <span class="n">ones</span><span class="p">(</span><span class="n">m</span><span class="p">)</span> <span class="o">*</span> <span class="n">S0</span> <span class="n">final_motion</span> <span class="o">=</span> <span class="n">ones</span><span class="p">(</span><span class="n">m</span><span class="p">)</span> <span class="o">*</span> <span class="n">S0</span>
<span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">T</span> <span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">T</span>
<span class="n">μ</span> <span class="o">=</span> <span class="p">(</span><span class="n">forward</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span> <span class="o">-</span> <span class="n">q</span><span class="p">)</span> <span class="err">μ</span> <span class="o">=</span> <span class="p">(</span><span class="n">forward</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span> <span class="o">-</span> <span class="n">q</span><span class="p">)</span>
<span class="n">year_motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">final_motion</span><span class="p">[:,</span><span class="k">end</span><span class="p">],</span> <span class="n">μ</span><span class="p">,</span> <span class="n">σ</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span> <span class="n">year_motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">final_motion</span><span class="p">[:,</span><span class="k">end</span><span class="p">],</span> <span class="err">μ</span><span class="p">,</span> <span class="err">σ</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
<span class="n">final_motion</span> <span class="o">=</span> <span class="n">hcat</span><span class="p">(</span><span class="n">final_motion</span><span class="p">,</span> <span class="n">year_motion</span><span class="p">)</span> <span class="n">final_motion</span> <span class="o">=</span> <span class="n">hcat</span><span class="p">(</span><span class="n">final_motion</span><span class="p">,</span> <span class="n">year_motion</span><span class="p">)</span>
<span class="k">end</span> <span class="k">end</span>
<span class="k">return</span> <span class="n">final_motion</span> <span class="k">return</span> <span class="n">final_motion</span>
@@ -3517,7 +3517,7 @@ fig.select("#fig-0378e04b897742b597befd2e8e1c169e-element-20")
<span class="n">tic</span><span class="p">()</span> <span class="n">tic</span><span class="p">()</span>
<span class="n">full_simulation</span><span class="p">(</span><span class="n">S0</span><span class="p">,</span> <span class="n">T</span><span class="p">,</span> <span class="n">n</span><span class="p">,</span> <span class="n">m</span><span class="p">,</span> <span class="n">term</span><span class="p">)</span> <span class="n">full_simulation</span><span class="p">(</span><span class="n">S0</span><span class="p">,</span> <span class="n">T</span><span class="p">,</span> <span class="n">n</span><span class="p">,</span> <span class="n">m</span><span class="p">,</span> <span class="n">term</span><span class="p">)</span>
<span class="n">time</span> <span class="o">=</span> <span class="n">toq</span><span class="p">()</span> <span class="n">time</span> <span class="o">=</span> <span class="n">toq</span><span class="p">()</span>
<span class="p">@</span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;Time to run simulation: </span><span class="si">%.2f</span><span class="s">s&quot;</span><span class="p">,</span> <span class="n">time</span><span class="p">)</span> <span class="p">@</span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;Time to run simulation: %.2fs&quot;</span><span class="p">,</span> <span class="n">time</span><span class="p">)</span>
</pre></div> </pre></div>
</div> </div>
@@ -3562,7 +3562,7 @@ fig.select("#fig-0378e04b897742b597befd2e8e1c169e-element-20")
<div class="prompt input_prompt">In&nbsp;[9]:</div> <div class="prompt input_prompt">In&nbsp;[9]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-julia"><pre><span></span><span class="n">call_barrier</span> <span class="o">=</span> <span class="n">S0</span> <div class=" highlight hl-julia"><pre><span class="n">call_barrier</span> <span class="o">=</span> <span class="n">S0</span>
<span class="n">strike</span> <span class="o">=</span> <span class="n">S0</span> <span class="n">strike</span> <span class="o">=</span> <span class="n">S0</span>
<span class="n">protection_barrier</span> <span class="o">=</span> <span class="n">S0</span> <span class="o">*</span> <span class="o">.</span><span class="mi">6</span> <span class="n">protection_barrier</span> <span class="o">=</span> <span class="n">S0</span> <span class="o">*</span> <span class="o">.</span><span class="mi">6</span>
<span class="n">coupon</span> <span class="o">=</span> <span class="n">nominal</span> <span class="o">*</span> <span class="o">.</span><span class="mi">07</span> <span class="n">coupon</span> <span class="o">=</span> <span class="n">nominal</span> <span class="o">*</span> <span class="o">.</span><span class="mi">07</span>
@@ -3576,7 +3576,7 @@ fig.select("#fig-0378e04b897742b597befd2e8e1c169e-element-20")
<span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">t</span> <span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">t</span>
<span class="n">price</span> <span class="o">=</span> <span class="n">year_prices</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="n">price</span> <span class="o">=</span> <span class="n">year_prices</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
<span class="k">if</span> <span class="n">price</span> <span class="n">≥</span> <span class="n">call_barrier</span> <span class="k">if</span> <span class="n">price</span> <span class="err">≥</span> <span class="n">call_barrier</span>
<span class="k">return</span> <span class="p">(</span><span class="n">nominal</span> <span class="o">+</span> <span class="n">coupon</span><span class="o">*</span><span class="n">i</span><span class="p">)</span> <span class="o">*</span> <span class="n">exp</span><span class="p">((</span><span class="n">prod</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">[</span><span class="n">i</span><span class="p">:</span><span class="k">end</span><span class="p">])</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">*</span><span class="p">(</span><span class="n">t</span><span class="o">-</span><span class="n">i</span><span class="p">))</span> <span class="k">return</span> <span class="p">(</span><span class="n">nominal</span> <span class="o">+</span> <span class="n">coupon</span><span class="o">*</span><span class="n">i</span><span class="p">)</span> <span class="o">*</span> <span class="n">exp</span><span class="p">((</span><span class="n">prod</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">[</span><span class="n">i</span><span class="p">:</span><span class="k">end</span><span class="p">])</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">*</span><span class="p">(</span><span class="n">t</span><span class="o">-</span><span class="n">i</span><span class="p">))</span>
<span class="k">end</span> <span class="k">end</span>
<span class="k">end</span> <span class="k">end</span>
@@ -3606,13 +3606,13 @@ fig.select("#fig-0378e04b897742b597befd2e8e1c169e-element-20")
<span class="n">tic</span><span class="p">()</span> <span class="n">tic</span><span class="p">()</span>
<span class="n">mean_payoffs</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="n">athena</span><span class="p">()</span> <span class="n">mean_payoffs</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="n">athena</span><span class="p">()</span>
<span class="n">time</span> <span class="o">=</span> <span class="n">toq</span><span class="p">()</span> <span class="n">time</span> <span class="o">=</span> <span class="n">toq</span><span class="p">()</span>
<span class="p">@</span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;Mean of simulation </span><span class="si">%i</span><span class="s">: \$</span><span class="si">%.4f</span><span class="s">; Simulation time: </span><span class="si">%.2f</span><span class="s">s</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">,</span> <span class="n">i</span><span class="p">,</span> <span class="n">mean_payoffs</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> <span class="n">time</span><span class="p">)</span> <span class="p">@</span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;Mean of simulation %i: \</span><span class="si">$</span><span class="s">%.4f; Simulation time: %.2fs</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">,</span> <span class="n">i</span><span class="p">,</span> <span class="n">mean_payoffs</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> <span class="n">time</span><span class="p">)</span>
<span class="k">end</span> <span class="k">end</span>
<span class="n">final_mean</span> <span class="o">=</span> <span class="n">mean</span><span class="p">(</span><span class="n">mean_payoffs</span><span class="p">)</span> <span class="n">final_mean</span> <span class="o">=</span> <span class="n">mean</span><span class="p">(</span><span class="n">mean_payoffs</span><span class="p">)</span>
<span class="n">println</span><span class="p">(</span><span class="s">&quot;Mean over </span><span class="si">$num_simulations</span><span class="s"> simulations: </span><span class="si">$(mean(mean_payoffs))</span><span class="s">&quot;</span><span class="p">)</span> <span class="n">println</span><span class="p">(</span><span class="s">&quot;Mean over </span><span class="si">$</span><span class="s">num_simulations simulations: </span><span class="si">$</span><span class="s">(mean(mean_payoffs))&quot;</span><span class="p">)</span>
<span class="n">pv</span> <span class="o">=</span> <span class="n">final_mean</span> <span class="o">*</span> <span class="p">(</span><span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="p">(</span><span class="n">prod</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">*</span><span class="n">T</span><span class="p">))</span> <span class="n">pv</span> <span class="o">=</span> <span class="n">final_mean</span> <span class="o">*</span> <span class="p">(</span><span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="p">(</span><span class="n">prod</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">*</span><span class="n">T</span><span class="p">))</span>
<span class="p">@</span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;Present value of Athena note: \$</span><span class="si">%.2f</span><span class="s">, notional: \$</span><span class="si">%.2f</span><span class="s">&quot;</span><span class="p">,</span> <span class="n">pv</span><span class="p">,</span> <span class="n">nominal</span><span class="p">)</span> <span class="p">@</span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;Present value of Athena note: \</span><span class="si">$</span><span class="s">%.2f, notional: \</span><span class="si">$</span><span class="s">%.2f&quot;</span><span class="p">,</span> <span class="n">pv</span><span class="p">,</span> <span class="n">nominal</span><span class="p">)</span>
</pre></div> </pre></div>
</div> </div>
@@ -3673,7 +3673,7 @@ Present value of Athena note: $95.00, notional: $100.00</pre>
<div class="prompt input_prompt">In&nbsp;[10]:</div> <div class="prompt input_prompt">In&nbsp;[10]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-julia"><pre><span></span><span class="n">call_barrier</span> <span class="o">=</span> <span class="n">S0</span> <div class=" highlight hl-julia"><pre><span class="n">call_barrier</span> <span class="o">=</span> <span class="n">S0</span>
<span class="n">coupon_barrier</span> <span class="o">=</span> <span class="n">S0</span> <span class="o">*</span> <span class="o">.</span><span class="mi">8</span> <span class="n">coupon_barrier</span> <span class="o">=</span> <span class="n">S0</span> <span class="o">*</span> <span class="o">.</span><span class="mi">8</span>
<span class="n">protection_barrier</span> <span class="o">=</span> <span class="n">S0</span> <span class="o">*</span> <span class="o">.</span><span class="mi">6</span> <span class="n">protection_barrier</span> <span class="o">=</span> <span class="n">S0</span> <span class="o">*</span> <span class="o">.</span><span class="mi">6</span>
<span class="n">coupon</span> <span class="o">=</span> <span class="n">nominal</span> <span class="o">*</span> <span class="o">.</span><span class="mi">06</span> <span class="n">coupon</span> <span class="o">=</span> <span class="n">nominal</span> <span class="o">*</span> <span class="o">.</span><span class="mi">06</span>
@@ -3686,9 +3686,9 @@ Present value of Athena note: $95.00, notional: $100.00</pre>
<span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">t</span> <span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">t</span>
<span class="n">price</span> <span class="o">=</span> <span class="n">year_prices</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="n">price</span> <span class="o">=</span> <span class="n">year_prices</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
<span class="k">if</span> <span class="n">price</span> <span class="n">≥</span> <span class="n">call_barrier</span> <span class="k">if</span> <span class="n">price</span> <span class="err">≥</span> <span class="n">call_barrier</span>
<span class="k">return</span> <span class="p">(</span><span class="n">nominal</span> <span class="o">+</span> <span class="n">coupon</span> <span class="o">+</span> <span class="n">total_coupons</span><span class="p">)</span><span class="o">*</span><span class="n">exp</span><span class="p">((</span><span class="n">prod</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">[</span><span class="n">i</span><span class="p">:</span><span class="k">end</span><span class="p">])</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">*</span><span class="p">(</span><span class="n">t</span><span class="o">-</span><span class="n">i</span><span class="p">))</span> <span class="k">return</span> <span class="p">(</span><span class="n">nominal</span> <span class="o">+</span> <span class="n">coupon</span> <span class="o">+</span> <span class="n">total_coupons</span><span class="p">)</span><span class="o">*</span><span class="n">exp</span><span class="p">((</span><span class="n">prod</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">[</span><span class="n">i</span><span class="p">:</span><span class="k">end</span><span class="p">])</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">*</span><span class="p">(</span><span class="n">t</span><span class="o">-</span><span class="n">i</span><span class="p">))</span>
<span class="k">elseif</span> <span class="n">price</span> <span class="n">≥</span> <span class="n">coupon_barrier</span> <span class="k">elseif</span> <span class="n">price</span> <span class="err">≥</span> <span class="n">coupon_barrier</span>
<span class="n">total_coupons</span> <span class="o">=</span> <span class="n">total_coupons</span> <span class="o">*</span> <span class="n">exp</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span> <span class="o">+</span> <span class="n">coupon</span> <span class="n">total_coupons</span> <span class="o">=</span> <span class="n">total_coupons</span> <span class="o">*</span> <span class="n">exp</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span> <span class="o">+</span> <span class="n">coupon</span>
<span class="k">else</span> <span class="k">else</span>
<span class="n">total_coupons</span> <span class="o">*=</span> <span class="n">exp</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span> <span class="n">total_coupons</span> <span class="o">*=</span> <span class="n">exp</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>
@@ -3720,13 +3720,13 @@ Present value of Athena note: $95.00, notional: $100.00</pre>
<span class="n">tic</span><span class="p">()</span> <span class="n">tic</span><span class="p">()</span>
<span class="n">mean_payoffs</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="n">phoenix_no_memory</span><span class="p">()</span> <span class="n">mean_payoffs</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="n">phoenix_no_memory</span><span class="p">()</span>
<span class="n">time</span> <span class="o">=</span> <span class="n">toq</span><span class="p">()</span> <span class="n">time</span> <span class="o">=</span> <span class="n">toq</span><span class="p">()</span>
<span class="p">@</span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;Mean of simulation </span><span class="si">%i</span><span class="s">: \$</span><span class="si">%.4f</span><span class="s">; Simulation time: </span><span class="si">%.2f</span><span class="s">s</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">,</span> <span class="n">i</span><span class="p">,</span> <span class="n">mean_payoffs</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> <span class="n">time</span><span class="p">)</span> <span class="p">@</span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;Mean of simulation %i: \</span><span class="si">$</span><span class="s">%.4f; Simulation time: %.2fs</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">,</span> <span class="n">i</span><span class="p">,</span> <span class="n">mean_payoffs</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> <span class="n">time</span><span class="p">)</span>
<span class="k">end</span> <span class="k">end</span>
<span class="n">final_mean</span> <span class="o">=</span> <span class="n">mean</span><span class="p">(</span><span class="n">mean_payoffs</span><span class="p">)</span> <span class="n">final_mean</span> <span class="o">=</span> <span class="n">mean</span><span class="p">(</span><span class="n">mean_payoffs</span><span class="p">)</span>
<span class="n">println</span><span class="p">(</span><span class="s">&quot;Mean over </span><span class="si">$num_simulations</span><span class="s"> simulations: </span><span class="si">$(mean(mean_payoffs))</span><span class="s">&quot;</span><span class="p">)</span> <span class="n">println</span><span class="p">(</span><span class="s">&quot;Mean over </span><span class="si">$</span><span class="s">num_simulations simulations: </span><span class="si">$</span><span class="s">(mean(mean_payoffs))&quot;</span><span class="p">)</span>
<span class="n">pv</span> <span class="o">=</span> <span class="n">final_mean</span> <span class="o">*</span> <span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="p">(</span><span class="n">prod</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">*</span><span class="p">(</span><span class="n">T</span><span class="p">))</span> <span class="n">pv</span> <span class="o">=</span> <span class="n">final_mean</span> <span class="o">*</span> <span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="p">(</span><span class="n">prod</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">*</span><span class="p">(</span><span class="n">T</span><span class="p">))</span>
<span class="p">@</span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;Present value of Phoenix without memory note: \$</span><span class="si">%.2f</span><span class="s">&quot;</span><span class="p">,</span> <span class="n">pv</span><span class="p">)</span> <span class="p">@</span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;Present value of Phoenix without memory note: \</span><span class="si">$</span><span class="s">%.2f&quot;</span><span class="p">,</span> <span class="n">pv</span><span class="p">)</span>
</pre></div> </pre></div>
</div> </div>
@@ -3773,7 +3773,7 @@ Present value of Phoenix without memory note: $97.44</pre>
<div class="prompt input_prompt">In&nbsp;[11]:</div> <div class="prompt input_prompt">In&nbsp;[11]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-julia"><pre><span></span><span class="n">call_barrier</span> <span class="o">=</span> <span class="n">S0</span> <div class=" highlight hl-julia"><pre><span class="n">call_barrier</span> <span class="o">=</span> <span class="n">S0</span>
<span class="n">coupon_barrier</span> <span class="o">=</span> <span class="n">S0</span> <span class="o">*</span> <span class="o">.</span><span class="mi">8</span> <span class="n">coupon_barrier</span> <span class="o">=</span> <span class="n">S0</span> <span class="o">*</span> <span class="o">.</span><span class="mi">8</span>
<span class="n">protection_barrier</span> <span class="o">=</span> <span class="n">S0</span> <span class="o">*</span> <span class="o">.</span><span class="mi">6</span> <span class="n">protection_barrier</span> <span class="o">=</span> <span class="n">S0</span> <span class="o">*</span> <span class="o">.</span><span class="mi">6</span>
<span class="n">coupon</span> <span class="o">=</span> <span class="n">nominal</span> <span class="o">*</span> <span class="o">.</span><span class="mi">07</span> <span class="n">coupon</span> <span class="o">=</span> <span class="n">nominal</span> <span class="o">*</span> <span class="o">.</span><span class="mi">07</span>
@@ -3827,14 +3827,14 @@ Present value of Phoenix without memory note: $97.44</pre>
<span class="n">tic</span><span class="p">()</span> <span class="n">tic</span><span class="p">()</span>
<span class="n">mean_payoffs</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="n">phoenix_with_memory</span><span class="p">()</span> <span class="n">mean_payoffs</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="n">phoenix_with_memory</span><span class="p">()</span>
<span class="n">time</span> <span class="o">=</span> <span class="n">toq</span><span class="p">()</span> <span class="n">time</span> <span class="o">=</span> <span class="n">toq</span><span class="p">()</span>
<span class="p">@</span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;Mean of simulation </span><span class="si">%i</span><span class="s">: \$</span><span class="si">%.4f</span><span class="s">; Simulation time: </span><span class="si">%.2f</span><span class="s">s</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">,</span> <span class="p">@</span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;Mean of simulation %i: \</span><span class="si">$</span><span class="s">%.4f; Simulation time: %.2fs</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">,</span>
<span class="n">i</span><span class="p">,</span> <span class="n">mean_payoffs</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> <span class="n">time</span><span class="p">)</span> <span class="n">i</span><span class="p">,</span> <span class="n">mean_payoffs</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> <span class="n">time</span><span class="p">)</span>
<span class="k">end</span> <span class="k">end</span>
<span class="n">final_mean</span> <span class="o">=</span> <span class="n">mean</span><span class="p">(</span><span class="n">mean_payoffs</span><span class="p">)</span> <span class="n">final_mean</span> <span class="o">=</span> <span class="n">mean</span><span class="p">(</span><span class="n">mean_payoffs</span><span class="p">)</span>
<span class="n">println</span><span class="p">(</span><span class="s">&quot;Mean over </span><span class="si">$num_simulations</span><span class="s"> simulations: </span><span class="si">$(mean(mean_payoffs))</span><span class="s">&quot;</span><span class="p">)</span> <span class="n">println</span><span class="p">(</span><span class="s">&quot;Mean over </span><span class="si">$</span><span class="s">num_simulations simulations: </span><span class="si">$</span><span class="s">(mean(mean_payoffs))&quot;</span><span class="p">)</span>
<span class="n">pv</span> <span class="o">=</span> <span class="n">final_mean</span> <span class="o">*</span> <span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="p">(</span><span class="n">prod</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">*</span><span class="p">(</span><span class="n">T</span><span class="p">))</span> <span class="n">pv</span> <span class="o">=</span> <span class="n">final_mean</span> <span class="o">*</span> <span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="p">(</span><span class="n">prod</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">*</span><span class="p">(</span><span class="n">T</span><span class="p">))</span>
<span class="p">@</span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;Present value of Phoenix with memory note: \$</span><span class="si">%.2f</span><span class="s">&quot;</span><span class="p">,</span> <span class="n">pv</span><span class="p">)</span> <span class="p">@</span><span class="n">printf</span><span class="p">(</span><span class="s">&quot;Present value of Phoenix with memory note: \</span><span class="si">$</span><span class="s">%.2f&quot;</span><span class="p">,</span> <span class="n">pv</span><span class="p">)</span>
</pre></div> </pre></div>
</div> </div>
categories.html
+2
View File
@@ -82,6 +82,8 @@
<div class="container content archive"> <div class="container content archive">
<h2><a href="https://bspeice.github.io/category/blog.html">Blog</a></h2> <h2><a href="https://bspeice.github.io/category/blog.html">Blog</a></h2>
<dl class="dl-horizontal"> <dl class="dl-horizontal">
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
<dt>Sun 15 May 2016</dt> <dt>Sun 15 May 2016</dt>
<dd><a href="https://bspeice.github.io/the-unfair-casino.html">The Unfair Casino</a></dd> <dd><a href="https://bspeice.github.io/the-unfair-casino.html">The Unfair Casino</a></dd>
<dt>Wed 06 April 2016</dt> <dt>Wed 06 April 2016</dt>
category/blog.html
+2
View File
@@ -83,6 +83,8 @@
<div class="container content archive"> <div class="container content archive">
<h2><a href="https://bspeice.github.io/category/blog.html">Blog</a></h2> <h2><a href="https://bspeice.github.io/category/blog.html">Blog</a></h2>
<dl class="dl-horizontal"> <dl class="dl-horizontal">
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
<dt>Sun 15 May 2016</dt> <dt>Sun 15 May 2016</dt>
<dd><a href="https://bspeice.github.io/the-unfair-casino.html">The Unfair Casino</a></dd> <dd><a href="https://bspeice.github.io/the-unfair-casino.html">The Unfair Casino</a></dd>
<dt>Wed 06 April 2016</dt> <dt>Wed 06 April 2016</dt>
category/blog2.html
+2
View File
@@ -83,6 +83,8 @@
<div class="container content archive"> <div class="container content archive">
<h2><a href="https://bspeice.github.io/category/blog.html">Blog</a></h2> <h2><a href="https://bspeice.github.io/category/blog.html">Blog</a></h2>
<dl class="dl-horizontal"> <dl class="dl-horizontal">
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
<dt>Sun 15 May 2016</dt> <dt>Sun 15 May 2016</dt>
<dd><a href="https://bspeice.github.io/the-unfair-casino.html">The Unfair Casino</a></dd> <dd><a href="https://bspeice.github.io/the-unfair-casino.html">The Unfair Casino</a></dd>
<dt>Wed 06 April 2016</dt> <dt>Wed 06 April 2016</dt>
cloudy-in-seattle.html
+29 -29
View File
@@ -4,7 +4,7 @@
<meta charset="utf-8"> <meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="In [1]: import pickle import pandas as pd import numpy as np from bokeh.plotting import output_notebook, figure, show from bokeh.palettes import RdBu4 as Palette from datetime import datetime ..."> <meta name="description" content="In [1]: import pickle import pandas as pd import numpy as np from bokeh.plotting import output_notebook, figure, show from bokeh.palettes import RdBu4 as Palette from datetime import datetime import ...">
<meta name="keywords" content="data science, weather"> <meta name="keywords" content="data science, weather">
<link rel="icon" href="https://bspeice.github.io/favicon.ico"> <link rel="icon" href="https://bspeice.github.io/favicon.ico">
@@ -92,7 +92,7 @@
<div class="prompt input_prompt">In&nbsp;[1]:</div> <div class="prompt input_prompt">In&nbsp;[1]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">pickle</span> <div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">pickle</span>
<span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span> <span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span> <span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">from</span> <span class="nn">bokeh.plotting</span> <span class="k">import</span> <span class="n">output_notebook</span><span class="p">,</span> <span class="n">figure</span><span class="p">,</span> <span class="n">show</span> <span class="kn">from</span> <span class="nn">bokeh.plotting</span> <span class="k">import</span> <span class="n">output_notebook</span><span class="p">,</span> <span class="n">figure</span><span class="p">,</span> <span class="n">show</span>
@@ -359,7 +359,7 @@
<div class="prompt input_prompt">In&nbsp;[2]:</div> <div class="prompt input_prompt">In&nbsp;[2]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">city_forecasts</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;city_forecasts.p&#39;</span><span class="p">,</span> <span class="s1">&#39;rb&#39;</span><span class="p">))</span> <div class=" highlight hl-ipython3"><pre><span class="n">city_forecasts</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="nb">open</span><span class="p">(</span><span class="s">&#39;city_forecasts.p&#39;</span><span class="p">,</span> <span class="s">&#39;rb&#39;</span><span class="p">))</span>
<span class="n">forecasts_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">city_forecasts</span><span class="p">)</span> <span class="n">forecasts_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">city_forecasts</span><span class="p">)</span>
</pre></div> </pre></div>
@@ -373,24 +373,24 @@
<div class="prompt input_prompt">In&nbsp;[3]:</div> <div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">cities</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;binghamton&#39;</span><span class="p">,</span> <span class="s1">&#39;cary&#39;</span><span class="p">,</span> <span class="s1">&#39;nyc&#39;</span><span class="p">,</span> <span class="s1">&#39;seattle&#39;</span><span class="p">]</span> <div class=" highlight hl-ipython3"><pre><span class="n">cities</span> <span class="o">=</span> <span class="p">[</span><span class="s">&#39;binghamton&#39;</span><span class="p">,</span> <span class="s">&#39;cary&#39;</span><span class="p">,</span> <span class="s">&#39;nyc&#39;</span><span class="p">,</span> <span class="s">&#39;seattle&#39;</span><span class="p">]</span>
<span class="n">city_colors</span> <span class="o">=</span> <span class="p">{</span><span class="n">cities</span><span class="p">[</span><span class="n">i</span><span class="p">]:</span> <span class="n">Palette</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">4</span><span class="p">)}</span> <span class="n">city_colors</span> <span class="o">=</span> <span class="p">{</span><span class="n">cities</span><span class="p">[</span><span class="n">i</span><span class="p">]:</span> <span class="n">Palette</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">4</span><span class="p">)}</span>
<span class="k">def</span> <span class="nf">safe_cover</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span> <span class="k">def</span> <span class="nf">safe_cover</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s1">&#39;cloudCover&#39;</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span> <span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s">&#39;cloudCover&#39;</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s1">&#39;cloudCover&#39;</span><span class="p">]</span> <span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s">&#39;cloudCover&#39;</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span> <span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span> <span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span>
<span class="k">def</span> <span class="nf">monthly_avg_cloudcover</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span> <span class="k">def</span> <span class="nf">monthly_avg_cloudcover</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span> <span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span> <span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">freq</span><span class="o">=</span><span class="s1">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">&#39;left&#39;</span><span class="p">)</span> <span class="n">freq</span><span class="o">=</span><span class="s">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">&#39;left&#39;</span><span class="p">)</span>
<span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">forecasts_df</span><span class="p">[</span><span class="n">city</span><span class="p">][</span><span class="n">x</span><span class="p">][</span><span class="s1">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span> <span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">forecasts_df</span><span class="p">[</span><span class="n">city</span><span class="p">][</span><span class="n">x</span><span class="p">][</span><span class="s">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">cloud_cover_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">cloud_cover_vals</span><span class="p">)))</span> <span class="n">cloud_cover_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">cloud_cover_vals</span><span class="p">)))</span>
<span class="c1"># Ignore an issue with nanmean having all NaN values. We&#39;ll discuss the data issues below.</span> <span class="c"># Ignore an issue with nanmean having all NaN values. We&#39;ll discuss the data issues below.</span>
<span class="k">with</span> <span class="n">warnings</span><span class="o">.</span><span class="n">catch_warnings</span><span class="p">():</span> <span class="k">with</span> <span class="n">warnings</span><span class="o">.</span><span class="n">catch_warnings</span><span class="p">():</span>
<span class="n">warnings</span><span class="o">.</span><span class="n">simplefilter</span><span class="p">(</span><span class="s1">&#39;ignore&#39;</span><span class="p">)</span> <span class="n">warnings</span><span class="o">.</span><span class="n">simplefilter</span><span class="p">(</span><span class="s">&#39;ignore&#39;</span><span class="p">)</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">nanmean</span><span class="p">(</span><span class="n">cloud_cover_vals</span><span class="p">),</span> <span class="n">cloud_cover_samples</span> <span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">nanmean</span><span class="p">(</span><span class="n">cloud_cover_vals</span><span class="p">),</span> <span class="n">cloud_cover_samples</span>
</pre></div> </pre></div>
@@ -404,23 +404,23 @@
<div class="prompt input_prompt">In&nbsp;[4]:</div> <div class="prompt input_prompt">In&nbsp;[4]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">years</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2016</span><span class="p">)</span> <div class=" highlight hl-ipython3"><pre><span class="n">years</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2016</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">city_avg_cc</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span> <span class="k">def</span> <span class="nf">city_avg_cc</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
<span class="k">return</span> <span class="p">[</span><span class="n">monthly_avg_cloudcover</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">month</span><span class="p">)</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">return</span> <span class="p">[</span><span class="n">monthly_avg_cloudcover</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">month</span><span class="p">)</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span>
<span class="n">months</span> <span class="o">=</span> <span class="p">[</span> <span class="n">months</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">(</span><span class="s1">&#39;July&#39;</span><span class="p">,</span> <span class="mi">7</span><span class="p">),</span> <span class="p">(</span><span class="s">&#39;July&#39;</span><span class="p">,</span> <span class="mi">7</span><span class="p">),</span>
<span class="p">(</span><span class="s1">&#39;August&#39;</span><span class="p">,</span> <span class="mi">8</span><span class="p">),</span> <span class="p">(</span><span class="s">&#39;August&#39;</span><span class="p">,</span> <span class="mi">8</span><span class="p">),</span>
<span class="p">(</span><span class="s1">&#39;September&#39;</span><span class="p">,</span> <span class="mi">9</span><span class="p">),</span> <span class="p">(</span><span class="s">&#39;September&#39;</span><span class="p">,</span> <span class="mi">9</span><span class="p">),</span>
<span class="p">(</span><span class="s1">&#39;October&#39;</span><span class="p">,</span> <span class="mi">10</span><span class="p">),</span> <span class="p">(</span><span class="s">&#39;October&#39;</span><span class="p">,</span> <span class="mi">10</span><span class="p">),</span>
<span class="p">(</span><span class="s1">&#39;November&#39;</span><span class="p">,</span> <span class="mi">11</span><span class="p">)</span> <span class="p">(</span><span class="s">&#39;November&#39;</span><span class="p">,</span> <span class="mi">11</span><span class="p">)</span>
<span class="p">]</span> <span class="p">]</span>
<span class="k">for</span> <span class="n">month</span><span class="p">,</span> <span class="n">month_id</span> <span class="ow">in</span> <span class="n">months</span><span class="p">:</span> <span class="k">for</span> <span class="n">month</span><span class="p">,</span> <span class="n">month_id</span> <span class="ow">in</span> <span class="n">months</span><span class="p">:</span>
<span class="n">month_averages</span> <span class="o">=</span> <span class="p">{</span><span class="n">city</span><span class="p">:</span> <span class="n">city_avg_cc</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">month_id</span><span class="p">)</span> <span class="k">for</span> <span class="n">city</span> <span class="ow">in</span> <span class="n">cities</span><span class="p">}</span> <span class="n">month_averages</span> <span class="o">=</span> <span class="p">{</span><span class="n">city</span><span class="p">:</span> <span class="n">city_avg_cc</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">month_id</span><span class="p">)</span> <span class="k">for</span> <span class="n">city</span> <span class="ow">in</span> <span class="n">cities</span><span class="p">}</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s2">&quot;</span><span class="si">{}</span><span class="s2"> Average Cloud Cover&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">month</span><span class="p">),</span> <span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">&quot;{} Average Cloud Cover&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">month</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">&#39;Year&#39;</span><span class="p">,</span> <span class="n">x_axis_label</span><span class="o">=</span><span class="s">&#39;Year&#39;</span><span class="p">,</span>
<span class="n">y_axis_label</span><span class="o">=</span><span class="s1">&#39;Cloud Cover Percentage&#39;</span><span class="p">)</span> <span class="n">y_axis_label</span><span class="o">=</span><span class="s">&#39;Cloud Cover Percentage&#39;</span><span class="p">)</span>
<span class="k">for</span> <span class="n">city</span> <span class="ow">in</span> <span class="n">cities</span><span class="p">:</span> <span class="k">for</span> <span class="n">city</span> <span class="ow">in</span> <span class="n">cities</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="p">[</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">month_averages</span><span class="p">[</span><span class="n">city</span><span class="p">]],</span> <span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="p">[</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">month_averages</span><span class="p">[</span><span class="n">city</span><span class="p">]],</span>
<span class="n">legend</span><span class="o">=</span><span class="n">city</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="n">city_colors</span><span class="p">[</span><span class="n">city</span><span class="p">])</span> <span class="n">legend</span><span class="o">=</span><span class="n">city</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="n">city_colors</span><span class="p">[</span><span class="n">city</span><span class="p">])</span>
@@ -609,21 +609,21 @@
<div class="prompt input_prompt">In&nbsp;[5]:</div> <div class="prompt input_prompt">In&nbsp;[5]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">safe_precip</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span> <div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">safe_precip</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s1">&#39;precipProbability&#39;</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span> <span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s">&#39;precipProbability&#39;</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s1">&#39;precipProbability&#39;</span><span class="p">]</span> <span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s">&#39;precipProbability&#39;</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span> <span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span> <span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span>
<span class="k">def</span> <span class="nf">monthly_avg_precip</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span> <span class="k">def</span> <span class="nf">monthly_avg_precip</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span> <span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span> <span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">freq</span><span class="o">=</span><span class="s1">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">&#39;left&#39;</span><span class="p">)</span> <span class="n">freq</span><span class="o">=</span><span class="s">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">&#39;left&#39;</span><span class="p">)</span>
<span class="n">precip_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">forecasts_df</span><span class="p">[</span><span class="n">city</span><span class="p">][</span><span class="n">x</span><span class="p">][</span><span class="s1">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span> <span class="n">precip_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">forecasts_df</span><span class="p">[</span><span class="n">city</span><span class="p">][</span><span class="n">x</span><span class="p">][</span><span class="s">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">precip_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">precip_vals</span><span class="p">)))</span> <span class="n">precip_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">precip_vals</span><span class="p">)))</span>
<span class="c1"># Ignore an issue with nanmean having all NaN values. We&#39;ll discuss the data issues below.</span> <span class="c"># Ignore an issue with nanmean having all NaN values. We&#39;ll discuss the data issues below.</span>
<span class="k">with</span> <span class="n">warnings</span><span class="o">.</span><span class="n">catch_warnings</span><span class="p">():</span> <span class="k">with</span> <span class="n">warnings</span><span class="o">.</span><span class="n">catch_warnings</span><span class="p">():</span>
<span class="n">warnings</span><span class="o">.</span><span class="n">simplefilter</span><span class="p">(</span><span class="s1">&#39;ignore&#39;</span><span class="p">)</span> <span class="n">warnings</span><span class="o">.</span><span class="n">simplefilter</span><span class="p">(</span><span class="s">&#39;ignore&#39;</span><span class="p">)</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">nanmean</span><span class="p">(</span><span class="n">precip_vals</span><span class="p">),</span> <span class="n">precip_samples</span> <span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">nanmean</span><span class="p">(</span><span class="n">precip_vals</span><span class="p">),</span> <span class="n">precip_samples</span>
<span class="k">def</span> <span class="nf">city_avg_precip</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span> <span class="k">def</span> <span class="nf">city_avg_precip</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
@@ -631,9 +631,9 @@
<span class="k">for</span> <span class="n">month</span><span class="p">,</span> <span class="n">month_id</span> <span class="ow">in</span> <span class="n">months</span><span class="p">:</span> <span class="k">for</span> <span class="n">month</span><span class="p">,</span> <span class="n">month_id</span> <span class="ow">in</span> <span class="n">months</span><span class="p">:</span>
<span class="n">month_averages</span> <span class="o">=</span> <span class="p">{</span><span class="n">city</span><span class="p">:</span> <span class="n">city_avg_cc</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">month_id</span><span class="p">)</span> <span class="k">for</span> <span class="n">city</span> <span class="ow">in</span> <span class="n">cities</span><span class="p">}</span> <span class="n">month_averages</span> <span class="o">=</span> <span class="p">{</span><span class="n">city</span><span class="p">:</span> <span class="n">city_avg_cc</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">month_id</span><span class="p">)</span> <span class="k">for</span> <span class="n">city</span> <span class="ow">in</span> <span class="n">cities</span><span class="p">}</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s2">&quot;</span><span class="si">{}</span><span class="s2"> Average Precipitation Chance&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">month</span><span class="p">),</span> <span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">&quot;{} Average Precipitation Chance&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">month</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">&#39;Year&#39;</span><span class="p">,</span> <span class="n">x_axis_label</span><span class="o">=</span><span class="s">&#39;Year&#39;</span><span class="p">,</span>
<span class="n">y_axis_label</span><span class="o">=</span><span class="s1">&#39;Precipitation Chance Percentage&#39;</span><span class="p">)</span> <span class="n">y_axis_label</span><span class="o">=</span><span class="s">&#39;Precipitation Chance Percentage&#39;</span><span class="p">)</span>
<span class="k">for</span> <span class="n">city</span> <span class="ow">in</span> <span class="n">cities</span><span class="p">:</span> <span class="k">for</span> <span class="n">city</span> <span class="ow">in</span> <span class="n">cities</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="p">[</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">month_averages</span><span class="p">[</span><span class="n">city</span><span class="p">]],</span> <span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="p">[</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">month_averages</span><span class="p">[</span><span class="n">city</span><span class="p">]],</span>
<span class="n">legend</span><span class="o">=</span><span class="n">city</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="n">city_colors</span><span class="p">[</span><span class="n">city</span><span class="p">])</span> <span class="n">legend</span><span class="o">=</span><span class="n">city</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="n">city_colors</span><span class="p">[</span><span class="n">city</span><span class="p">])</span>
complaining-about-the-weather.html
+58 -58
View File
@@ -92,7 +92,7 @@
<div class="prompt input_prompt">In&nbsp;[1]:</div> <div class="prompt input_prompt">In&nbsp;[1]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">bokeh.plotting</span> <span class="k">import</span> <span class="n">figure</span><span class="p">,</span> <span class="n">output_notebook</span><span class="p">,</span> <span class="n">show</span> <div class=" highlight hl-ipython3"><pre><span class="kn">from</span> <span class="nn">bokeh.plotting</span> <span class="k">import</span> <span class="n">figure</span><span class="p">,</span> <span class="n">output_notebook</span><span class="p">,</span> <span class="n">show</span>
<span class="kn">from</span> <span class="nn">bokeh.palettes</span> <span class="k">import</span> <span class="n">PuBuGn9</span> <span class="k">as</span> <span class="n">Palette</span> <span class="kn">from</span> <span class="nn">bokeh.palettes</span> <span class="k">import</span> <span class="n">PuBuGn9</span> <span class="k">as</span> <span class="n">Palette</span>
<span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span> <span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span> <span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
@@ -352,7 +352,7 @@
<div class="prompt input_prompt">In&nbsp;[2]:</div> <div class="prompt input_prompt">In&nbsp;[2]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">city_forecasts</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;city_forecasts.p&#39;</span><span class="p">,</span> <span class="s1">&#39;rb&#39;</span><span class="p">))</span> <div class=" highlight hl-ipython3"><pre><span class="n">city_forecasts</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="nb">open</span><span class="p">(</span><span class="s">&#39;city_forecasts.p&#39;</span><span class="p">,</span> <span class="s">&#39;rb&#39;</span><span class="p">))</span>
<span class="n">forecast_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">city_forecasts</span><span class="p">)</span> <span class="n">forecast_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">city_forecasts</span><span class="p">)</span>
</pre></div> </pre></div>
@@ -366,31 +366,31 @@
<div class="prompt input_prompt">In&nbsp;[3]:</div> <div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">cary_forecast</span> <span class="o">=</span> <span class="n">forecast_df</span><span class="p">[</span><span class="s1">&#39;cary&#39;</span><span class="p">]</span> <div class=" highlight hl-ipython3"><pre><span class="n">cary_forecast</span> <span class="o">=</span> <span class="n">forecast_df</span><span class="p">[</span><span class="s">&#39;cary&#39;</span><span class="p">]</span>
<span class="n">years</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2016</span><span class="p">)</span> <span class="n">years</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2016</span><span class="p">)</span>
<span class="n">months</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">7</span><span class="p">,</span> <span class="mi">12</span><span class="p">)</span> <span class="n">months</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">7</span><span class="p">,</span> <span class="mi">12</span><span class="p">)</span>
<span class="n">months_str</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;July&#39;</span><span class="p">,</span> <span class="s1">&#39;August&#39;</span><span class="p">,</span> <span class="s1">&#39;September&#39;</span><span class="p">,</span> <span class="s1">&#39;October&#39;</span><span class="p">,</span> <span class="s1">&#39;November&#39;</span><span class="p">]</span> <span class="n">months_str</span> <span class="o">=</span> <span class="p">[</span><span class="s">&#39;July&#39;</span><span class="p">,</span> <span class="s">&#39;August&#39;</span><span class="p">,</span> <span class="s">&#39;September&#39;</span><span class="p">,</span> <span class="s">&#39;October&#39;</span><span class="p">,</span> <span class="s">&#39;November&#39;</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">safe_cover</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span> <span class="k">def</span> <span class="nf">safe_cover</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s1">&#39;cloudCover&#39;</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span> <span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s">&#39;cloudCover&#39;</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s1">&#39;cloudCover&#39;</span><span class="p">]</span> <span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s">&#39;cloudCover&#39;</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span> <span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span> <span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span>
<span class="k">def</span> <span class="nf">monthly_avg_cloudcover</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span> <span class="k">def</span> <span class="nf">monthly_avg_cloudcover</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span> <span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span> <span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">freq</span><span class="o">=</span><span class="s1">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">&#39;left&#39;</span><span class="p">)</span> <span class="n">freq</span><span class="o">=</span><span class="s">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">&#39;left&#39;</span><span class="p">)</span>
<span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s1">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span> <span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">cloud_cover_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">cloud_cover_vals</span><span class="p">)))</span> <span class="n">cloud_cover_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">cloud_cover_vals</span><span class="p">)))</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">nanmean</span><span class="p">(</span><span class="n">cloud_cover_vals</span><span class="p">),</span> <span class="n">cloud_cover_samples</span> <span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">nanmean</span><span class="p">(</span><span class="n">cloud_cover_vals</span><span class="p">),</span> <span class="n">cloud_cover_samples</span>
<span class="n">monthly_cover_vals</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_avg_cloudcover</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span> <span class="n">monthly_cover_vals</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_avg_cloudcover</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">&#39;Monthly Average Cloud Cover&#39;</span><span class="p">,</span> <span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">&#39;Monthly Average Cloud Cover&#39;</span><span class="p">,</span>
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span> <span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">&#39;Year&#39;</span><span class="p">)</span> <span class="n">x_axis_label</span><span class="o">=</span><span class="s">&#39;Year&#39;</span><span class="p">)</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_cover_vals</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span> <span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_cover_vals</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span> <span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
@@ -454,11 +454,11 @@
<div class="prompt input_prompt">In&nbsp;[4]:</div> <div class="prompt input_prompt">In&nbsp;[4]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">monthly_cloudy_days</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span> <div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">monthly_cloudy_days</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span> <span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span> <span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">freq</span><span class="o">=</span><span class="s1">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">&#39;left&#39;</span><span class="p">)</span> <span class="n">freq</span><span class="o">=</span><span class="s">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">&#39;left&#39;</span><span class="p">)</span>
<span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s1">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span> <span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">cloud_cover_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">cloud_cover_vals</span><span class="p">)))</span> <span class="n">cloud_cover_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">cloud_cover_vals</span><span class="p">)))</span>
<span class="n">cloudy_days</span> <span class="o">=</span> <span class="p">[</span><span class="n">cover</span> <span class="o">&gt;</span> <span class="o">.</span><span class="mi">5</span> <span class="k">for</span> <span class="n">cover</span> <span class="ow">in</span> <span class="n">cloud_cover_vals</span><span class="p">]</span> <span class="n">cloudy_days</span> <span class="o">=</span> <span class="p">[</span><span class="n">cover</span> <span class="o">&gt;</span> <span class="o">.</span><span class="mi">5</span> <span class="k">for</span> <span class="n">cover</span> <span class="ow">in</span> <span class="n">cloud_cover_vals</span><span class="p">]</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">count_nonzero</span><span class="p">(</span><span class="n">cloudy_days</span><span class="p">),</span> <span class="n">cloud_cover_samples</span> <span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">count_nonzero</span><span class="p">(</span><span class="n">cloudy_days</span><span class="p">),</span> <span class="n">cloud_cover_samples</span>
@@ -466,16 +466,16 @@
<span class="n">monthly_days_vals</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_cloudy_days</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span> <span class="n">monthly_days_vals</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_cloudy_days</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
<span class="n">monthly_cover_samples</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_cloudy_days</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span> <span class="n">monthly_cover_samples</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_cloudy_days</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">&#39;Monthly Cloudy Days&#39;</span><span class="p">,</span> <span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">&#39;Monthly Cloudy Days&#39;</span><span class="p">,</span>
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span> <span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">&#39;Year&#39;</span><span class="p">)</span> <span class="n">x_axis_label</span><span class="o">=</span><span class="s">&#39;Year&#39;</span><span class="p">)</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_days_vals</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span> <span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_days_vals</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span> <span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">&#39;Monthly Cloud Cover Samples&#39;</span><span class="p">,</span> <span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">&#39;Monthly Cloud Cover Samples&#39;</span><span class="p">,</span>
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span> <span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">&#39;Year&#39;</span><span class="p">,</span> <span class="n">x_axis_label</span><span class="o">=</span><span class="s">&#39;Year&#39;</span><span class="p">,</span>
<span class="n">height</span><span class="o">=</span><span class="mi">300</span><span class="p">)</span> <span class="n">height</span><span class="o">=</span><span class="mi">300</span><span class="p">)</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_cover_samples</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span> <span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_cover_samples</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
@@ -570,25 +570,25 @@
<div class="prompt input_prompt">In&nbsp;[5]:</div> <div class="prompt input_prompt">In&nbsp;[5]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">safe_precip</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span> <div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">safe_precip</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s1">&#39;precipProbability&#39;</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span> <span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s">&#39;precipProbability&#39;</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s1">&#39;precipProbability&#39;</span><span class="p">]</span> <span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s">&#39;precipProbability&#39;</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span> <span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span> <span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span>
<span class="k">def</span> <span class="nf">monthly_avg_precip</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span> <span class="k">def</span> <span class="nf">monthly_avg_precip</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span> <span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span> <span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">freq</span><span class="o">=</span><span class="s1">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">&#39;left&#39;</span><span class="p">)</span> <span class="n">freq</span><span class="o">=</span><span class="s">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">&#39;left&#39;</span><span class="p">)</span>
<span class="n">precip_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s1">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span> <span class="n">precip_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">precip_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">precip_vals</span><span class="p">)))</span> <span class="n">precip_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">precip_vals</span><span class="p">)))</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">nanmean</span><span class="p">(</span><span class="n">precip_vals</span><span class="p">),</span> <span class="n">precip_samples</span> <span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">nanmean</span><span class="p">(</span><span class="n">precip_vals</span><span class="p">),</span> <span class="n">precip_samples</span>
<span class="n">monthly_avg_precip_vals</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_avg_precip</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span> <span class="n">monthly_avg_precip_vals</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_avg_precip</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">&#39;Monthly Average Precipitation Chance&#39;</span><span class="p">,</span> <span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">&#39;Monthly Average Precipitation Chance&#39;</span><span class="p">,</span>
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span> <span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">&#39;Year&#39;</span><span class="p">)</span> <span class="n">x_axis_label</span><span class="o">=</span><span class="s">&#39;Year&#39;</span><span class="p">)</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_avg_precip_vals</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span> <span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_avg_precip_vals</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span> <span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
@@ -651,11 +651,11 @@
<div class="prompt input_prompt">In&nbsp;[6]:</div> <div class="prompt input_prompt">In&nbsp;[6]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">monthly_rainy_days</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span> <div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">monthly_rainy_days</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span> <span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span> <span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">freq</span><span class="o">=</span><span class="s1">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">&#39;left&#39;</span><span class="p">)</span> <span class="n">freq</span><span class="o">=</span><span class="s">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">&#39;left&#39;</span><span class="p">)</span>
<span class="n">precip_prob_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s1">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span> <span class="n">precip_prob_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">precip_prob_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">precip_prob_vals</span><span class="p">)))</span> <span class="n">precip_prob_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">precip_prob_vals</span><span class="p">)))</span>
<span class="n">precip_days</span> <span class="o">=</span> <span class="p">[</span><span class="n">prob</span> <span class="o">&gt;</span> <span class="o">.</span><span class="mi">25</span> <span class="k">for</span> <span class="n">prob</span> <span class="ow">in</span> <span class="n">precip_prob_vals</span><span class="p">]</span> <span class="n">precip_days</span> <span class="o">=</span> <span class="p">[</span><span class="n">prob</span> <span class="o">&gt;</span> <span class="o">.</span><span class="mi">25</span> <span class="k">for</span> <span class="n">prob</span> <span class="ow">in</span> <span class="n">precip_prob_vals</span><span class="p">]</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">count_nonzero</span><span class="p">(</span><span class="n">precip_days</span><span class="p">),</span> <span class="n">precip_prob_samples</span> <span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">count_nonzero</span><span class="p">(</span><span class="n">precip_days</span><span class="p">),</span> <span class="n">precip_prob_samples</span>
@@ -663,16 +663,16 @@
<span class="n">monthly_precip_days_vals</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_rainy_days</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span> <span class="n">monthly_precip_days_vals</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_rainy_days</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
<span class="n">monthly_precip_samples</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_rainy_days</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span> <span class="n">monthly_precip_samples</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_rainy_days</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">&#39;Monthly Rainy Days&#39;</span><span class="p">,</span> <span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">&#39;Monthly Rainy Days&#39;</span><span class="p">,</span>
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span> <span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">&#39;Year&#39;</span><span class="p">)</span> <span class="n">x_axis_label</span><span class="o">=</span><span class="s">&#39;Year&#39;</span><span class="p">)</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_precip_days_vals</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span> <span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_precip_days_vals</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span> <span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">&#39;Monthly Rainy Days Samples&#39;</span><span class="p">,</span> <span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">&#39;Monthly Rainy Days Samples&#39;</span><span class="p">,</span>
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span> <span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">&#39;Year&#39;</span><span class="p">,</span> <span class="n">x_axis_label</span><span class="o">=</span><span class="s">&#39;Year&#39;</span><span class="p">,</span>
<span class="n">height</span><span class="o">=</span><span class="mi">300</span><span class="p">)</span> <span class="n">height</span><span class="o">=</span><span class="mi">300</span><span class="p">)</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_precip_samples</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span> <span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_precip_samples</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
@@ -770,69 +770,69 @@
<div class="inner_cell"> <div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html"> <div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Generating-the-Forecast-file">Generating the Forecast file<a class="anchor-link" href="#Generating-the-Forecast-file">&#182;</a></h1><p>The following code was generates the file that was used throughout the blog post. Please note that I'm retrieving data for other cities to use in a future blog post, only Cary data was used for this post.</p> <h1 id="Generating-the-Forecast-file">Generating the Forecast file<a class="anchor-link" href="#Generating-the-Forecast-file">&#182;</a></h1><p>The following code was generates the file that was used throughout the blog post. Please note that I'm retrieving data for other cities to use in a future blog post, only Cary data was used for this post.</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">pandas</span> <span class="kn">as</span> <span class="nn">pd</span> <div class="highlight"><pre><span class="kn">import</span> <span class="nn">pandas</span> <span class="kn">as</span> <span class="nn">pd</span>
<span class="kn">from</span> <span class="nn">functools</span> <span class="kn">import</span> <span class="nb">reduce</span> <span class="kn">from</span> <span class="nn">functools</span> <span class="kn">import</span> <span class="nb">reduce</span>
<span class="kn">import</span> <span class="nn">requests</span> <span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">from</span> <span class="nn">datetime</span> <span class="kn">import</span> <span class="n">datetime</span> <span class="kn">from</span> <span class="nn">datetime</span> <span class="kn">import</span> <span class="n">datetime</span>
<span class="c1"># Coordinate data from http://itouchmap.com/latlong.html</span> <span class="c"># Coordinate data from http://itouchmap.com/latlong.html</span>
<span class="n">cary_loc</span> <span class="o">=</span> <span class="p">(</span><span class="mf">35.79154</span><span class="p">,</span><span class="o">-</span><span class="mf">78.781117</span><span class="p">)</span> <span class="n">cary_loc</span> <span class="o">=</span> <span class="p">(</span><span class="mf">35.79154</span><span class="p">,</span><span class="o">-</span><span class="mf">78.781117</span><span class="p">)</span>
<span class="n">nyc_loc</span> <span class="o">=</span> <span class="p">(</span><span class="mf">40.78306</span><span class="p">,</span><span class="o">-</span><span class="mf">73.971249</span><span class="p">)</span> <span class="n">nyc_loc</span> <span class="o">=</span> <span class="p">(</span><span class="mf">40.78306</span><span class="p">,</span><span class="o">-</span><span class="mf">73.971249</span><span class="p">)</span>
<span class="n">seattle_loc</span> <span class="o">=</span> <span class="p">(</span><span class="mf">47.60621</span><span class="p">,</span><span class="o">-</span><span class="mf">122.332071</span><span class="p">)</span> <span class="n">seattle_loc</span> <span class="o">=</span> <span class="p">(</span><span class="mf">47.60621</span><span class="p">,</span><span class="o">-</span><span class="mf">122.332071</span><span class="p">)</span>
<span class="n">binghamton_loc</span> <span class="o">=</span> <span class="p">(</span><span class="mf">42.098687</span><span class="p">,</span><span class="o">-</span><span class="mf">75.917974</span><span class="p">)</span> <span class="n">binghamton_loc</span> <span class="o">=</span> <span class="p">(</span><span class="mf">42.098687</span><span class="p">,</span><span class="o">-</span><span class="mf">75.917974</span><span class="p">)</span>
<span class="n">cities</span> <span class="o">=</span> <span class="p">{</span> <span class="n">cities</span> <span class="o">=</span> <span class="p">{</span>
<span class="s1">&#39;cary&#39;</span><span class="p">:</span> <span class="n">cary_loc</span><span class="p">,</span> <span class="s">&#39;cary&#39;</span><span class="p">:</span> <span class="n">cary_loc</span><span class="p">,</span>
<span class="s1">&#39;nyc&#39;</span><span class="p">:</span> <span class="n">nyc_loc</span><span class="p">,</span> <span class="s">&#39;nyc&#39;</span><span class="p">:</span> <span class="n">nyc_loc</span><span class="p">,</span>
<span class="s1">&#39;seattle&#39;</span><span class="p">:</span> <span class="n">seattle_loc</span><span class="p">,</span> <span class="s">&#39;seattle&#39;</span><span class="p">:</span> <span class="n">seattle_loc</span><span class="p">,</span>
<span class="s1">&#39;binghamton&#39;</span><span class="p">:</span> <span class="n">binghamton_loc</span> <span class="s">&#39;binghamton&#39;</span><span class="p">:</span> <span class="n">binghamton_loc</span>
<span class="p">}</span> <span class="p">}</span>
<span class="n">apikey</span> <span class="o">=</span> <span class="s1">&#39;&#39;</span> <span class="c1"># My super-secret API Key</span> <span class="n">apikey</span> <span class="o">=</span> <span class="s">&#39;&#39;</span> <span class="c"># My super-secret API Key</span>
<span class="k">def</span> <span class="nf">get_forecast</span><span class="p">(</span><span class="n">lat</span><span class="p">,</span> <span class="nb">long</span><span class="p">,</span> <span class="n">date</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span> <span class="k">def</span> <span class="nf">get_forecast</span><span class="p">(</span><span class="n">lat</span><span class="p">,</span> <span class="nb">long</span><span class="p">,</span> <span class="n">date</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span>
<span class="n">forecast_base</span> <span class="o">=</span> <span class="s2">&quot;https://api.forecast.io/forecast/&quot;</span> <span class="n">forecast_base</span> <span class="o">=</span> <span class="s">&quot;https://api.forecast.io/forecast/&quot;</span>
<span class="k">if</span> <span class="n">date</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span> <span class="k">if</span> <span class="n">date</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
<span class="n">url</span> <span class="o">=</span> <span class="n">forecast_base</span> <span class="o">+</span> <span class="n">apikey</span> <span class="o">+</span> <span class="s1">&#39;/{},{}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">lat</span><span class="p">,</span> <span class="nb">long</span><span class="p">)</span> <span class="n">url</span> <span class="o">=</span> <span class="n">forecast_base</span> <span class="o">+</span> <span class="n">apikey</span> <span class="o">+</span> <span class="s">&#39;/{},{}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">lat</span><span class="p">,</span> <span class="nb">long</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span> <span class="k">else</span><span class="p">:</span>
<span class="n">epoch</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">date</span><span class="o">.</span><span class="n">timestamp</span><span class="p">())</span> <span class="n">epoch</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">date</span><span class="o">.</span><span class="n">timestamp</span><span class="p">())</span>
<span class="n">url</span> <span class="o">=</span> <span class="n">forecast_base</span> <span class="o">+</span> <span class="n">apikey</span> <span class="o">+</span> <span class="s1">&#39;/{},{},{}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">lat</span><span class="p">,</span> <span class="nb">long</span><span class="p">,</span> <span class="n">epoch</span><span class="p">)</span> <span class="n">url</span> <span class="o">=</span> <span class="n">forecast_base</span> <span class="o">+</span> <span class="n">apikey</span> <span class="o">+</span> <span class="s">&#39;/{},{},{}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">lat</span><span class="p">,</span> <span class="nb">long</span><span class="p">,</span> <span class="n">epoch</span><span class="p">)</span>
<span class="k">return</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span><span class="o">.</span><span class="n">json</span><span class="p">()</span> <span class="k">return</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span><span class="o">.</span><span class="n">json</span><span class="p">()</span>
<span class="n">years</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span><span class="mi">2016</span><span class="p">)</span> <span class="n">years</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span><span class="mi">2016</span><span class="p">)</span>
<span class="c1"># For datetimes, the 12 is for getting the weather at noon.</span> <span class="c"># For datetimes, the 12 is for getting the weather at noon.</span>
<span class="c1"># We&#39;re doing this over midnight because we&#39;re more concerned</span> <span class="c"># We&#39;re doing this over midnight because we&#39;re more concerned</span>
<span class="c1"># with what people see, and people don&#39;t typically see the weather</span> <span class="c"># with what people see, and people don&#39;t typically see the weather</span>
<span class="c1"># at midnight.</span> <span class="c"># at midnight.</span>
<span class="n">dt_indices</span> <span class="o">=</span> <span class="p">[</span><span class="n">pd</span><span class="o">.</span><span class="n">date_range</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span> <span class="n">dt_indices</span> <span class="o">=</span> <span class="p">[</span><span class="n">pd</span><span class="o">.</span><span class="n">date_range</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">30</span><span class="p">,</span> <span class="mi">12</span><span class="p">))</span> <span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">30</span><span class="p">,</span> <span class="mi">12</span><span class="p">))</span>
<span class="k">for</span> <span class="n">year</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">year</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span>
<span class="n">dt_merge</span> <span class="o">=</span> <span class="nb">reduce</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">union</span><span class="p">(</span><span class="n">y</span><span class="p">),</span> <span class="n">dt_indices</span><span class="p">)</span> <span class="n">dt_merge</span> <span class="o">=</span> <span class="nb">reduce</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">union</span><span class="p">(</span><span class="n">y</span><span class="p">),</span> <span class="n">dt_indices</span><span class="p">)</span>
<span class="c1"># Because we have to pay a little bit to use the API, we use for loops here</span> <span class="c"># Because we have to pay a little bit to use the API, we use for loops here</span>
<span class="c1"># instead of a comprehension - if something breaks, we want to preserve the</span> <span class="c"># instead of a comprehension - if something breaks, we want to preserve the</span>
<span class="c1"># data already retrieved</span> <span class="c"># data already retrieved</span>
<span class="n">city_forecasts</span> <span class="o">=</span> <span class="p">{}</span> <span class="n">city_forecasts</span> <span class="o">=</span> <span class="p">{}</span>
<span class="k">for</span> <span class="n">city</span><span class="p">,</span> <span class="n">loc</span> <span class="ow">in</span> <span class="n">cities</span><span class="o">.</span><span class="n">items</span><span class="p">():</span> <span class="k">for</span> <span class="n">city</span><span class="p">,</span> <span class="n">loc</span> <span class="ow">in</span> <span class="n">cities</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
<span class="k">print</span><span class="p">(</span><span class="s2">&quot;Retrieving data for {} starting at {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="k">print</span><span class="p">(</span><span class="s">&quot;Retrieving data for {} starting at {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">city</span><span class="p">,</span>
<span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s2">&quot;%I:%M:%S %p&quot;</span><span class="p">)))</span> <span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s">&quot;%I:%M:%S %p&quot;</span><span class="p">)))</span>
<span class="k">for</span> <span class="n">dt</span> <span class="ow">in</span> <span class="n">dt_merge</span><span class="p">:</span> <span class="k">for</span> <span class="n">dt</span> <span class="ow">in</span> <span class="n">dt_merge</span><span class="p">:</span>
<span class="k">try</span><span class="p">:</span> <span class="k">try</span><span class="p">:</span>
<span class="n">city_forecasts</span><span class="p">[(</span><span class="n">city</span><span class="p">,</span> <span class="n">dt</span><span class="p">)]</span> <span class="o">=</span> <span class="n">get_forecast</span><span class="p">(</span><span class="o">*</span><span class="n">loc</span><span class="p">,</span> <span class="n">dt</span><span class="p">)</span> <span class="n">city_forecasts</span><span class="p">[(</span><span class="n">city</span><span class="p">,</span> <span class="n">dt</span><span class="p">)]</span> <span class="o">=</span> <span class="n">get_forecast</span><span class="p">(</span><span class="o">*</span><span class="n">loc</span><span class="p">,</span> <span class="n">dt</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span> <span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
<span class="k">print</span><span class="p">(</span><span class="n">e</span><span class="p">)</span> <span class="k">print</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
<span class="n">city_forecasts</span><span class="p">[(</span><span class="n">city</span><span class="p">,</span> <span class="n">dt</span><span class="p">)]</span> <span class="o">=</span> <span class="bp">None</span> <span class="n">city_forecasts</span><span class="p">[(</span><span class="n">city</span><span class="p">,</span> <span class="n">dt</span><span class="p">)]</span> <span class="o">=</span> <span class="bp">None</span>
<span class="k">print</span><span class="p">(</span><span class="s2">&quot;End forecast retrieval: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s2">&quot;%I:%M:%S %p&quot;</span><span class="p">)))</span> <span class="k">print</span><span class="p">(</span><span class="s">&quot;End forecast retrieval: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s">&quot;%I:%M:%S %p&quot;</span><span class="p">)))</span>
<span class="kn">import</span> <span class="nn">pickle</span> <span class="kn">import</span> <span class="nn">pickle</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">city_forecasts</span><span class="p">,</span> <span class="nb">open</span><span class="p">(</span><span class="s1">&#39;city_forecasts.p&#39;</span><span class="p">,</span> <span class="s1">&#39;wb&#39;</span><span class="p">))</span> <span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">city_forecasts</span><span class="p">,</span> <span class="nb">open</span><span class="p">(</span><span class="s">&#39;city_forecasts.p&#39;</span><span class="p">,</span> <span class="s">&#39;wb&#39;</span><span class="p">))</span>
<span class="c1">### Output:</span> <span class="c">### Output:</span>
<span class="c1"># Retrieving data for binghamton starting at 05:13:42 PM</span> <span class="c"># Retrieving data for binghamton starting at 05:13:42 PM</span>
<span class="c1"># Retrieving data for seattle starting at 05:30:51 PM</span> <span class="c"># Retrieving data for seattle starting at 05:30:51 PM</span>
<span class="c1"># Retrieving data for nyc starting at 05:48:30 PM</span> <span class="c"># Retrieving data for nyc starting at 05:48:30 PM</span>
<span class="c1"># Retrieving data for cary starting at 06:08:32 PM</span> <span class="c"># Retrieving data for cary starting at 06:08:32 PM</span>
<span class="c1"># End forecast retrieval: 06:25:21 PM</span> <span class="c"># End forecast retrieval: 06:25:21 PM</span>
</pre></div> </pre></div>
</div> </div>
event-studies-and-earnings-releases.html
+5377
View File
File diff suppressed because it is too large Load Diff
feeds/all.atom.xml
+5656 -420
View File
File diff suppressed because it is too large Load Diff
feeds/blog.atom.xml
+5656 -420
View File
File diff suppressed because it is too large Load Diff
guaranteed-money-maker.html
+6 -6
View File
@@ -153,7 +153,7 @@ d_n &> \frac{o_n}{c_n - o_n} \sum_{i=1}^{n-1} d_i(1 - \frac{1}{o_i})
<div class="prompt input_prompt">In&nbsp;[1]:</div> <div class="prompt input_prompt">In&nbsp;[1]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-julia"><pre><span></span><span class="k">using</span> <span class="n">Quandl</span> <div class=" highlight hl-julia"><pre><span class="k">using</span> <span class="n">Quandl</span>
<span class="n">api_key</span> <span class="o">=</span> <span class="s">&quot;&quot;</span> <span class="n">api_key</span> <span class="o">=</span> <span class="s">&quot;&quot;</span>
<span class="n">daily_investment</span> <span class="o">=</span> <span class="n">function</span><span class="p">(</span><span class="n">current_open</span><span class="p">,</span> <span class="n">current_close</span><span class="p">,</span> <span class="n">purchase_history</span><span class="p">,</span> <span class="n">open_history</span><span class="p">)</span> <span class="n">daily_investment</span> <span class="o">=</span> <span class="n">function</span><span class="p">(</span><span class="n">current_open</span><span class="p">,</span> <span class="n">current_close</span><span class="p">,</span> <span class="n">purchase_history</span><span class="p">,</span> <span class="n">open_history</span><span class="p">)</span>
<span class="c"># We&#39;re not going to safeguard against divide by 0 - that&#39;s the user&#39;s responsibility</span> <span class="c"># We&#39;re not going to safeguard against divide by 0 - that&#39;s the user&#39;s responsibility</span>
@@ -183,7 +183,7 @@ d_n &> \frac{o_n}{c_n - o_n} \sum_{i=1}^{n-1} d_i(1 - \frac{1}{o_i})
<div class="prompt input_prompt">In&nbsp;[2]:</div> <div class="prompt input_prompt">In&nbsp;[2]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-julia"><pre><span></span><span class="n">is_profitable</span> <span class="o">=</span> <span class="n">function</span><span class="p">(</span><span class="n">current_price</span><span class="p">,</span> <span class="n">purchase_history</span><span class="p">,</span> <span class="n">open_history</span><span class="p">)</span> <div class=" highlight hl-julia"><pre><span class="n">is_profitable</span> <span class="o">=</span> <span class="n">function</span><span class="p">(</span><span class="n">current_price</span><span class="p">,</span> <span class="n">purchase_history</span><span class="p">,</span> <span class="n">open_history</span><span class="p">)</span>
<span class="n">shares</span> <span class="o">=</span> <span class="n">sum</span><span class="p">(</span><span class="n">purchase_history</span> <span class="o">./</span> <span class="n">open_history</span><span class="p">)</span> <span class="n">shares</span> <span class="o">=</span> <span class="n">sum</span><span class="p">(</span><span class="n">purchase_history</span> <span class="o">./</span> <span class="n">open_history</span><span class="p">)</span>
<span class="k">return</span> <span class="n">current_price</span><span class="o">*</span><span class="n">shares</span> <span class="o">&gt;</span> <span class="n">sum</span><span class="p">(</span><span class="n">purchase_history</span><span class="p">)</span> <span class="k">return</span> <span class="n">current_price</span><span class="o">*</span><span class="n">shares</span> <span class="o">&gt;</span> <span class="n">sum</span><span class="p">(</span><span class="n">purchase_history</span><span class="p">)</span>
<span class="k">end</span> <span class="k">end</span>
@@ -218,8 +218,8 @@ d_n &> \frac{o_n}{c_n - o_n} \sum_{i=1}^{n-1} d_i(1 - \frac{1}{o_i})
<span class="n">leverages</span> <span class="o">=</span> <span class="p">[</span><span class="n">sum</span><span class="p">(</span><span class="n">investments</span><span class="p">[</span><span class="mi">1</span><span class="p">:</span><span class="n">i</span><span class="p">])</span> <span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">length</span><span class="p">(</span><span class="n">investments</span><span class="p">)]</span> <span class="n">leverages</span> <span class="o">=</span> <span class="p">[</span><span class="n">sum</span><span class="p">(</span><span class="n">investments</span><span class="p">[</span><span class="mi">1</span><span class="p">:</span><span class="n">i</span><span class="p">])</span> <span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">length</span><span class="p">(</span><span class="n">investments</span><span class="p">)]</span>
<span class="n">max_leverage</span> <span class="o">=</span> <span class="n">maximum</span><span class="p">(</span><span class="n">leverages</span><span class="p">)</span> <span class="o">/</span> <span class="n">investments</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="n">max_leverage</span> <span class="o">=</span> <span class="n">maximum</span><span class="p">(</span><span class="n">leverages</span><span class="p">)</span> <span class="o">/</span> <span class="n">investments</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="n">println</span><span class="p">(</span><span class="s">&quot;Max leverage: </span><span class="si">$(max_leverage)</span><span class="s">&quot;</span><span class="p">)</span> <span class="n">println</span><span class="p">(</span><span class="s">&quot;Max leverage: </span><span class="si">$(max_leverage)</span><span class="s">&quot;</span><span class="p">)</span>
<span class="n">println</span><span class="p">(</span><span class="s">&quot;Days invested: </span><span class="si">$(length(investments))</span><span class="s">&quot;</span><span class="p">)</span> <span class="n">println</span><span class="p">(</span><span class="s">&quot;Days invested: </span><span class="si">$</span><span class="s">(length(investments))&quot;</span><span class="p">)</span>
<span class="n">println</span><span class="p">(</span><span class="s">&quot;Profit: </span><span class="si">$profit</span><span class="s">&quot;</span><span class="p">)</span> <span class="n">println</span><span class="p">(</span><span class="s">&quot;Profit: </span><span class="si">$</span><span class="s">profit&quot;</span><span class="p">)</span>
<span class="k">end</span><span class="p">;</span> <span class="k">end</span><span class="p">;</span>
</pre></div> </pre></div>
@@ -246,7 +246,7 @@ d_n &> \frac{o_n}{c_n - o_n} \sum_{i=1}^{n-1} d_i(1 - \frac{1}{o_i})
<div class="prompt input_prompt">In&nbsp;[3]:</div> <div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-julia"><pre><span></span><span class="n">investments</span><span class="p">,</span> <span class="n">profit</span> <span class="o">=</span> <span class="n">simulate</span><span class="p">(</span><span class="s">&quot;YAHOO/LMT&quot;</span><span class="p">,</span> <span class="n">Date</span><span class="p">(</span><span class="mi">2015</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">29</span><span class="p">),</span> <span class="mi">100</span><span class="p">,</span> <span class="mf">1.01</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span> <div class=" highlight hl-julia"><pre><span class="n">investments</span><span class="p">,</span> <span class="n">profit</span> <span class="o">=</span> <span class="n">simulate</span><span class="p">(</span><span class="s">&quot;YAHOO/LMT&quot;</span><span class="p">,</span> <span class="n">Date</span><span class="p">(</span><span class="mi">2015</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">29</span><span class="p">),</span> <span class="mi">100</span><span class="p">,</span> <span class="mf">1.01</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span>
<span class="n">sim_summary</span><span class="p">(</span><span class="n">investments</span><span class="p">,</span> <span class="n">profit</span><span class="p">)</span> <span class="n">sim_summary</span><span class="p">(</span><span class="n">investments</span><span class="p">,</span> <span class="n">profit</span><span class="p">)</span>
</pre></div> </pre></div>
@@ -289,7 +289,7 @@ Profit: 0.6894803101560001
<div class="prompt input_prompt">In&nbsp;[4]:</div> <div class="prompt input_prompt">In&nbsp;[4]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-julia"><pre><span></span><span class="n">investments</span><span class="p">,</span> <span class="n">profit</span> <span class="o">=</span> <span class="n">simulate</span><span class="p">(</span><span class="s">&quot;YAHOO/LMT&quot;</span><span class="p">,</span> <span class="n">Date</span><span class="p">(</span><span class="mi">2015</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">29</span><span class="p">),</span> <span class="mi">100</span><span class="p">,</span> <span class="mf">1.02</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span> <div class=" highlight hl-julia"><pre><span class="n">investments</span><span class="p">,</span> <span class="n">profit</span> <span class="o">=</span> <span class="n">simulate</span><span class="p">(</span><span class="s">&quot;YAHOO/LMT&quot;</span><span class="p">,</span> <span class="n">Date</span><span class="p">(</span><span class="mi">2015</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">29</span><span class="p">),</span> <span class="mi">100</span><span class="p">,</span> <span class="mf">1.02</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span>
<span class="n">sim_summary</span><span class="p">(</span><span class="n">investments</span><span class="p">,</span> <span class="n">profit</span><span class="p">)</span> <span class="n">sim_summary</span><span class="p">(</span><span class="n">investments</span><span class="p">,</span> <span class="n">profit</span><span class="p">)</span>
</pre></div> </pre></div>
index.html
+2 -2
View File
@@ -83,6 +83,8 @@
<div class="container content archive"> <div class="container content archive">
<h2><a href="https://bspeice.github.io/index.html"> </a></h2> <h2><a href="https://bspeice.github.io/index.html"> </a></h2>
<dl class="dl-horizontal"> <dl class="dl-horizontal">
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
<dt>Sun 15 May 2016</dt> <dt>Sun 15 May 2016</dt>
<dd><a href="https://bspeice.github.io/the-unfair-casino.html">The Unfair Casino</a></dd> <dd><a href="https://bspeice.github.io/the-unfair-casino.html">The Unfair Casino</a></dd>
<dt>Wed 06 April 2016</dt> <dt>Wed 06 April 2016</dt>
@@ -101,8 +103,6 @@
<dd><a href="https://bspeice.github.io/complaining-about-the-weather.html">Complaining about the Weather</a></dd> <dd><a href="https://bspeice.github.io/complaining-about-the-weather.html">Complaining about the Weather</a></dd>
<dt>Sat 26 December 2015</dt> <dt>Sat 26 December 2015</dt>
<dd><a href="https://bspeice.github.io/testing-cramer.html">Testing Cramer</a></dd> <dd><a href="https://bspeice.github.io/testing-cramer.html">Testing Cramer</a></dd>
<dt>Fri 27 November 2015</dt>
<dd><a href="https://bspeice.github.io/autocallable-bonds.html">Autocallable Bonds</a></dd>
</dl> </dl>
</div> </div>
</div> </div>
index2.html
+2
View File
@@ -83,6 +83,8 @@
<div class="container content archive"> <div class="container content archive">
<h2><a href="https://bspeice.github.io/index2.html"> <small>- page 2</small></a></h2> <h2><a href="https://bspeice.github.io/index2.html"> <small>- page 2</small></a></h2>
<dl class="dl-horizontal"> <dl class="dl-horizontal">
<dt>Fri 27 November 2015</dt>
<dd><a href="https://bspeice.github.io/autocallable-bonds.html">Autocallable Bonds</a></dd>
<dt>Thu 19 November 2015</dt> <dt>Thu 19 November 2015</dt>
<dd><a href="https://bspeice.github.io/welcome-and-an-algorithm.html">Welcome, and an algorithm</a></dd> <dd><a href="https://bspeice.github.io/welcome-and-an-algorithm.html">Welcome, and an algorithm</a></dd>
</dl> </dl>
predicting-santander-customer-happiness.html
+27 -27
View File
@@ -103,18 +103,18 @@
<div class="prompt input_prompt">In&nbsp;[1]:</div> <div class="prompt input_prompt">In&nbsp;[1]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span> <div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span> <span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span> <span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
<span class="o">%</span><span class="k">matplotlib</span> inline <span class="o">%</span><span class="k">matplotlib</span> inline
<span class="c1"># Record how long it takes to run the notebook - I&#39;m curious.</span> <span class="c"># Record how long it takes to run the notebook - I&#39;m curious.</span>
<span class="kn">from</span> <span class="nn">datetime</span> <span class="k">import</span> <span class="n">datetime</span> <span class="kn">from</span> <span class="nn">datetime</span> <span class="k">import</span> <span class="n">datetime</span>
<span class="n">start</span> <span class="o">=</span> <span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span> <span class="n">start</span> <span class="o">=</span> <span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span>
<span class="n">dataset</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s1">&#39;split_train.csv&#39;</span><span class="p">)</span> <span class="n">dataset</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s">&#39;split_train.csv&#39;</span><span class="p">)</span>
<span class="n">dataset</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">ID</span> <span class="n">dataset</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">ID</span>
<span class="n">X</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">drop</span><span class="p">([</span><span class="s1">&#39;TARGET&#39;</span><span class="p">,</span> <span class="s1">&#39;ID&#39;</span><span class="p">,</span> <span class="s1">&#39;ID.1&#39;</span><span class="p">],</span> <span class="mi">1</span><span class="p">)</span> <span class="n">X</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">drop</span><span class="p">([</span><span class="s">&#39;TARGET&#39;</span><span class="p">,</span> <span class="s">&#39;ID&#39;</span><span class="p">,</span> <span class="s">&#39;ID.1&#39;</span><span class="p">],</span> <span class="mi">1</span><span class="p">)</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">TARGET</span> <span class="n">y</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">TARGET</span>
</pre></div> </pre></div>
@@ -128,7 +128,7 @@
<div class="prompt input_prompt">In&nbsp;[2]:</div> <div class="prompt input_prompt">In&nbsp;[2]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">y</span><span class="o">.</span><span class="n">unique</span><span class="p">()</span> <div class=" highlight hl-ipython3"><pre><span class="n">y</span><span class="o">.</span><span class="n">unique</span><span class="p">()</span>
</pre></div> </pre></div>
</div> </div>
@@ -157,7 +157,7 @@
<div class="prompt input_prompt">In&nbsp;[3]:</div> <div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="nb">len</span><span class="p">(</span><span class="n">X</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span> <div class=" highlight hl-ipython3"><pre><span class="nb">len</span><span class="p">(</span><span class="n">X</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
</pre></div> </pre></div>
</div> </div>
@@ -199,7 +199,7 @@
<div class="prompt input_prompt">In&nbsp;[4]:</div> <div class="prompt input_prompt">In&nbsp;[4]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">cols</span> <span class="o">=</span> <span class="n">X</span><span class="o">.</span><span class="n">columns</span> <div class=" highlight hl-ipython3"><pre><span class="n">cols</span> <span class="o">=</span> <span class="n">X</span><span class="o">.</span><span class="n">columns</span>
<span class="n">b_class</span> <span class="o">=</span> <span class="p">[]</span> <span class="n">b_class</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">cols</span><span class="p">:</span> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">cols</span><span class="p">:</span>
<span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">X</span><span class="p">[</span><span class="n">c</span><span class="p">]</span><span class="o">.</span><span class="n">unique</span><span class="p">())</span> <span class="o">==</span> <span class="mi">2</span><span class="p">:</span> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">X</span><span class="p">[</span><span class="n">c</span><span class="p">]</span><span class="o">.</span><span class="n">unique</span><span class="p">())</span> <span class="o">==</span> <span class="mi">2</span><span class="p">:</span>
@@ -244,13 +244,13 @@
<div class="prompt input_prompt">In&nbsp;[5]:</div> <div class="prompt input_prompt">In&nbsp;[5]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># First we need to `binarize` the data to 0-1; some of the labels are {0, 1},</span> <div class=" highlight hl-ipython3"><pre><span class="c"># First we need to `binarize` the data to 0-1; some of the labels are {0, 1},</span>
<span class="c1"># some are {0, 3}, etc.</span> <span class="c"># some are {0, 3}, etc.</span>
<span class="kn">from</span> <span class="nn">sklearn.preprocessing</span> <span class="k">import</span> <span class="n">binarize</span> <span class="kn">from</span> <span class="nn">sklearn.preprocessing</span> <span class="k">import</span> <span class="n">binarize</span>
<span class="n">X_bin</span> <span class="o">=</span> <span class="n">binarize</span><span class="p">(</span><span class="n">X</span><span class="p">[</span><span class="n">b_class</span><span class="p">])</span> <span class="n">X_bin</span> <span class="o">=</span> <span class="n">binarize</span><span class="p">(</span><span class="n">X</span><span class="p">[</span><span class="n">b_class</span><span class="p">])</span>
<span class="n">accuracy</span> <span class="o">=</span> <span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">X_bin</span><span class="p">[:,</span><span class="n">i</span><span class="p">]</span> <span class="o">==</span> <span class="n">y</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">b_class</span><span class="p">))]</span> <span class="n">accuracy</span> <span class="o">=</span> <span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">X_bin</span><span class="p">[:,</span><span class="n">i</span><span class="p">]</span> <span class="o">==</span> <span class="n">y</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">b_class</span><span class="p">))]</span>
<span class="n">acc_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">({</span><span class="s2">&quot;Accuracy&quot;</span><span class="p">:</span> <span class="n">accuracy</span><span class="p">},</span> <span class="n">index</span><span class="o">=</span><span class="n">b_class</span><span class="p">)</span> <span class="n">acc_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">({</span><span class="s">&quot;Accuracy&quot;</span><span class="p">:</span> <span class="n">accuracy</span><span class="p">},</span> <span class="n">index</span><span class="o">=</span><span class="n">b_class</span><span class="p">)</span>
<span class="n">acc_df</span><span class="o">.</span><span class="n">describe</span><span class="p">()</span> <span class="n">acc_df</span><span class="o">.</span><span class="n">describe</span><span class="p">()</span>
</pre></div> </pre></div>
@@ -333,10 +333,10 @@
<div class="prompt input_prompt">In&nbsp;[6]:</div> <div class="prompt input_prompt">In&nbsp;[6]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">unsat</span> <span class="o">=</span> <span class="n">y</span><span class="p">[</span><span class="n">y</span> <span class="o">==</span> <span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">count</span><span class="p">()</span> <div class=" highlight hl-ipython3"><pre><span class="n">unsat</span> <span class="o">=</span> <span class="n">y</span><span class="p">[</span><span class="n">y</span> <span class="o">==</span> <span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">count</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Satisfied customers: </span><span class="si">{}</span><span class="s2">; Unsatisfied customers: </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">y</span><span class="p">)</span> <span class="o">-</span> <span class="n">unsat</span><span class="p">,</span> <span class="n">unsat</span><span class="p">))</span> <span class="nb">print</span><span class="p">(</span><span class="s">&quot;Satisfied customers: {}; Unsatisfied customers: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">y</span><span class="p">)</span> <span class="o">-</span> <span class="n">unsat</span><span class="p">,</span> <span class="n">unsat</span><span class="p">))</span>
<span class="n">naive_guess</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">y</span> <span class="o">==</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">y</span><span class="p">)))</span> <span class="n">naive_guess</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">y</span> <span class="o">==</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">y</span><span class="p">)))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Naive guess accuracy: </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">))</span> <span class="nb">print</span><span class="p">(</span><span class="s">&quot;Naive guess accuracy: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">))</span>
</pre></div> </pre></div>
</div> </div>
@@ -383,7 +383,7 @@ Naive guess accuracy: 0.9608561656706882
<div class="prompt input_prompt">In&nbsp;[7]:</div> <div class="prompt input_prompt">In&nbsp;[7]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.naive_bayes</span> <span class="k">import</span> <span class="n">GaussianNB</span> <div class=" highlight hl-ipython3"><pre><span class="kn">from</span> <span class="nn">sklearn.naive_bayes</span> <span class="k">import</span> <span class="n">GaussianNB</span>
<span class="kn">from</span> <span class="nn">sklearn.decomposition</span> <span class="k">import</span> <span class="n">PCA</span> <span class="kn">from</span> <span class="nn">sklearn.decomposition</span> <span class="k">import</span> <span class="n">PCA</span>
<span class="n">X_no_bin</span> <span class="o">=</span> <span class="n">X</span><span class="o">.</span><span class="n">drop</span><span class="p">(</span><span class="n">b_class</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span> <span class="n">X_no_bin</span> <span class="o">=</span> <span class="n">X</span><span class="o">.</span><span class="n">drop</span><span class="p">(</span><span class="n">b_class</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
@@ -397,9 +397,9 @@ Naive guess accuracy: 0.9608561656706882
<span class="k">return</span> <span class="n">gnb</span><span class="o">.</span><span class="n">score</span><span class="p">(</span><span class="n">X_xform</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span> <span class="k">return</span> <span class="n">gnb</span><span class="o">.</span><span class="n">score</span><span class="p">(</span><span class="n">X_xform</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
<span class="n">dim_range</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">21</span><span class="p">)</span> <span class="n">dim_range</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">21</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">dim_range</span><span class="p">,</span> <span class="p">[</span><span class="n">evaluate_gnb</span><span class="p">(</span><span class="n">dim</span><span class="p">)</span> <span class="k">for</span> <span class="n">dim</span> <span class="ow">in</span> <span class="n">dim_range</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;Gaussian NB Accuracy&quot;</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">dim_range</span><span class="p">,</span> <span class="p">[</span><span class="n">evaluate_gnb</span><span class="p">(</span><span class="n">dim</span><span class="p">)</span> <span class="k">for</span> <span class="n">dim</span> <span class="ow">in</span> <span class="n">dim_range</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s">&quot;Gaussian NB Accuracy&quot;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;Naive Guess&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s1">&#39;k&#39;</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&quot;Naive Guess&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s">&#39;k&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="mi">1</span> <span class="o">-</span> <span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;Inverse Naive Guess&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s1">&#39;k&#39;</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="mi">1</span> <span class="o">-</span> <span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&quot;Inverse Naive Guess&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s">&#39;k&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">();</span> <span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">();</span>
</pre></div> </pre></div>
@@ -674,7 +674,7 @@ rkJggg==
<div class="prompt input_prompt">In&nbsp;[8]:</div> <div class="prompt input_prompt">In&nbsp;[8]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">evaluate_gnb_full</span><span class="p">(</span><span class="n">dims</span><span class="p">):</span> <div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">evaluate_gnb_full</span><span class="p">(</span><span class="n">dims</span><span class="p">):</span>
<span class="n">pca</span> <span class="o">=</span> <span class="n">PCA</span><span class="p">(</span><span class="n">n_components</span><span class="o">=</span><span class="n">dims</span><span class="p">)</span> <span class="n">pca</span> <span class="o">=</span> <span class="n">PCA</span><span class="p">(</span><span class="n">n_components</span><span class="o">=</span><span class="n">dims</span><span class="p">)</span>
<span class="n">X_xform</span> <span class="o">=</span> <span class="n">pca</span><span class="o">.</span><span class="n">fit_transform</span><span class="p">(</span><span class="n">X</span><span class="p">)</span> <span class="n">X_xform</span> <span class="o">=</span> <span class="n">pca</span><span class="o">.</span><span class="n">fit_transform</span><span class="p">(</span><span class="n">X</span><span class="p">)</span>
@@ -683,9 +683,9 @@ rkJggg==
<span class="k">return</span> <span class="n">gnb</span><span class="o">.</span><span class="n">score</span><span class="p">(</span><span class="n">X_xform</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span> <span class="k">return</span> <span class="n">gnb</span><span class="o">.</span><span class="n">score</span><span class="p">(</span><span class="n">X_xform</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
<span class="n">dim_range</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">21</span><span class="p">)</span> <span class="n">dim_range</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">21</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">dim_range</span><span class="p">,</span> <span class="p">[</span><span class="n">evaluate_gnb</span><span class="p">(</span><span class="n">dim</span><span class="p">)</span> <span class="k">for</span> <span class="n">dim</span> <span class="ow">in</span> <span class="n">dim_range</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;Gaussian NB Accuracy&quot;</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">dim_range</span><span class="p">,</span> <span class="p">[</span><span class="n">evaluate_gnb</span><span class="p">(</span><span class="n">dim</span><span class="p">)</span> <span class="k">for</span> <span class="n">dim</span> <span class="ow">in</span> <span class="n">dim_range</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s">&quot;Gaussian NB Accuracy&quot;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;Naive Guess&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s1">&#39;k&#39;</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&quot;Naive Guess&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s">&#39;k&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="mi">1</span> <span class="o">-</span> <span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;Inverse Naive Guess&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s1">&#39;k&#39;</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="mi">1</span> <span class="o">-</span> <span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&quot;Inverse Naive Guess&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s">&#39;k&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">();</span> <span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">();</span>
</pre></div> </pre></div>
@@ -960,8 +960,8 @@ rkJggg==
<div class="prompt input_prompt">In&nbsp;[9]:</div> <div class="prompt input_prompt">In&nbsp;[9]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">end</span> <span class="o">=</span> <span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span> <div class=" highlight hl-ipython3"><pre><span class="n">end</span> <span class="o">=</span> <span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Running time: </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">end</span> <span class="o">-</span> <span class="n">start</span><span class="p">))</span> <span class="nb">print</span><span class="p">(</span><span class="s">&quot;Running time: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">end</span> <span class="o">-</span> <span class="n">start</span><span class="p">))</span>
</pre></div> </pre></div>
</div> </div>
@@ -989,15 +989,15 @@ rkJggg==
<div class="inner_cell"> <div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html"> <div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Appendix">Appendix<a class="anchor-link" href="#Appendix">&#182;</a></h1><p>Code used to split the initial training data:</p> <h1 id="Appendix">Appendix<a class="anchor-link" href="#Appendix">&#182;</a></h1><p>Code used to split the initial training data:</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.cross_validation</span> <span class="kn">import</span> <span class="n">train_test_split</span> <div class="highlight"><pre><span class="kn">from</span> <span class="nn">sklearn.cross_validation</span> <span class="kn">import</span> <span class="n">train_test_split</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s1">&#39;train.csv&#39;</span><span class="p">)</span> <span class="n">data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s">&#39;train.csv&#39;</span><span class="p">)</span>
<span class="n">data</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">ID</span> <span class="n">data</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">ID</span>
<span class="n">data_train</span><span class="p">,</span> <span class="n">data_validate</span> <span class="o">=</span> <span class="n">train_test_split</span><span class="p">(</span> <span class="n">data_train</span><span class="p">,</span> <span class="n">data_validate</span> <span class="o">=</span> <span class="n">train_test_split</span><span class="p">(</span>
<span class="n">data</span><span class="p">,</span> <span class="n">train_size</span><span class="o">=.</span><span class="mi">7</span><span class="p">)</span> <span class="n">data</span><span class="p">,</span> <span class="n">train_size</span><span class="o">=.</span><span class="mi">7</span><span class="p">)</span>
<span class="n">data_train</span><span class="o">.</span><span class="n">to_csv</span><span class="p">(</span><span class="s1">&#39;split_train.csv&#39;</span><span class="p">)</span> <span class="n">data_train</span><span class="o">.</span><span class="n">to_csv</span><span class="p">(</span><span class="s">&#39;split_train.csv&#39;</span><span class="p">)</span>
<span class="n">data_validate</span><span class="o">.</span><span class="n">to_csv</span><span class="p">(</span><span class="s1">&#39;split_validate.csv&#39;</span><span class="p">)</span> <span class="n">data_validate</span><span class="o">.</span><span class="n">to_csv</span><span class="p">(</span><span class="s">&#39;split_validate.csv&#39;</span><span class="p">)</span>
</pre></div> </pre></div>
</div> </div>
profitability-using-the-investment-formula.html
+30 -30
View File
@@ -118,12 +118,12 @@
<div class="prompt input_prompt">In&nbsp;[19]:</div> <div class="prompt input_prompt">In&nbsp;[19]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span> <div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span> <span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span> <span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
<span class="kn">from</span> <span class="nn">Quandl</span> <span class="k">import</span> <span class="n">get</span> <span class="k">as</span> <span class="n">qget</span> <span class="kn">from</span> <span class="nn">Quandl</span> <span class="k">import</span> <span class="n">get</span> <span class="k">as</span> <span class="n">qget</span>
<span class="o">%</span><span class="k">matplotlib</span> inline <span class="o">%</span><span class="k">matplotlib</span> inline
<span class="n">api_key</span> <span class="o">=</span> <span class="s1">&#39;&#39;</span> <span class="n">api_key</span> <span class="o">=</span> <span class="s">&#39;&#39;</span>
<span class="n">profitability</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">p</span><span class="p">,</span> <span class="n">i</span><span class="p">,</span> <span class="n">m</span><span class="p">,</span> <span class="n">d</span><span class="p">:</span> <span class="mi">1000</span><span class="o">*</span><span class="n">p</span> <span class="o">/</span> <span class="p">(</span><span class="n">m</span> <span class="o">+</span> <span class="n">i</span><span class="o">*</span><span class="n">d</span><span class="p">)</span> <span class="n">profitability</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">p</span><span class="p">,</span> <span class="n">i</span><span class="p">,</span> <span class="n">m</span><span class="p">,</span> <span class="n">d</span><span class="p">:</span> <span class="mi">1000</span><span class="o">*</span><span class="n">p</span> <span class="o">/</span> <span class="p">(</span><span class="n">m</span> <span class="o">+</span> <span class="n">i</span><span class="o">*</span><span class="n">d</span><span class="p">)</span>
@@ -147,15 +147,15 @@
<span class="n">expected_close</span> <span class="o">=</span> <span class="n">open_vals</span><span class="p">[</span><span class="n">day</span><span class="p">]</span> <span class="o">*</span> <span class="n">expected</span> <span class="n">expected_close</span> <span class="o">=</span> <span class="n">open_vals</span><span class="p">[</span><span class="n">day</span><span class="p">]</span> <span class="o">*</span> <span class="n">expected</span>
<span class="n">todays_purchase</span> <span class="o">=</span> <span class="n">daily_investment</span><span class="p">(</span><span class="n">open_vals</span><span class="p">[</span><span class="n">day</span><span class="p">],</span> <span class="n">expected_close</span><span class="p">,</span> <span class="n">invested</span><span class="p">,</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="n">day</span><span class="p">])</span> <span class="n">todays_purchase</span> <span class="o">=</span> <span class="n">daily_investment</span><span class="p">(</span><span class="n">open_vals</span><span class="p">[</span><span class="n">day</span><span class="p">],</span> <span class="n">expected_close</span><span class="p">,</span> <span class="n">invested</span><span class="p">,</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="n">day</span><span class="p">])</span>
<span class="n">invested</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">invested</span><span class="p">,</span> <span class="n">todays_purchase</span> <span class="o">+</span> <span class="n">bias</span><span class="p">)</span> <span class="n">invested</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">invested</span><span class="p">,</span> <span class="n">todays_purchase</span> <span class="o">+</span> <span class="n">bias</span><span class="p">)</span>
<span class="c1"># expected_profit = expected_close * (invested / open_vals[0:len(invested)]).sum() - invested.sum()</span> <span class="c"># expected_profit = expected_close * (invested / open_vals[0:len(invested)]).sum() - invested.sum()</span>
<span class="n">day</span> <span class="o">+=</span> <span class="mi">1</span> <span class="n">day</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="n">profitable</span> <span class="o">=</span> <span class="n">is_profitable</span><span class="p">(</span><span class="n">close_vals</span><span class="p">[</span><span class="n">day</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span> <span class="n">invested</span><span class="p">,</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="nb">len</span><span class="p">(</span><span class="n">invested</span><span class="p">)])</span> \ <span class="n">profitable</span> <span class="o">=</span> <span class="n">is_profitable</span><span class="p">(</span><span class="n">close_vals</span><span class="p">[</span><span class="n">day</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span> <span class="n">invested</span><span class="p">,</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="nb">len</span><span class="p">(</span><span class="n">invested</span><span class="p">)])</span> \
<span class="ow">or</span> <span class="n">is_profitable</span><span class="p">(</span><span class="n">open_vals</span><span class="p">[</span><span class="n">day</span><span class="p">],</span> <span class="n">invested</span><span class="p">,</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="nb">len</span><span class="p">(</span><span class="n">invested</span><span class="p">)])</span> <span class="ow">or</span> <span class="n">is_profitable</span><span class="p">(</span><span class="n">open_vals</span><span class="p">[</span><span class="n">day</span><span class="p">],</span> <span class="n">invested</span><span class="p">,</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="nb">len</span><span class="p">(</span><span class="n">invested</span><span class="p">)])</span>
<span class="n">shares</span> <span class="o">=</span> <span class="p">(</span><span class="n">invested</span> <span class="o">/</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="nb">len</span><span class="p">(</span><span class="n">invested</span><span class="p">)])</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span> <span class="n">shares</span> <span class="o">=</span> <span class="p">(</span><span class="n">invested</span> <span class="o">/</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="nb">len</span><span class="p">(</span><span class="n">invested</span><span class="p">)])</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
<span class="c1"># Make sure we can&#39;t see into the future - we know either today&#39;s close or tomorrow&#39;s open</span> <span class="c"># Make sure we can&#39;t see into the future - we know either today&#39;s close or tomorrow&#39;s open</span>
<span class="c1"># will be profitable, but we need to check which one.</span> <span class="c"># will be profitable, but we need to check which one.</span>
<span class="k">if</span> <span class="n">is_profitable</span><span class="p">(</span><span class="n">close_vals</span><span class="p">[</span><span class="n">day</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span> <span class="n">invested</span><span class="p">,</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="nb">len</span><span class="p">(</span><span class="n">invested</span><span class="p">)]):</span> <span class="k">if</span> <span class="n">is_profitable</span><span class="p">(</span><span class="n">close_vals</span><span class="p">[</span><span class="n">day</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span> <span class="n">invested</span><span class="p">,</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="nb">len</span><span class="p">(</span><span class="n">invested</span><span class="p">)]):</span>
<span class="n">ending_price</span> <span class="o">=</span> <span class="n">close_vals</span><span class="p">[</span><span class="n">day</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span> <span class="n">ending_price</span> <span class="o">=</span> <span class="n">close_vals</span><span class="p">[</span><span class="n">day</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span> <span class="k">else</span><span class="p">:</span>
@@ -168,21 +168,21 @@
<span class="n">ticker_info</span> <span class="o">=</span> <span class="n">qget</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start</span><span class="p">,</span> <span class="n">api_key</span><span class="o">=</span><span class="n">api_key</span><span class="p">)</span> <span class="n">ticker_info</span> <span class="o">=</span> <span class="n">qget</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start</span><span class="p">,</span> <span class="n">api_key</span><span class="o">=</span><span class="n">api_key</span><span class="p">)</span>
<span class="n">evaluation_times</span> <span class="o">=</span> <span class="n">ticker_info</span><span class="p">[:</span><span class="n">end</span><span class="p">]</span><span class="o">.</span><span class="n">index</span> <span class="n">evaluation_times</span> <span class="o">=</span> <span class="n">ticker_info</span><span class="p">[:</span><span class="n">end</span><span class="p">]</span><span class="o">.</span><span class="n">index</span>
<span class="c1"># Handle Google vs. YFinance data</span> <span class="c"># Handle Google vs. YFinance data</span>
<span class="k">if</span> <span class="s2">&quot;Adjusted Close&quot;</span> <span class="ow">in</span> <span class="n">ticker_info</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span> <span class="k">if</span> <span class="s">&quot;Adjusted Close&quot;</span> <span class="ow">in</span> <span class="n">ticker_info</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
<span class="n">close_column</span> <span class="o">=</span> <span class="s2">&quot;Adjusted Close&quot;</span> <span class="n">close_column</span> <span class="o">=</span> <span class="s">&quot;Adjusted Close&quot;</span>
<span class="k">else</span><span class="p">:</span> <span class="k">else</span><span class="p">:</span>
<span class="n">close_column</span> <span class="o">=</span> <span class="s2">&quot;Close&quot;</span> <span class="n">close_column</span> <span class="o">=</span> <span class="s">&quot;Close&quot;</span>
<span class="n">sim</span> <span class="o">=</span> <span class="p">{</span><span class="n">d</span><span class="p">:</span> <span class="n">simulate_day</span><span class="p">(</span><span class="n">ticker_info</span><span class="p">[</span><span class="n">d</span><span class="p">:][</span><span class="s2">&quot;Open&quot;</span><span class="p">],</span> <span class="n">ticker_info</span><span class="p">[</span><span class="n">d</span><span class="p">:][</span><span class="n">close_column</span><span class="p">],</span> <span class="n">sim</span> <span class="o">=</span> <span class="p">{</span><span class="n">d</span><span class="p">:</span> <span class="n">simulate_day</span><span class="p">(</span><span class="n">ticker_info</span><span class="p">[</span><span class="n">d</span><span class="p">:][</span><span class="s">&quot;Open&quot;</span><span class="p">],</span> <span class="n">ticker_info</span><span class="p">[</span><span class="n">d</span><span class="p">:][</span><span class="n">close_column</span><span class="p">],</span>
<span class="mi">100</span><span class="p">,</span> <span class="mf">1.02</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">evaluation_times</span><span class="p">}</span> <span class="mi">100</span><span class="p">,</span> <span class="mf">1.02</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">evaluation_times</span><span class="p">}</span>
<span class="n">sim_series</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">sim</span><span class="p">)</span> <span class="n">sim_series</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">sim</span><span class="p">)</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">()</span> <span class="n">result</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">()</span>
<span class="n">result</span><span class="p">[</span><span class="s2">&quot;profit&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">])</span> <span class="n">result</span><span class="p">[</span><span class="s">&quot;profit&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">])</span>
<span class="n">result</span><span class="p">[</span><span class="s2">&quot;max&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">max</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]))</span> <span class="n">result</span><span class="p">[</span><span class="s">&quot;max&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">max</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]))</span>
<span class="n">result</span><span class="p">[</span><span class="s2">&quot;days&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]))</span> <span class="n">result</span><span class="p">[</span><span class="s">&quot;days&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]))</span>
<span class="n">result</span><span class="p">[</span><span class="s2">&quot;score&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">profitability</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">0</span><span class="p">],</span> <span class="nb">max</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]),</span> <span class="nb">len</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">])))</span> <span class="n">result</span><span class="p">[</span><span class="s">&quot;score&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">profitability</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">0</span><span class="p">],</span> <span class="nb">max</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]),</span> <span class="nb">len</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">])))</span>
<span class="n">result</span><span class="p">[</span><span class="s2">&quot;investments&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="n">result</span><span class="p">[</span><span class="s">&quot;investments&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
<span class="k">return</span> <span class="n">result</span> <span class="k">return</span> <span class="n">result</span>
@@ -217,7 +217,7 @@
<div class="prompt input_prompt">In&nbsp;[7]:</div> <div class="prompt input_prompt">In&nbsp;[7]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">fang_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s2">&quot;YAHOO/FB&quot;</span><span class="p">,</span> <span class="s2">&quot;YAHOO/AAPL&quot;</span><span class="p">,</span> <span class="s2">&quot;YAHOO/NFLX&quot;</span><span class="p">,</span> <span class="s2">&quot;YAHOO/GOOG&quot;</span><span class="p">])</span> <div class=" highlight hl-ipython3"><pre><span class="n">fang_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s">&quot;YAHOO/FB&quot;</span><span class="p">,</span> <span class="s">&quot;YAHOO/AAPL&quot;</span><span class="p">,</span> <span class="s">&quot;YAHOO/NFLX&quot;</span><span class="p">,</span> <span class="s">&quot;YAHOO/GOOG&quot;</span><span class="p">])</span>
</pre></div> </pre></div>
</div> </div>
@@ -230,9 +230,9 @@
<div class="prompt input_prompt">In&nbsp;[8]:</div> <div class="prompt input_prompt">In&nbsp;[8]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">fang_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">&#39;days&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span> <div class=" highlight hl-ipython3"><pre><span class="n">fang_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">&#39;days&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">8</span><span class="p">);</span> <span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">8</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">&quot;Distribution of Days Until Profitability&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span> <span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">&quot;Distribution of Days Until Profitability&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
</pre></div> </pre></div>
</div> </div>
@@ -704,9 +704,9 @@ kiSpNScYJEmSJElSa04wSJIkSZKk1pxgkCRJkiRJrf1/uiw9D8O0d6EAAAAASUVORK5CYII=
<div class="prompt input_prompt">In&nbsp;[10]:</div> <div class="prompt input_prompt">In&nbsp;[10]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">fang_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">&#39;score&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span> <div class=" highlight hl-ipython3"><pre><span class="n">fang_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">&#39;score&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">&quot;Profitability score over time&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span> <span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">&quot;Profitability score over time&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
</pre></div> </pre></div>
</div> </div>
@@ -2343,7 +2343,7 @@ EZIkSZIkSZIkSZIk6RIpIiRJkiRJkiRJkiRJ0iX+D91jLOlExGw8AAAAAElFTkSuQmCC
<div class="prompt input_prompt">In&nbsp;[13]:</div> <div class="prompt input_prompt">In&nbsp;[13]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">cyclic_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s2">&quot;YAHOO/X&quot;</span><span class="p">,</span> <span class="s2">&quot;YAHOO/CAT&quot;</span><span class="p">,</span> <span class="s2">&quot;YAHOO/NFLX&quot;</span><span class="p">,</span> <span class="s2">&quot;YAHOO/GOOG&quot;</span><span class="p">])</span> <div class=" highlight hl-ipython3"><pre><span class="n">cyclic_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s">&quot;YAHOO/X&quot;</span><span class="p">,</span> <span class="s">&quot;YAHOO/CAT&quot;</span><span class="p">,</span> <span class="s">&quot;YAHOO/NFLX&quot;</span><span class="p">,</span> <span class="s">&quot;YAHOO/GOOG&quot;</span><span class="p">])</span>
</pre></div> </pre></div>
</div> </div>
@@ -2356,9 +2356,9 @@ EZIkSZIkSZIkSZIk6RIpIiRJkiRJkiRJkiRJ0iX+D91jLOlExGw8AAAAAElFTkSuQmCC
<div class="prompt input_prompt">In&nbsp;[14]:</div> <div class="prompt input_prompt">In&nbsp;[14]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">cyclic_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">&#39;days&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span> <div class=" highlight hl-ipython3"><pre><span class="n">cyclic_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">&#39;days&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">8</span><span class="p">);</span> <span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">8</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">&quot;Distribution of Days Until Profitability&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span> <span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">&quot;Distribution of Days Until Profitability&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
</pre></div> </pre></div>
</div> </div>
@@ -2844,9 +2844,9 @@ JEmSJEnSwGwwSJIkSZKkgdlgkCRJkiRJA7PBIEmSJEmSBvb/AfKAew4+oiojAAAAAElFTkSuQmCC
<div class="prompt input_prompt">In&nbsp;[15]:</div> <div class="prompt input_prompt">In&nbsp;[15]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">cyclic_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">&#39;score&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span> <div class=" highlight hl-ipython3"><pre><span class="n">cyclic_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">&#39;score&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">&quot;Profitability score over time&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span> <span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">&quot;Profitability score over time&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
</pre></div> </pre></div>
</div> </div>
@@ -4629,7 +4629,7 @@ hUKhUCgUpor/A9AnO0PoIItTAAAAAElFTkSuQmCC
<div class="prompt input_prompt">In&nbsp;[21]:</div> <div class="prompt input_prompt">In&nbsp;[21]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">biotech_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s1">&#39;YAHOO/REGN&#39;</span><span class="p">,</span> <span class="s1">&#39;YAHOO/CELG&#39;</span><span class="p">,</span> <span class="s1">&#39;GOOG/NASDAQ_BIB&#39;</span><span class="p">,</span> <span class="s1">&#39;GOOG/NASDAQ_IBB&#39;</span><span class="p">])</span> <div class=" highlight hl-ipython3"><pre><span class="n">biotech_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s">&#39;YAHOO/REGN&#39;</span><span class="p">,</span> <span class="s">&#39;YAHOO/CELG&#39;</span><span class="p">,</span> <span class="s">&#39;GOOG/NASDAQ_BIB&#39;</span><span class="p">,</span> <span class="s">&#39;GOOG/NASDAQ_IBB&#39;</span><span class="p">])</span>
</pre></div> </pre></div>
</div> </div>
@@ -4642,9 +4642,9 @@ hUKhUCgUpor/A9AnO0PoIItTAAAAAElFTkSuQmCC
<div class="prompt input_prompt">In&nbsp;[22]:</div> <div class="prompt input_prompt">In&nbsp;[22]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">biotech_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">&#39;days&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span> <div class=" highlight hl-ipython3"><pre><span class="n">biotech_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">&#39;days&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">8</span><span class="p">);</span> <span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">8</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">&quot;Distribution of Days Until Profitability&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span> <span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">&quot;Distribution of Days Until Profitability&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
</pre></div> </pre></div>
</div> </div>
@@ -5195,9 +5195,9 @@ WKBTQVcCAAAAAElFTkSuQmCC
<div class="prompt input_prompt">In&nbsp;[23]:</div> <div class="prompt input_prompt">In&nbsp;[23]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">biotech_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">&#39;score&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span> <div class=" highlight hl-ipython3"><pre><span class="n">biotech_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">&#39;score&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">&quot;Profitability score over time&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span> <span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">&quot;Profitability score over time&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
</pre></div> </pre></div>
</div> </div>
tag/earnings.html
+123
View File
@@ -0,0 +1,123 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content=" earnings">
<meta name="keywords" content="">
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
<title> earnings - Bradlee Speice</title>
<!-- Stylesheets -->
<link href="https://bspeice.github.io/theme/css/bootstrap.min.css" rel="stylesheet">
<link href="https://bspeice.github.io/theme/css/fonts.css" rel="stylesheet">
<link href="https://bspeice.github.io/theme/css/nest.css" rel="stylesheet">
<link href="https://bspeice.github.io/theme/css/pygment.css" rel="stylesheet">
<!-- /Stylesheets -->
<!-- RSS Feeds -->
<link href="https://bspeice.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate" title="Bradlee Speice Full Atom Feed" />
<!-- /RSS Feeds -->
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-74711362-1', 'auto');
ga('send', 'pageview');
</script>
<!-- /Google Analytics -->
</head>
<body>
<!-- Header -->
<div class="header-container gradient">
<!-- Static navbar -->
<div class="container">
<div class="header-nav">
<div class="header-logo">
<a class="pull-left" href="https://bspeice.github.io/"><img class="mr20" src="https://bspeice.github.io/images/logo.svg" alt="logo">Bradlee Speice</a>
</div>
<div class="nav pull-right">
</div>
</div>
</div>
<!-- /Static navbar -->
<!-- Header -->
<div class="container header-wrapper">
<div class="row">
<div class="col-lg-12">
<div class="header-content">
<h1 class="header-title text-uppercase"> : #earnings</h1>
<div class="header-underline"></div>
<p class="header-subtitle header-subtitle-homepage"> #earnings</p>
</div>
</div>
</div>
</div>
<!-- /Header -->
</div>
<!-- /Header -->
<!-- Content -->
<div class="archive-container">
<div class="container content archive">
<h2><a href="https://bspeice.github.io/tag/earnings.html">earnings</a></h2>
<dl class="dl-horizontal">
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
</dl>
</div>
</div>
<!-- /Content -->
<!-- Footer -->
<div class="footer gradient-2">
<div class="container footer-container ">
<div class="row">
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
<div class="footer-title"></div>
<ul class="list-unstyled">
<li><a href="https://bspeice.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate"></a></li>
</ul>
</div>
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
<div class="footer-title"></div>
<ul class="list-unstyled">
<li><a href="https://github.com/bspeice" target="_blank">Github</a></li>
<li><a href="https://www.linkedin.com/in/bradleespeice" target="_blank">LinkedIn</a></li>
</ul>
</div>
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
</div>
<div class="col-xs-12 col-sm-3 col-md-3 col-lg-3">
<p class="pull-right text-right">
<small><em>Proudly powered by <a href="http://docs.getpelican.com/" target="_blank">pelican</a></em></small><br/>
<small><em>Theme and code by <a href="https://github.com/molivier" target="_blank">molivier</a></em></small><br/>
<small></small>
</p>
</div>
</div>
</div>
</div>
<!-- /Footer -->
</body>
</html>
tag/event-study.html
+123
View File
@@ -0,0 +1,123 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content=" event study">
<meta name="keywords" content="">
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
<title> event study - Bradlee Speice</title>
<!-- Stylesheets -->
<link href="https://bspeice.github.io/theme/css/bootstrap.min.css" rel="stylesheet">
<link href="https://bspeice.github.io/theme/css/fonts.css" rel="stylesheet">
<link href="https://bspeice.github.io/theme/css/nest.css" rel="stylesheet">
<link href="https://bspeice.github.io/theme/css/pygment.css" rel="stylesheet">
<!-- /Stylesheets -->
<!-- RSS Feeds -->
<link href="https://bspeice.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate" title="Bradlee Speice Full Atom Feed" />
<!-- /RSS Feeds -->
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-74711362-1', 'auto');
ga('send', 'pageview');
</script>
<!-- /Google Analytics -->
</head>
<body>
<!-- Header -->
<div class="header-container gradient">
<!-- Static navbar -->
<div class="container">
<div class="header-nav">
<div class="header-logo">
<a class="pull-left" href="https://bspeice.github.io/"><img class="mr20" src="https://bspeice.github.io/images/logo.svg" alt="logo">Bradlee Speice</a>
</div>
<div class="nav pull-right">
</div>
</div>
</div>
<!-- /Static navbar -->
<!-- Header -->
<div class="container header-wrapper">
<div class="row">
<div class="col-lg-12">
<div class="header-content">
<h1 class="header-title text-uppercase"> : #event study</h1>
<div class="header-underline"></div>
<p class="header-subtitle header-subtitle-homepage"> #event study</p>
</div>
</div>
</div>
</div>
<!-- /Header -->
</div>
<!-- /Header -->
<!-- Content -->
<div class="archive-container">
<div class="container content archive">
<h2><a href="https://bspeice.github.io/tag/event-study.html">event study</a></h2>
<dl class="dl-horizontal">
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
</dl>
</div>
</div>
<!-- /Content -->
<!-- Footer -->
<div class="footer gradient-2">
<div class="container footer-container ">
<div class="row">
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
<div class="footer-title"></div>
<ul class="list-unstyled">
<li><a href="https://bspeice.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate"></a></li>
</ul>
</div>
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
<div class="footer-title"></div>
<ul class="list-unstyled">
<li><a href="https://github.com/bspeice" target="_blank">Github</a></li>
<li><a href="https://www.linkedin.com/in/bradleespeice" target="_blank">LinkedIn</a></li>
</ul>
</div>
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
</div>
<div class="col-xs-12 col-sm-3 col-md-3 col-lg-3">
<p class="pull-right text-right">
<small><em>Proudly powered by <a href="http://docs.getpelican.com/" target="_blank">pelican</a></em></small><br/>
<small><em>Theme and code by <a href="https://github.com/molivier" target="_blank">molivier</a></em></small><br/>
<small></small>
</p>
</div>
</div>
</div>
</div>
<!-- /Footer -->
</body>
</html>
tags.html
+4
View File
@@ -89,8 +89,12 @@
<dt><span class="label label-default">3</span> articles </dt> <dt><span class="label label-default">3</span> articles </dt>
<dd><a href="https://bspeice.github.io/tag/data-science.html">data science</a></dd> <dd><a href="https://bspeice.github.io/tag/data-science.html">data science</a></dd>
<dt><span class="label label-default">1</span> article </dt> <dt><span class="label label-default">1</span> article </dt>
<dd><a href="https://bspeice.github.io/tag/earnings.html">earnings</a></dd>
<dt><span class="label label-default">1</span> article </dt>
<dd><a href="https://bspeice.github.io/tag/em.html">em</a></dd> <dd><a href="https://bspeice.github.io/tag/em.html">em</a></dd>
<dt><span class="label label-default">1</span> article </dt> <dt><span class="label label-default">1</span> article </dt>
<dd><a href="https://bspeice.github.io/tag/event-study.html">event study</a></dd>
<dt><span class="label label-default">1</span> article </dt>
<dd><a href="https://bspeice.github.io/tag/finance.html">finance</a></dd> <dd><a href="https://bspeice.github.io/tag/finance.html">finance</a></dd>
<dt><span class="label label-default">1</span> article </dt> <dt><span class="label label-default">1</span> article </dt>
<dd><a href="https://bspeice.github.io/tag/fitbit.html">fitbit</a></dd> <dd><a href="https://bspeice.github.io/tag/fitbit.html">fitbit</a></dd>
testing-cramer.html
+72 -72
View File
@@ -4,7 +4,7 @@
<meta charset="utf-8"> <meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="In [1]: import requests import pandas as pd import numpy as np from dateutil import parser as dtparser from dateutil.relativedelta import relativedelta from datetime import datetime from ..."> <meta name="description" content="In [1]: import requests import pandas as pd import numpy as np from dateutil import parser as dtparser from dateutil.relativedelta import relativedelta from datetime import datetime from html.parser ...">
<meta name="keywords" content="data science, futures"> <meta name="keywords" content="data science, futures">
<link rel="icon" href="https://bspeice.github.io/favicon.ico"> <link rel="icon" href="https://bspeice.github.io/favicon.ico">
@@ -92,7 +92,7 @@
<div class="prompt input_prompt">In&nbsp;[1]:</div> <div class="prompt input_prompt">In&nbsp;[1]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">requests</span> <div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span> <span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span> <span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">from</span> <span class="nn">dateutil</span> <span class="k">import</span> <span class="n">parser</span> <span class="k">as</span> <span class="n">dtparser</span> <span class="kn">from</span> <span class="nn">dateutil</span> <span class="k">import</span> <span class="n">parser</span> <span class="k">as</span> <span class="n">dtparser</span>
@@ -127,26 +127,26 @@
<div class="prompt input_prompt">In&nbsp;[2]:</div> <div class="prompt input_prompt">In&nbsp;[2]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">ArticleListParser</span><span class="p">(</span><span class="n">HTMLParser</span><span class="p">):</span> <div class=" highlight hl-ipython3"><pre><span class="k">class</span> <span class="nc">ArticleListParser</span><span class="p">(</span><span class="n">HTMLParser</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;Given a web page with articles on it, parse out the article links&quot;&quot;&quot;</span> <span class="sd">&quot;&quot;&quot;Given a web page with articles on it, parse out the article links&quot;&quot;&quot;</span>
<span class="n">articles</span> <span class="o">=</span> <span class="p">[]</span> <span class="n">articles</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">def</span> <span class="nf">handle_starttag</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">,</span> <span class="n">attrs</span><span class="p">):</span> <span class="k">def</span> <span class="nf">handle_starttag</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">,</span> <span class="n">attrs</span><span class="p">):</span>
<span class="c1">#if tag == &#39;div&#39; and (&quot;id&quot;, &quot;author_articles_wrapper&quot;) in attrs:</span> <span class="c">#if tag == &#39;div&#39; and (&quot;id&quot;, &quot;author_articles_wrapper&quot;) in attrs:</span>
<span class="c1"># self.fetch_links = True</span> <span class="c"># self.fetch_links = True</span>
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s1">&#39;a&#39;</span> <span class="ow">and</span> <span class="p">(</span><span class="s1">&#39;class&#39;</span><span class="p">,</span> <span class="s1">&#39;dashboard_article_link&#39;</span><span class="p">)</span> <span class="ow">in</span> <span class="n">attrs</span><span class="p">:</span> <span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s">&#39;a&#39;</span> <span class="ow">and</span> <span class="p">(</span><span class="s">&#39;class&#39;</span><span class="p">,</span> <span class="s">&#39;dashboard_article_link&#39;</span><span class="p">)</span> <span class="ow">in</span> <span class="n">attrs</span><span class="p">:</span>
<span class="n">href</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s1">&#39;href&#39;</span><span class="p">,</span> <span class="n">attrs</span><span class="p">))[</span><span class="mi">0</span><span class="p">][</span><span class="mi">1</span><span class="p">]</span> <span class="n">href</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s">&#39;href&#39;</span><span class="p">,</span> <span class="n">attrs</span><span class="p">))[</span><span class="mi">0</span><span class="p">][</span><span class="mi">1</span><span class="p">]</span>
<span class="bp">self</span><span class="o">.</span><span class="n">articles</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">href</span><span class="p">)</span> <span class="bp">self</span><span class="o">.</span><span class="n">articles</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">href</span><span class="p">)</span>
<span class="n">base_url</span> <span class="o">=</span> <span class="s2">&quot;http://seekingalpha.com/author/wall-street-breakfast/articles&quot;</span> <span class="n">base_url</span> <span class="o">=</span> <span class="s">&quot;http://seekingalpha.com/author/wall-street-breakfast/articles&quot;</span>
<span class="n">article_page_urls</span> <span class="o">=</span> <span class="p">[</span><span class="n">base_url</span><span class="p">]</span> <span class="o">+</span> <span class="p">[</span><span class="n">base_url</span> <span class="o">+</span> <span class="s1">&#39;/</span><span class="si">{}</span><span class="s1">&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">20</span><span class="p">)]</span> <span class="n">article_page_urls</span> <span class="o">=</span> <span class="p">[</span><span class="n">base_url</span><span class="p">]</span> <span class="o">+</span> <span class="p">[</span><span class="n">base_url</span> <span class="o">+</span> <span class="s">&#39;/{}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">20</span><span class="p">)]</span>
<span class="n">global_articles</span> <span class="o">=</span> <span class="p">[]</span> <span class="n">global_articles</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">page</span> <span class="ow">in</span> <span class="n">article_page_urls</span><span class="p">:</span> <span class="k">for</span> <span class="n">page</span> <span class="ow">in</span> <span class="n">article_page_urls</span><span class="p">:</span>
<span class="c1"># We need to switch the user agent, as SA blocks the standard requests agent</span> <span class="c"># We need to switch the user agent, as SA blocks the standard requests agent</span>
<span class="n">articles_html</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">page</span><span class="p">,</span> <span class="n">articles_html</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">page</span><span class="p">,</span>
<span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="s2">&quot;User-Agent&quot;</span><span class="p">:</span> <span class="s2">&quot;Wget/1.13.4&quot;</span><span class="p">})</span> <span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="s">&quot;User-Agent&quot;</span><span class="p">:</span> <span class="s">&quot;Wget/1.13.4&quot;</span><span class="p">})</span>
<span class="n">parser</span> <span class="o">=</span> <span class="n">ArticleListParser</span><span class="p">()</span> <span class="n">parser</span> <span class="o">=</span> <span class="n">ArticleListParser</span><span class="p">()</span>
<span class="n">parser</span><span class="o">.</span><span class="n">feed</span><span class="p">(</span><span class="n">articles_html</span><span class="o">.</span><span class="n">text</span><span class="p">)</span> <span class="n">parser</span><span class="o">.</span><span class="n">feed</span><span class="p">(</span><span class="n">articles_html</span><span class="o">.</span><span class="n">text</span><span class="p">)</span>
<span class="n">global_articles</span> <span class="o">+=</span> <span class="p">(</span><span class="n">parser</span><span class="o">.</span><span class="n">articles</span><span class="p">)</span> <span class="n">global_articles</span> <span class="o">+=</span> <span class="p">(</span><span class="n">parser</span><span class="o">.</span><span class="n">articles</span><span class="p">)</span>
@@ -162,25 +162,25 @@
<div class="prompt input_prompt">In&nbsp;[3]:</div> <div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">ArticleReturnParser</span><span class="p">(</span><span class="n">HTMLParser</span><span class="p">):</span> <div class=" highlight hl-ipython3"><pre><span class="k">class</span> <span class="nc">ArticleReturnParser</span><span class="p">(</span><span class="n">HTMLParser</span><span class="p">):</span>
<span class="s2">&quot;Given an article, parse out the futures returns in it&quot;</span> <span class="s">&quot;Given an article, parse out the futures returns in it&quot;</span>
<span class="n">record_font_tags</span> <span class="o">=</span> <span class="kc">False</span> <span class="n">record_font_tags</span> <span class="o">=</span> <span class="k">False</span>
<span class="n">in_font_tag</span> <span class="o">=</span> <span class="kc">False</span> <span class="n">in_font_tag</span> <span class="o">=</span> <span class="k">False</span>
<span class="n">counter</span> <span class="o">=</span> <span class="mi">0</span> <span class="n">counter</span> <span class="o">=</span> <span class="mi">0</span>
<span class="c1"># data = {} # See __init__</span> <span class="c"># data = {} # See __init__</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">__init__</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">__init__</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span> <span class="o">=</span> <span class="p">{}</span> <span class="bp">self</span><span class="o">.</span><span class="n">data</span> <span class="o">=</span> <span class="p">{}</span>
<span class="k">def</span> <span class="nf">handle_starttag</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">,</span> <span class="n">attrs</span><span class="p">):</span> <span class="k">def</span> <span class="nf">handle_starttag</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">,</span> <span class="n">attrs</span><span class="p">):</span>
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s1">&#39;span&#39;</span> <span class="ow">and</span> <span class="p">(</span><span class="s1">&#39;itemprop&#39;</span><span class="p">,</span> <span class="s1">&#39;datePublished&#39;</span><span class="p">)</span> <span class="ow">in</span> <span class="n">attrs</span><span class="p">:</span> <span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s">&#39;span&#39;</span> <span class="ow">and</span> <span class="p">(</span><span class="s">&#39;itemprop&#39;</span><span class="p">,</span> <span class="s">&#39;datePublished&#39;</span><span class="p">)</span> <span class="ow">in</span> <span class="n">attrs</span><span class="p">:</span>
<span class="n">date_string</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s1">&#39;content&#39;</span><span class="p">,</span> <span class="n">attrs</span><span class="p">))[</span><span class="mi">0</span><span class="p">][</span><span class="mi">1</span><span class="p">]</span> <span class="n">date_string</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s">&#39;content&#39;</span><span class="p">,</span> <span class="n">attrs</span><span class="p">))[</span><span class="mi">0</span><span class="p">][</span><span class="mi">1</span><span class="p">]</span>
<span class="n">date</span> <span class="o">=</span> <span class="n">dtparser</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="n">date_string</span><span class="p">)</span> <span class="n">date</span> <span class="o">=</span> <span class="n">dtparser</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="n">date_string</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">&#39;date&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">date</span> <span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">&#39;date&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">date</span>
<span class="bp">self</span><span class="o">.</span><span class="n">in_font_tag</span> <span class="o">=</span> <span class="n">tag</span> <span class="o">==</span> <span class="s1">&#39;font&#39;</span> <span class="bp">self</span><span class="o">.</span><span class="n">in_font_tag</span> <span class="o">=</span> <span class="n">tag</span> <span class="o">==</span> <span class="s">&#39;font&#39;</span>
<span class="k">def</span> <span class="nf">safe_float</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">string</span><span class="p">):</span> <span class="k">def</span> <span class="nf">safe_float</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">string</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span> <span class="k">try</span><span class="p">:</span>
@@ -189,43 +189,43 @@
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span> <span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span>
<span class="k">def</span> <span class="nf">handle_data</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">content</span><span class="p">):</span> <span class="k">def</span> <span class="nf">handle_data</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">content</span><span class="p">):</span>
<span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">record_font_tags</span> <span class="ow">and</span> <span class="s2">&quot;Futures at 6&quot;</span> <span class="ow">in</span> <span class="n">content</span><span class="p">:</span> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">record_font_tags</span> <span class="ow">and</span> <span class="s">&quot;Futures at 6&quot;</span> <span class="ow">in</span> <span class="n">content</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">record_font_tags</span> <span class="o">=</span> <span class="kc">True</span> <span class="bp">self</span><span class="o">.</span><span class="n">record_font_tags</span> <span class="o">=</span> <span class="k">True</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">record_font_tags</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">in_font_tag</span><span class="p">:</span> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">record_font_tags</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">in_font_tag</span><span class="p">:</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">&#39;DOW&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span> <span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">&#39;DOW&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span> <span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">&#39;S&amp;P&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span> <span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">&#39;S&amp;P&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">2</span><span class="p">:</span> <span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">2</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">&#39;NASDAQ&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span> <span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">&#39;NASDAQ&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">3</span><span class="p">:</span> <span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">3</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">&#39;Crude&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span> <span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">&#39;Crude&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">4</span><span class="p">:</span> <span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">4</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">&#39;Gold&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span> <span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">&#39;Gold&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">+=</span> <span class="mi">1</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">def</span> <span class="nf">handle_endtag</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">):</span> <span class="k">def</span> <span class="nf">handle_endtag</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">in_font_tag</span> <span class="o">=</span> <span class="kc">False</span> <span class="bp">self</span><span class="o">.</span><span class="n">in_font_tag</span> <span class="o">=</span> <span class="k">False</span>
<span class="k">def</span> <span class="nf">retrieve_data</span><span class="p">(</span><span class="n">url</span><span class="p">):</span> <span class="k">def</span> <span class="nf">retrieve_data</span><span class="p">(</span><span class="n">url</span><span class="p">):</span>
<span class="n">sa</span> <span class="o">=</span> <span class="s2">&quot;http://seekingalpha.com&quot;</span> <span class="n">sa</span> <span class="o">=</span> <span class="s">&quot;http://seekingalpha.com&quot;</span>
<span class="n">article_html</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">sa</span> <span class="o">+</span> <span class="n">url</span><span class="p">,</span> <span class="n">article_html</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">sa</span> <span class="o">+</span> <span class="n">url</span><span class="p">,</span>
<span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="s2">&quot;User-Agent&quot;</span><span class="p">:</span> <span class="s2">&quot;Wget/1.13.4&quot;</span><span class="p">})</span> <span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="s">&quot;User-Agent&quot;</span><span class="p">:</span> <span class="s">&quot;Wget/1.13.4&quot;</span><span class="p">})</span>
<span class="n">parser</span> <span class="o">=</span> <span class="n">ArticleReturnParser</span><span class="p">()</span> <span class="n">parser</span> <span class="o">=</span> <span class="n">ArticleReturnParser</span><span class="p">()</span>
<span class="n">parser</span><span class="o">.</span><span class="n">feed</span><span class="p">(</span><span class="n">article_html</span><span class="o">.</span><span class="n">text</span><span class="p">)</span> <span class="n">parser</span><span class="o">.</span><span class="n">feed</span><span class="p">(</span><span class="n">article_html</span><span class="o">.</span><span class="n">text</span><span class="p">)</span>
<span class="n">parser</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">update</span><span class="p">({</span><span class="s2">&quot;url&quot;</span><span class="p">:</span> <span class="n">url</span><span class="p">})</span> <span class="n">parser</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">update</span><span class="p">({</span><span class="s">&quot;url&quot;</span><span class="p">:</span> <span class="n">url</span><span class="p">})</span>
<span class="n">parser</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">update</span><span class="p">({</span><span class="s2">&quot;text&quot;</span><span class="p">:</span> <span class="n">article_html</span><span class="o">.</span><span class="n">text</span><span class="p">})</span> <span class="n">parser</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">update</span><span class="p">({</span><span class="s">&quot;text&quot;</span><span class="p">:</span> <span class="n">article_html</span><span class="o">.</span><span class="n">text</span><span class="p">})</span>
<span class="k">return</span> <span class="n">parser</span><span class="o">.</span><span class="n">data</span> <span class="k">return</span> <span class="n">parser</span><span class="o">.</span><span class="n">data</span>
<span class="c1"># This copy **MUST** be in place. I&#39;m not sure why,</span> <span class="c"># This copy **MUST** be in place. I&#39;m not sure why,</span>
<span class="c1"># as you&#39;d think that the data being returned would already</span> <span class="c"># as you&#39;d think that the data being returned would already</span>
<span class="c1"># represent a different memory location. Even so, it blows up</span> <span class="c"># represent a different memory location. Even so, it blows up</span>
<span class="c1"># if you don&#39;t do this.</span> <span class="c"># if you don&#39;t do this.</span>
<span class="n">article_list</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">global_articles</span><span class="p">))</span> <span class="n">article_list</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">global_articles</span><span class="p">))</span>
<span class="n">article_data</span> <span class="o">=</span> <span class="p">[</span><span class="n">copy</span><span class="p">(</span><span class="n">retrieve_data</span><span class="p">(</span><span class="n">url</span><span class="p">))</span> <span class="k">for</span> <span class="n">url</span> <span class="ow">in</span> <span class="n">article_list</span><span class="p">]</span> <span class="n">article_data</span> <span class="o">=</span> <span class="p">[</span><span class="n">copy</span><span class="p">(</span><span class="n">retrieve_data</span><span class="p">(</span><span class="n">url</span><span class="p">))</span> <span class="k">for</span> <span class="n">url</span> <span class="ow">in</span> <span class="n">article_list</span><span class="p">]</span>
<span class="c1"># If there&#39;s an issue downloading the article, drop it.</span> <span class="c"># If there&#39;s an issue downloading the article, drop it.</span>
<span class="n">article_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">article_data</span><span class="p">)</span><span class="o">.</span><span class="n">dropna</span><span class="p">()</span> <span class="n">article_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">article_data</span><span class="p">)</span><span class="o">.</span><span class="n">dropna</span><span class="p">()</span>
</pre></div> </pre></div>
@@ -249,12 +249,12 @@
<div class="prompt input_prompt">In&nbsp;[4]:</div> <div class="prompt input_prompt">In&nbsp;[4]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># article_df is sorted by date, so we get the first row.</span> <div class=" highlight hl-ipython3"><pre><span class="c"># article_df is sorted by date, so we get the first row.</span>
<span class="n">start_date</span> <span class="o">=</span> <span class="n">article_df</span><span class="o">.</span><span class="n">sort_values</span><span class="p">(</span><span class="n">by</span><span class="o">=</span><span class="s1">&#39;date&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="s1">&#39;date&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="n">relativedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span> <span class="n">start_date</span> <span class="o">=</span> <span class="n">article_df</span><span class="o">.</span><span class="n">sort_values</span><span class="p">(</span><span class="n">by</span><span class="o">=</span><span class="s">&#39;date&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="s">&#39;date&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="n">relativedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="n">SPY</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;GOOG/NYSE_SPY&quot;</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span> <span class="n">SPY</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">&quot;GOOG/NYSE_SPY&quot;</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
<span class="n">DJIA</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;GOOG/AMS_DIA&quot;</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span> <span class="n">DJIA</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">&quot;GOOG/AMS_DIA&quot;</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
<span class="n">RUSS</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;GOOG/AMEX_IWM&quot;</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span> <span class="n">RUSS</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">&quot;GOOG/AMEX_IWM&quot;</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
<span class="n">NASDAQ</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;GOOG/EPA_QQQ&quot;</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span> <span class="n">NASDAQ</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">&quot;GOOG/EPA_QQQ&quot;</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
</pre></div> </pre></div>
</div> </div>
@@ -278,14 +278,14 @@
<div class="prompt input_prompt">In&nbsp;[5]:</div> <div class="prompt input_prompt">In&nbsp;[5]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">calculate_opening_ret</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span> <div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">calculate_opening_ret</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
<span class="c1"># I&#39;m not a huge fan of the appending for loop,</span> <span class="c"># I&#39;m not a huge fan of the appending for loop,</span>
<span class="c1"># but it&#39;s a bit verbose for a comprehension</span> <span class="c"># but it&#39;s a bit verbose for a comprehension</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">{}</span> <span class="n">data</span> <span class="o">=</span> <span class="p">{}</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">frame</span><span class="p">)):</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">frame</span><span class="p">)):</span>
<span class="n">date</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">name</span> <span class="n">date</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
<span class="n">prior_close</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="o">-</span><span class="mi">1</span><span class="p">][</span><span class="s1">&#39;Close&#39;</span><span class="p">]</span> <span class="n">prior_close</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="o">-</span><span class="mi">1</span><span class="p">][</span><span class="s">&#39;Close&#39;</span><span class="p">]</span>
<span class="n">open_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s1">&#39;Open&#39;</span><span class="p">]</span> <span class="n">open_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s">&#39;Open&#39;</span><span class="p">]</span>
<span class="n">data</span><span class="p">[</span><span class="n">date</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">open_val</span> <span class="o">-</span> <span class="n">prior_close</span><span class="p">)</span> <span class="o">/</span> <span class="n">prior_close</span> <span class="n">data</span><span class="p">[</span><span class="n">date</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">open_val</span> <span class="o">-</span> <span class="n">prior_close</span><span class="p">)</span> <span class="o">/</span> <span class="n">prior_close</span>
<span class="k">return</span> <span class="n">data</span> <span class="k">return</span> <span class="n">data</span>
@@ -296,8 +296,8 @@
<span class="n">NASDAQ_open_ret</span> <span class="o">=</span> <span class="n">calculate_opening_ret</span><span class="p">(</span><span class="n">NASDAQ</span><span class="p">)</span> <span class="n">NASDAQ_open_ret</span> <span class="o">=</span> <span class="n">calculate_opening_ret</span><span class="p">(</span><span class="n">NASDAQ</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">signs_match</span><span class="p">(</span><span class="n">list_1</span><span class="p">,</span> <span class="n">list_2</span><span class="p">):</span> <span class="k">def</span> <span class="nf">signs_match</span><span class="p">(</span><span class="n">list_1</span><span class="p">,</span> <span class="n">list_2</span><span class="p">):</span>
<span class="c1"># This is a surprisingly difficult task - we have to match</span> <span class="c"># This is a surprisingly difficult task - we have to match</span>
<span class="c1"># up the dates in order to check if opening returns actually match</span> <span class="c"># up the dates in order to check if opening returns actually match</span>
<span class="n">index_dict_dt</span> <span class="o">=</span> <span class="p">{</span><span class="n">key</span><span class="o">.</span><span class="n">to_datetime</span><span class="p">():</span> <span class="n">list_2</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">list_2</span><span class="o">.</span><span class="n">keys</span><span class="p">()}</span> <span class="n">index_dict_dt</span> <span class="o">=</span> <span class="p">{</span><span class="n">key</span><span class="o">.</span><span class="n">to_datetime</span><span class="p">():</span> <span class="n">list_2</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">list_2</span><span class="o">.</span><span class="n">keys</span><span class="p">()}</span>
<span class="n">matches</span> <span class="o">=</span> <span class="p">[]</span> <span class="n">matches</span> <span class="o">=</span> <span class="p">[]</span>
@@ -313,26 +313,26 @@
<span class="n">matches</span> <span class="o">+=</span> <span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="n">matches</span> <span class="o">+=</span> <span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span> <span class="k">else</span><span class="p">:</span>
<span class="n">matches</span> <span class="o">+=</span> <span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="n">matches</span> <span class="o">+=</span> <span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="c1">#print(&quot;{}&quot;.format(list_2[index_dt]))</span> <span class="c">#print(&quot;{}&quot;.format(list_2[index_dt]))</span>
<span class="k">return</span> <span class="n">matches</span> <span class="k">return</span> <span class="n">matches</span>
<span class="n">prediction_dict</span> <span class="o">=</span> <span class="p">{}</span> <span class="n">prediction_dict</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">matches_dict</span> <span class="o">=</span> <span class="p">{}</span> <span class="n">matches_dict</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">count_dict</span> <span class="o">=</span> <span class="p">{}</span> <span class="n">count_dict</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">index_dict</span> <span class="o">=</span> <span class="p">{</span><span class="s2">&quot;SPY&quot;</span><span class="p">:</span> <span class="n">SPY_open_ret</span><span class="p">,</span> <span class="s2">&quot;DJIA&quot;</span><span class="p">:</span> <span class="n">DJIA_open_ret</span><span class="p">,</span> <span class="s2">&quot;RUSS&quot;</span><span class="p">:</span> <span class="n">RUSS_open_ret</span><span class="p">,</span> <span class="s2">&quot;NASDAQ&quot;</span><span class="p">:</span> <span class="n">NASDAQ_open_ret</span><span class="p">}</span> <span class="n">index_dict</span> <span class="o">=</span> <span class="p">{</span><span class="s">&quot;SPY&quot;</span><span class="p">:</span> <span class="n">SPY_open_ret</span><span class="p">,</span> <span class="s">&quot;DJIA&quot;</span><span class="p">:</span> <span class="n">DJIA_open_ret</span><span class="p">,</span> <span class="s">&quot;RUSS&quot;</span><span class="p">:</span> <span class="n">RUSS_open_ret</span><span class="p">,</span> <span class="s">&quot;NASDAQ&quot;</span><span class="p">:</span> <span class="n">NASDAQ_open_ret</span><span class="p">}</span>
<span class="n">indices</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;SPY&quot;</span><span class="p">,</span> <span class="s2">&quot;DJIA&quot;</span><span class="p">,</span> <span class="s2">&quot;RUSS&quot;</span><span class="p">,</span> <span class="s2">&quot;NASDAQ&quot;</span><span class="p">]</span> <span class="n">indices</span> <span class="o">=</span> <span class="p">[</span><span class="s">&quot;SPY&quot;</span><span class="p">,</span> <span class="s">&quot;DJIA&quot;</span><span class="p">,</span> <span class="s">&quot;RUSS&quot;</span><span class="p">,</span> <span class="s">&quot;NASDAQ&quot;</span><span class="p">]</span>
<span class="n">futures</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;Crude&quot;</span><span class="p">,</span> <span class="s2">&quot;Gold&quot;</span><span class="p">,</span> <span class="s2">&quot;DOW&quot;</span><span class="p">,</span> <span class="s2">&quot;NASDAQ&quot;</span><span class="p">,</span> <span class="s2">&quot;S&amp;P&quot;</span><span class="p">]</span> <span class="n">futures</span> <span class="o">=</span> <span class="p">[</span><span class="s">&quot;Crude&quot;</span><span class="p">,</span> <span class="s">&quot;Gold&quot;</span><span class="p">,</span> <span class="s">&quot;DOW&quot;</span><span class="p">,</span> <span class="s">&quot;NASDAQ&quot;</span><span class="p">,</span> <span class="s">&quot;S&amp;P&quot;</span><span class="p">]</span>
<span class="k">for</span> <span class="n">index</span> <span class="ow">in</span> <span class="n">indices</span><span class="p">:</span> <span class="k">for</span> <span class="n">index</span> <span class="ow">in</span> <span class="n">indices</span><span class="p">:</span>
<span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">signs_match</span><span class="p">(</span><span class="n">article_df</span><span class="p">[[</span><span class="n">future</span><span class="p">,</span> <span class="s1">&#39;date&#39;</span><span class="p">]],</span> <span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">signs_match</span><span class="p">(</span><span class="n">article_df</span><span class="p">[[</span><span class="n">future</span><span class="p">,</span> <span class="s">&#39;date&#39;</span><span class="p">]],</span>
<span class="n">index_dict</span><span class="p">[</span><span class="n">index</span><span class="p">])</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span> <span class="n">index_dict</span><span class="p">[</span><span class="n">index</span><span class="p">])</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
<span class="n">count_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">future</span><span class="p">])</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span> <span class="n">count_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">future</span><span class="p">])</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
<span class="n">prediction_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">future</span><span class="p">])</span> <span class="n">prediction_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">future</span><span class="p">])</span>
<span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Articles Checked: &quot;</span><span class="p">)</span> <span class="nb">print</span><span class="p">(</span><span class="s">&quot;Articles Checked: &quot;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">count_dict</span><span class="p">))</span> <span class="nb">print</span><span class="p">(</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">count_dict</span><span class="p">))</span>
<span class="nb">print</span><span class="p">()</span> <span class="nb">print</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Prediction Accuracy:&quot;</span><span class="p">)</span> <span class="nb">print</span><span class="p">(</span><span class="s">&quot;Prediction Accuracy:&quot;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">prediction_dict</span><span class="p">))</span> <span class="nb">print</span><span class="p">(</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">prediction_dict</span><span class="p">))</span>
</pre></div> </pre></div>
@@ -391,14 +391,14 @@ S&amp;P 0.604478 0.597015 0.811808 0.848708
<div class="prompt input_prompt">In&nbsp;[6]:</div> <div class="prompt input_prompt">In&nbsp;[6]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">calculate_closing_ret</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span> <div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">calculate_closing_ret</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
<span class="c1"># I&#39;m not a huge fan of the appending for loop,</span> <span class="c"># I&#39;m not a huge fan of the appending for loop,</span>
<span class="c1"># but it&#39;s a bit verbose for a comprehension</span> <span class="c"># but it&#39;s a bit verbose for a comprehension</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">{}</span> <span class="n">data</span> <span class="o">=</span> <span class="p">{}</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">frame</span><span class="p">)):</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">frame</span><span class="p">)):</span>
<span class="n">date</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">name</span> <span class="n">date</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
<span class="n">open_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s1">&#39;Open&#39;</span><span class="p">]</span> <span class="n">open_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s">&#39;Open&#39;</span><span class="p">]</span>
<span class="n">close_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s1">&#39;Close&#39;</span><span class="p">]</span> <span class="n">close_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s">&#39;Close&#39;</span><span class="p">]</span>
<span class="n">data</span><span class="p">[</span><span class="n">date</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">close_val</span> <span class="o">-</span> <span class="n">open_val</span><span class="p">)</span> <span class="o">/</span> <span class="n">open_val</span> <span class="n">data</span><span class="p">[</span><span class="n">date</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">close_val</span> <span class="o">-</span> <span class="n">open_val</span><span class="p">)</span> <span class="o">/</span> <span class="n">open_val</span>
<span class="k">return</span> <span class="n">data</span> <span class="k">return</span> <span class="n">data</span>
@@ -409,8 +409,8 @@ S&amp;P 0.604478 0.597015 0.811808 0.848708
<span class="n">NASDAQ_close_ret</span> <span class="o">=</span> <span class="n">calculate_closing_ret</span><span class="p">(</span><span class="n">NASDAQ</span><span class="p">)</span> <span class="n">NASDAQ_close_ret</span> <span class="o">=</span> <span class="n">calculate_closing_ret</span><span class="p">(</span><span class="n">NASDAQ</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">signs_match</span><span class="p">(</span><span class="n">list_1</span><span class="p">,</span> <span class="n">list_2</span><span class="p">):</span> <span class="k">def</span> <span class="nf">signs_match</span><span class="p">(</span><span class="n">list_1</span><span class="p">,</span> <span class="n">list_2</span><span class="p">):</span>
<span class="c1"># This is a surprisingly difficult task - we have to match</span> <span class="c"># This is a surprisingly difficult task - we have to match</span>
<span class="c1"># up the dates in order to check if opening returns actually match</span> <span class="c"># up the dates in order to check if opening returns actually match</span>
<span class="n">index_dict_dt</span> <span class="o">=</span> <span class="p">{</span><span class="n">key</span><span class="o">.</span><span class="n">to_datetime</span><span class="p">():</span> <span class="n">list_2</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">list_2</span><span class="o">.</span><span class="n">keys</span><span class="p">()}</span> <span class="n">index_dict_dt</span> <span class="o">=</span> <span class="p">{</span><span class="n">key</span><span class="o">.</span><span class="n">to_datetime</span><span class="p">():</span> <span class="n">list_2</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">list_2</span><span class="o">.</span><span class="n">keys</span><span class="p">()}</span>
<span class="n">matches</span> <span class="o">=</span> <span class="p">[]</span> <span class="n">matches</span> <span class="o">=</span> <span class="p">[]</span>
@@ -426,28 +426,28 @@ S&amp;P 0.604478 0.597015 0.811808 0.848708
<span class="n">matches</span> <span class="o">+=</span> <span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="n">matches</span> <span class="o">+=</span> <span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span> <span class="k">else</span><span class="p">:</span>
<span class="n">matches</span> <span class="o">+=</span> <span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="n">matches</span> <span class="o">+=</span> <span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="c1">#print(&quot;{}&quot;.format(list_2[index_dt]))</span> <span class="c">#print(&quot;{}&quot;.format(list_2[index_dt]))</span>
<span class="k">return</span> <span class="n">matches</span> <span class="k">return</span> <span class="n">matches</span>
<span class="n">matches_dict</span> <span class="o">=</span> <span class="p">{}</span> <span class="n">matches_dict</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">count_dict</span> <span class="o">=</span> <span class="p">{}</span> <span class="n">count_dict</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">prediction_dict</span> <span class="o">=</span> <span class="p">{}</span> <span class="n">prediction_dict</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">index_dict</span> <span class="o">=</span> <span class="p">{</span><span class="s2">&quot;SPY&quot;</span><span class="p">:</span> <span class="n">SPY_close_ret</span><span class="p">,</span> <span class="s2">&quot;DJIA&quot;</span><span class="p">:</span> <span class="n">DJIA_close_ret</span><span class="p">,</span> <span class="n">index_dict</span> <span class="o">=</span> <span class="p">{</span><span class="s">&quot;SPY&quot;</span><span class="p">:</span> <span class="n">SPY_close_ret</span><span class="p">,</span> <span class="s">&quot;DJIA&quot;</span><span class="p">:</span> <span class="n">DJIA_close_ret</span><span class="p">,</span>
<span class="s2">&quot;RUSS&quot;</span><span class="p">:</span> <span class="n">RUSS_close_ret</span><span class="p">,</span> <span class="s2">&quot;NASDAQ&quot;</span><span class="p">:</span> <span class="n">NASDAQ_close_ret</span><span class="p">}</span> <span class="s">&quot;RUSS&quot;</span><span class="p">:</span> <span class="n">RUSS_close_ret</span><span class="p">,</span> <span class="s">&quot;NASDAQ&quot;</span><span class="p">:</span> <span class="n">NASDAQ_close_ret</span><span class="p">}</span>
<span class="n">indices</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;SPY&quot;</span><span class="p">,</span> <span class="s2">&quot;DJIA&quot;</span><span class="p">,</span> <span class="s2">&quot;RUSS&quot;</span><span class="p">,</span> <span class="s2">&quot;NASDAQ&quot;</span><span class="p">]</span> <span class="n">indices</span> <span class="o">=</span> <span class="p">[</span><span class="s">&quot;SPY&quot;</span><span class="p">,</span> <span class="s">&quot;DJIA&quot;</span><span class="p">,</span> <span class="s">&quot;RUSS&quot;</span><span class="p">,</span> <span class="s">&quot;NASDAQ&quot;</span><span class="p">]</span>
<span class="n">futures</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;Crude&quot;</span><span class="p">,</span> <span class="s2">&quot;Gold&quot;</span><span class="p">,</span> <span class="s2">&quot;DOW&quot;</span><span class="p">,</span> <span class="s2">&quot;NASDAQ&quot;</span><span class="p">,</span> <span class="s2">&quot;S&amp;P&quot;</span><span class="p">]</span> <span class="n">futures</span> <span class="o">=</span> <span class="p">[</span><span class="s">&quot;Crude&quot;</span><span class="p">,</span> <span class="s">&quot;Gold&quot;</span><span class="p">,</span> <span class="s">&quot;DOW&quot;</span><span class="p">,</span> <span class="s">&quot;NASDAQ&quot;</span><span class="p">,</span> <span class="s">&quot;S&amp;P&quot;</span><span class="p">]</span>
<span class="k">for</span> <span class="n">index</span> <span class="ow">in</span> <span class="n">indices</span><span class="p">:</span> <span class="k">for</span> <span class="n">index</span> <span class="ow">in</span> <span class="n">indices</span><span class="p">:</span>
<span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">signs_match</span><span class="p">(</span><span class="n">article_df</span><span class="p">[[</span><span class="n">future</span><span class="p">,</span> <span class="s1">&#39;date&#39;</span><span class="p">]],</span> <span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">signs_match</span><span class="p">(</span><span class="n">article_df</span><span class="p">[[</span><span class="n">future</span><span class="p">,</span> <span class="s">&#39;date&#39;</span><span class="p">]],</span>
<span class="n">index_dict</span><span class="p">[</span><span class="n">index</span><span class="p">])</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span> <span class="n">index_dict</span><span class="p">[</span><span class="n">index</span><span class="p">])</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
<span class="n">count_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">future</span><span class="p">])</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span> <span class="n">count_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">future</span><span class="p">])</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
<span class="n">prediction_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">future</span><span class="p">])</span> <span class="n">prediction_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">future</span><span class="p">])</span>
<span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Articles Checked:&quot;</span><span class="p">)</span> <span class="nb">print</span><span class="p">(</span><span class="s">&quot;Articles Checked:&quot;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">count_dict</span><span class="p">))</span> <span class="nb">print</span><span class="p">(</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">count_dict</span><span class="p">))</span>
<span class="nb">print</span><span class="p">()</span> <span class="nb">print</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Prediction Accuracy:&quot;</span><span class="p">)</span> <span class="nb">print</span><span class="p">(</span><span class="s">&quot;Prediction Accuracy:&quot;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">prediction_dict</span><span class="p">))</span> <span class="nb">print</span><span class="p">(</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">prediction_dict</span><span class="p">))</span>
</pre></div> </pre></div>
the-unfair-casino.html
+50 -50
View File
@@ -4,7 +4,7 @@
<meta charset="utf-8"> <meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="Or, how to get thrown out of a casino because you&#39;re a mathematician. In the ongoing eternal cycle of mathematicians asking generally useless questions about probability, I dreamt up another one. ..."> <meta name="description" content="Or, how to get thrown out of a casino because you&#39;re a mathematician. In the ongoing eternal cycle of mathematicians asking generally useless questions about probability, I dreamt up another one. The ...">
<meta name="keywords" content="casino, em, machine learning, probability, simulated annealing"> <meta name="keywords" content="casino, em, machine learning, probability, simulated annealing">
<link rel="icon" href="https://bspeice.github.io/favicon.ico"> <link rel="icon" href="https://bspeice.github.io/favicon.ico">
@@ -106,7 +106,7 @@
<div class="prompt input_prompt">In&nbsp;[1]:</div> <div class="prompt input_prompt">In&nbsp;[1]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span> <div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span> <span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span> <span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
<span class="o">%</span><span class="k">matplotlib</span> inline <span class="o">%</span><span class="k">matplotlib</span> inline
@@ -114,8 +114,8 @@
<span class="n">fair_1</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span> <span class="n">fair_1</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
<span class="n">fair_2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span> <span class="n">fair_2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
<span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">fair_1</span> <span class="o">+</span> <span class="n">fair_2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s1">&#39;hist&#39;</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">);</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">fair_1</span> <span class="o">+</span> <span class="n">fair_2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s">&#39;hist&#39;</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">&#39;Fair Distribution&#39;</span><span class="p">);</span> <span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">&#39;Fair Distribution&#39;</span><span class="p">);</span>
</pre></div> </pre></div>
</div> </div>
@@ -316,7 +316,7 @@ D_2 = \left\{
<div class="prompt input_prompt">In&nbsp;[2]:</div> <div class="prompt input_prompt">In&nbsp;[2]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">unfair_die</span><span class="p">(</span><span class="n">p_vals</span><span class="p">,</span> <span class="n">n</span><span class="p">):</span> <div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">unfair_die</span><span class="p">(</span><span class="n">p_vals</span><span class="p">,</span> <span class="n">n</span><span class="p">):</span>
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">multinomial</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">p_vals</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span> <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">multinomial</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">p_vals</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
<span class="k">return</span> <span class="n">x</span><span class="o">.</span><span class="n">nonzero</span><span class="p">()[</span><span class="mi">1</span><span class="p">]</span> <span class="o">+</span> <span class="mi">1</span> <span class="k">return</span> <span class="n">x</span><span class="o">.</span><span class="n">nonzero</span><span class="p">()[</span><span class="mi">1</span><span class="p">]</span> <span class="o">+</span> <span class="mi">1</span>
@@ -326,8 +326,8 @@ D_2 = \left\{
<span class="n">x1</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d1</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span> <span class="n">x1</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d1</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
<span class="n">x2</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d2</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span> <span class="n">x2</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d2</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
<span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">x1</span> <span class="o">+</span> <span class="n">x2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s1">&#39;hist&#39;</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">);</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">x1</span> <span class="o">+</span> <span class="n">x2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s">&#39;hist&#39;</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">&#39;$D_1$ biased low, $D_2$ biased high&#39;</span><span class="p">);</span> <span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">&#39;$D_1$ biased low, $D_2$ biased high&#39;</span><span class="p">);</span>
</pre></div> </pre></div>
</div> </div>
@@ -519,14 +519,14 @@ D_2 = \left\{
<div class="prompt input_prompt">In&nbsp;[3]:</div> <div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">d1</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">]</span> <div class=" highlight hl-ipython3"><pre><span class="n">d1</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">]</span>
<span class="n">d2</span> <span class="o">=</span> <span class="n">d1</span> <span class="n">d2</span> <span class="o">=</span> <span class="n">d1</span>
<span class="n">x1</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d1</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span> <span class="n">x1</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d1</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
<span class="n">x2</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d2</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span> <span class="n">x2</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d2</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
<span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">x1</span> <span class="o">+</span> <span class="n">x2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s1">&#39;hist&#39;</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">)</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">x1</span> <span class="o">+</span> <span class="n">x2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s">&#39;hist&#39;</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s2">&quot;$D_1$ and $D_2$ biased to 1 and 6&quot;</span><span class="p">);</span> <span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">&quot;$D_1$ and $D_2$ biased to 1 and 6&quot;</span><span class="p">);</span>
</pre></div> </pre></div>
</div> </div>
@@ -742,22 +742,22 @@ and continue the next iteration.</li>
<div class="prompt input_prompt">In&nbsp;[5]:</div> <div class="prompt input_prompt">In&nbsp;[5]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span> <div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">from</span> <span class="nn">numpy</span> <span class="k">import</span> <span class="n">polynomial</span> <span class="kn">from</span> <span class="nn">numpy</span> <span class="k">import</span> <span class="n">polynomial</span>
<span class="k">def</span> <span class="nf">density_coef</span><span class="p">(</span><span class="n">d1_density</span><span class="p">,</span> <span class="n">d2_density</span><span class="p">):</span> <span class="k">def</span> <span class="nf">density_coef</span><span class="p">(</span><span class="n">d1_density</span><span class="p">,</span> <span class="n">d2_density</span><span class="p">):</span>
<span class="c1"># Calculating the probabilities of each outcome was taken</span> <span class="c"># Calculating the probabilities of each outcome was taken</span>
<span class="c1"># from this author: http://math.stackexchange.com/a/1710392/320784</span> <span class="c"># from this author: http://math.stackexchange.com/a/1710392/320784</span>
<span class="n">d1_p</span> <span class="o">=</span> <span class="n">polynomial</span><span class="o">.</span><span class="n">Polynomial</span><span class="p">(</span><span class="n">d1_density</span><span class="p">)</span> <span class="n">d1_p</span> <span class="o">=</span> <span class="n">polynomial</span><span class="o">.</span><span class="n">Polynomial</span><span class="p">(</span><span class="n">d1_density</span><span class="p">)</span>
<span class="n">d2_p</span> <span class="o">=</span> <span class="n">polynomial</span><span class="o">.</span><span class="n">Polynomial</span><span class="p">(</span><span class="n">d2_density</span><span class="p">)</span> <span class="n">d2_p</span> <span class="o">=</span> <span class="n">polynomial</span><span class="o">.</span><span class="n">Polynomial</span><span class="p">(</span><span class="n">d2_density</span><span class="p">)</span>
<span class="n">coefs</span> <span class="o">=</span> <span class="p">(</span><span class="n">d1_p</span> <span class="o">*</span> <span class="n">d2_p</span><span class="p">)</span><span class="o">.</span><span class="n">coef</span> <span class="n">coefs</span> <span class="o">=</span> <span class="p">(</span><span class="n">d1_p</span> <span class="o">*</span> <span class="n">d2_p</span><span class="p">)</span><span class="o">.</span><span class="n">coef</span>
<span class="k">return</span> <span class="n">coefs</span> <span class="k">return</span> <span class="n">coefs</span>
<span class="k">def</span> <span class="nf">score</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">d1_density</span><span class="p">,</span> <span class="n">d2_density</span><span class="p">):</span> <span class="k">def</span> <span class="nf">score</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">d1_density</span><span class="p">,</span> <span class="n">d2_density</span><span class="p">):</span>
<span class="c1"># We&#39;ve now got the probabilities of each event, but we need</span> <span class="c"># We&#39;ve now got the probabilities of each event, but we need</span>
<span class="c1"># to shift the array a bit so we can use the x values to actually</span> <span class="c"># to shift the array a bit so we can use the x values to actually</span>
<span class="c1"># index into it. This will allow us to do all the calculations</span> <span class="c"># index into it. This will allow us to do all the calculations</span>
<span class="c1"># incredibly quickly</span> <span class="c"># incredibly quickly</span>
<span class="n">coefs</span> <span class="o">=</span> <span class="n">density_coef</span><span class="p">(</span><span class="n">d1_density</span><span class="p">,</span> <span class="n">d2_density</span><span class="p">)</span> <span class="n">coefs</span> <span class="o">=</span> <span class="n">density_coef</span><span class="p">(</span><span class="n">d1_density</span><span class="p">,</span> <span class="n">d2_density</span><span class="p">)</span>
<span class="n">coefs</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">hstack</span><span class="p">((</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="n">coefs</span><span class="p">))</span> <span class="n">coefs</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">hstack</span><span class="p">((</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="n">coefs</span><span class="p">))</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">coefs</span><span class="p">[</span><span class="n">x</span><span class="p">])</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span> <span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">coefs</span><span class="p">[</span><span class="n">x</span><span class="p">])</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
@@ -783,24 +783,24 @@ and continue the next iteration.</li>
<div class="prompt input_prompt">In&nbsp;[6]:</div> <div class="prompt input_prompt">In&nbsp;[6]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">permute</span><span class="p">(</span><span class="n">d1_density</span><span class="p">,</span> <span class="n">d2_density</span><span class="p">):</span> <div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">permute</span><span class="p">(</span><span class="n">d1_density</span><span class="p">,</span> <span class="n">d2_density</span><span class="p">):</span>
<span class="c1"># To ensure we have legitimate densities, we will randomly</span> <span class="c"># To ensure we have legitimate densities, we will randomly</span>
<span class="c1"># increase one die face probability by `change`,</span> <span class="c"># increase one die face probability by `change`,</span>
<span class="c1"># and decrease one by `change`.</span> <span class="c"># and decrease one by `change`.</span>
<span class="c1"># This means there are something less than (1/`change`)^12 possibilities</span> <span class="c"># This means there are something less than (1/`change`)^12 possibilities</span>
<span class="c1"># we are trying to search over.</span> <span class="c"># we are trying to search over.</span>
<span class="n">change</span> <span class="o">=</span> <span class="o">.</span><span class="mi">01</span> <span class="n">change</span> <span class="o">=</span> <span class="o">.</span><span class="mi">01</span>
<span class="n">d1_index1</span><span class="p">,</span> <span class="n">d1_index2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span> <span class="n">d1_index1</span><span class="p">,</span> <span class="n">d1_index2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
<span class="n">d2_index1</span><span class="p">,</span> <span class="n">d2_index2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span> <span class="n">d2_index1</span><span class="p">,</span> <span class="n">d2_index2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
<span class="c1"># Also make sure to copy. I&#39;ve had some weird aliasing issues</span> <span class="c"># Also make sure to copy. I&#39;ve had some weird aliasing issues</span>
<span class="c1"># in the past that made everything blow up.</span> <span class="c"># in the past that made everything blow up.</span>
<span class="n">new_d1</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">float64</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="n">d1_density</span><span class="p">))</span> <span class="n">new_d1</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">float64</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="n">d1_density</span><span class="p">))</span>
<span class="n">new_d2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">float64</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="n">d2_density</span><span class="p">))</span> <span class="n">new_d2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">float64</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="n">d2_density</span><span class="p">))</span>
<span class="c1"># While this doesn&#39;t account for the possibility that some</span> <span class="c"># While this doesn&#39;t account for the possibility that some</span>
<span class="c1"># values go negative, in practice this never happens</span> <span class="c"># values go negative, in practice this never happens</span>
<span class="n">new_d1</span><span class="p">[</span><span class="n">d1_index1</span><span class="p">]</span> <span class="o">+=</span> <span class="n">change</span> <span class="n">new_d1</span><span class="p">[</span><span class="n">d1_index1</span><span class="p">]</span> <span class="o">+=</span> <span class="n">change</span>
<span class="n">new_d1</span><span class="p">[</span><span class="n">d1_index2</span><span class="p">]</span> <span class="o">-=</span> <span class="n">change</span> <span class="n">new_d1</span><span class="p">[</span><span class="n">d1_index2</span><span class="p">]</span> <span class="o">-=</span> <span class="n">change</span>
<span class="n">new_d2</span><span class="p">[</span><span class="n">d2_index1</span><span class="p">]</span> <span class="o">+=</span> <span class="n">change</span> <span class="n">new_d2</span><span class="p">[</span><span class="n">d2_index1</span><span class="p">]</span> <span class="o">+=</span> <span class="n">change</span>
@@ -829,28 +829,28 @@ and continue the next iteration.</li>
<div class="prompt input_prompt">In&nbsp;[7]:</div> <div class="prompt input_prompt">In&nbsp;[7]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">optimize</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">conv_count</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">max_iter</span><span class="o">=</span><span class="mi">1</span><span class="n">e4</span><span class="p">):</span> <div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">optimize</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">conv_count</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">max_iter</span><span class="o">=</span><span class="mi">1</span><span class="n">e4</span><span class="p">):</span>
<span class="n">switch_failures</span> <span class="o">=</span> <span class="mi">0</span> <span class="n">switch_failures</span> <span class="o">=</span> <span class="mi">0</span>
<span class="n">iter_count</span> <span class="o">=</span> <span class="mi">0</span> <span class="n">iter_count</span> <span class="o">=</span> <span class="mi">0</span>
<span class="c1"># Start with guessing fair dice</span> <span class="c"># Start with guessing fair dice</span>
<span class="n">cur_d1</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="mi">1</span><span class="o">/</span><span class="mi">6</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span> <span class="n">cur_d1</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="mi">1</span><span class="o">/</span><span class="mi">6</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
<span class="n">cur_d2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="mi">1</span><span class="o">/</span><span class="mi">6</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span> <span class="n">cur_d2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="mi">1</span><span class="o">/</span><span class="mi">6</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
<span class="n">cur_score</span> <span class="o">=</span> <span class="n">score</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">cur_d1</span><span class="p">,</span> <span class="n">cur_d2</span><span class="p">)</span> <span class="n">cur_score</span> <span class="o">=</span> <span class="n">score</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">cur_d1</span><span class="p">,</span> <span class="n">cur_d2</span><span class="p">)</span>
<span class="c1"># Keep track of our best guesses - may not be</span> <span class="c"># Keep track of our best guesses - may not be</span>
<span class="c1"># what we end with</span> <span class="c"># what we end with</span>
<span class="n">max_score</span> <span class="o">=</span> <span class="n">cur_score</span> <span class="n">max_score</span> <span class="o">=</span> <span class="n">cur_score</span>
<span class="n">max_d1</span> <span class="o">=</span> <span class="n">cur_d1</span> <span class="n">max_d1</span> <span class="o">=</span> <span class="n">cur_d1</span>
<span class="n">max_d2</span> <span class="o">=</span> <span class="n">cur_d2</span> <span class="n">max_d2</span> <span class="o">=</span> <span class="n">cur_d2</span>
<span class="c1"># Optimization stops when we have failed to switch `conv_count`</span> <span class="c"># Optimization stops when we have failed to switch `conv_count`</span>
<span class="c1"># times (presumably because we have a great guess), or we reach</span> <span class="c"># times (presumably because we have a great guess), or we reach</span>
<span class="c1"># the maximum number of iterations.</span> <span class="c"># the maximum number of iterations.</span>
<span class="k">while</span> <span class="n">switch_failures</span> <span class="o">&lt;</span> <span class="n">conv_count</span> <span class="ow">and</span> <span class="n">iter_count</span> <span class="o">&lt;</span> <span class="n">max_iter</span><span class="p">:</span> <span class="k">while</span> <span class="n">switch_failures</span> <span class="o">&lt;</span> <span class="n">conv_count</span> <span class="ow">and</span> <span class="n">iter_count</span> <span class="o">&lt;</span> <span class="n">max_iter</span><span class="p">:</span>
<span class="n">iter_count</span> <span class="o">+=</span> <span class="mi">1</span> <span class="n">iter_count</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">if</span> <span class="n">iter_count</span> <span class="o">%</span> <span class="p">(</span><span class="n">max_iter</span> <span class="o">/</span> <span class="mi">10</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span> <span class="k">if</span> <span class="n">iter_count</span> <span class="o">%</span> <span class="p">(</span><span class="n">max_iter</span> <span class="o">/</span> <span class="mi">10</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Iteration: </span><span class="si">{}</span><span class="s1">; Current score (higher is better): </span><span class="si">{}</span><span class="s1">&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span> <span class="nb">print</span><span class="p">(</span><span class="s">&#39;Iteration: {}; Current score (higher is better): {}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
<span class="n">iter_count</span><span class="p">,</span> <span class="n">cur_score</span><span class="p">))</span> <span class="n">iter_count</span><span class="p">,</span> <span class="n">cur_score</span><span class="p">))</span>
<span class="n">new_d1</span><span class="p">,</span> <span class="n">new_d2</span> <span class="o">=</span> <span class="n">permute</span><span class="p">(</span><span class="n">cur_d1</span><span class="p">,</span> <span class="n">cur_d2</span><span class="p">)</span> <span class="n">new_d1</span><span class="p">,</span> <span class="n">new_d2</span> <span class="o">=</span> <span class="n">permute</span><span class="p">(</span><span class="n">cur_d1</span><span class="p">,</span> <span class="n">cur_d2</span><span class="p">)</span>
@@ -862,19 +862,19 @@ and continue the next iteration.</li>
<span class="n">max_d2</span> <span class="o">=</span> <span class="n">new_d2</span> <span class="n">max_d2</span> <span class="o">=</span> <span class="n">new_d2</span>
<span class="k">if</span> <span class="n">new_score</span> <span class="o">&gt;</span> <span class="n">cur_score</span><span class="p">:</span> <span class="k">if</span> <span class="n">new_score</span> <span class="o">&gt;</span> <span class="n">cur_score</span><span class="p">:</span>
<span class="c1"># If the new permutation beats the old one,</span> <span class="c"># If the new permutation beats the old one,</span>
<span class="c1"># automatically select it.</span> <span class="c"># automatically select it.</span>
<span class="n">cur_score</span> <span class="o">=</span> <span class="n">new_score</span> <span class="n">cur_score</span> <span class="o">=</span> <span class="n">new_score</span>
<span class="n">cur_d1</span> <span class="o">=</span> <span class="n">new_d1</span> <span class="n">cur_d1</span> <span class="o">=</span> <span class="n">new_d1</span>
<span class="n">cur_d2</span> <span class="o">=</span> <span class="n">new_d2</span> <span class="n">cur_d2</span> <span class="o">=</span> <span class="n">new_d2</span>
<span class="n">switch_failures</span> <span class="o">=</span> <span class="mi">0</span> <span class="n">switch_failures</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">else</span><span class="p">:</span> <span class="k">else</span><span class="p">:</span>
<span class="c1"># We didn&#39;t beat the current score, but allow</span> <span class="c"># We didn&#39;t beat the current score, but allow</span>
<span class="c1"># for possibly switching anyways.</span> <span class="c"># for possibly switching anyways.</span>
<span class="n">accept_prob</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="n">new_score</span> <span class="o">-</span> <span class="n">cur_score</span><span class="p">)</span> <span class="n">accept_prob</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="n">new_score</span> <span class="o">-</span> <span class="n">cur_score</span><span class="p">)</span>
<span class="n">coin_toss</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">()</span> <span class="n">coin_toss</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">()</span>
<span class="k">if</span> <span class="n">coin_toss</span> <span class="o">&lt;</span> <span class="n">accept_prob</span><span class="p">:</span> <span class="k">if</span> <span class="n">coin_toss</span> <span class="o">&lt;</span> <span class="n">accept_prob</span><span class="p">:</span>
<span class="c1"># We randomly switch to the new distribution</span> <span class="c"># We randomly switch to the new distribution</span>
<span class="n">cur_score</span> <span class="o">=</span> <span class="n">new_score</span> <span class="n">cur_score</span> <span class="o">=</span> <span class="n">new_score</span>
<span class="n">cur_d1</span> <span class="o">=</span> <span class="n">new_d1</span> <span class="n">cur_d1</span> <span class="o">=</span> <span class="n">new_d1</span>
<span class="n">cur_d2</span> <span class="o">=</span> <span class="n">new_d2</span> <span class="n">cur_d2</span> <span class="o">=</span> <span class="n">new_d2</span>
@@ -882,7 +882,7 @@ and continue the next iteration.</li>
<span class="k">else</span><span class="p">:</span> <span class="k">else</span><span class="p">:</span>
<span class="n">switch_failures</span> <span class="o">+=</span> <span class="mi">1</span> <span class="n">switch_failures</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="c1"># Return both our best guess, and the ending guess</span> <span class="c"># Return both our best guess, and the ending guess</span>
<span class="k">return</span> <span class="n">max_d1</span><span class="p">,</span> <span class="n">max_d2</span><span class="p">,</span> <span class="n">cur_d1</span><span class="p">,</span> <span class="n">cur_d2</span> <span class="k">return</span> <span class="n">max_d1</span><span class="p">,</span> <span class="n">max_d2</span><span class="p">,</span> <span class="n">cur_d1</span><span class="p">,</span> <span class="n">cur_d2</span>
</pre></div> </pre></div>
@@ -908,7 +908,7 @@ and continue the next iteration.</li>
<div class="prompt input_prompt">In&nbsp;[8]:</div> <div class="prompt input_prompt">In&nbsp;[8]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">time</span> <div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">time</span>
<span class="k">def</span> <span class="nf">simulate_casino</span><span class="p">(</span><span class="n">d1_dist</span><span class="p">,</span> <span class="n">d2_dist</span><span class="p">,</span> <span class="n">n</span><span class="o">=</span><span class="mi">10000</span><span class="p">):</span> <span class="k">def</span> <span class="nf">simulate_casino</span><span class="p">(</span><span class="n">d1_dist</span><span class="p">,</span> <span class="n">d2_dist</span><span class="p">,</span> <span class="n">n</span><span class="o">=</span><span class="mi">10000</span><span class="p">):</span>
<span class="n">d1_vals</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d1_dist</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span> <span class="n">d1_vals</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d1_dist</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
<span class="n">d2_vals</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d2_dist</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span> <span class="n">d2_vals</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d2_dist</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
@@ -916,20 +916,20 @@ and continue the next iteration.</li>
<span class="n">start</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">perf_counter</span><span class="p">()</span> <span class="n">start</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">perf_counter</span><span class="p">()</span>
<span class="n">max_d1</span><span class="p">,</span> <span class="n">max_d2</span><span class="p">,</span> <span class="n">final_d1</span><span class="p">,</span> <span class="n">final_d2</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span><span class="n">d1_vals</span> <span class="o">+</span> <span class="n">d2_vals</span><span class="p">)</span> <span class="n">max_d1</span><span class="p">,</span> <span class="n">max_d2</span><span class="p">,</span> <span class="n">final_d1</span><span class="p">,</span> <span class="n">final_d2</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span><span class="n">d1_vals</span> <span class="o">+</span> <span class="n">d2_vals</span><span class="p">)</span>
<span class="n">end</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">perf_counter</span><span class="p">()</span> <span class="n">end</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">perf_counter</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Simulated Annealing time: </span><span class="si">{:.02f}</span><span class="s2">s&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">end</span> <span class="o">-</span> <span class="n">start</span><span class="p">))</span> <span class="nb">print</span><span class="p">(</span><span class="s">&quot;Simulated Annealing time: {:.02f}s&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">end</span> <span class="o">-</span> <span class="n">start</span><span class="p">))</span>
<span class="n">coef_range</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">13</span><span class="p">)</span> <span class="o">-</span> <span class="o">.</span><span class="mi">5</span> <span class="n">coef_range</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">13</span><span class="p">)</span> <span class="o">-</span> <span class="o">.</span><span class="mi">5</span>
<span class="n">plt</span><span class="o">.</span><span class="n">subplot</span><span class="p">(</span><span class="mi">221</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplot</span><span class="p">(</span><span class="mi">221</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">bar</span><span class="p">(</span><span class="n">coef_range</span><span class="p">,</span> <span class="n">density_coef</span><span class="p">(</span><span class="n">d1_dist</span><span class="p">,</span> <span class="n">d2_dist</span><span class="p">),</span> <span class="n">width</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">bar</span><span class="p">(</span><span class="n">coef_range</span><span class="p">,</span> <span class="n">density_coef</span><span class="p">(</span><span class="n">d1_dist</span><span class="p">,</span> <span class="n">d2_dist</span><span class="p">),</span> <span class="n">width</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">&#39;True Distribution&#39;</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">&#39;True Distribution&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">subplot</span><span class="p">(</span><span class="mi">222</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplot</span><span class="p">(</span><span class="mi">222</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">hist</span><span class="p">(</span><span class="n">d1_vals</span> <span class="o">+</span> <span class="n">d2_vals</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">hist</span><span class="p">(</span><span class="n">d1_vals</span> <span class="o">+</span> <span class="n">d2_vals</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">&#39;Empirical Distribution&#39;</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">&#39;Empirical Distribution&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">subplot</span><span class="p">(</span><span class="mi">223</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplot</span><span class="p">(</span><span class="mi">223</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">bar</span><span class="p">(</span><span class="n">coef_range</span><span class="p">,</span> <span class="n">density_coef</span><span class="p">(</span><span class="n">max_d1</span><span class="p">,</span> <span class="n">max_d2</span><span class="p">),</span> <span class="n">width</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">bar</span><span class="p">(</span><span class="n">coef_range</span><span class="p">,</span> <span class="n">density_coef</span><span class="p">(</span><span class="n">max_d1</span><span class="p">,</span> <span class="n">max_d2</span><span class="p">),</span> <span class="n">width</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">&#39;Recovered Distribution&#39;</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">&#39;Recovered Distribution&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span>
@@ -1310,7 +1310,7 @@ m+QFBkHwS1PQ018D5wD/PHxt/03H/cxXeHtR0jLn4qiSJEkNdH17UZIkaUUwdEmSJDVg6JIkSWrA
<div class="prompt input_prompt">In&nbsp;[9]:</div> <div class="prompt input_prompt">In&nbsp;[9]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">simulate_casino</span><span class="p">([</span><span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">],</span> <div class=" highlight hl-ipython3"><pre><span class="n">simulate_casino</span><span class="p">([</span><span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">],</span>
<span class="p">[</span><span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">])</span> <span class="p">[</span><span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">])</span>
</pre></div> </pre></div>
@@ -1638,7 +1638,7 @@ jAU+qF/bLxXO081keTEiTnJpjhoRERHRgNLLixEREREjQoquiIiIiAak6IqIiIhoQIquiIiIiAak
<div class="prompt input_prompt">In&nbsp;[10]:</div> <div class="prompt input_prompt">In&nbsp;[10]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">simulate_casino</span><span class="p">([</span><span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">],</span> <div class=" highlight hl-ipython3"><pre><span class="n">simulate_casino</span><span class="p">([</span><span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">],</span>
<span class="p">[</span><span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">])</span> <span class="p">[</span><span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">12</span><span class="p">])</span>
</pre></div> </pre></div>
@@ -2012,7 +2012,7 @@ RK5CYII=
<div class="prompt input_prompt">In&nbsp;[11]:</div> <div class="prompt input_prompt">In&nbsp;[11]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">simulate_casino</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="mi">1</span><span class="o">/</span><span class="mi">6</span><span class="p">,</span> <span class="mi">6</span><span class="p">),</span> <span class="n">np</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="mi">1</span><span class="o">/</span><span class="mi">6</span><span class="p">,</span> <span class="mi">6</span><span class="p">))</span> <div class=" highlight hl-ipython3"><pre><span class="n">simulate_casino</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="mi">1</span><span class="o">/</span><span class="mi">6</span><span class="p">,</span> <span class="mi">6</span><span class="p">),</span> <span class="n">np</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="mi">1</span><span class="o">/</span><span class="mi">6</span><span class="p">,</span> <span class="mi">6</span><span class="p">))</span>
</pre></div> </pre></div>
</div> </div>
@@ -2403,7 +2403,7 @@ hoSz1PA0AAAAAElFTkSuQmCC
<div class="prompt input_prompt">In&nbsp;[12]:</div> <div class="prompt input_prompt">In&nbsp;[12]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">simulate_casino</span><span class="p">([</span><span class="mi">1</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">2</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">2</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">2</span><span class="o">/</span><span class="mi">9</span><span class="p">],</span> <div class=" highlight hl-ipython3"><pre><span class="n">simulate_casino</span><span class="p">([</span><span class="mi">1</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">2</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">2</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">2</span><span class="o">/</span><span class="mi">9</span><span class="p">],</span>
<span class="p">[</span><span class="mi">1</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">2</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">2</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">2</span><span class="o">/</span><span class="mi">9</span><span class="p">],</span> <span class="p">[</span><span class="mi">1</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">1</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">2</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">2</span><span class="o">/</span><span class="mi">9</span><span class="p">,</span> <span class="mi">2</span><span class="o">/</span><span class="mi">9</span><span class="p">],</span>
<span class="n">n</span><span class="o">=</span><span class="mi">250</span><span class="p">)</span> <span class="n">n</span><span class="o">=</span><span class="mi">250</span><span class="p">)</span>
</pre></div> </pre></div>
tick-tock.html
+48 -48
View File
@@ -105,7 +105,7 @@
<div class="prompt input_prompt">In&nbsp;[1]:</div> <div class="prompt input_prompt">In&nbsp;[1]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">total_heartbeats</span> <span class="o">=</span> <span class="mi">2500000000</span> <div class=" highlight hl-ipython3"><pre><span class="n">total_heartbeats</span> <span class="o">=</span> <span class="mi">2500000000</span>
</pre></div> </pre></div>
</div> </div>
@@ -137,7 +137,7 @@
<div class="prompt input_prompt">In&nbsp;[2]:</div> <div class="prompt input_prompt">In&nbsp;[2]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># Import all the OAuth secret information from a local file</span> <div class=" highlight hl-ipython3"><pre><span class="c"># Import all the OAuth secret information from a local file</span>
<span class="kn">from</span> <span class="nn">secrets</span> <span class="k">import</span> <span class="n">CLIENT_SECRET</span><span class="p">,</span> <span class="n">CLIENT_ID</span><span class="p">,</span> <span class="n">CALLBACK_URL</span> <span class="kn">from</span> <span class="nn">secrets</span> <span class="k">import</span> <span class="n">CLIENT_SECRET</span><span class="p">,</span> <span class="n">CLIENT_ID</span><span class="p">,</span> <span class="n">CALLBACK_URL</span>
</pre></div> </pre></div>
@@ -176,19 +176,19 @@ Application: Thank you, now give me all the data.</code></pre>
<div class="prompt input_prompt">In&nbsp;[3]:</div> <div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">urllib</span> <div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">urllib</span>
<span class="n">FITBIT_URI</span> <span class="o">=</span> <span class="s1">&#39;https://www.fitbit.com/oauth2/authorize&#39;</span> <span class="n">FITBIT_URI</span> <span class="o">=</span> <span class="s">&#39;https://www.fitbit.com/oauth2/authorize&#39;</span>
<span class="n">params</span> <span class="o">=</span> <span class="p">{</span> <span class="n">params</span> <span class="o">=</span> <span class="p">{</span>
<span class="c1"># If we need more than one scope, must be a CSV string</span> <span class="c"># If we need more than one scope, must be a CSV string</span>
<span class="s1">&#39;scope&#39;</span><span class="p">:</span> <span class="s1">&#39;heartrate&#39;</span><span class="p">,</span> <span class="s">&#39;scope&#39;</span><span class="p">:</span> <span class="s">&#39;heartrate&#39;</span><span class="p">,</span>
<span class="s1">&#39;response_type&#39;</span><span class="p">:</span> <span class="s1">&#39;token&#39;</span><span class="p">,</span> <span class="s">&#39;response_type&#39;</span><span class="p">:</span> <span class="s">&#39;token&#39;</span><span class="p">,</span>
<span class="s1">&#39;expires_in&#39;</span><span class="p">:</span> <span class="mi">86400</span><span class="p">,</span> <span class="c1"># 1 day</span> <span class="s">&#39;expires_in&#39;</span><span class="p">:</span> <span class="mi">86400</span><span class="p">,</span> <span class="c"># 1 day</span>
<span class="s1">&#39;redirect_uri&#39;</span><span class="p">:</span> <span class="n">CALLBACK_URL</span><span class="p">,</span> <span class="s">&#39;redirect_uri&#39;</span><span class="p">:</span> <span class="n">CALLBACK_URL</span><span class="p">,</span>
<span class="s1">&#39;client_id&#39;</span><span class="p">:</span> <span class="n">CLIENT_ID</span> <span class="s">&#39;client_id&#39;</span><span class="p">:</span> <span class="n">CLIENT_ID</span>
<span class="p">}</span> <span class="p">}</span>
<span class="n">request_url</span> <span class="o">=</span> <span class="n">FITBIT_URI</span> <span class="o">+</span> <span class="s1">&#39;?&#39;</span> <span class="o">+</span> <span class="n">urllib</span><span class="o">.</span><span class="n">parse</span><span class="o">.</span><span class="n">urlencode</span><span class="p">(</span><span class="n">params</span><span class="p">)</span> <span class="n">request_url</span> <span class="o">=</span> <span class="n">FITBIT_URI</span> <span class="o">+</span> <span class="s">&#39;?&#39;</span> <span class="o">+</span> <span class="n">urllib</span><span class="o">.</span><span class="n">parse</span><span class="o">.</span><span class="n">urlencode</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
</pre></div> </pre></div>
</div> </div>
@@ -211,10 +211,10 @@ Application: Thank you, now give me all the data.</code></pre>
<div class="prompt input_prompt">In&nbsp;[6]:</div> <div class="prompt input_prompt">In&nbsp;[6]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># The `response_url` variable contains the full URL that</span> <div class=" highlight hl-ipython3"><pre><span class="c"># The `response_url` variable contains the full URL that</span>
<span class="c1"># FitBit sent back to us, but most importantly,</span> <span class="c"># FitBit sent back to us, but most importantly,</span>
<span class="c1"># contains the token we need for authorization.</span> <span class="c"># contains the token we need for authorization.</span>
<span class="n">access_token</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">urllib</span><span class="o">.</span><span class="n">parse</span><span class="o">.</span><span class="n">parse_qsl</span><span class="p">(</span><span class="n">response_url</span><span class="p">))[</span><span class="s1">&#39;access_token&#39;</span><span class="p">]</span> <span class="n">access_token</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">urllib</span><span class="o">.</span><span class="n">parse</span><span class="o">.</span><span class="n">parse_qsl</span><span class="p">(</span><span class="n">response_url</span><span class="p">))[</span><span class="s">&#39;access_token&#39;</span><span class="p">]</span>
</pre></div> </pre></div>
</div> </div>
@@ -238,46 +238,46 @@ Application: Thank you, now give me all the data.</code></pre>
<div class="prompt input_prompt">In&nbsp;[7]:</div> <div class="prompt input_prompt">In&nbsp;[7]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">requests_oauthlib</span> <span class="k">import</span> <span class="n">OAuth2Session</span> <div class=" highlight hl-ipython3"><pre><span class="kn">from</span> <span class="nn">requests_oauthlib</span> <span class="k">import</span> <span class="n">OAuth2Session</span>
<span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span> <span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
<span class="kn">from</span> <span class="nn">datetime</span> <span class="k">import</span> <span class="n">datetime</span> <span class="kn">from</span> <span class="nn">datetime</span> <span class="k">import</span> <span class="n">datetime</span>
<span class="n">session</span> <span class="o">=</span> <span class="n">OAuth2Session</span><span class="p">(</span><span class="n">token</span><span class="o">=</span><span class="p">{</span> <span class="n">session</span> <span class="o">=</span> <span class="n">OAuth2Session</span><span class="p">(</span><span class="n">token</span><span class="o">=</span><span class="p">{</span>
<span class="s1">&#39;access_token&#39;</span><span class="p">:</span> <span class="n">access_token</span><span class="p">,</span> <span class="s">&#39;access_token&#39;</span><span class="p">:</span> <span class="n">access_token</span><span class="p">,</span>
<span class="s1">&#39;token_type&#39;</span><span class="p">:</span> <span class="s1">&#39;Bearer&#39;</span> <span class="s">&#39;token_type&#39;</span><span class="p">:</span> <span class="s">&#39;Bearer&#39;</span>
<span class="p">})</span> <span class="p">})</span>
<span class="n">format_str</span> <span class="o">=</span> <span class="s1">&#39;%Y-%m-</span><span class="si">%d</span><span class="s1">&#39;</span> <span class="n">format_str</span> <span class="o">=</span> <span class="s">&#39;%Y-%m-%d&#39;</span>
<span class="n">start_date</span> <span class="o">=</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span> <span class="n">start_date</span> <span class="o">=</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="n">end_date</span> <span class="o">=</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">31</span><span class="p">)</span> <span class="n">end_date</span> <span class="o">=</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">31</span><span class="p">)</span>
<span class="n">dr</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">date_range</span><span class="p">(</span><span class="n">start_date</span><span class="p">,</span> <span class="n">end_date</span><span class="p">)</span> <span class="n">dr</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">date_range</span><span class="p">(</span><span class="n">start_date</span><span class="p">,</span> <span class="n">end_date</span><span class="p">)</span>
<span class="n">url</span> <span class="o">=</span> <span class="s1">&#39;https://api.fitbit.com/1/user/-/activities/heart/date/</span><span class="si">{0}</span><span class="s1">/1d/1min.json&#39;</span> <span class="n">url</span> <span class="o">=</span> <span class="s">&#39;https://api.fitbit.com/1/user/-/activities/heart/date/{0}/1d/1min.json&#39;</span>
<span class="n">hr_responses</span> <span class="o">=</span> <span class="p">[</span><span class="n">session</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">d</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="n">format_str</span><span class="p">)))</span> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">dr</span><span class="p">]</span> <span class="n">hr_responses</span> <span class="o">=</span> <span class="p">[</span><span class="n">session</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">d</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="n">format_str</span><span class="p">)))</span> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">dr</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">record_to_df</span><span class="p">(</span><span class="n">record</span><span class="p">):</span> <span class="k">def</span> <span class="nf">record_to_df</span><span class="p">(</span><span class="n">record</span><span class="p">):</span>
<span class="k">if</span> <span class="s1">&#39;activities-heart&#39;</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">record</span><span class="p">:</span> <span class="k">if</span> <span class="s">&#39;activities-heart&#39;</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">record</span><span class="p">:</span>
<span class="k">return</span> <span class="kc">None</span> <span class="k">return</span> <span class="k">None</span>
<span class="n">date_str</span> <span class="o">=</span> <span class="n">record</span><span class="p">[</span><span class="s1">&#39;activities-heart&#39;</span><span class="p">][</span><span class="mi">0</span><span class="p">][</span><span class="s1">&#39;dateTime&#39;</span><span class="p">]</span> <span class="n">date_str</span> <span class="o">=</span> <span class="n">record</span><span class="p">[</span><span class="s">&#39;activities-heart&#39;</span><span class="p">][</span><span class="mi">0</span><span class="p">][</span><span class="s">&#39;dateTime&#39;</span><span class="p">]</span>
<span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">record</span><span class="p">[</span><span class="s1">&#39;activities-heart-intraday&#39;</span><span class="p">][</span><span class="s1">&#39;dataset&#39;</span><span class="p">])</span> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">record</span><span class="p">[</span><span class="s">&#39;activities-heart-intraday&#39;</span><span class="p">][</span><span class="s">&#39;dataset&#39;</span><span class="p">])</span>
<span class="n">df</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="s1">&#39;time&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span> <span class="n">df</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="s">&#39;time&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span>
<span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">strptime</span><span class="p">(</span><span class="n">date_str</span> <span class="o">+</span> <span class="s1">&#39; &#39;</span> <span class="o">+</span> <span class="n">x</span><span class="p">,</span> <span class="s1">&#39;%Y-%m-</span><span class="si">%d</span><span class="s1"> %H:%M:%S&#39;</span><span class="p">))</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">strptime</span><span class="p">(</span><span class="n">date_str</span> <span class="o">+</span> <span class="s">&#39; &#39;</span> <span class="o">+</span> <span class="n">x</span><span class="p">,</span> <span class="s">&#39;%Y-%m-%d %H:%M:%S&#39;</span><span class="p">))</span>
<span class="k">return</span> <span class="n">df</span> <span class="k">return</span> <span class="n">df</span>
<span class="n">hr_dataframes</span> <span class="o">=</span> <span class="p">[</span><span class="n">record_to_df</span><span class="p">(</span><span class="n">record</span><span class="o">.</span><span class="n">json</span><span class="p">())</span> <span class="k">for</span> <span class="n">record</span> <span class="ow">in</span> <span class="n">hr_responses</span><span class="p">]</span> <span class="n">hr_dataframes</span> <span class="o">=</span> <span class="p">[</span><span class="n">record_to_df</span><span class="p">(</span><span class="n">record</span><span class="o">.</span><span class="n">json</span><span class="p">())</span> <span class="k">for</span> <span class="n">record</span> <span class="ow">in</span> <span class="n">hr_responses</span><span class="p">]</span>
<span class="n">hr_df_concat</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">concat</span><span class="p">(</span><span class="n">hr_dataframes</span><span class="p">)</span> <span class="n">hr_df_concat</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">concat</span><span class="p">(</span><span class="n">hr_dataframes</span><span class="p">)</span>
<span class="c1"># There are some minutes with missing data, so we need to correct that</span> <span class="c"># There are some minutes with missing data, so we need to correct that</span>
<span class="n">full_daterange</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">date_range</span><span class="p">(</span><span class="n">hr_df_concat</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">full_daterange</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">date_range</span><span class="p">(</span><span class="n">hr_df_concat</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span>
<span class="n">hr_df_concat</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span> <span class="n">hr_df_concat</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span>
<span class="n">freq</span><span class="o">=</span><span class="s1">&#39;min&#39;</span><span class="p">)</span> <span class="n">freq</span><span class="o">=</span><span class="s">&#39;min&#39;</span><span class="p">)</span>
<span class="n">hr_df_full</span> <span class="o">=</span> <span class="n">hr_df_concat</span><span class="o">.</span><span class="n">reindex</span><span class="p">(</span><span class="n">full_daterange</span><span class="p">,</span> <span class="n">method</span><span class="o">=</span><span class="s1">&#39;nearest&#39;</span><span class="p">)</span> <span class="n">hr_df_full</span> <span class="o">=</span> <span class="n">hr_df_concat</span><span class="o">.</span><span class="n">reindex</span><span class="p">(</span><span class="n">full_daterange</span><span class="p">,</span> <span class="n">method</span><span class="o">=</span><span class="s">&#39;nearest&#39;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Heartbeats from </span><span class="si">{}</span><span class="s2"> to </span><span class="si">{}</span><span class="s2">: </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="nb">print</span><span class="p">(</span><span class="s">&quot;Heartbeats from {} to {}: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span>
<span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span> <span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span>
<span class="n">hr_df_full</span><span class="p">[</span><span class="s1">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()))</span> <span class="n">hr_df_full</span><span class="p">[</span><span class="s">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()))</span>
</pre></div> </pre></div>
</div> </div>
@@ -325,9 +325,9 @@ hr_q \cdot n - hr_d \cdot (n-m)
<div class="prompt input_prompt">In&nbsp;[8]:</div> <div class="prompt input_prompt">In&nbsp;[8]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">quarterly_count</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[</span><span class="s1">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span> <div class=" highlight hl-ipython3"><pre><span class="n">quarterly_count</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[</span><span class="s">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
<span class="n">leap_day_count</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">month</span> <span class="o">==</span> <span class="mi">2</span><span class="p">)</span> <span class="o">&amp;</span> <span class="n">leap_day_count</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">month</span> <span class="o">==</span> <span class="mi">2</span><span class="p">)</span> <span class="o">&amp;</span>
<span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">day</span> <span class="o">==</span> <span class="mi">29</span><span class="p">)][</span><span class="s1">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span> <span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">day</span> <span class="o">==</span> <span class="mi">29</span><span class="p">)][</span><span class="s">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
<span class="n">num_quarters</span> <span class="o">=</span> <span class="mi">96</span> <span class="n">num_quarters</span> <span class="o">=</span> <span class="mi">96</span>
<span class="n">leap_days</span> <span class="o">=</span> <span class="mi">6</span> <span class="n">leap_days</span> <span class="o">=</span> <span class="mi">6</span>
@@ -372,17 +372,17 @@ hr_q \cdot n - hr_d \cdot (n-m)
<div class="prompt input_prompt">In&nbsp;[9]:</div> <div class="prompt input_prompt">In&nbsp;[9]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">minute_mean</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[</span><span class="s1">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">mean</span><span class="p">()</span> <div class=" highlight hl-ipython3"><pre><span class="n">minute_mean</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[</span><span class="s">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">mean</span><span class="p">()</span>
<span class="c1"># Don&#39;t you wish you knew?</span> <span class="c"># Don&#39;t you wish you knew?</span>
<span class="c1"># birthday_minutes = ???</span> <span class="c"># birthday_minutes = ???</span>
<span class="n">birthday_heartbeats</span> <span class="o">=</span> <span class="n">birthday_minutes</span> <span class="o">*</span> <span class="n">minute_mean</span> <span class="n">birthday_heartbeats</span> <span class="o">=</span> <span class="n">birthday_minutes</span> <span class="o">*</span> <span class="n">minute_mean</span>
<span class="n">heartbeats_until_2016</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">birthday_heartbeats</span> <span class="o">+</span> <span class="n">jan_92_jan_16</span><span class="p">)</span> <span class="n">heartbeats_until_2016</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">birthday_heartbeats</span> <span class="o">+</span> <span class="n">jan_92_jan_16</span><span class="p">)</span>
<span class="n">remaining_2016</span> <span class="o">=</span> <span class="n">total_heartbeats</span> <span class="o">-</span> <span class="n">heartbeats_until_2016</span> <span class="n">remaining_2016</span> <span class="o">=</span> <span class="n">total_heartbeats</span> <span class="o">-</span> <span class="n">heartbeats_until_2016</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Heartbeats so far: </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">heartbeats_until_2016</span><span class="p">))</span> <span class="nb">print</span><span class="p">(</span><span class="s">&quot;Heartbeats so far: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">heartbeats_until_2016</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Remaining heartbeats: </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_2016</span><span class="p">))</span> <span class="nb">print</span><span class="p">(</span><span class="s">&quot;Remaining heartbeats: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_2016</span><span class="p">))</span>
</pre></div> </pre></div>
</div> </div>
@@ -430,7 +430,7 @@ hr_q \cdot n - hr_d \cdot (n - m)
<div class="prompt input_prompt">In&nbsp;[10]:</div> <div class="prompt input_prompt">In&nbsp;[10]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">heartbeats_4year</span> <span class="o">=</span> <span class="n">quarterly_count</span> <span class="o">*</span> <span class="mi">16</span> <span class="o">-</span> <span class="n">leap_day_count</span> <span class="o">*</span> <span class="p">(</span><span class="mi">16</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <div class=" highlight hl-ipython3"><pre><span class="n">heartbeats_4year</span> <span class="o">=</span> <span class="n">quarterly_count</span> <span class="o">*</span> <span class="mi">16</span> <span class="o">-</span> <span class="n">leap_day_count</span> <span class="o">*</span> <span class="p">(</span><span class="mi">16</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span>
<span class="n">heartbeats_4year</span> <span class="n">heartbeats_4year</span>
</pre></div> </pre></div>
@@ -470,11 +470,11 @@ hr_q \cdot n - hr_d \cdot (n - m)
<div class="prompt input_prompt">In&nbsp;[11]:</div> <div class="prompt input_prompt">In&nbsp;[11]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">four_year_periods</span> <span class="o">=</span> <span class="n">remaining_2016</span> <span class="o">//</span> <span class="n">heartbeats_4year</span> <div class=" highlight hl-ipython3"><pre><span class="n">four_year_periods</span> <span class="o">=</span> <span class="n">remaining_2016</span> <span class="o">//</span> <span class="n">heartbeats_4year</span>
<span class="n">remaining_4y</span> <span class="o">=</span> <span class="n">remaining_2016</span> <span class="o">-</span> <span class="n">four_year_periods</span> <span class="o">*</span> <span class="n">heartbeats_4year</span> <span class="n">remaining_4y</span> <span class="o">=</span> <span class="n">remaining_2016</span> <span class="o">-</span> <span class="n">four_year_periods</span> <span class="o">*</span> <span class="n">heartbeats_4year</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Four year periods remaining: </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">four_year_periods</span><span class="p">))</span> <span class="nb">print</span><span class="p">(</span><span class="s">&quot;Four year periods remaining: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">four_year_periods</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Remaining heartbeats after 4 year periods: </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_4y</span><span class="p">))</span> <span class="nb">print</span><span class="p">(</span><span class="s">&quot;Remaining heartbeats after 4 year periods: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_4y</span><span class="p">))</span>
</pre></div> </pre></div>
</div> </div>
@@ -512,15 +512,15 @@ Remaining heartbeats after 4 year periods: 48041640
<div class="prompt input_prompt">In&nbsp;[12]:</div> <div class="prompt input_prompt">In&nbsp;[12]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">remaining_leap</span> <span class="o">=</span> <span class="n">remaining_4y</span> <span class="o">-</span> <span class="n">leap_day_count</span> <div class=" highlight hl-ipython3"><pre><span class="n">remaining_leap</span> <span class="o">=</span> <span class="n">remaining_4y</span> <span class="o">-</span> <span class="n">leap_day_count</span>
<span class="c1"># Ignore leap day in the data set</span> <span class="c"># Ignore leap day in the data set</span>
<span class="n">heartbeats_quarter</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">month</span> <span class="o">!=</span> <span class="mi">2</span><span class="p">)</span> <span class="o">&amp;</span> <span class="n">heartbeats_quarter</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">month</span> <span class="o">!=</span> <span class="mi">2</span><span class="p">)</span> <span class="o">&amp;</span>
<span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">day</span> <span class="o">!=</span> <span class="mi">29</span><span class="p">)][</span><span class="s1">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span> <span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">day</span> <span class="o">!=</span> <span class="mi">29</span><span class="p">)][</span><span class="s">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
<span class="n">quarters_left</span> <span class="o">=</span> <span class="n">remaining_leap</span> <span class="o">//</span> <span class="n">heartbeats_quarter</span> <span class="n">quarters_left</span> <span class="o">=</span> <span class="n">remaining_leap</span> <span class="o">//</span> <span class="n">heartbeats_quarter</span>
<span class="n">remaining_year</span> <span class="o">=</span> <span class="n">remaining_leap</span> <span class="o">-</span> <span class="n">quarters_left</span> <span class="o">*</span> <span class="n">heartbeats_quarter</span> <span class="n">remaining_year</span> <span class="o">=</span> <span class="n">remaining_leap</span> <span class="o">-</span> <span class="n">quarters_left</span> <span class="o">*</span> <span class="n">heartbeats_quarter</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Quarters left starting 2068: </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">quarters_left</span><span class="p">))</span> <span class="nb">print</span><span class="p">(</span><span class="s">&quot;Quarters left starting 2068: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">quarters_left</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Remaining heartbeats after that: </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_year</span><span class="p">))</span> <span class="nb">print</span><span class="p">(</span><span class="s">&quot;Remaining heartbeats after that: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_year</span><span class="p">))</span>
</pre></div> </pre></div>
</div> </div>
@@ -558,7 +558,7 @@ Remaining heartbeats after that: 4760716
<div class="prompt input_prompt">In&nbsp;[13]:</div> <div class="prompt input_prompt">In&nbsp;[13]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">datetime</span> <span class="k">import</span> <span class="n">timedelta</span> <div class=" highlight hl-ipython3"><pre><span class="kn">from</span> <span class="nn">datetime</span> <span class="k">import</span> <span class="n">timedelta</span>
<span class="n">base</span> <span class="o">=</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2070</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span> <span class="n">base</span> <span class="o">=</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2070</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="n">minutes_left</span> <span class="o">=</span> <span class="n">remaining_year</span> <span class="o">//</span> <span class="n">minute_mean</span> <span class="n">minutes_left</span> <span class="o">=</span> <span class="n">remaining_year</span> <span class="o">//</span> <span class="n">minute_mean</span>
tweet-like-me.html
+27 -27
View File
@@ -4,7 +4,7 @@
<meta charset="utf-8"> <meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="An experiment in creating a robot that will imitate me on Twitter. So, I&#39;m taking a Machine Learning course this semester in school, and one of the topics we keep coming back to is natural ..."> <meta name="description" content="An experiment in creating a robot that will imitate me on Twitter. So, I&#39;m taking a Machine Learning course this semester in school, and one of the topics we keep coming back to is natural language ...">
<meta name="keywords" content="MCMC, twitter"> <meta name="keywords" content="MCMC, twitter">
<link rel="icon" href="https://bspeice.github.io/favicon.ico"> <link rel="icon" href="https://bspeice.github.io/favicon.ico">
@@ -126,13 +126,13 @@
<div class="prompt input_prompt">In&nbsp;[1]:</div> <div class="prompt input_prompt">In&nbsp;[1]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span> <div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
<span class="n">tweets</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s1">&#39;tweets.csv&#39;</span><span class="p">)</span> <span class="n">tweets</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s">&#39;tweets.csv&#39;</span><span class="p">)</span>
<span class="n">text</span> <span class="o">=</span> <span class="n">tweets</span><span class="o">.</span><span class="n">text</span> <span class="n">text</span> <span class="o">=</span> <span class="n">tweets</span><span class="o">.</span><span class="n">text</span>
<span class="c1"># Don&#39;t include tweets in reply to or mentioning people</span> <span class="c"># Don&#39;t include tweets in reply to or mentioning people</span>
<span class="n">replies</span> <span class="o">=</span> <span class="n">text</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">contains</span><span class="p">(</span><span class="s1">&#39;@&#39;</span><span class="p">)</span> <span class="n">replies</span> <span class="o">=</span> <span class="n">text</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">contains</span><span class="p">(</span><span class="s">&#39;@&#39;</span><span class="p">)</span>
<span class="n">text_norep</span> <span class="o">=</span> <span class="n">text</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="o">~</span><span class="n">replies</span><span class="p">]</span> <span class="n">text_norep</span> <span class="o">=</span> <span class="n">text</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="o">~</span><span class="n">replies</span><span class="p">]</span>
</pre></div> </pre></div>
@@ -156,7 +156,7 @@
<div class="prompt input_prompt">In&nbsp;[2]:</div> <div class="prompt input_prompt">In&nbsp;[2]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">nltk.tokenize</span> <span class="k">import</span> <span class="n">TweetTokenizer</span> <div class=" highlight hl-ipython3"><pre><span class="kn">from</span> <span class="nn">nltk.tokenize</span> <span class="k">import</span> <span class="n">TweetTokenizer</span>
<span class="n">tknzr</span> <span class="o">=</span> <span class="n">TweetTokenizer</span><span class="p">()</span> <span class="n">tknzr</span> <span class="o">=</span> <span class="n">TweetTokenizer</span><span class="p">()</span>
<span class="n">tokens</span> <span class="o">=</span> <span class="n">text_norep</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">tknzr</span><span class="o">.</span><span class="n">tokenize</span><span class="p">)</span> <span class="n">tokens</span> <span class="o">=</span> <span class="n">text_norep</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">tknzr</span><span class="o">.</span><span class="n">tokenize</span><span class="p">)</span>
@@ -185,12 +185,12 @@
<div class="prompt input_prompt">In&nbsp;[3]:</div> <div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">functools</span> <span class="k">import</span> <span class="n">reduce</span> <div class=" highlight hl-ipython3"><pre><span class="kn">from</span> <span class="nn">functools</span> <span class="k">import</span> <span class="n">reduce</span>
<span class="c1"># Get all possible words</span> <span class="c"># Get all possible words</span>
<span class="n">all_words</span> <span class="o">=</span> <span class="n">reduce</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span> <span class="n">x</span><span class="o">+</span><span class="n">y</span><span class="p">,</span> <span class="n">tokens</span><span class="p">,</span> <span class="p">[])</span> <span class="n">all_words</span> <span class="o">=</span> <span class="n">reduce</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span> <span class="n">x</span><span class="o">+</span><span class="n">y</span><span class="p">,</span> <span class="n">tokens</span><span class="p">,</span> <span class="p">[])</span>
<span class="n">unique_words</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">all_words</span><span class="p">)</span> <span class="n">unique_words</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">all_words</span><span class="p">)</span>
<span class="n">actual_words</span> <span class="o">=</span> <span class="nb">set</span><span class="p">([</span><span class="n">x</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s1">&#39;.&#39;</span> <span class="k">else</span> <span class="kc">None</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">unique_words</span><span class="p">])</span> <span class="n">actual_words</span> <span class="o">=</span> <span class="nb">set</span><span class="p">([</span><span class="n">x</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s">&#39;.&#39;</span> <span class="k">else</span> <span class="k">None</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">unique_words</span><span class="p">])</span>
<span class="n">word_dist</span> <span class="o">=</span> <span class="p">{}</span> <span class="n">word_dist</span> <span class="o">=</span> <span class="p">{}</span>
<span class="k">for</span> <span class="n">word</span> <span class="ow">in</span> <span class="nb">iter</span><span class="p">(</span><span class="n">actual_words</span><span class="p">):</span> <span class="k">for</span> <span class="n">word</span> <span class="ow">in</span> <span class="nb">iter</span><span class="p">(</span><span class="n">actual_words</span><span class="p">):</span>
@@ -219,12 +219,12 @@
<div class="prompt input_prompt">In&nbsp;[4]:</div> <div class="prompt input_prompt">In&nbsp;[4]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span> <div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
<span class="o">%</span><span class="k">matplotlib</span> inline <span class="o">%</span><span class="k">matplotlib</span> inline
<span class="n">hashtags</span> <span class="o">=</span> <span class="n">text_norep</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">count</span><span class="p">(</span><span class="s1">&#39;#&#39;</span><span class="p">)</span> <span class="n">hashtags</span> <span class="o">=</span> <span class="n">text_norep</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">count</span><span class="p">(</span><span class="s">&#39;#&#39;</span><span class="p">)</span>
<span class="n">bins</span> <span class="o">=</span> <span class="n">hashtags</span><span class="o">.</span><span class="n">unique</span><span class="p">()</span><span class="o">.</span><span class="n">max</span><span class="p">()</span> <span class="n">bins</span> <span class="o">=</span> <span class="n">hashtags</span><span class="o">.</span><span class="n">unique</span><span class="p">()</span><span class="o">.</span><span class="n">max</span><span class="p">()</span>
<span class="n">hashtags</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s1">&#39;hist&#39;</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="n">bins</span><span class="p">)</span> <span class="n">hashtags</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s">&#39;hist&#39;</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="n">bins</span><span class="p">)</span>
</pre></div> </pre></div>
</div> </div>
@@ -356,7 +356,7 @@ AElFTkSuQmCC
<div class="prompt input_prompt">In&nbsp;[5]:</div> <div class="prompt input_prompt">In&nbsp;[5]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">mle</span> <span class="o">=</span> <span class="n">hashtags</span><span class="o">.</span><span class="n">mean</span><span class="p">()</span> <div class=" highlight hl-ipython3"><pre><span class="n">mle</span> <span class="o">=</span> <span class="n">hashtags</span><span class="o">.</span><span class="n">mean</span><span class="p">()</span>
<span class="n">mle</span> <span class="n">mle</span>
</pre></div> </pre></div>
@@ -396,12 +396,12 @@ AElFTkSuQmCC
<div class="prompt input_prompt">In&nbsp;[6]:</div> <div class="prompt input_prompt">In&nbsp;[6]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">hashtags</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">all_words</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s1">&#39;#&#39;</span><span class="p">]</span> <div class=" highlight hl-ipython3"><pre><span class="n">hashtags</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">all_words</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s">&#39;#&#39;</span><span class="p">]</span>
<span class="n">n_hashtags</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">hashtags</span><span class="p">)</span> <span class="n">n_hashtags</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">hashtags</span><span class="p">)</span>
<span class="n">unique_hashtags</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">([</span><span class="n">x</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">unique_words</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s1">&#39;#&#39;</span><span class="p">]))</span> <span class="n">unique_hashtags</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">([</span><span class="n">x</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">unique_words</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s">&#39;#&#39;</span><span class="p">]))</span>
<span class="n">hashtag_dist</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">({</span><span class="s1">&#39;hashtags&#39;</span><span class="p">:</span> <span class="n">unique_hashtags</span><span class="p">,</span> <span class="n">hashtag_dist</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">({</span><span class="s">&#39;hashtags&#39;</span><span class="p">:</span> <span class="n">unique_hashtags</span><span class="p">,</span>
<span class="s1">&#39;prob&#39;</span><span class="p">:</span> <span class="p">[</span><span class="n">all_words</span><span class="o">.</span><span class="n">count</span><span class="p">(</span><span class="n">h</span><span class="p">)</span> <span class="o">/</span> <span class="n">n_hashtags</span> <span class="s">&#39;prob&#39;</span><span class="p">:</span> <span class="p">[</span><span class="n">all_words</span><span class="o">.</span><span class="n">count</span><span class="p">(</span><span class="n">h</span><span class="p">)</span> <span class="o">/</span> <span class="n">n_hashtags</span>
<span class="k">for</span> <span class="n">h</span> <span class="ow">in</span> <span class="n">unique_hashtags</span><span class="p">]})</span> <span class="k">for</span> <span class="n">h</span> <span class="ow">in</span> <span class="n">unique_hashtags</span><span class="p">]})</span>
<span class="nb">len</span><span class="p">(</span><span class="n">hashtag_dist</span><span class="p">)</span> <span class="nb">len</span><span class="p">(</span><span class="n">hashtag_dist</span><span class="p">)</span>
</pre></div> </pre></div>
@@ -454,12 +454,12 @@ z: .17</code></pre>
<div class="prompt input_prompt">In&nbsp;[7]:</div> <div class="prompt input_prompt">In&nbsp;[7]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span> <div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="k">def</span> <span class="nf">multinom_sim</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">vals</span><span class="p">,</span> <span class="n">probs</span><span class="p">):</span> <span class="k">def</span> <span class="nf">multinom_sim</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">vals</span><span class="p">,</span> <span class="n">probs</span><span class="p">):</span>
<span class="n">occurrences</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">multinomial</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">probs</span><span class="p">)</span> <span class="n">occurrences</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">multinomial</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">probs</span><span class="p">)</span>
<span class="n">results</span> <span class="o">=</span> <span class="n">occurrences</span> <span class="o">*</span> <span class="n">vals</span> <span class="n">results</span> <span class="o">=</span> <span class="n">occurrences</span> <span class="o">*</span> <span class="n">vals</span>
<span class="k">return</span> <span class="s1">&#39; &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">results</span><span class="p">[</span><span class="n">results</span> <span class="o">!=</span> <span class="s1">&#39;&#39;</span><span class="p">])</span> <span class="k">return</span> <span class="s">&#39; &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">results</span><span class="p">[</span><span class="n">results</span> <span class="o">!=</span> <span class="s">&#39;&#39;</span><span class="p">])</span>
<span class="k">def</span> <span class="nf">sim_n_hashtags</span><span class="p">(</span><span class="n">hashtag_freq</span><span class="p">):</span> <span class="k">def</span> <span class="nf">sim_n_hashtags</span><span class="p">(</span><span class="n">hashtag_freq</span><span class="p">):</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">poisson</span><span class="p">(</span><span class="n">hashtag_freq</span><span class="p">)</span> <span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">poisson</span><span class="p">(</span><span class="n">hashtag_freq</span><span class="p">)</span>
@@ -469,7 +469,7 @@ z: .17</code></pre>
<span class="k">def</span> <span class="nf">sim_first_word</span><span class="p">(</span><span class="n">first_word_dist</span><span class="p">):</span> <span class="k">def</span> <span class="nf">sim_first_word</span><span class="p">(</span><span class="n">first_word_dist</span><span class="p">):</span>
<span class="n">probs</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">float64</span><span class="p">(</span><span class="n">first_word_dist</span><span class="o">.</span><span class="n">values</span><span class="p">)</span> <span class="n">probs</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">float64</span><span class="p">(</span><span class="n">first_word_dist</span><span class="o">.</span><span class="n">values</span><span class="p">)</span>
<span class="k">return</span> <span class="n">multinom_sim</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">first_word_dist</span><span class="o">.</span><span class="n">reset_index</span><span class="p">()[</span><span class="s1">&#39;index&#39;</span><span class="p">],</span> <span class="n">probs</span><span class="p">)</span> <span class="k">return</span> <span class="n">multinom_sim</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">first_word_dist</span><span class="o">.</span><span class="n">reset_index</span><span class="p">()[</span><span class="s">&#39;index&#39;</span><span class="p">],</span> <span class="n">probs</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">sim_next_word</span><span class="p">(</span><span class="n">current</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">):</span> <span class="k">def</span> <span class="nf">sim_next_word</span><span class="p">(</span><span class="n">current</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">):</span>
<span class="n">dist</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">word_dist</span><span class="p">[</span><span class="n">current</span><span class="p">])</span> <span class="n">dist</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">word_dist</span><span class="p">[</span><span class="n">current</span><span class="p">])</span>
@@ -497,14 +497,14 @@ z: .17</code></pre>
<div class="prompt input_prompt">In&nbsp;[8]:</div> <div class="prompt input_prompt">In&nbsp;[8]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">first</span> <span class="o">=</span> <span class="n">sim_first_word</span><span class="p">(</span><span class="n">first_word_dist</span><span class="p">)</span> <div class=" highlight hl-ipython3"><pre><span class="n">first</span> <span class="o">=</span> <span class="n">sim_first_word</span><span class="p">(</span><span class="n">first_word_dist</span><span class="p">)</span>
<span class="n">second</span> <span class="o">=</span> <span class="n">sim_next_word</span><span class="p">(</span><span class="n">first</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">)</span> <span class="n">second</span> <span class="o">=</span> <span class="n">sim_next_word</span><span class="p">(</span><span class="n">first</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">)</span>
<span class="n">third</span> <span class="o">=</span> <span class="n">sim_next_word</span><span class="p">(</span><span class="n">second</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">)</span> <span class="n">third</span> <span class="o">=</span> <span class="n">sim_next_word</span><span class="p">(</span><span class="n">second</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">)</span>
<span class="n">fourth</span> <span class="o">=</span> <span class="n">sim_next_word</span><span class="p">(</span><span class="n">third</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">)</span> <span class="n">fourth</span> <span class="o">=</span> <span class="n">sim_next_word</span><span class="p">(</span><span class="n">third</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">)</span>
<span class="n">fifth</span> <span class="o">=</span> <span class="n">sim_next_word</span><span class="p">(</span><span class="n">fourth</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">)</span> <span class="n">fifth</span> <span class="o">=</span> <span class="n">sim_next_word</span><span class="p">(</span><span class="n">fourth</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">)</span>
<span class="n">hashtag</span> <span class="o">=</span> <span class="n">sim_hashtags</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">hashtag_dist</span><span class="p">)</span> <span class="n">hashtag</span> <span class="o">=</span> <span class="n">sim_hashtags</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">hashtag_dist</span><span class="p">)</span>
<span class="s1">&#39; &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">((</span><span class="n">first</span><span class="p">,</span> <span class="n">second</span><span class="p">,</span> <span class="n">third</span><span class="p">,</span> <span class="n">fourth</span><span class="p">,</span> <span class="n">fifth</span><span class="p">,</span> <span class="n">hashtag</span><span class="p">))</span> <span class="s">&#39; &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">((</span><span class="n">first</span><span class="p">,</span> <span class="n">second</span><span class="p">,</span> <span class="n">third</span><span class="p">,</span> <span class="n">fourth</span><span class="p">,</span> <span class="n">fifth</span><span class="p">,</span> <span class="n">hashtag</span><span class="p">))</span>
</pre></div> </pre></div>
</div> </div>
@@ -543,7 +543,7 @@ z: .17</code></pre>
<div class="prompt input_prompt">In&nbsp;[9]:</div> <div class="prompt input_prompt">In&nbsp;[9]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">simulate_tweet</span><span class="p">():</span> <div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">simulate_tweet</span><span class="p">():</span>
<span class="n">chars_remaining</span> <span class="o">=</span> <span class="mi">140</span> <span class="n">chars_remaining</span> <span class="o">=</span> <span class="mi">140</span>
<span class="n">first</span> <span class="o">=</span> <span class="n">sim_first_word</span><span class="p">(</span><span class="n">first_word_dist</span><span class="p">)</span> <span class="n">first</span> <span class="o">=</span> <span class="n">sim_first_word</span><span class="p">(</span><span class="n">first_word_dist</span><span class="p">)</span>
<span class="n">n_hash</span> <span class="o">=</span> <span class="n">sim_n_hashtags</span><span class="p">(</span><span class="n">mle</span><span class="p">)</span> <span class="n">n_hash</span> <span class="o">=</span> <span class="n">sim_n_hashtags</span><span class="p">(</span><span class="n">mle</span><span class="p">)</span>
@@ -553,13 +553,13 @@ z: .17</code></pre>
<span class="n">tweet</span> <span class="o">=</span> <span class="n">first</span> <span class="n">tweet</span> <span class="o">=</span> <span class="n">first</span>
<span class="n">current</span> <span class="o">=</span> <span class="n">first</span> <span class="n">current</span> <span class="o">=</span> <span class="n">first</span>
<span class="k">while</span> <span class="n">chars_remaining</span> <span class="o">&gt;</span> <span class="nb">len</span><span class="p">(</span><span class="n">tweet</span><span class="p">)</span> <span class="o">+</span> <span class="nb">len</span><span class="p">(</span><span class="n">hashtags</span><span class="p">)</span> <span class="ow">and</span> <span class="n">current</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s1">&#39;.&#39;</span> <span class="ow">and</span> <span class="n">current</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s1">&#39;!&#39;</span><span class="p">:</span> <span class="k">while</span> <span class="n">chars_remaining</span> <span class="o">&gt;</span> <span class="nb">len</span><span class="p">(</span><span class="n">tweet</span><span class="p">)</span> <span class="o">+</span> <span class="nb">len</span><span class="p">(</span><span class="n">hashtags</span><span class="p">)</span> <span class="ow">and</span> <span class="n">current</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s">&#39;.&#39;</span> <span class="ow">and</span> <span class="n">current</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s">&#39;!&#39;</span><span class="p">:</span>
<span class="n">current</span> <span class="o">=</span> <span class="n">sim_next_word</span><span class="p">(</span><span class="n">current</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">)</span> <span class="n">current</span> <span class="o">=</span> <span class="n">sim_next_word</span><span class="p">(</span><span class="n">current</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">)</span>
<span class="n">tweet</span> <span class="o">+=</span> <span class="s1">&#39; &#39;</span> <span class="o">+</span> <span class="n">current</span> <span class="n">tweet</span> <span class="o">+=</span> <span class="s">&#39; &#39;</span> <span class="o">+</span> <span class="n">current</span>
<span class="n">tweet</span> <span class="o">=</span> <span class="n">tweet</span><span class="p">[:</span><span class="o">-</span><span class="mi">2</span><span class="p">]</span> <span class="o">+</span> <span class="n">tweet</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span> <span class="n">tweet</span> <span class="o">=</span> <span class="n">tweet</span><span class="p">[:</span><span class="o">-</span><span class="mi">2</span><span class="p">]</span> <span class="o">+</span> <span class="n">tweet</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
<span class="k">return</span> <span class="s1">&#39; &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">((</span><span class="n">tweet</span><span class="p">,</span> <span class="n">hashtags</span><span class="p">))</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span> <span class="k">return</span> <span class="s">&#39; &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">((</span><span class="n">tweet</span><span class="p">,</span> <span class="n">hashtags</span><span class="p">))</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
</pre></div> </pre></div>
</div> </div>
@@ -582,7 +582,7 @@ z: .17</code></pre>
<div class="prompt input_prompt">In&nbsp;[12]:</div> <div class="prompt input_prompt">In&nbsp;[12]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">20</span><span class="p">):</span> <div class=" highlight hl-ipython3"><pre><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">20</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="n">simulate_tweet</span><span class="p">())</span> <span class="nb">print</span><span class="p">(</span><span class="n">simulate_tweet</span><span class="p">())</span>
<span class="nb">print</span><span class="p">()</span> <span class="nb">print</span><span class="p">()</span>
</pre></div> </pre></div>
welcome-and-an-algorithm.html
+35 -35
View File
@@ -4,7 +4,7 @@
<meta charset="utf-8"> <meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="Hello! Glad to meet you. I&#39;m currently a student at Columbia University studying Financial Engineering, and want to give an overview of the projects I&#39;m working on! To start things off, Columbia ..."> <meta name="description" content="Hello! Glad to meet you. I&#39;m currently a student at Columbia University studying Financial Engineering, and want to give an overview of the projects I&#39;m working on! To start things off, Columbia has ...">
<meta name="keywords" content="introduction, trading"> <meta name="keywords" content="introduction, trading">
<link rel="icon" href="https://bspeice.github.io/favicon.ico"> <link rel="icon" href="https://bspeice.github.io/favicon.ico">
@@ -157,16 +157,16 @@ to make some tweaks over the coming weeks, and do another forward test in Januar
<div class="prompt input_prompt">In&nbsp;[1]:</div> <div class="prompt input_prompt">In&nbsp;[1]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">IPython.display</span> <span class="k">import</span> <span class="n">display</span> <div class=" highlight hl-ipython3"><pre><span class="kn">from</span> <span class="nn">IPython.display</span> <span class="k">import</span> <span class="n">display</span>
<span class="kn">import</span> <span class="nn">Quandl</span> <span class="kn">import</span> <span class="nn">Quandl</span>
<span class="kn">from</span> <span class="nn">datetime</span> <span class="k">import</span> <span class="n">datetime</span><span class="p">,</span> <span class="n">timedelta</span> <span class="kn">from</span> <span class="nn">datetime</span> <span class="k">import</span> <span class="n">datetime</span><span class="p">,</span> <span class="n">timedelta</span>
<span class="n">tickers</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;XOM&#39;</span><span class="p">,</span> <span class="s1">&#39;CVX&#39;</span><span class="p">,</span> <span class="s1">&#39;CLB&#39;</span><span class="p">,</span> <span class="s1">&#39;OXY&#39;</span><span class="p">,</span> <span class="s1">&#39;SLB&#39;</span><span class="p">]</span> <span class="n">tickers</span> <span class="o">=</span> <span class="p">[</span><span class="s">&#39;XOM&#39;</span><span class="p">,</span> <span class="s">&#39;CVX&#39;</span><span class="p">,</span> <span class="s">&#39;CLB&#39;</span><span class="p">,</span> <span class="s">&#39;OXY&#39;</span><span class="p">,</span> <span class="s">&#39;SLB&#39;</span><span class="p">]</span>
<span class="n">market_ticker</span> <span class="o">=</span> <span class="s1">&#39;GOOG/NYSE_VOO&#39;</span> <span class="n">market_ticker</span> <span class="o">=</span> <span class="s">&#39;GOOG/NYSE_VOO&#39;</span>
<span class="n">lookback</span> <span class="o">=</span> <span class="mi">30</span> <span class="n">lookback</span> <span class="o">=</span> <span class="mi">30</span>
<span class="n">d_col</span> <span class="o">=</span> <span class="s1">&#39;Close&#39;</span> <span class="n">d_col</span> <span class="o">=</span> <span class="s">&#39;Close&#39;</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">{</span><span class="n">tick</span><span class="p">:</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">&#39;YAHOO/</span><span class="si">{}</span><span class="s1">&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">tick</span><span class="p">))[</span><span class="o">-</span><span class="n">lookback</span><span class="p">:]</span> <span class="k">for</span> <span class="n">tick</span> <span class="ow">in</span> <span class="n">tickers</span><span class="p">}</span> <span class="n">data</span> <span class="o">=</span> <span class="p">{</span><span class="n">tick</span><span class="p">:</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">&#39;YAHOO/{}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">tick</span><span class="p">))[</span><span class="o">-</span><span class="n">lookback</span><span class="p">:]</span> <span class="k">for</span> <span class="n">tick</span> <span class="ow">in</span> <span class="n">tickers</span><span class="p">}</span>
<span class="n">market</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">market_ticker</span><span class="p">)</span> <span class="n">market</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">market_ticker</span><span class="p">)</span>
</pre></div> </pre></div>
@@ -190,7 +190,7 @@ to make some tweaks over the coming weeks, and do another forward test in Januar
<div class="prompt input_prompt">In&nbsp;[2]:</div> <div class="prompt input_prompt">In&nbsp;[2]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">returns</span> <span class="o">=</span> <span class="p">{</span><span class="n">tick</span><span class="p">:</span> <span class="n">data</span><span class="p">[</span><span class="n">tick</span><span class="p">][</span><span class="n">d_col</span><span class="p">]</span><span class="o">.</span><span class="n">pct_change</span><span class="p">()</span> <span class="k">for</span> <span class="n">tick</span> <span class="ow">in</span> <span class="n">tickers</span><span class="p">}</span> <div class=" highlight hl-ipython3"><pre><span class="n">returns</span> <span class="o">=</span> <span class="p">{</span><span class="n">tick</span><span class="p">:</span> <span class="n">data</span><span class="p">[</span><span class="n">tick</span><span class="p">][</span><span class="n">d_col</span><span class="p">]</span><span class="o">.</span><span class="n">pct_change</span><span class="p">()</span> <span class="k">for</span> <span class="n">tick</span> <span class="ow">in</span> <span class="n">tickers</span><span class="p">}</span>
<span class="n">display</span><span class="p">({</span><span class="n">tick</span><span class="p">:</span> <span class="n">returns</span><span class="p">[</span><span class="n">tick</span><span class="p">]</span><span class="o">.</span><span class="n">mean</span><span class="p">()</span> <span class="k">for</span> <span class="n">tick</span> <span class="ow">in</span> <span class="n">tickers</span><span class="p">})</span> <span class="n">display</span><span class="p">({</span><span class="n">tick</span><span class="p">:</span> <span class="n">returns</span><span class="p">[</span><span class="n">tick</span><span class="p">]</span><span class="o">.</span><span class="n">mean</span><span class="p">()</span> <span class="k">for</span> <span class="n">tick</span> <span class="ow">in</span> <span class="n">tickers</span><span class="p">})</span>
</pre></div> </pre></div>
@@ -236,7 +236,7 @@ to make some tweaks over the coming weeks, and do another forward test in Januar
<div class="prompt input_prompt">In&nbsp;[3]:</div> <div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">market_returns</span> <span class="o">=</span> <span class="n">market</span><span class="o">.</span><span class="n">pct_change</span><span class="p">()</span> <div class=" highlight hl-ipython3"><pre><span class="n">market_returns</span> <span class="o">=</span> <span class="n">market</span><span class="o">.</span><span class="n">pct_change</span><span class="p">()</span>
<span class="n">sharpe</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">ret</span><span class="p">:</span> <span class="p">(</span><span class="n">ret</span><span class="o">.</span><span class="n">mean</span><span class="p">()</span> <span class="o">-</span> <span class="n">market_returns</span><span class="p">[</span><span class="n">d_col</span><span class="p">]</span><span class="o">.</span><span class="n">mean</span><span class="p">())</span> <span class="o">/</span> <span class="n">ret</span><span class="o">.</span><span class="n">std</span><span class="p">()</span> <span class="n">sharpe</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">ret</span><span class="p">:</span> <span class="p">(</span><span class="n">ret</span><span class="o">.</span><span class="n">mean</span><span class="p">()</span> <span class="o">-</span> <span class="n">market_returns</span><span class="p">[</span><span class="n">d_col</span><span class="p">]</span><span class="o">.</span><span class="n">mean</span><span class="p">())</span> <span class="o">/</span> <span class="n">ret</span><span class="o">.</span><span class="n">std</span><span class="p">()</span>
<span class="n">sharpes</span> <span class="o">=</span> <span class="p">{</span><span class="n">tick</span><span class="p">:</span> <span class="n">sharpe</span><span class="p">(</span><span class="n">returns</span><span class="p">[</span><span class="n">tick</span><span class="p">])</span> <span class="k">for</span> <span class="n">tick</span> <span class="ow">in</span> <span class="n">tickers</span><span class="p">}</span> <span class="n">sharpes</span> <span class="o">=</span> <span class="p">{</span><span class="n">tick</span><span class="p">:</span> <span class="n">sharpe</span><span class="p">(</span><span class="n">returns</span><span class="p">[</span><span class="n">tick</span><span class="p">])</span> <span class="k">for</span> <span class="n">tick</span> <span class="ow">in</span> <span class="n">tickers</span><span class="p">}</span>
@@ -284,7 +284,7 @@ to make some tweaks over the coming weeks, and do another forward test in Januar
<div class="prompt input_prompt">In&nbsp;[4]:</div> <div class="prompt input_prompt">In&nbsp;[4]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">drawdown</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">ret</span><span class="p">:</span> <span class="n">ret</span><span class="o">.</span><span class="n">abs</span><span class="p">()</span><span class="o">.</span><span class="n">max</span><span class="p">()</span> <div class=" highlight hl-ipython3"><pre><span class="n">drawdown</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">ret</span><span class="p">:</span> <span class="n">ret</span><span class="o">.</span><span class="n">abs</span><span class="p">()</span><span class="o">.</span><span class="n">max</span><span class="p">()</span>
<span class="n">drawdowns</span> <span class="o">=</span> <span class="p">{</span><span class="n">tick</span><span class="p">:</span> <span class="n">drawdown</span><span class="p">(</span><span class="n">returns</span><span class="p">[</span><span class="n">tick</span><span class="p">])</span> <span class="k">for</span> <span class="n">tick</span> <span class="ow">in</span> <span class="n">tickers</span><span class="p">}</span> <span class="n">drawdowns</span> <span class="o">=</span> <span class="p">{</span><span class="n">tick</span><span class="p">:</span> <span class="n">drawdown</span><span class="p">(</span><span class="n">returns</span><span class="p">[</span><span class="n">tick</span><span class="p">])</span> <span class="k">for</span> <span class="n">tick</span> <span class="ow">in</span> <span class="n">tickers</span><span class="p">}</span>
<span class="n">display</span><span class="p">(</span><span class="n">drawdowns</span><span class="p">)</span> <span class="n">display</span><span class="p">(</span><span class="n">drawdowns</span><span class="p">)</span>
@@ -337,25 +337,25 @@ s.t.\ \ & \vec{1} \omega = 1\\
<div class="prompt input_prompt">In&nbsp;[5]:</div> <div class="prompt input_prompt">In&nbsp;[5]:</div>
<div class="inner_cell"> <div class="inner_cell">
<div class="input_area"> <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span> <div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">from</span> <span class="nn">scipy.optimize</span> <span class="k">import</span> <span class="n">minimize</span> <span class="kn">from</span> <span class="nn">scipy.optimize</span> <span class="k">import</span> <span class="n">minimize</span>
<span class="c1">#sharpe_limit = .1</span> <span class="c">#sharpe_limit = .1</span>
<span class="n">drawdown_limit</span> <span class="o">=</span> <span class="o">.</span><span class="mi">05</span> <span class="n">drawdown_limit</span> <span class="o">=</span> <span class="o">.</span><span class="mi">05</span>
<span class="n">leverage</span> <span class="o">=</span> <span class="mi">250</span> <span class="n">leverage</span> <span class="o">=</span> <span class="mi">250</span>
<span class="c1"># Use the map so we can guarantee we maintain the correct order</span> <span class="c"># Use the map so we can guarantee we maintain the correct order</span>
<span class="c1"># sharpe_a = np.array(list(map(lambda tick: sharpes[tick], tickers))) * -1 # So we can write as upper-bound</span> <span class="c"># sharpe_a = np.array(list(map(lambda tick: sharpes[tick], tickers))) * -1 # So we can write as upper-bound</span>
<span class="n">dd_a</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">tick</span><span class="p">:</span> <span class="n">drawdowns</span><span class="p">[</span><span class="n">tick</span><span class="p">],</span> <span class="n">tickers</span><span class="p">)))</span> <span class="n">dd_a</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">tick</span><span class="p">:</span> <span class="n">drawdowns</span><span class="p">[</span><span class="n">tick</span><span class="p">],</span> <span class="n">tickers</span><span class="p">)))</span>
<span class="n">returns_a</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">tick</span><span class="p">:</span> <span class="n">returns</span><span class="p">[</span><span class="n">tick</span><span class="p">]</span><span class="o">.</span><span class="n">mean</span><span class="p">(),</span> <span class="n">tickers</span><span class="p">)))</span> <span class="c1"># Because minimizing</span> <span class="n">returns_a</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">tick</span><span class="p">:</span> <span class="n">returns</span><span class="p">[</span><span class="n">tick</span><span class="p">]</span><span class="o">.</span><span class="n">mean</span><span class="p">(),</span> <span class="n">tickers</span><span class="p">)))</span> <span class="c"># Because minimizing</span>
<span class="n">meets_sharpe</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="o">*</span> <span class="n">sharpe_a</span><span class="p">)</span> <span class="o">-</span> <span class="n">sharpe_limit</span> <span class="n">meets_sharpe</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="o">*</span> <span class="n">sharpe_a</span><span class="p">)</span> <span class="o">-</span> <span class="n">sharpe_limit</span>
<span class="k">def</span> <span class="nf">meets_dd</span><span class="p">(</span><span class="n">x</span><span class="p">):</span> <span class="k">def</span> <span class="nf">meets_dd</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="n">portfolio</span> <span class="o">=</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">))</span> <span class="n">portfolio</span> <span class="o">=</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">))</span>
<span class="k">if</span> <span class="n">portfolio</span> <span class="o">&lt;</span> <span class="o">.</span><span class="mi">1</span><span class="p">:</span> <span class="k">if</span> <span class="n">portfolio</span> <span class="o">&lt;</span> <span class="o">.</span><span class="mi">1</span><span class="p">:</span>
<span class="c1"># If there are no stocks in the portfolio,</span> <span class="c"># If there are no stocks in the portfolio,</span>
<span class="c1"># we can accidentally induce division by 0,</span> <span class="c"># we can accidentally induce division by 0,</span>
<span class="c1"># or division by something small enough to cause infinity</span> <span class="c"># or division by something small enough to cause infinity</span>
<span class="k">return</span> <span class="mi">0</span> <span class="k">return</span> <span class="mi">0</span>
<span class="k">return</span> <span class="n">drawdown_limit</span> <span class="o">-</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="o">*</span> <span class="n">dd_a</span><span class="p">)</span> <span class="o">/</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">))</span> <span class="k">return</span> <span class="n">drawdown_limit</span> <span class="o">-</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="o">*</span> <span class="n">dd_a</span><span class="p">)</span> <span class="o">/</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">))</span>
@@ -365,41 +365,41 @@ s.t.\ \ & \vec{1} \omega = 1\\
<span class="k">def</span> <span class="nf">within_leverage</span><span class="p">(</span><span class="n">x</span><span class="p">):</span> <span class="k">def</span> <span class="nf">within_leverage</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">leverage</span> <span class="o">-</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">))</span> <span class="k">return</span> <span class="n">leverage</span> <span class="o">-</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">))</span>
<span class="n">objective</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">sum</span><span class="p">(</span><span class="n">x</span> <span class="o">*</span> <span class="n">returns_a</span><span class="p">)</span> <span class="o">*</span> <span class="o">-</span><span class="mi">1</span> <span class="c1"># Because we&#39;re minimizing</span> <span class="n">objective</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">sum</span><span class="p">(</span><span class="n">x</span> <span class="o">*</span> <span class="n">returns_a</span><span class="p">)</span> <span class="o">*</span> <span class="o">-</span><span class="mi">1</span> <span class="c"># Because we&#39;re minimizing</span>
<span class="n">bounds</span> <span class="o">=</span> <span class="p">((</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">),)</span> <span class="o">*</span> <span class="nb">len</span><span class="p">(</span><span class="n">tickers</span><span class="p">)</span> <span class="n">bounds</span> <span class="o">=</span> <span class="p">((</span><span class="k">None</span><span class="p">,</span> <span class="k">None</span><span class="p">),)</span> <span class="o">*</span> <span class="nb">len</span><span class="p">(</span><span class="n">tickers</span><span class="p">)</span>
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">tickers</span><span class="p">))</span> <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">tickers</span><span class="p">))</span>
<span class="n">constraints</span> <span class="o">=</span> <span class="p">[</span> <span class="n">constraints</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">{</span> <span class="p">{</span>
<span class="s1">&#39;type&#39;</span><span class="p">:</span> <span class="s1">&#39;eq&#39;</span><span class="p">,</span> <span class="s">&#39;type&#39;</span><span class="p">:</span> <span class="s">&#39;eq&#39;</span><span class="p">,</span>
<span class="s1">&#39;fun&#39;</span><span class="p">:</span> <span class="n">is_portfolio</span> <span class="s">&#39;fun&#39;</span><span class="p">:</span> <span class="n">is_portfolio</span>
<span class="p">},</span> <span class="p">{</span> <span class="p">},</span> <span class="p">{</span>
<span class="s1">&#39;type&#39;</span><span class="p">:</span> <span class="s1">&#39;ineq&#39;</span><span class="p">,</span> <span class="s">&#39;type&#39;</span><span class="p">:</span> <span class="s">&#39;ineq&#39;</span><span class="p">,</span>
<span class="s1">&#39;fun&#39;</span><span class="p">:</span> <span class="n">within_leverage</span> <span class="s">&#39;fun&#39;</span><span class="p">:</span> <span class="n">within_leverage</span>
<span class="c1">#}, {</span> <span class="c">#}, {</span>
<span class="c1"># &#39;type&#39;: &#39;ineq&#39;,</span> <span class="c"># &#39;type&#39;: &#39;ineq&#39;,</span>
<span class="c1"># &#39;fun&#39;: meets_sharpe</span> <span class="c"># &#39;fun&#39;: meets_sharpe</span>
<span class="p">},</span> <span class="p">{</span> <span class="p">},</span> <span class="p">{</span>
<span class="s1">&#39;type&#39;</span><span class="p">:</span> <span class="s1">&#39;ineq&#39;</span><span class="p">,</span> <span class="s">&#39;type&#39;</span><span class="p">:</span> <span class="s">&#39;ineq&#39;</span><span class="p">,</span>
<span class="s1">&#39;fun&#39;</span><span class="p">:</span> <span class="n">meets_dd</span> <span class="s">&#39;fun&#39;</span><span class="p">:</span> <span class="n">meets_dd</span>
<span class="p">}</span> <span class="p">}</span>
<span class="p">]</span> <span class="p">]</span>
<span class="n">optimal</span> <span class="o">=</span> <span class="n">minimize</span><span class="p">(</span><span class="n">objective</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">bounds</span><span class="o">=</span><span class="n">bounds</span><span class="p">,</span> <span class="n">constraints</span><span class="o">=</span><span class="n">constraints</span><span class="p">,</span> <span class="n">optimal</span> <span class="o">=</span> <span class="n">minimize</span><span class="p">(</span><span class="n">objective</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">bounds</span><span class="o">=</span><span class="n">bounds</span><span class="p">,</span> <span class="n">constraints</span><span class="o">=</span><span class="n">constraints</span><span class="p">,</span>
<span class="n">options</span><span class="o">=</span><span class="p">{</span><span class="s1">&#39;maxiter&#39;</span><span class="p">:</span> <span class="mi">500</span><span class="p">})</span> <span class="n">options</span><span class="o">=</span><span class="p">{</span><span class="s">&#39;maxiter&#39;</span><span class="p">:</span> <span class="mi">500</span><span class="p">})</span>
<span class="c1"># Optimization time!</span> <span class="c"># Optimization time!</span>
<span class="n">display</span><span class="p">(</span><span class="n">optimal</span><span class="o">.</span><span class="n">message</span><span class="p">)</span> <span class="n">display</span><span class="p">(</span><span class="n">optimal</span><span class="o">.</span><span class="n">message</span><span class="p">)</span>
<span class="n">display</span><span class="p">(</span><span class="s2">&quot;Holdings: </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">zip</span><span class="p">(</span><span class="n">tickers</span><span class="p">,</span> <span class="n">optimal</span><span class="o">.</span><span class="n">x</span><span class="p">))))</span> <span class="n">display</span><span class="p">(</span><span class="s">&quot;Holdings: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">zip</span><span class="p">(</span><span class="n">tickers</span><span class="p">,</span> <span class="n">optimal</span><span class="o">.</span><span class="n">x</span><span class="p">))))</span>
<span class="n">expected_return</span> <span class="o">=</span> <span class="n">optimal</span><span class="o">.</span><span class="n">fun</span> <span class="o">*</span> <span class="o">-</span><span class="mi">100</span> <span class="c1"># multiply by -100 to scale, and compensate for minimizing</span> <span class="n">expected_return</span> <span class="o">=</span> <span class="n">optimal</span><span class="o">.</span><span class="n">fun</span> <span class="o">*</span> <span class="o">-</span><span class="mi">100</span> <span class="c"># multiply by -100 to scale, and compensate for minimizing</span>
<span class="n">display</span><span class="p">(</span><span class="s2">&quot;Expected Return: </span><span class="si">{:.3f}</span><span class="s2">%&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">expected_return</span><span class="p">))</span> <span class="n">display</span><span class="p">(</span><span class="s">&quot;Expected Return: {:.3f}%&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">expected_return</span><span class="p">))</span>
<span class="n">expected_drawdown</span> <span class="o">=</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">optimal</span><span class="o">.</span><span class="n">x</span><span class="p">)</span> <span class="o">*</span> <span class="n">dd_a</span><span class="p">)</span> <span class="o">/</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">optimal</span><span class="o">.</span><span class="n">x</span><span class="p">))</span> <span class="o">*</span> <span class="mi">100</span> <span class="n">expected_drawdown</span> <span class="o">=</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">optimal</span><span class="o">.</span><span class="n">x</span><span class="p">)</span> <span class="o">*</span> <span class="n">dd_a</span><span class="p">)</span> <span class="o">/</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">optimal</span><span class="o">.</span><span class="n">x</span><span class="p">))</span> <span class="o">*</span> <span class="mi">100</span>
<span class="n">display</span><span class="p">(</span><span class="s2">&quot;Expected Max Drawdown: </span><span class="si">{0:.2f}</span><span class="s2">%&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">expected_drawdown</span><span class="p">))</span> <span class="n">display</span><span class="p">(</span><span class="s">&quot;Expected Max Drawdown: {0:.2f}%&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">expected_drawdown</span><span class="p">))</span>
<span class="c1"># TODO: Calculate expected Sharpe</span> <span class="c"># TODO: Calculate expected Sharpe</span>
</pre></div> </pre></div>
</div> </div>