Add a new Rust article!

This commit is contained in:
Bradlee Speice 2016-10-22 22:29:18 -04:00
parent f9e8c08491
commit 6910e70b66
26 changed files with 2728 additions and 1520 deletions

View File

@ -0,0 +1,408 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="I listen to a lot of Drum and Bass music, because it&#39;s beautiful music. And there&#39;s a particular site, Bassdrive.com that hosts a lot of great content. Specifically, the archives section of the ...">
<meta name="keywords" content="nutone, Rust">
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
<title>A Rustic Re-Podcasting Server (Part 1) - Bradlee Speice</title>
<!-- Stylesheets -->
<link href="https://bspeice.github.io/theme/css/bootstrap.min.css" rel="stylesheet">
<link href="https://bspeice.github.io/theme/css/fonts.css" rel="stylesheet">
<link href="https://bspeice.github.io/theme/css/nest.css" rel="stylesheet">
<link href="https://bspeice.github.io/theme/css/pygment.css" rel="stylesheet">
<!-- /Stylesheets -->
<!-- RSS Feeds -->
<link href="https://bspeice.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate" title="Bradlee Speice Full Atom Feed" />
<link href="https://bspeice.github.io/feeds/blog.atom.xml" type="application/atom+xml" rel="alternate" title="Bradlee Speice Categories Atom Feed" />
<!-- /RSS Feeds -->
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-74711362-1', 'auto');
ga('send', 'pageview');
</script>
<!-- /Google Analytics -->
</head>
<body>
<!-- Header -->
<div class="header-container gradient">
<!-- Static navbar -->
<div class="container">
<div class="header-nav">
<div class="header-logo">
<a class="pull-left" href="https://bspeice.github.io/"><img class="mr20" src="https://bspeice.github.io/images/logo.svg" alt="logo">Bradlee Speice</a>
</div>
<div class="nav pull-right">
</div>
</div>
</div>
<!-- /Static navbar -->
<!-- Header -->
<!-- Header -->
<div class="container header-wrapper">
<div class="row">
<div class="col-lg-12">
<div class="header-content">
<h1 class="header-title">A Rustic Re-Podcasting Server (Part 1)</h1>
<p class="header-date"> <a href="https://bspeice.github.io/author/bradlee-speice.html">Bradlee Speice</a>, Sat 22 October 2016, <a href="https://bspeice.github.io/category/blog.html">Blog</a></p>
<div class="header-underline"></div>
<div class="clearfix"></div>
<p class="pull-right header-tags">
<span class="glyphicon glyphicon-tags mr5" aria-hidden="true"></span>
<a href="https://bspeice.github.io/tag/nutone.html">nutone</a>, <a href="https://bspeice.github.io/tag/rust.html">Rust</a> </p>
</div>
</div>
</div>
</div>
<!-- /Header -->
<!-- /Header -->
</div>
<!-- /Header -->
<!-- Content -->
<div class="container content">
<p>I listen to a lot of Drum and Bass music, because it's beautiful music. And
there's a particular site, <a href="http://bassdrive.com/">Bassdrive.com</a> that hosts
a lot of great content. Specifically, the
<a href="http://archives.bassdrivearchive.com/">archives</a> section of the site has a
list of the past shows that you can download and listen to. The issue is, it's
just a <a href="http://archives.bassdrivearchive.com/6%20-%20Saturday/Electronic%20Warfare%20-%20The%20Overfiend/">giant list of links to download</a>. I'd really like
this in a podcast format to take with me on the road, etc.</p>
<p>So I wrote the <a href="https://github.com/bspeice/elektricity">elektricity</a> web
application to actually accomplish all that. Whenever you request a feed, it
goes out to Bassdrive, processes all the links on a page, and serves up some
fresh, tasty RSS to satisfy your ears. I hosted it on Heroku using the free
tier because it's really not resource-intensive at all.</p>
<p><strong>The issue so far</strong> is that I keep running out of free tier hours during a
month because my podcasting application likes to have a server scan for new
episodes constantly. Not sure why it's doing that, but I don't have a whole
lot of control over it. It's a phenomenal application otherwise.</p>
<p><strong>My (over-engineered) solution</strong>: Re-write the application using the
<a href="https://www.rust-lang.org/en-US/">Rust</a> programming language. I'd like to run
this on a small hacker board I own, and doing this in Rust would allow me to
easily cross-compile it. Plus, I've been very interested in the Rust language
for a while and this would be a great opportunity to really learn it well.
The code is available <a href="https://github.com/bspeice/nutone">here</a> as development
progresses.</p>
<h1>The Setup</h1>
<p>We'll be using the <a href="http://ironframework.io/">iron</a> library to handle the
server, and <a href="http://hyper.rs/">hyper</a> to fetch the data we need from elsewhere
on the interwebs. <a href="http://doc.servo.org/html5ever/index.html">HTML5Ever</a> allows
us to ingest the content that will be coming from Bassdrive, and finally,
output is done with <a href="http://sunng87.github.io/handlebars-rust/handlebars/index.html">handlebars-rust</a>.</p>
<p>It will ultimately be interesting to see how much more work must be done to
actually get this working over another language like Python. Coming from a
dynamic state of mind it's super easy to just chain stuff together, ship it out,
and call it a day. I think I'm going to end up getting much dirtier trying to
write all of this out.</p>
<h1>Issue 1: Strings</h1>
<p>Strings in Rust are hard. I acknowledge Python can get away with some things
that make strings super easy (and Python 3 has gotten better at cracking down
on some bad cases, <code>str &lt;-&gt; bytes</code> specifically), but Rust is hard.</p>
<p>Let's take for example the <code>404</code> error handler I'm trying to write. The result
should be incredibly simple: All I want is to echo back
<code>Didn't find URL: &lt;url&gt;</code>. Shouldn't be that hard right? In Python I'd just do
something like:</p>
<div class="highlight"><pre><span class="k">def</span> <span class="nf">echo_handler</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="k">return</span> <span class="s2">&quot;You&#39;re visiting: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">request</span><span class="o">.</span><span class="n">uri</span><span class="p">)</span>
</pre></div>
<p>And we'd call it a day. Rust isn't so simple. Let's start with the trivial
examples people post online:</p>
<div class="highlight"><pre><span class="k">fn</span><span class="w"> </span><span class="n">hello_world</span><span class="p">(</span><span class="n">req</span><span class="o">:</span><span class="w"> </span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="p">)</span><span class="w"> </span><span class="o">-&gt;</span><span class="w"> </span><span class="n">IronResult</span><span class="o">&lt;</span><span class="n">Response</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">Response</span><span class="o">::</span><span class="n">with</span><span class="p">((</span><span class="n">status</span><span class="o">::</span><span class="nb">Ok</span><span class="p">,</span><span class="w"> </span><span class="s">&quot;You found the server!&quot;</span><span class="p">)))</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>
<p>Doesn't look too bad right? In fact, it's essentially the same as the Python
version! All we need to do is just send back a string of some form. So, we
look up the documentation for <a href="http://ironframework.io/doc/iron/request/struct.Request.html"><code>Request</code></a> and see a <code>url</code> field that will contain
what we want. Let's try the first iteration:</p>
<div class="highlight"><pre><span class="k">fn</span><span class="w"> </span><span class="n">hello_world</span><span class="p">(</span><span class="n">req</span><span class="o">:</span><span class="w"> </span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="p">)</span><span class="w"> </span><span class="o">-&gt;</span><span class="w"> </span><span class="n">IronResult</span><span class="o">&lt;</span><span class="n">Response</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">Response</span><span class="o">::</span><span class="n">with</span><span class="p">((</span><span class="n">status</span><span class="o">::</span><span class="nb">Ok</span><span class="p">,</span><span class="w"> </span><span class="s">&quot;You found the URL: &quot;</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">req</span><span class="p">.</span><span class="n">url</span><span class="p">)))</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>
<p>Which yields the error:</p>
<div class="highlight"><pre>error[E0369]: binary operation `+` cannot be applied to type `&amp;&#39;static str`
</pre></div>
<p>OK, what's going on here? Time to start Googling for <a href="https://www.google.com/#q=concatenate+strings+in+rust">"concatenate strings in Rust"</a>. That's what we
want to do right? Concatenate a static string and the URL.</p>
<p>After Googling, we come across a helpful <a href="https://doc.rust-lang.org/std/macro.concat!.html"><code>concat!</code></a> macro that looks really nice! Let's try that one:</p>
<div class="highlight"><pre><span class="k">fn</span><span class="w"> </span><span class="n">hello_world</span><span class="p">(</span><span class="n">req</span><span class="o">:</span><span class="w"> </span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="p">)</span><span class="w"> </span><span class="o">-&gt;</span><span class="w"> </span><span class="n">IronResult</span><span class="o">&lt;</span><span class="n">Response</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">Response</span><span class="o">::</span><span class="n">with</span><span class="p">((</span><span class="n">status</span><span class="o">::</span><span class="nb">Ok</span><span class="p">,</span><span class="w"> </span><span class="n">concat</span><span class="o">!</span><span class="p">(</span><span class="s">&quot;You found the URL: &quot;</span><span class="p">,</span><span class="w"> </span><span class="n">req</span><span class="p">.</span><span class="n">url</span><span class="p">))))</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>
<p>And the error:</p>
<p><code>error: expected a literal</code></p>
<p>Turns out Rust actually blows up because the <code>concat!</code> macro expects us to know
at compile time what <code>req.url</code> is. Which, in my outsider opinion, is a bit
strange. <code>println!</code> and <code>format!</code>, etc., all handle values they don't know at
compile time. Why can't <code>concat!</code>? By any means, we need a new plan of attack.
How about we try formatting strings?</p>
<div class="highlight"><pre><span class="k">fn</span><span class="w"> </span><span class="n">hello_world</span><span class="p">(</span><span class="n">req</span><span class="o">:</span><span class="w"> </span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="p">)</span><span class="w"> </span><span class="o">-&gt;</span><span class="w"> </span><span class="n">IronResult</span><span class="o">&lt;</span><span class="n">Response</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">Response</span><span class="o">::</span><span class="n">with</span><span class="p">((</span><span class="n">status</span><span class="o">::</span><span class="nb">Ok</span><span class="p">,</span><span class="w"> </span><span class="n">format</span><span class="o">!</span><span class="p">(</span><span class="s">&quot;You found the URL: {}&quot;</span><span class="p">,</span><span class="w"> </span><span class="n">req</span><span class="p">.</span><span class="n">url</span><span class="p">))))</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>
<p>And at long last, it works. Onwards!</p>
<h1>Issue 2: Fighting with the borrow checker</h1>
<p>Rust's single coolest feature is how the compiler can guarantee safety in your
program. As long as you don't use <code>unsafe</code> pointers in Rust, you're guaranteed
safety. And not having truly manual memory management is really cool; I'm
totally OK with never having to write <code>malloc()</code> again.</p>
<p>That said, even <a href="https://doc.rust-lang.org/book/ownership.html">the Rust documentation</a> makes a specific note:</p>
<blockquote>
<p>Many new users to Rust experience something we like to call
fighting with the borrow checker, where the Rust compiler refuses to
compile a program that the author thinks is valid.</p>
</blockquote>
<p>If you have to put it in the documentation, it's not a helpful note:
it's hazing.</p>
<p>So now that we have a handler which works with information from the request, we
want to start making something that looks like an actual web application.
The router provided by <code>iron</code> isn't terribly difficult so I won't cover it.
Instead, the thing that had me stumped for a couple hours was trying to
dynamically create routes.</p>
<p>The unfortunate thing with Rust (in my limited experience at the moment) is that
there is a severe lack of non-trivial examples. Using the router is easy when
you want to give an example of a static function. But how do you you start
working on things that are a bit more complex?</p>
<p>We're going to cover that here. Our first try: creating a function which returns
other functions. This is a principle called <a href="http://stackoverflow.com/a/36321/1454178">currying</a>. We set up a function that allows us to keep some data in scope
for another function to come later.</p>
<div class="highlight"><pre><span class="k">fn</span><span class="w"> </span><span class="n">build_handler</span><span class="p">(</span><span class="n">message</span><span class="o">:</span><span class="w"> </span><span class="nb">String</span><span class="p">)</span><span class="w"> </span><span class="o">-&gt;</span><span class="w"> </span><span class="nb">Fn</span><span class="p">(</span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="p">)</span><span class="w"> </span><span class="o">-&gt;</span><span class="w"> </span><span class="n">IronResult</span><span class="o">&lt;</span><span class="n">Response</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="n">move</span><span class="w"> </span><span class="o">|</span><span class="n">_</span><span class="o">:</span><span class="w"> </span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="o">|</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">Response</span><span class="o">::</span><span class="n">with</span><span class="p">((</span><span class="n">status</span><span class="o">::</span><span class="nb">Ok</span><span class="p">,</span><span class="w"> </span><span class="n">message</span><span class="p">)))</span><span class="w"></span>
<span class="w"> </span><span class="p">}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>
<p>We've simply set up a function that returns another anonymous function with the
<code>message</code> parameter scoped in. If you compile this, you get not 1, not 2, but 5
new errors. 4 of them are the same though:</p>
<div class="highlight"><pre>error[E0277]: the trait bound `for&lt;&#39;r, &#39;r, &#39;r&gt; std::ops::Fn(&amp;&#39;r mut iron::Request&lt;&#39;r, &#39;r&gt;) -&gt; std::result::Result&lt;iron::Response, iron::IronError&gt; + &#39;static: std::marker::Sized` is not satisfied
</pre></div>
<p>...oookay. I for one, am not going to spend time trying to figure out what's
going on there.</p>
<p>And it is here that I will save the audience many hours of frustrated effort.
At this point, I decided to switch from <code>iron</code> to pure <code>hyper</code> since using
<code>hyper</code> would give me a much simpler API. All I would have to do is build a
function that took two parameters as input, and we're done. That said, it
ultimately posed many more issues because I started getting into a weird fight
with the <code>'static</code> <a href="https://doc.rust-lang.org/book/lifetimes.html">lifetime</a>
and being a Rust newbie I just gave up on trying to understand it.</p>
<p>Instead, we will abandon (mostly) the curried function attempt, and instead
take advantage of something Rust actually intends us to use: <code>struct</code> and
<code>trait</code>.</p>
<p>Remember when I talked about a lack of non-trivial examples on the Internet?
This is what I was talking about. I could only find <em>one</em> example of this
available online, and it was incredibly complex and contained code we honestly
don't need or care about. There was no documentation of how to build routes that
didn't use static functions, etc. But, I'm assuming you don't really care about
my whining, so let's get to it.</p>
<p>The <code>iron</code> documentation mentions the <a href="http://ironframework.io/doc/iron/middleware/trait.Handler.html"><code>Handler</code></a> trait as being something we can implement.
Does the function signature for that <code>handle()</code> method look familiar? It's what
we've been working with so far.</p>
<p>The principle is that we need to define a new <code>struct</code> to hold our data, then
implement that <code>handle()</code> method to return the result. Something that looks
like this might do:</p>
<div class="highlight"><pre><span class="k">struct</span><span class="w"> </span><span class="n">EchoHandler</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="n">message</span><span class="o">:</span><span class="w"> </span><span class="nb">String</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
<span class="k">impl</span><span class="w"> </span><span class="n">Handler</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">EchoHandler</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="n">handle</span><span class="p">(</span><span class="o">&amp;</span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">_</span><span class="o">:</span><span class="w"> </span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="p">)</span><span class="w"> </span><span class="o">-&gt;</span><span class="w"> </span><span class="n">IronResult</span><span class="o">&lt;</span><span class="n">Response</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">Response</span><span class="o">::</span><span class="n">with</span><span class="p">((</span><span class="n">status</span><span class="o">::</span><span class="nb">Ok</span><span class="p">,</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">message</span><span class="p">)))</span><span class="w"></span>
<span class="w"> </span><span class="p">}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
<span class="c1">// Later in the code when we set up the router...</span>
<span class="kd">let</span><span class="w"> </span><span class="n">echo</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">EchoHandler</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w"> </span><span class="n">message</span><span class="o">:</span><span class="w"> </span><span class="s">&quot;Is it working yet?&quot;</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
<span class="n">router</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">&quot;/&quot;</span><span class="p">,</span><span class="w"> </span><span class="n">echo</span><span class="p">.</span><span class="n">handle</span><span class="p">,</span><span class="w"> </span><span class="s">&quot;index&quot;</span><span class="p">);</span><span class="w"></span>
</pre></div>
<p>We attempt to build a struct, and give its <code>handle</code> method off to the router
so the router knows what to do.</p>
<p>You guessed it, more errors:</p>
<div class="highlight"><pre><span class="n">error</span><span class="o">:</span> <span class="n">attempted</span> <span class="n">to</span> <span class="n">take</span> <span class="n">value</span> <span class="n">of</span> <span class="n">method</span> <span class="err">`</span><span class="n">handle</span><span class="err">`</span> <span class="n">on</span> <span class="n">type</span> <span class="err">`</span><span class="n">EchoHandler</span><span class="err">`</span>
</pre></div>
<p>Now, the Rust compiler is actually a really nice fellow, and offers us help:</p>
<div class="highlight"><pre><span class="n">help</span><span class="o">:</span> <span class="n">maybe</span> <span class="n">a</span> <span class="err">`</span><span class="o">()</span><span class="err">`</span> <span class="n">to</span> <span class="n">call</span> <span class="n">it</span> <span class="k">is</span> <span class="n">missing</span><span class="o">?</span> <span class="n">If</span> <span class="n">not</span><span class="o">,</span> <span class="k">try</span> <span class="n">an</span> <span class="n">anonymous</span> <span class="kd">function</span>
</pre></div>
<p>We definitely don't want to call that function, so maybe try an anonymous
function as it recommends?</p>
<div class="highlight"><pre><span class="n">router</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">&quot;/&quot;</span><span class="p">,</span><span class="w"> </span><span class="o">|</span><span class="n">req</span><span class="o">:</span><span class="w"> </span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="o">|</span><span class="w"> </span><span class="n">echo</span><span class="p">.</span><span class="n">handle</span><span class="p">(</span><span class="n">req</span><span class="p">),</span><span class="w"> </span><span class="s">&quot;index&quot;</span><span class="p">);</span><span class="w"></span>
</pre></div>
<p>Another error:</p>
<div class="highlight"><pre>error[E0373]: closure may outlive the current function, but it borrows `echo`, which is owned by the current function
</pre></div>
<p>Another helpful message:</p>
<div class="highlight"><pre><span class="n">help</span><span class="o">:</span> <span class="n">to</span> <span class="n">force</span> <span class="n">the</span> <span class="n">closure</span> <span class="n">to</span> <span class="n">take</span> <span class="n">ownership</span> <span class="n">of</span> <span class="err">`</span><span class="n">echo</span><span class="err">`</span> <span class="o">(</span><span class="n">and</span> <span class="n">any</span> <span class="n">other</span> <span class="n">referenced</span> <span class="n">variables</span><span class="o">),</span> <span class="n">use</span> <span class="n">the</span> <span class="err">`</span><span class="n">move</span><span class="err">`</span> <span class="n">keyword</span>
</pre></div>
<p>We're getting closer though! Let's implement this change:</p>
<div class="highlight"><pre><span class="n">router</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">&quot;/&quot;</span><span class="p">,</span><span class="w"> </span><span class="n">move</span><span class="w"> </span><span class="o">|</span><span class="n">req</span><span class="o">:</span><span class="w"> </span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="o">|</span><span class="w"> </span><span class="n">echo</span><span class="p">.</span><span class="n">handle</span><span class="p">(</span><span class="n">req</span><span class="p">),</span><span class="w"> </span><span class="s">&quot;index&quot;</span><span class="p">);</span><span class="w"></span>
</pre></div>
<p>And here's where things get strange:</p>
<div class="highlight"><pre>error[E0507]: cannot move out of borrowed content
--&gt; src/main.rs:18:40
|
18 | Ok(Response::with((status::Ok, self.message)))
| ^^^^ cannot move out of borrowed content
</pre></div>
<p>Now, this took me another couple hours to figure out. I'm going to explain it,
but <strong>keep this in mind: Rust only allows one reference at a time</strong> (exceptions
apply of course).</p>
<p>When we attempt to use <code>self.message</code> as it has been created in the earlier
<code>struct</code>, we essentially are trying to give it away to another piece of code.
Rust's semantics then state that <em>we may no longer access it</em> unless it is
returned to us (which <code>iron</code>'s code does not do). There are two ways to fix
this:</p>
<ol>
<li>Only give away references (i.e. <code>&amp;self.message</code> instead of <code>self.message</code>)
instead of transferring ownership</li>
<li>Make a copy of the underlying value which will be safe to give away</li>
</ol>
<p>I didn't know these were the two options originally, so I hope this helps the
audience out. Because <code>iron</code> won't accept a reference, we are forced into the
second option: making a copy. To do so, we just need to change the function
to look like this:</p>
<div class="highlight"><pre><span class="nb">Ok</span><span class="p">(</span><span class="n">Response</span><span class="o">::</span><span class="n">with</span><span class="p">((</span><span class="n">status</span><span class="o">::</span><span class="nb">Ok</span><span class="p">,</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">message</span><span class="p">.</span><span class="n">clone</span><span class="p">())))</span><span class="w"></span>
</pre></div>
<p>Not so bad, huh? My only complaint is that it took so long to figure out exactly
what was going on.</p>
<p>And now we have a small server that we can configure dynamically. At long last.</p>
<blockquote>
<p>Final sidenote: You can actually do this without anonymous functions. Just
change the router line to:
<code>router.get("/", echo, "index");</code></p>
<p>Rust's type system seems to figure out that we want to use the <code>handle()</code> method.</p>
</blockquote>
<h1>Conclusion</h1>
<p>After a good long days' work, we now have the routing functionality set up on
our application. We should be able to scale this pretty well in the future:
the RSS content we need to deliver in the future can be treated as a string, so
the building blocks are in place.</p>
<p>There are two important things I learned starting with Rust today:</p>
<ol>
<li>Rust is a new language, and while the code is high-quality, the mindshare is coming.</li>
<li>I'm a terrible programmer.</li>
</ol>
<p>Number 1 is pretty obvious and not surprising to anyone. Number two caught me
off guard. I've gotten used to having either a garbage collector (Java, Python,
etc.) or playing a little fast and loose with scoping rules (C, C++). You don't
have to worry about object lifetime there. With Rust, it's forcing me to fully
understand and use well the memory in my applications. In the final mistake I
fixed (using <code>.clone()</code>) I would have been fine in C++ to just give away that
reference and never use it again. I wouldn't have run into a "use-after-free"
error, but I would have potentially been leaking memory. Rust forced me to be
incredibly precise about how I use it.</p>
<p>All said I'm excited for using Rust more. I think it's super cool, it's just
going to take me a lot longer to do this than I originally thought.</p>
<div class="comments">
<div id="disqus_thread"></div>
<script type="text/javascript">
var disqus_shortname = 'bradleespeice';
var disqus_identifier = 'a-rustic-re-podcasting-server-part-1.html';
var disqus_url = 'https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html';
(function() {
var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js';
(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
})();
</script>
<noscript>Please enable JavaScript to view the comments.</noscript>
</div>
</div>
<!-- /Content -->
<!-- Footer -->
<div class="footer gradient-2">
<div class="container footer-container ">
<div class="row">
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
<div class="footer-title"></div>
<ul class="list-unstyled">
<li><a href="https://bspeice.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate"></a></li>
</ul>
</div>
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
<div class="footer-title"></div>
<ul class="list-unstyled">
<li><a href="https://github.com/bspeice" target="_blank">Github</a></li>
<li><a href="https://www.linkedin.com/in/bradleespeice" target="_blank">LinkedIn</a></li>
</ul>
</div>
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
</div>
<div class="col-xs-12 col-sm-3 col-md-3 col-lg-3">
<p class="pull-right text-right">
<small><em>Proudly powered by <a href="http://docs.getpelican.com/" target="_blank">pelican</a></em></small><br/>
<small><em>Theme and code by <a href="https://github.com/molivier" target="_blank">molivier</a></em></small><br/>
<small></small>
</p>
</div>
</div>
</div>
</div>
<!-- /Footer -->
</body>
</html>

View File

@ -82,6 +82,8 @@
<div class="container content archive">
<h2><a href="https://bspeice.github.io/archives.html"></a></h2>
<dl class="dl-horizontal">
<dt>Sat 22 October 2016</dt>
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
<dt>Sun 15 May 2016</dt>

View File

@ -82,6 +82,8 @@
<div class="container content archive">
<h2><a href="https://bspeice.github.io/author/bradlee-speice.html"></a></h2>
<dl class="dl-horizontal">
<dt>Sat 22 October 2016</dt>
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
<dt>Sun 15 May 2016</dt>

View File

@ -82,6 +82,8 @@
<div class="container content archive">
<h2><a href="https://bspeice.github.io/author/bradlee-speice.html"></a></h2>
<dl class="dl-horizontal">
<dt>Sat 22 October 2016</dt>
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
<dt>Sun 15 May 2016</dt>

View File

@ -82,6 +82,8 @@
<div class="container content archive">
<h2><a href="https://bspeice.github.io/author/bradlee-speice.html">Bradlee Speice</a></h2>
<dl class="dl-horizontal">
<dt>Sat 22 October 2016</dt>
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
<dt>Sun 15 May 2016</dt>

View File

@ -134,7 +134,7 @@ Because these are all very similar, we decided to demonstrate all 3 products at
<div class=" highlight hl-julia"><pre><span class="n">S0</span> <span class="o">=</span> <span class="mf">102.2</span>
<span class="n">nominal</span> <span class="o">=</span> <span class="mi">100</span>
<span class="n">q</span> <span class="o">=</span> <span class="mf">2.84</span> <span class="o">/</span> <span class="mi">100</span>
<span class="err">σ</span> <span class="o">=</span> <span class="mf">15.37</span> <span class="o">/</span> <span class="mi">100</span>
<span class="n">σ</span> <span class="o">=</span> <span class="mf">15.37</span> <span class="o">/</span> <span class="mi">100</span>
<span class="n">term</span> <span class="o">=</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="o">.</span><span class="mi">49</span><span class="p">,</span> <span class="o">.</span><span class="mi">9</span><span class="p">,</span> <span class="mf">1.21</span><span class="p">,</span> <span class="mf">1.45</span><span class="p">,</span> <span class="mf">1.69</span><span class="p">]</span> <span class="o">/</span> <span class="mi">100</span> <span class="o">+</span> <span class="mi">1</span>
<span class="c">###</span>
@ -187,7 +187,7 @@ Because these are all very similar, we decided to demonstrate all 3 products at
<div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-julia"><pre><span class="n">simulate_gbm</span> <span class="o">=</span> <span class="n">function</span><span class="p">(</span><span class="n">S0</span><span class="p">,</span> <span class="err">μ</span><span class="p">,</span> <span class="err">σ</span><span class="p">,</span> <span class="n">T</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
<div class=" highlight hl-julia"><pre><span class="n">simulate_gbm</span> <span class="o">=</span> <span class="n">function</span><span class="p">(</span><span class="n">S0</span><span class="p">,</span> <span class="n">μ</span><span class="p">,</span> <span class="n">σ</span><span class="p">,</span> <span class="n">T</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
<span class="c"># Set the initial state</span>
<span class="n">m</span> <span class="o">=</span> <span class="n">length</span><span class="p">(</span><span class="n">S0</span><span class="p">)</span>
<span class="n">t</span> <span class="o">=</span> <span class="n">T</span> <span class="o">/</span> <span class="n">n</span>
@ -196,7 +196,7 @@ Because these are all very similar, we decided to demonstrate all 3 products at
<span class="c"># Build out all states</span>
<span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:(</span><span class="n">n</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>
<span class="n">motion</span><span class="p">[:,</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="n">motion</span><span class="p">[:,</span><span class="n">i</span><span class="p">]</span> <span class="o">.*</span> <span class="n">exp</span><span class="p">((</span><span class="err">μ</span> <span class="o">-</span> <span class="err">σ</span><span class="o">^</span><span class="mi">2</span><span class="o">/</span><span class="mi">2</span><span class="p">)</span><span class="o">*</span><span class="n">t</span><span class="p">)</span> <span class="o">.*</span> <span class="n">exp</span><span class="p">(</span><span class="n">sqrt</span><span class="p">(</span><span class="n">t</span><span class="p">)</span> <span class="o">*</span> <span class="err">σ</span> <span class="o">.*</span> <span class="n">randn</span><span class="p">(</span><span class="n">m</span><span class="p">))</span>
<span class="n">motion</span><span class="p">[:,</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="n">motion</span><span class="p">[:,</span><span class="n">i</span><span class="p">]</span> <span class="o">.*</span> <span class="n">exp</span><span class="p">((</span><span class="n">μ</span> <span class="o">-</span> <span class="n">σ</span><span class="o">^</span><span class="mi">2</span><span class="o">/</span><span class="mi">2</span><span class="p">)</span><span class="o">*</span><span class="n">t</span><span class="p">)</span> <span class="o">.*</span> <span class="n">exp</span><span class="p">(</span><span class="n">sqrt</span><span class="p">(</span><span class="n">t</span><span class="p">)</span> <span class="o">*</span> <span class="n">σ</span> <span class="o">.*</span> <span class="n">randn</span><span class="p">(</span><span class="n">m</span><span class="p">))</span>
<span class="k">end</span>
<span class="k">return</span> <span class="n">motion</span>
@ -260,7 +260,7 @@ Because these are all very similar, we decided to demonstrate all 3 products at
<div class="input_area">
<div class=" highlight hl-julia"><pre><span class="n">initial</span> <span class="o">=</span> <span class="n">ones</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span> <span class="o">*</span> <span class="n">S0</span>
<span class="c"># Using μ=0, T=.25 for now, we&#39;ll use the proper values later</span>
<span class="n">motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">initial</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="err">σ</span><span class="p">,</span> <span class="o">.</span><span class="mi">25</span><span class="p">,</span> <span class="mi">200</span><span class="p">)</span>
<span class="n">motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">initial</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="n">σ</span><span class="p">,</span> <span class="o">.</span><span class="mi">25</span><span class="p">,</span> <span class="mi">200</span><span class="p">)</span>
<span class="n">display_motion</span><span class="p">(</span><span class="n">motion</span><span class="p">,</span> <span class="o">.</span><span class="mi">25</span><span class="p">)</span>
</pre></div>
@ -1943,8 +1943,8 @@ fig.select("#fig-3a6dd25ad25c4037a166889ee51bb151-element-20")
<div class=" highlight hl-julia"><pre><span class="n">full_motion</span> <span class="o">=</span> <span class="n">ones</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span> <span class="o">*</span> <span class="n">S0</span>
<span class="n">full_term</span> <span class="o">=</span> <span class="n">vcat</span><span class="p">(</span><span class="n">term</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">forward_yield</span><span class="p">)</span>
<span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">T</span>
<span class="err">μ</span> <span class="o">=</span> <span class="p">(</span><span class="n">full_term</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span> <span class="o">-</span> <span class="n">q</span><span class="p">)</span>
<span class="n">year_motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">full_motion</span><span class="p">[:,</span><span class="k">end</span><span class="p">],</span> <span class="err">μ</span><span class="p">,</span> <span class="err">σ</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
<span class="n">μ</span> <span class="o">=</span> <span class="p">(</span><span class="n">full_term</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span> <span class="o">-</span> <span class="n">q</span><span class="p">)</span>
<span class="n">year_motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">full_motion</span><span class="p">[:,</span><span class="k">end</span><span class="p">],</span> <span class="n">μ</span><span class="p">,</span> <span class="n">σ</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
<span class="n">full_motion</span> <span class="o">=</span> <span class="n">hcat</span><span class="p">(</span><span class="n">full_motion</span><span class="p">,</span> <span class="n">year_motion</span><span class="p">)</span>
<span class="k">end</span>
@ -3507,8 +3507,8 @@ fig.select("#fig-0378e04b897742b597befd2e8e1c169e-element-20")
<span class="c"># And an S0 to kick things off.</span>
<span class="n">final_motion</span> <span class="o">=</span> <span class="n">ones</span><span class="p">(</span><span class="n">m</span><span class="p">)</span> <span class="o">*</span> <span class="n">S0</span>
<span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">T</span>
<span class="err">μ</span> <span class="o">=</span> <span class="p">(</span><span class="n">forward</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span> <span class="o">-</span> <span class="n">q</span><span class="p">)</span>
<span class="n">year_motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">final_motion</span><span class="p">[:,</span><span class="k">end</span><span class="p">],</span> <span class="err">μ</span><span class="p">,</span> <span class="err">σ</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
<span class="n">μ</span> <span class="o">=</span> <span class="p">(</span><span class="n">forward</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span> <span class="o">-</span> <span class="n">q</span><span class="p">)</span>
<span class="n">year_motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">final_motion</span><span class="p">[:,</span><span class="k">end</span><span class="p">],</span> <span class="n">μ</span><span class="p">,</span> <span class="n">σ</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
<span class="n">final_motion</span> <span class="o">=</span> <span class="n">hcat</span><span class="p">(</span><span class="n">final_motion</span><span class="p">,</span> <span class="n">year_motion</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">return</span> <span class="n">final_motion</span>
@ -3576,7 +3576,7 @@ fig.select("#fig-0378e04b897742b597befd2e8e1c169e-element-20")
<span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">t</span>
<span class="n">price</span> <span class="o">=</span> <span class="n">year_prices</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
<span class="k">if</span> <span class="n">price</span> <span class="err"></span> <span class="n">call_barrier</span>
<span class="k">if</span> <span class="n">price</span> <span class="n"></span> <span class="n">call_barrier</span>
<span class="k">return</span> <span class="p">(</span><span class="n">nominal</span> <span class="o">+</span> <span class="n">coupon</span><span class="o">*</span><span class="n">i</span><span class="p">)</span> <span class="o">*</span> <span class="n">exp</span><span class="p">((</span><span class="n">prod</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">[</span><span class="n">i</span><span class="p">:</span><span class="k">end</span><span class="p">])</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">*</span><span class="p">(</span><span class="n">t</span><span class="o">-</span><span class="n">i</span><span class="p">))</span>
<span class="k">end</span>
<span class="k">end</span>
@ -3686,9 +3686,9 @@ Present value of Athena note: $95.00, notional: $100.00</pre>
<span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">t</span>
<span class="n">price</span> <span class="o">=</span> <span class="n">year_prices</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
<span class="k">if</span> <span class="n">price</span> <span class="err"></span> <span class="n">call_barrier</span>
<span class="k">if</span> <span class="n">price</span> <span class="n"></span> <span class="n">call_barrier</span>
<span class="k">return</span> <span class="p">(</span><span class="n">nominal</span> <span class="o">+</span> <span class="n">coupon</span> <span class="o">+</span> <span class="n">total_coupons</span><span class="p">)</span><span class="o">*</span><span class="n">exp</span><span class="p">((</span><span class="n">prod</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">[</span><span class="n">i</span><span class="p">:</span><span class="k">end</span><span class="p">])</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">*</span><span class="p">(</span><span class="n">t</span><span class="o">-</span><span class="n">i</span><span class="p">))</span>
<span class="k">elseif</span> <span class="n">price</span> <span class="err"></span> <span class="n">coupon_barrier</span>
<span class="k">elseif</span> <span class="n">price</span> <span class="n"></span> <span class="n">coupon_barrier</span>
<span class="n">total_coupons</span> <span class="o">=</span> <span class="n">total_coupons</span> <span class="o">*</span> <span class="n">exp</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span> <span class="o">+</span> <span class="n">coupon</span>
<span class="k">else</span>
<span class="n">total_coupons</span> <span class="o">*=</span> <span class="n">exp</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>

View File

@ -82,6 +82,8 @@
<div class="container content archive">
<h2><a href="https://bspeice.github.io/category/blog.html">Blog</a></h2>
<dl class="dl-horizontal">
<dt>Sat 22 October 2016</dt>
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
<dt>Sun 15 May 2016</dt>

View File

@ -83,6 +83,8 @@
<div class="container content archive">
<h2><a href="https://bspeice.github.io/category/blog.html">Blog</a></h2>
<dl class="dl-horizontal">
<dt>Sat 22 October 2016</dt>
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
<dt>Sun 15 May 2016</dt>

View File

@ -83,6 +83,8 @@
<div class="container content archive">
<h2><a href="https://bspeice.github.io/category/blog.html">Blog</a></h2>
<dl class="dl-horizontal">
<dt>Sat 22 October 2016</dt>
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
<dt>Sun 15 May 2016</dt>

View File

@ -4,7 +4,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="In [1]: import pickle import pandas as pd import numpy as np from bokeh.plotting import output_notebook, figure, show from bokeh.palettes import RdBu4 as Palette from datetime import datetime import ...">
<meta name="description" content="In [1]: import pickle import pandas as pd import numpy as np from bokeh.plotting import output_notebook, figure, show from bokeh.palettes import RdBu4 as Palette from datetime import datetime ...">
<meta name="keywords" content="data science, weather">
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
@ -359,7 +359,7 @@
<div class="prompt input_prompt">In&nbsp;[2]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">city_forecasts</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="nb">open</span><span class="p">(</span><span class="s">&#39;city_forecasts.p&#39;</span><span class="p">,</span> <span class="s">&#39;rb&#39;</span><span class="p">))</span>
<div class=" highlight hl-ipython3"><pre><span class="n">city_forecasts</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;city_forecasts.p&#39;</span><span class="p">,</span> <span class="s1">&#39;rb&#39;</span><span class="p">))</span>
<span class="n">forecasts_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">city_forecasts</span><span class="p">)</span>
</pre></div>
@ -373,24 +373,24 @@
<div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">cities</span> <span class="o">=</span> <span class="p">[</span><span class="s">&#39;binghamton&#39;</span><span class="p">,</span> <span class="s">&#39;cary&#39;</span><span class="p">,</span> <span class="s">&#39;nyc&#39;</span><span class="p">,</span> <span class="s">&#39;seattle&#39;</span><span class="p">]</span>
<div class=" highlight hl-ipython3"><pre><span class="n">cities</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;binghamton&#39;</span><span class="p">,</span> <span class="s1">&#39;cary&#39;</span><span class="p">,</span> <span class="s1">&#39;nyc&#39;</span><span class="p">,</span> <span class="s1">&#39;seattle&#39;</span><span class="p">]</span>
<span class="n">city_colors</span> <span class="o">=</span> <span class="p">{</span><span class="n">cities</span><span class="p">[</span><span class="n">i</span><span class="p">]:</span> <span class="n">Palette</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">4</span><span class="p">)}</span>
<span class="k">def</span> <span class="nf">safe_cover</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s">&#39;cloudCover&#39;</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s">&#39;cloudCover&#39;</span><span class="p">]</span>
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s1">&#39;cloudCover&#39;</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s1">&#39;cloudCover&#39;</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span>
<span class="k">def</span> <span class="nf">monthly_avg_cloudcover</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">freq</span><span class="o">=</span><span class="s">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">&#39;left&#39;</span><span class="p">)</span>
<span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">forecasts_df</span><span class="p">[</span><span class="n">city</span><span class="p">][</span><span class="n">x</span><span class="p">][</span><span class="s">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">freq</span><span class="o">=</span><span class="s1">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">&#39;left&#39;</span><span class="p">)</span>
<span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">forecasts_df</span><span class="p">[</span><span class="n">city</span><span class="p">][</span><span class="n">x</span><span class="p">][</span><span class="s1">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">cloud_cover_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">cloud_cover_vals</span><span class="p">)))</span>
<span class="c"># Ignore an issue with nanmean having all NaN values. We&#39;ll discuss the data issues below.</span>
<span class="c1"># Ignore an issue with nanmean having all NaN values. We&#39;ll discuss the data issues below.</span>
<span class="k">with</span> <span class="n">warnings</span><span class="o">.</span><span class="n">catch_warnings</span><span class="p">():</span>
<span class="n">warnings</span><span class="o">.</span><span class="n">simplefilter</span><span class="p">(</span><span class="s">&#39;ignore&#39;</span><span class="p">)</span>
<span class="n">warnings</span><span class="o">.</span><span class="n">simplefilter</span><span class="p">(</span><span class="s1">&#39;ignore&#39;</span><span class="p">)</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">nanmean</span><span class="p">(</span><span class="n">cloud_cover_vals</span><span class="p">),</span> <span class="n">cloud_cover_samples</span>
</pre></div>
@ -409,18 +409,18 @@
<span class="k">return</span> <span class="p">[</span><span class="n">monthly_avg_cloudcover</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">month</span><span class="p">)</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span>
<span class="n">months</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">(</span><span class="s">&#39;July&#39;</span><span class="p">,</span> <span class="mi">7</span><span class="p">),</span>
<span class="p">(</span><span class="s">&#39;August&#39;</span><span class="p">,</span> <span class="mi">8</span><span class="p">),</span>
<span class="p">(</span><span class="s">&#39;September&#39;</span><span class="p">,</span> <span class="mi">9</span><span class="p">),</span>
<span class="p">(</span><span class="s">&#39;October&#39;</span><span class="p">,</span> <span class="mi">10</span><span class="p">),</span>
<span class="p">(</span><span class="s">&#39;November&#39;</span><span class="p">,</span> <span class="mi">11</span><span class="p">)</span>
<span class="p">(</span><span class="s1">&#39;July&#39;</span><span class="p">,</span> <span class="mi">7</span><span class="p">),</span>
<span class="p">(</span><span class="s1">&#39;August&#39;</span><span class="p">,</span> <span class="mi">8</span><span class="p">),</span>
<span class="p">(</span><span class="s1">&#39;September&#39;</span><span class="p">,</span> <span class="mi">9</span><span class="p">),</span>
<span class="p">(</span><span class="s1">&#39;October&#39;</span><span class="p">,</span> <span class="mi">10</span><span class="p">),</span>
<span class="p">(</span><span class="s1">&#39;November&#39;</span><span class="p">,</span> <span class="mi">11</span><span class="p">)</span>
<span class="p">]</span>
<span class="k">for</span> <span class="n">month</span><span class="p">,</span> <span class="n">month_id</span> <span class="ow">in</span> <span class="n">months</span><span class="p">:</span>
<span class="n">month_averages</span> <span class="o">=</span> <span class="p">{</span><span class="n">city</span><span class="p">:</span> <span class="n">city_avg_cc</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">month_id</span><span class="p">)</span> <span class="k">for</span> <span class="n">city</span> <span class="ow">in</span> <span class="n">cities</span><span class="p">}</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">&quot;{} Average Cloud Cover&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">month</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s">&#39;Year&#39;</span><span class="p">,</span>
<span class="n">y_axis_label</span><span class="o">=</span><span class="s">&#39;Cloud Cover Percentage&#39;</span><span class="p">)</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s2">&quot;{} Average Cloud Cover&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">month</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">&#39;Year&#39;</span><span class="p">,</span>
<span class="n">y_axis_label</span><span class="o">=</span><span class="s1">&#39;Cloud Cover Percentage&#39;</span><span class="p">)</span>
<span class="k">for</span> <span class="n">city</span> <span class="ow">in</span> <span class="n">cities</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="p">[</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">month_averages</span><span class="p">[</span><span class="n">city</span><span class="p">]],</span>
<span class="n">legend</span><span class="o">=</span><span class="n">city</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="n">city_colors</span><span class="p">[</span><span class="n">city</span><span class="p">])</span>
@ -610,20 +610,20 @@
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">safe_precip</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s">&#39;precipProbability&#39;</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s">&#39;precipProbability&#39;</span><span class="p">]</span>
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s1">&#39;precipProbability&#39;</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s1">&#39;precipProbability&#39;</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span>
<span class="k">def</span> <span class="nf">monthly_avg_precip</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">freq</span><span class="o">=</span><span class="s">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">&#39;left&#39;</span><span class="p">)</span>
<span class="n">precip_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">forecasts_df</span><span class="p">[</span><span class="n">city</span><span class="p">][</span><span class="n">x</span><span class="p">][</span><span class="s">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">freq</span><span class="o">=</span><span class="s1">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">&#39;left&#39;</span><span class="p">)</span>
<span class="n">precip_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">forecasts_df</span><span class="p">[</span><span class="n">city</span><span class="p">][</span><span class="n">x</span><span class="p">][</span><span class="s1">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">precip_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">precip_vals</span><span class="p">)))</span>
<span class="c"># Ignore an issue with nanmean having all NaN values. We&#39;ll discuss the data issues below.</span>
<span class="c1"># Ignore an issue with nanmean having all NaN values. We&#39;ll discuss the data issues below.</span>
<span class="k">with</span> <span class="n">warnings</span><span class="o">.</span><span class="n">catch_warnings</span><span class="p">():</span>
<span class="n">warnings</span><span class="o">.</span><span class="n">simplefilter</span><span class="p">(</span><span class="s">&#39;ignore&#39;</span><span class="p">)</span>
<span class="n">warnings</span><span class="o">.</span><span class="n">simplefilter</span><span class="p">(</span><span class="s1">&#39;ignore&#39;</span><span class="p">)</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">nanmean</span><span class="p">(</span><span class="n">precip_vals</span><span class="p">),</span> <span class="n">precip_samples</span>
<span class="k">def</span> <span class="nf">city_avg_precip</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
@ -631,9 +631,9 @@
<span class="k">for</span> <span class="n">month</span><span class="p">,</span> <span class="n">month_id</span> <span class="ow">in</span> <span class="n">months</span><span class="p">:</span>
<span class="n">month_averages</span> <span class="o">=</span> <span class="p">{</span><span class="n">city</span><span class="p">:</span> <span class="n">city_avg_cc</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">month_id</span><span class="p">)</span> <span class="k">for</span> <span class="n">city</span> <span class="ow">in</span> <span class="n">cities</span><span class="p">}</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">&quot;{} Average Precipitation Chance&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">month</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s">&#39;Year&#39;</span><span class="p">,</span>
<span class="n">y_axis_label</span><span class="o">=</span><span class="s">&#39;Precipitation Chance Percentage&#39;</span><span class="p">)</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s2">&quot;{} Average Precipitation Chance&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">month</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">&#39;Year&#39;</span><span class="p">,</span>
<span class="n">y_axis_label</span><span class="o">=</span><span class="s1">&#39;Precipitation Chance Percentage&#39;</span><span class="p">)</span>
<span class="k">for</span> <span class="n">city</span> <span class="ow">in</span> <span class="n">cities</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="p">[</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">month_averages</span><span class="p">[</span><span class="n">city</span><span class="p">]],</span>
<span class="n">legend</span><span class="o">=</span><span class="n">city</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="n">city_colors</span><span class="p">[</span><span class="n">city</span><span class="p">])</span>

View File

@ -352,7 +352,7 @@
<div class="prompt input_prompt">In&nbsp;[2]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">city_forecasts</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="nb">open</span><span class="p">(</span><span class="s">&#39;city_forecasts.p&#39;</span><span class="p">,</span> <span class="s">&#39;rb&#39;</span><span class="p">))</span>
<div class=" highlight hl-ipython3"><pre><span class="n">city_forecasts</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;city_forecasts.p&#39;</span><span class="p">,</span> <span class="s1">&#39;rb&#39;</span><span class="p">))</span>
<span class="n">forecast_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">city_forecasts</span><span class="p">)</span>
</pre></div>
@ -366,31 +366,31 @@
<div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">cary_forecast</span> <span class="o">=</span> <span class="n">forecast_df</span><span class="p">[</span><span class="s">&#39;cary&#39;</span><span class="p">]</span>
<div class=" highlight hl-ipython3"><pre><span class="n">cary_forecast</span> <span class="o">=</span> <span class="n">forecast_df</span><span class="p">[</span><span class="s1">&#39;cary&#39;</span><span class="p">]</span>
<span class="n">years</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2016</span><span class="p">)</span>
<span class="n">months</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">7</span><span class="p">,</span> <span class="mi">12</span><span class="p">)</span>
<span class="n">months_str</span> <span class="o">=</span> <span class="p">[</span><span class="s">&#39;July&#39;</span><span class="p">,</span> <span class="s">&#39;August&#39;</span><span class="p">,</span> <span class="s">&#39;September&#39;</span><span class="p">,</span> <span class="s">&#39;October&#39;</span><span class="p">,</span> <span class="s">&#39;November&#39;</span><span class="p">]</span>
<span class="n">months_str</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;July&#39;</span><span class="p">,</span> <span class="s1">&#39;August&#39;</span><span class="p">,</span> <span class="s1">&#39;September&#39;</span><span class="p">,</span> <span class="s1">&#39;October&#39;</span><span class="p">,</span> <span class="s1">&#39;November&#39;</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">safe_cover</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s">&#39;cloudCover&#39;</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s">&#39;cloudCover&#39;</span><span class="p">]</span>
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s1">&#39;cloudCover&#39;</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s1">&#39;cloudCover&#39;</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span>
<span class="k">def</span> <span class="nf">monthly_avg_cloudcover</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">freq</span><span class="o">=</span><span class="s">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">&#39;left&#39;</span><span class="p">)</span>
<span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">freq</span><span class="o">=</span><span class="s1">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">&#39;left&#39;</span><span class="p">)</span>
<span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s1">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">cloud_cover_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">cloud_cover_vals</span><span class="p">)))</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">nanmean</span><span class="p">(</span><span class="n">cloud_cover_vals</span><span class="p">),</span> <span class="n">cloud_cover_samples</span>
<span class="n">monthly_cover_vals</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_avg_cloudcover</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">&#39;Monthly Average Cloud Cover&#39;</span><span class="p">,</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">&#39;Monthly Average Cloud Cover&#39;</span><span class="p">,</span>
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s">&#39;Year&#39;</span><span class="p">)</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">&#39;Year&#39;</span><span class="p">)</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_cover_vals</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
@ -457,8 +457,8 @@
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">monthly_cloudy_days</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">freq</span><span class="o">=</span><span class="s">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">&#39;left&#39;</span><span class="p">)</span>
<span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">freq</span><span class="o">=</span><span class="s1">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">&#39;left&#39;</span><span class="p">)</span>
<span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s1">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">cloud_cover_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">cloud_cover_vals</span><span class="p">)))</span>
<span class="n">cloudy_days</span> <span class="o">=</span> <span class="p">[</span><span class="n">cover</span> <span class="o">&gt;</span> <span class="o">.</span><span class="mi">5</span> <span class="k">for</span> <span class="n">cover</span> <span class="ow">in</span> <span class="n">cloud_cover_vals</span><span class="p">]</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">count_nonzero</span><span class="p">(</span><span class="n">cloudy_days</span><span class="p">),</span> <span class="n">cloud_cover_samples</span>
@ -466,16 +466,16 @@
<span class="n">monthly_days_vals</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_cloudy_days</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
<span class="n">monthly_cover_samples</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_cloudy_days</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">&#39;Monthly Cloudy Days&#39;</span><span class="p">,</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">&#39;Monthly Cloudy Days&#39;</span><span class="p">,</span>
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s">&#39;Year&#39;</span><span class="p">)</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">&#39;Year&#39;</span><span class="p">)</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_days_vals</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">&#39;Monthly Cloud Cover Samples&#39;</span><span class="p">,</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">&#39;Monthly Cloud Cover Samples&#39;</span><span class="p">,</span>
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s">&#39;Year&#39;</span><span class="p">,</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">&#39;Year&#39;</span><span class="p">,</span>
<span class="n">height</span><span class="o">=</span><span class="mi">300</span><span class="p">)</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_cover_samples</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
@ -571,24 +571,24 @@
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">safe_precip</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s">&#39;precipProbability&#39;</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s">&#39;precipProbability&#39;</span><span class="p">]</span>
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s1">&#39;precipProbability&#39;</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s1">&#39;precipProbability&#39;</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span>
<span class="k">def</span> <span class="nf">monthly_avg_precip</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">freq</span><span class="o">=</span><span class="s">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">&#39;left&#39;</span><span class="p">)</span>
<span class="n">precip_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">freq</span><span class="o">=</span><span class="s1">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">&#39;left&#39;</span><span class="p">)</span>
<span class="n">precip_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s1">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">precip_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">precip_vals</span><span class="p">)))</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">nanmean</span><span class="p">(</span><span class="n">precip_vals</span><span class="p">),</span> <span class="n">precip_samples</span>
<span class="n">monthly_avg_precip_vals</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_avg_precip</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">&#39;Monthly Average Precipitation Chance&#39;</span><span class="p">,</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">&#39;Monthly Average Precipitation Chance&#39;</span><span class="p">,</span>
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s">&#39;Year&#39;</span><span class="p">)</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">&#39;Year&#39;</span><span class="p">)</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_avg_precip_vals</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
@ -654,8 +654,8 @@
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">monthly_rainy_days</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">freq</span><span class="o">=</span><span class="s">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">&#39;left&#39;</span><span class="p">)</span>
<span class="n">precip_prob_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">freq</span><span class="o">=</span><span class="s1">&#39;D&#39;</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">&#39;left&#39;</span><span class="p">)</span>
<span class="n">precip_prob_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s1">&#39;currently&#39;</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
<span class="n">precip_prob_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">precip_prob_vals</span><span class="p">)))</span>
<span class="n">precip_days</span> <span class="o">=</span> <span class="p">[</span><span class="n">prob</span> <span class="o">&gt;</span> <span class="o">.</span><span class="mi">25</span> <span class="k">for</span> <span class="n">prob</span> <span class="ow">in</span> <span class="n">precip_prob_vals</span><span class="p">]</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">count_nonzero</span><span class="p">(</span><span class="n">precip_days</span><span class="p">),</span> <span class="n">precip_prob_samples</span>
@ -663,16 +663,16 @@
<span class="n">monthly_precip_days_vals</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_rainy_days</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
<span class="n">monthly_precip_samples</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_rainy_days</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">&#39;Monthly Rainy Days&#39;</span><span class="p">,</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">&#39;Monthly Rainy Days&#39;</span><span class="p">,</span>
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s">&#39;Year&#39;</span><span class="p">)</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">&#39;Year&#39;</span><span class="p">)</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_precip_days_vals</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">&#39;Monthly Rainy Days Samples&#39;</span><span class="p">,</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">&#39;Monthly Rainy Days Samples&#39;</span><span class="p">,</span>
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s">&#39;Year&#39;</span><span class="p">,</span>
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">&#39;Year&#39;</span><span class="p">,</span>
<span class="n">height</span><span class="o">=</span><span class="mi">300</span><span class="p">)</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_precip_samples</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
@ -775,64 +775,64 @@
<span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">from</span> <span class="nn">datetime</span> <span class="kn">import</span> <span class="n">datetime</span>
<span class="c"># Coordinate data from http://itouchmap.com/latlong.html</span>
<span class="c1"># Coordinate data from http://itouchmap.com/latlong.html</span>
<span class="n">cary_loc</span> <span class="o">=</span> <span class="p">(</span><span class="mf">35.79154</span><span class="p">,</span><span class="o">-</span><span class="mf">78.781117</span><span class="p">)</span>
<span class="n">nyc_loc</span> <span class="o">=</span> <span class="p">(</span><span class="mf">40.78306</span><span class="p">,</span><span class="o">-</span><span class="mf">73.971249</span><span class="p">)</span>
<span class="n">seattle_loc</span> <span class="o">=</span> <span class="p">(</span><span class="mf">47.60621</span><span class="p">,</span><span class="o">-</span><span class="mf">122.332071</span><span class="p">)</span>
<span class="n">binghamton_loc</span> <span class="o">=</span> <span class="p">(</span><span class="mf">42.098687</span><span class="p">,</span><span class="o">-</span><span class="mf">75.917974</span><span class="p">)</span>
<span class="n">cities</span> <span class="o">=</span> <span class="p">{</span>
<span class="s">&#39;cary&#39;</span><span class="p">:</span> <span class="n">cary_loc</span><span class="p">,</span>
<span class="s">&#39;nyc&#39;</span><span class="p">:</span> <span class="n">nyc_loc</span><span class="p">,</span>
<span class="s">&#39;seattle&#39;</span><span class="p">:</span> <span class="n">seattle_loc</span><span class="p">,</span>
<span class="s">&#39;binghamton&#39;</span><span class="p">:</span> <span class="n">binghamton_loc</span>
<span class="s1">&#39;cary&#39;</span><span class="p">:</span> <span class="n">cary_loc</span><span class="p">,</span>
<span class="s1">&#39;nyc&#39;</span><span class="p">:</span> <span class="n">nyc_loc</span><span class="p">,</span>
<span class="s1">&#39;seattle&#39;</span><span class="p">:</span> <span class="n">seattle_loc</span><span class="p">,</span>
<span class="s1">&#39;binghamton&#39;</span><span class="p">:</span> <span class="n">binghamton_loc</span>
<span class="p">}</span>
<span class="n">apikey</span> <span class="o">=</span> <span class="s">&#39;&#39;</span> <span class="c"># My super-secret API Key</span>
<span class="n">apikey</span> <span class="o">=</span> <span class="s1">&#39;&#39;</span> <span class="c1"># My super-secret API Key</span>
<span class="k">def</span> <span class="nf">get_forecast</span><span class="p">(</span><span class="n">lat</span><span class="p">,</span> <span class="nb">long</span><span class="p">,</span> <span class="n">date</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span>
<span class="n">forecast_base</span> <span class="o">=</span> <span class="s">&quot;https://api.forecast.io/forecast/&quot;</span>
<span class="n">forecast_base</span> <span class="o">=</span> <span class="s2">&quot;https://api.forecast.io/forecast/&quot;</span>
<span class="k">if</span> <span class="n">date</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
<span class="n">url</span> <span class="o">=</span> <span class="n">forecast_base</span> <span class="o">+</span> <span class="n">apikey</span> <span class="o">+</span> <span class="s">&#39;/{},{}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">lat</span><span class="p">,</span> <span class="nb">long</span><span class="p">)</span>
<span class="n">url</span> <span class="o">=</span> <span class="n">forecast_base</span> <span class="o">+</span> <span class="n">apikey</span> <span class="o">+</span> <span class="s1">&#39;/{},{}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">lat</span><span class="p">,</span> <span class="nb">long</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">epoch</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">date</span><span class="o">.</span><span class="n">timestamp</span><span class="p">())</span>
<span class="n">url</span> <span class="o">=</span> <span class="n">forecast_base</span> <span class="o">+</span> <span class="n">apikey</span> <span class="o">+</span> <span class="s">&#39;/{},{},{}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">lat</span><span class="p">,</span> <span class="nb">long</span><span class="p">,</span> <span class="n">epoch</span><span class="p">)</span>
<span class="n">url</span> <span class="o">=</span> <span class="n">forecast_base</span> <span class="o">+</span> <span class="n">apikey</span> <span class="o">+</span> <span class="s1">&#39;/{},{},{}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">lat</span><span class="p">,</span> <span class="nb">long</span><span class="p">,</span> <span class="n">epoch</span><span class="p">)</span>
<span class="k">return</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span><span class="o">.</span><span class="n">json</span><span class="p">()</span>
<span class="n">years</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span><span class="mi">2016</span><span class="p">)</span>
<span class="c"># For datetimes, the 12 is for getting the weather at noon.</span>
<span class="c"># We&#39;re doing this over midnight because we&#39;re more concerned</span>
<span class="c"># with what people see, and people don&#39;t typically see the weather</span>
<span class="c"># at midnight.</span>
<span class="c1"># For datetimes, the 12 is for getting the weather at noon.</span>
<span class="c1"># We&#39;re doing this over midnight because we&#39;re more concerned</span>
<span class="c1"># with what people see, and people don&#39;t typically see the weather</span>
<span class="c1"># at midnight.</span>
<span class="n">dt_indices</span> <span class="o">=</span> <span class="p">[</span><span class="n">pd</span><span class="o">.</span><span class="n">date_range</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">30</span><span class="p">,</span> <span class="mi">12</span><span class="p">))</span>
<span class="k">for</span> <span class="n">year</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span>
<span class="n">dt_merge</span> <span class="o">=</span> <span class="nb">reduce</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">union</span><span class="p">(</span><span class="n">y</span><span class="p">),</span> <span class="n">dt_indices</span><span class="p">)</span>
<span class="c"># Because we have to pay a little bit to use the API, we use for loops here</span>
<span class="c"># instead of a comprehension - if something breaks, we want to preserve the</span>
<span class="c"># data already retrieved</span>
<span class="c1"># Because we have to pay a little bit to use the API, we use for loops here</span>
<span class="c1"># instead of a comprehension - if something breaks, we want to preserve the</span>
<span class="c1"># data already retrieved</span>
<span class="n">city_forecasts</span> <span class="o">=</span> <span class="p">{}</span>
<span class="k">for</span> <span class="n">city</span><span class="p">,</span> <span class="n">loc</span> <span class="ow">in</span> <span class="n">cities</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
<span class="k">print</span><span class="p">(</span><span class="s">&quot;Retrieving data for {} starting at {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">city</span><span class="p">,</span>
<span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s">&quot;%I:%M:%S %p&quot;</span><span class="p">)))</span>
<span class="k">print</span><span class="p">(</span><span class="s2">&quot;Retrieving data for {} starting at {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">city</span><span class="p">,</span>
<span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s2">&quot;%I:%M:%S %p&quot;</span><span class="p">)))</span>
<span class="k">for</span> <span class="n">dt</span> <span class="ow">in</span> <span class="n">dt_merge</span><span class="p">:</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">city_forecasts</span><span class="p">[(</span><span class="n">city</span><span class="p">,</span> <span class="n">dt</span><span class="p">)]</span> <span class="o">=</span> <span class="n">get_forecast</span><span class="p">(</span><span class="o">*</span><span class="n">loc</span><span class="p">,</span> <span class="n">dt</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
<span class="k">print</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
<span class="n">city_forecasts</span><span class="p">[(</span><span class="n">city</span><span class="p">,</span> <span class="n">dt</span><span class="p">)]</span> <span class="o">=</span> <span class="bp">None</span>
<span class="k">print</span><span class="p">(</span><span class="s">&quot;End forecast retrieval: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s">&quot;%I:%M:%S %p&quot;</span><span class="p">)))</span>
<span class="k">print</span><span class="p">(</span><span class="s2">&quot;End forecast retrieval: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s2">&quot;%I:%M:%S %p&quot;</span><span class="p">)))</span>
<span class="kn">import</span> <span class="nn">pickle</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">city_forecasts</span><span class="p">,</span> <span class="nb">open</span><span class="p">(</span><span class="s">&#39;city_forecasts.p&#39;</span><span class="p">,</span> <span class="s">&#39;wb&#39;</span><span class="p">))</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">city_forecasts</span><span class="p">,</span> <span class="nb">open</span><span class="p">(</span><span class="s1">&#39;city_forecasts.p&#39;</span><span class="p">,</span> <span class="s1">&#39;wb&#39;</span><span class="p">))</span>
<span class="c">### Output:</span>
<span class="c"># Retrieving data for binghamton starting at 05:13:42 PM</span>
<span class="c"># Retrieving data for seattle starting at 05:30:51 PM</span>
<span class="c"># Retrieving data for nyc starting at 05:48:30 PM</span>
<span class="c"># Retrieving data for cary starting at 06:08:32 PM</span>
<span class="c"># End forecast retrieval: 06:25:21 PM</span>
<span class="c1">### Output:</span>
<span class="c1"># Retrieving data for binghamton starting at 05:13:42 PM</span>
<span class="c1"># Retrieving data for seattle starting at 05:30:51 PM</span>
<span class="c1"># Retrieving data for nyc starting at 05:48:30 PM</span>
<span class="c1"># Retrieving data for cary starting at 06:08:32 PM</span>
<span class="c1"># End forecast retrieval: 06:25:21 PM</span>
</pre></div>
</div>

View File

@ -4,7 +4,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="Or, being suspicious of market insiders. Use the button below to show the code I&#39;ve used to generate this article. Because there is a significant amount more code involved than most other posts I&#39;ve ...">
<meta name="description" content="Or, being suspicious of market insiders. Use the button below to show the code I&#39;ve used to generate this article. Because there is a significant amount more code involved than most other posts ...">
<meta name="keywords" content="earnings, event study">
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
@ -108,19 +108,19 @@
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="kn">from</span> <span class="nn">IPython.display</span> <span class="k">import</span> <span class="n">HTML</span>
<span class="n">HTML</span><span class="p">(</span><span class="s">&#39;&#39;&#39;&lt;script&gt;</span>
<span class="s">code_show=true; </span>
<span class="s">function code_toggle() {</span>
<span class="s"> if (code_show){</span>
<span class="s"> $(&#39;div.input&#39;).hide();</span>
<span class="s"> } else {</span>
<span class="s"> $(&#39;div.input&#39;).show();</span>
<span class="s"> }</span>
<span class="s"> code_show = !code_show</span>
<span class="s">} </span>
<span class="s">$( document ).ready(code_toggle);</span>
<span class="s">&lt;/script&gt;</span>
<span class="s">&lt;form action=&quot;javascript:code_toggle()&quot;&gt;&lt;input type=&quot;submit&quot; value=&quot;Click here to toggle on/off the raw code.&quot;&gt;&lt;/form&gt;&#39;&#39;&#39;</span><span class="p">)</span>
<span class="n">HTML</span><span class="p">(</span><span class="s1">&#39;&#39;&#39;&lt;script&gt;</span>
<span class="s1">code_show=true; </span>
<span class="s1">function code_toggle() {</span>
<span class="s1"> if (code_show){</span>
<span class="s1"> $(&#39;div.input&#39;).hide();</span>
<span class="s1"> } else {</span>
<span class="s1"> $(&#39;div.input&#39;).show();</span>
<span class="s1"> }</span>
<span class="s1"> code_show = !code_show</span>
<span class="s1">} </span>
<span class="s1">$( document ).ready(code_toggle);</span>
<span class="s1">&lt;/script&gt;</span>
<span class="s1">&lt;form action=&quot;javascript:code_toggle()&quot;&gt;&lt;input type=&quot;submit&quot; value=&quot;Click here to toggle on/off the raw code.&quot;&gt;&lt;/form&gt;&#39;&#39;&#39;</span><span class="p">)</span>
</pre></div>
</div>
@ -182,20 +182,20 @@ $( document ).ready(code_toggle);
<span class="o">%</span><span class="k">matplotlib</span> inline
<span class="k">def</span> <span class="nf">fetch_ticker</span><span class="p">(</span><span class="n">ticker</span><span class="p">,</span> <span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">):</span>
<span class="c"># Quandl is currently giving me issues with returning</span>
<span class="c"># the entire dataset and not slicing server-side.</span>
<span class="c"># So instead, we&#39;ll do it client-side!</span>
<span class="n">q_format</span> <span class="o">=</span> <span class="s">&#39;%Y-%m-%d&#39;</span>
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">&#39;YAHOO/&#39;</span> <span class="o">+</span> <span class="n">ticker</span><span class="p">,</span>
<span class="c1"># Quandl is currently giving me issues with returning</span>
<span class="c1"># the entire dataset and not slicing server-side.</span>
<span class="c1"># So instead, we&#39;ll do it client-side!</span>
<span class="n">q_format</span> <span class="o">=</span> <span class="s1">&#39;%Y-%m-</span><span class="si">%d</span><span class="s1">&#39;</span>
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">&#39;YAHOO/&#39;</span> <span class="o">+</span> <span class="n">ticker</span><span class="p">,</span>
<span class="n">start_date</span><span class="o">=</span><span class="n">start</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="n">q_format</span><span class="p">),</span>
<span class="n">end_date</span><span class="o">=</span><span class="n">end</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="n">q_format</span><span class="p">),</span>
<span class="n">authtoken</span><span class="o">=</span><span class="n">QUANDL_KEY</span><span class="p">)</span>
<span class="k">return</span> <span class="n">ticker_data</span>
<span class="k">def</span> <span class="nf">ohlc_dataframe</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">ax</span><span class="o">=</span><span class="k">None</span><span class="p">):</span>
<span class="c"># Much of this code re-used from:</span>
<span class="c"># http://matplotlib.org/examples/pylab_examples/finance_demo.html</span>
<span class="k">if</span> <span class="n">ax</span> <span class="ow">is</span> <span class="k">None</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">ohlc_dataframe</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">ax</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="c1"># Much of this code re-used from:</span>
<span class="c1"># http://matplotlib.org/examples/pylab_examples/finance_demo.html</span>
<span class="k">if</span> <span class="n">ax</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
<span class="n">f</span><span class="p">,</span> <span class="n">ax</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">()</span>
<span class="n">vals</span> <span class="o">=</span> <span class="p">[(</span><span class="n">date2num</span><span class="p">(</span><span class="n">date</span><span class="p">),</span> <span class="o">*</span><span class="p">(</span><span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">date</span><span class="p">]))</span>
@ -204,20 +204,20 @@ $( document ).ready(code_toggle);
<span class="n">mondays</span> <span class="o">=</span> <span class="n">WeekdayLocator</span><span class="p">(</span><span class="n">MONDAY</span><span class="p">)</span>
<span class="n">alldays</span> <span class="o">=</span> <span class="n">DayLocator</span><span class="p">()</span>
<span class="n">weekFormatter</span> <span class="o">=</span> <span class="n">DateFormatter</span><span class="p">(</span><span class="s">&#39;%b %d&#39;</span><span class="p">)</span>
<span class="n">weekFormatter</span> <span class="o">=</span> <span class="n">DateFormatter</span><span class="p">(</span><span class="s1">&#39;%b </span><span class="si">%d</span><span class="s1">&#39;</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">xaxis</span><span class="o">.</span><span class="n">set_major_locator</span><span class="p">(</span><span class="n">mondays</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">xaxis</span><span class="o">.</span><span class="n">set_minor_locator</span><span class="p">(</span><span class="n">alldays</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">xaxis</span><span class="o">.</span><span class="n">set_major_formatter</span><span class="p">(</span><span class="n">weekFormatter</span><span class="p">)</span>
<span class="k">return</span> <span class="n">ax</span>
<span class="n">AAPL</span> <span class="o">=</span> <span class="n">fetch_ticker</span><span class="p">(</span><span class="s">&#39;AAPL&#39;</span><span class="p">,</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span>
<span class="n">AAPL</span> <span class="o">=</span> <span class="n">fetch_ticker</span><span class="p">(</span><span class="s1">&#39;AAPL&#39;</span><span class="p">,</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span>
<span class="n">ax</span> <span class="o">=</span> <span class="n">ohlc_dataframe</span><span class="p">(</span><span class="n">AAPL</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">vlines</span><span class="p">(</span><span class="n">date2num</span><span class="p">(</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">26</span><span class="p">,</span> <span class="mi">12</span><span class="p">)),</span>
<span class="n">ax</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">0</span><span class="p">],</span> <span class="n">ax</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">1</span><span class="p">],</span>
<span class="n">color</span><span class="o">=</span><span class="s">&#39;b&#39;</span><span class="p">,</span>
<span class="n">label</span><span class="o">=</span><span class="s">&#39;Earnings Release&#39;</span><span class="p">)</span>
<span class="n">color</span><span class="o">=</span><span class="s1">&#39;b&#39;</span><span class="p">,</span>
<span class="n">label</span><span class="o">=</span><span class="s1">&#39;Earnings Release&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">3</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">&quot;Apple Price 3/1/2016 - 5/1/2016&quot;</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s2">&quot;Apple Price 3/1/2016 - 5/1/2016&quot;</span><span class="p">);</span>
</pre></div>
</div>
@ -439,12 +439,12 @@ rgAAAABJRU5ErkJggg==
<div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">FB</span> <span class="o">=</span> <span class="n">fetch_ticker</span><span class="p">(</span><span class="s">&#39;FB&#39;</span><span class="p">,</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">5</span><span class="p">))</span>
<div class=" highlight hl-ipython3"><pre><span class="n">FB</span> <span class="o">=</span> <span class="n">fetch_ticker</span><span class="p">(</span><span class="s1">&#39;FB&#39;</span><span class="p">,</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">5</span><span class="p">))</span>
<span class="n">ax</span> <span class="o">=</span> <span class="n">ohlc_dataframe</span><span class="p">(</span><span class="n">FB</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">vlines</span><span class="p">(</span><span class="n">date2num</span><span class="p">(</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">27</span><span class="p">,</span> <span class="mi">12</span><span class="p">)),</span>
<span class="n">ax</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">0</span><span class="p">],</span> <span class="n">ax</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">1</span><span class="p">],</span>
<span class="n">color</span><span class="o">=</span><span class="s">&#39;b&#39;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&#39;Earnings Release&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">&#39;Facebook Price 3/5/2016 - 5/5/2016&#39;</span><span class="p">)</span>
<span class="n">color</span><span class="o">=</span><span class="s1">&#39;b&#39;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">&#39;Earnings Release&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">&#39;Facebook Price 3/5/2016 - 5/5/2016&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">2</span><span class="p">);</span>
</pre></div>
@ -698,36 +698,36 @@ czHjid0552LGE7tzzsXM/wfgLII59KYqMQAAAABJRU5ErkJggg==
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">plot_hilo</span><span class="p">(</span><span class="n">ax</span><span class="p">,</span> <span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">data</span><span class="p">):</span>
<span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">([</span><span class="n">date2num</span><span class="p">(</span><span class="n">start</span><span class="p">),</span> <span class="n">date2num</span><span class="p">(</span><span class="n">end</span><span class="p">)],</span>
<span class="p">[</span><span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">start</span><span class="p">][</span><span class="s">&#39;High&#39;</span><span class="p">],</span> <span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">end</span><span class="p">][</span><span class="s">&#39;High&#39;</span><span class="p">]],</span>
<span class="n">color</span><span class="o">=</span><span class="s">&#39;b&#39;</span><span class="p">)</span>
<span class="p">[</span><span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">start</span><span class="p">][</span><span class="s1">&#39;High&#39;</span><span class="p">],</span> <span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">end</span><span class="p">][</span><span class="s1">&#39;High&#39;</span><span class="p">]],</span>
<span class="n">color</span><span class="o">=</span><span class="s1">&#39;b&#39;</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">([</span><span class="n">date2num</span><span class="p">(</span><span class="n">start</span><span class="p">),</span> <span class="n">date2num</span><span class="p">(</span><span class="n">end</span><span class="p">)],</span>
<span class="p">[</span><span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">start</span><span class="p">][</span><span class="s">&#39;Low&#39;</span><span class="p">],</span> <span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">end</span><span class="p">][</span><span class="s">&#39;Low&#39;</span><span class="p">]],</span>
<span class="n">color</span><span class="o">=</span><span class="s">&#39;b&#39;</span><span class="p">)</span>
<span class="p">[</span><span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">start</span><span class="p">][</span><span class="s1">&#39;Low&#39;</span><span class="p">],</span> <span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">end</span><span class="p">][</span><span class="s1">&#39;Low&#39;</span><span class="p">]],</span>
<span class="n">color</span><span class="o">=</span><span class="s1">&#39;b&#39;</span><span class="p">)</span>
<span class="n">f</span><span class="p">,</span> <span class="n">axarr</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
<span class="n">ax_aapl</span> <span class="o">=</span> <span class="n">axarr</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="n">ax_fb</span> <span class="o">=</span> <span class="n">axarr</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="c"># Plot the AAPL trend up and down</span>
<span class="c1"># Plot the AAPL trend up and down</span>
<span class="n">ohlc_dataframe</span><span class="p">(</span><span class="n">AAPL</span><span class="p">,</span> <span class="n">ax</span><span class="o">=</span><span class="n">ax_aapl</span><span class="p">)</span>
<span class="n">plot_hilo</span><span class="p">(</span><span class="n">ax_aapl</span><span class="p">,</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">15</span><span class="p">),</span> <span class="n">AAPL</span><span class="p">)</span>
<span class="n">plot_hilo</span><span class="p">(</span><span class="n">ax_aapl</span><span class="p">,</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">18</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">26</span><span class="p">),</span> <span class="n">AAPL</span><span class="p">)</span>
<span class="n">ax_aapl</span><span class="o">.</span><span class="n">vlines</span><span class="p">(</span><span class="n">date2num</span><span class="p">(</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">26</span><span class="p">,</span> <span class="mi">12</span><span class="p">)),</span>
<span class="n">ax_aapl</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">0</span><span class="p">],</span> <span class="n">ax_aapl</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">1</span><span class="p">],</span>
<span class="n">color</span><span class="o">=</span><span class="s">&#39;g&#39;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&#39;Earnings Release&#39;</span><span class="p">)</span>
<span class="n">color</span><span class="o">=</span><span class="s1">&#39;g&#39;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">&#39;Earnings Release&#39;</span><span class="p">)</span>
<span class="n">ax_aapl</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
<span class="n">ax_aapl</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s">&#39;AAPL Price History&#39;</span><span class="p">)</span>
<span class="n">ax_aapl</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s1">&#39;AAPL Price History&#39;</span><span class="p">)</span>
<span class="c"># Plot the FB trend down and up</span>
<span class="c1"># Plot the FB trend down and up</span>
<span class="n">ohlc_dataframe</span><span class="p">(</span><span class="n">FB</span><span class="p">,</span> <span class="n">ax</span><span class="o">=</span><span class="n">ax_fb</span><span class="p">)</span>
<span class="n">plot_hilo</span><span class="p">(</span><span class="n">ax_fb</span><span class="p">,</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">30</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">27</span><span class="p">),</span> <span class="n">FB</span><span class="p">)</span>
<span class="n">plot_hilo</span><span class="p">(</span><span class="n">ax_fb</span><span class="p">,</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">28</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">5</span><span class="p">),</span> <span class="n">FB</span><span class="p">)</span>
<span class="n">ax_fb</span><span class="o">.</span><span class="n">vlines</span><span class="p">(</span><span class="n">date2num</span><span class="p">(</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">27</span><span class="p">,</span> <span class="mi">12</span><span class="p">)),</span>
<span class="n">ax_fb</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">0</span><span class="p">],</span> <span class="n">ax_fb</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">1</span><span class="p">],</span>
<span class="n">color</span><span class="o">=</span><span class="s">&#39;g&#39;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&#39;Earnings Release&#39;</span><span class="p">)</span>
<span class="n">color</span><span class="o">=</span><span class="s1">&#39;g&#39;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">&#39;Earnings Release&#39;</span><span class="p">)</span>
<span class="n">ax_fb</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
<span class="n">ax_fb</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s">&#39;FB Price History&#39;</span><span class="p">)</span>
<span class="n">ax_fb</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s1">&#39;FB Price History&#39;</span><span class="p">)</span>
<span class="n">f</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
</pre></div>
@ -1385,17 +1385,17 @@ cAw4PwAAAABJRU5ErkJggg==
<span class="kn">from</span> <span class="nn">pandas.tseries.offsets</span> <span class="k">import</span> <span class="n">CustomBusinessDay</span>
<span class="kn">from</span> <span class="nn">datetime</span> <span class="k">import</span> <span class="n">datetime</span><span class="p">,</span> <span class="n">timedelta</span>
<span class="c"># If you remove rules, it removes them from *all* calendars</span>
<span class="c"># To ensure we don&#39;t pop rules we don&#39;t want to, first make</span>
<span class="c"># sure to fully copy the object</span>
<span class="c1"># If you remove rules, it removes them from *all* calendars</span>
<span class="c1"># To ensure we don&#39;t pop rules we don&#39;t want to, first make</span>
<span class="c1"># sure to fully copy the object</span>
<span class="n">trade_calendar</span> <span class="o">=</span> <span class="n">USFederalHolidayCalendar</span><span class="p">()</span>
<span class="n">trade_calendar</span><span class="o">.</span><span class="n">rules</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="mi">6</span><span class="p">)</span> <span class="c"># Remove Columbus day</span>
<span class="n">trade_calendar</span><span class="o">.</span><span class="n">rules</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="mi">7</span><span class="p">)</span> <span class="c"># Remove Veteran&#39;s day</span>
<span class="n">trade_calendar</span><span class="o">.</span><span class="n">rules</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="mi">6</span><span class="p">)</span> <span class="c1"># Remove Columbus day</span>
<span class="n">trade_calendar</span><span class="o">.</span><span class="n">rules</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="mi">7</span><span class="p">)</span> <span class="c1"># Remove Veteran&#39;s day</span>
<span class="n">TradeDay</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">days</span><span class="p">:</span> <span class="n">CustomBusinessDay</span><span class="p">(</span><span class="n">days</span><span class="p">,</span> <span class="n">calendar</span><span class="o">=</span><span class="n">trade_calendar</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">plot_study</span><span class="p">(</span><span class="n">array</span><span class="p">):</span>
<span class="c"># Given a 2-d array, we assume the event happens at index `lookback`,</span>
<span class="c"># and create all of our summary statistics from there.</span>
<span class="c1"># Given a 2-d array, we assume the event happens at index `lookback`,</span>
<span class="c1"># and create all of our summary statistics from there.</span>
<span class="n">lookback</span> <span class="o">=</span> <span class="nb">int</span><span class="p">((</span><span class="n">array</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">/</span> <span class="mi">2</span><span class="p">)</span>
<span class="n">norm_factor</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="n">array</span><span class="p">[:,</span><span class="n">lookback</span><span class="p">]</span><span class="o">.</span><span class="n">reshape</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="n">array</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="n">centered_data</span> <span class="o">=</span> <span class="n">array</span> <span class="o">/</span> <span class="n">norm_factor</span> <span class="o">-</span> <span class="mi">1</span>
@ -1414,17 +1414,17 @@ cAw4PwAAAABJRU5ErkJggg==
<span class="n">axarr</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">fill_between</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">range_end</span><span class="p">),</span>
<span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="p">:]</span> <span class="o">+</span> <span class="n">std_dev</span><span class="p">,</span>
<span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="p">:]</span> <span class="o">-</span> <span class="n">std_dev</span><span class="p">,</span>
<span class="n">alpha</span><span class="o">=.</span><span class="mi">5</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&quot;$\pm$ 1 s.d.&quot;</span><span class="p">)</span>
<span class="n">alpha</span><span class="o">=.</span><span class="mi">5</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;$\pm$ 1 s.d.&quot;</span><span class="p">)</span>
<span class="n">axarr</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">fill_between</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">range_end</span><span class="p">),</span>
<span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="p">:]</span> <span class="o">+</span> <span class="n">std_dev</span><span class="p">,</span>
<span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="p">:]</span> <span class="o">-</span> <span class="n">std_dev</span><span class="p">,</span>
<span class="n">alpha</span><span class="o">=.</span><span class="mi">5</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&quot;$\pm$ 1 s.d.&quot;</span><span class="p">)</span>
<span class="n">alpha</span><span class="o">=.</span><span class="mi">5</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;$\pm$ 1 s.d.&quot;</span><span class="p">)</span>
<span class="n">max_err</span> <span class="o">=</span> <span class="n">maxes</span> <span class="o">-</span> <span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="o">+</span><span class="mi">1</span><span class="p">:]</span>
<span class="n">min_err</span> <span class="o">=</span> <span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="o">+</span><span class="mi">1</span><span class="p">:]</span> <span class="o">-</span> <span class="n">mins</span>
<span class="n">axarr</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">errorbar</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">range_end</span><span class="p">),</span>
<span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="o">+</span><span class="mi">1</span><span class="p">:],</span>
<span class="n">yerr</span><span class="o">=</span><span class="p">[</span><span class="n">min_err</span><span class="p">,</span> <span class="n">max_err</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s">&#39;Max &amp; Min&#39;</span><span class="p">)</span>
<span class="n">yerr</span><span class="o">=</span><span class="p">[</span><span class="n">min_err</span><span class="p">,</span> <span class="n">max_err</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s1">&#39;Max &amp; Min&#39;</span><span class="p">)</span>
<span class="n">axarr</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
<span class="n">axarr</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
@ -1432,8 +1432,8 @@ cAw4PwAAAABJRU5ErkJggg==
<span class="n">axarr</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">set_xlim</span><span class="p">((</span><span class="o">-</span><span class="n">lookback</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="n">lookback</span><span class="o">+</span><span class="mi">1</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">plot_study_small</span><span class="p">(</span><span class="n">array</span><span class="p">):</span>
<span class="c"># Given a 2-d array, we assume the event happens at index `lookback`,</span>
<span class="c"># and create all of our summary statistics from there.</span>
<span class="c1"># Given a 2-d array, we assume the event happens at index `lookback`,</span>
<span class="c1"># and create all of our summary statistics from there.</span>
<span class="n">lookback</span> <span class="o">=</span> <span class="nb">int</span><span class="p">((</span><span class="n">array</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">/</span> <span class="mi">2</span><span class="p">)</span>
<span class="n">norm_factor</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="n">array</span><span class="p">[:,</span><span class="n">lookback</span><span class="p">]</span><span class="o">.</span><span class="n">reshape</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="n">array</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="n">centered_data</span> <span class="o">=</span> <span class="n">array</span> <span class="o">/</span> <span class="n">norm_factor</span> <span class="o">-</span> <span class="mi">1</span>
@ -1450,24 +1450,24 @@ cAw4PwAAAABJRU5ErkJggg==
<span class="n">plt</span><span class="o">.</span><span class="n">fill_between</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">range_end</span><span class="p">),</span>
<span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="p">:]</span> <span class="o">+</span> <span class="n">std_dev</span><span class="p">,</span>
<span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="p">:]</span> <span class="o">-</span> <span class="n">std_dev</span><span class="p">,</span>
<span class="n">alpha</span><span class="o">=.</span><span class="mi">5</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&quot;$\pm$ 1 s.d.&quot;</span><span class="p">)</span>
<span class="n">alpha</span><span class="o">=.</span><span class="mi">5</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;$\pm$ 1 s.d.&quot;</span><span class="p">)</span>
<span class="n">max_err</span> <span class="o">=</span> <span class="n">maxes</span> <span class="o">-</span> <span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="o">+</span><span class="mi">1</span><span class="p">:]</span>
<span class="n">min_err</span> <span class="o">=</span> <span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="o">+</span><span class="mi">1</span><span class="p">:]</span> <span class="o">-</span> <span class="n">mins</span>
<span class="n">plt</span><span class="o">.</span><span class="n">errorbar</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">range_end</span><span class="p">),</span>
<span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="o">+</span><span class="mi">1</span><span class="p">:],</span>
<span class="n">yerr</span><span class="o">=</span><span class="p">[</span><span class="n">min_err</span><span class="p">,</span> <span class="n">max_err</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s">&#39;Max &amp; Min&#39;</span><span class="p">)</span>
<span class="n">yerr</span><span class="o">=</span><span class="p">[</span><span class="n">min_err</span><span class="p">,</span> <span class="n">max_err</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s1">&#39;Max &amp; Min&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">xlim</span><span class="p">((</span><span class="o">-</span><span class="n">lookback</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="n">lookback</span><span class="o">+</span><span class="mi">1</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">fetch_event_data</span><span class="p">(</span><span class="n">ticker</span><span class="p">,</span> <span class="n">events</span><span class="p">,</span> <span class="n">horizon</span><span class="o">=</span><span class="mi">5</span><span class="p">):</span>
<span class="c"># Use horizon+1 to account for including the day of the event,</span>
<span class="c"># and half-open interval - that is, for a horizon of 5,</span>
<span class="c"># we should be including 11 events. Additionally, using the</span>
<span class="c"># CustomBusinessDay means we automatically handle issues if</span>
<span class="c"># for example a company reports Friday afternoon - the date</span>
<span class="c"># calculator will turn this into a &quot;Saturday&quot; release, but</span>
<span class="c"># we effectively shift that to Monday with the logic below.</span>
<span class="c1"># Use horizon+1 to account for including the day of the event,</span>
<span class="c1"># and half-open interval - that is, for a horizon of 5,</span>
<span class="c1"># we should be including 11 events. Additionally, using the</span>
<span class="c1"># CustomBusinessDay means we automatically handle issues if</span>
<span class="c1"># for example a company reports Friday afternoon - the date</span>
<span class="c1"># calculator will turn this into a &quot;Saturday&quot; release, but</span>
<span class="c1"># we effectively shift that to Monday with the logic below.</span>
<span class="n">td_back</span> <span class="o">=</span> <span class="n">TradeDay</span><span class="p">(</span><span class="n">horizon</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span>
<span class="n">td_forward</span> <span class="o">=</span> <span class="n">TradeDay</span><span class="p">(</span><span class="n">horizon</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span>
@ -1476,28 +1476,28 @@ cAw4PwAAAABJRU5ErkJggg==
<span class="n">total_data</span> <span class="o">=</span> <span class="n">fetch_ticker</span><span class="p">(</span><span class="n">ticker</span><span class="p">,</span> <span class="n">start_date</span><span class="p">,</span> <span class="n">end_date</span><span class="p">)</span>
<span class="n">event_data</span> <span class="o">=</span> <span class="p">[</span><span class="n">total_data</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="n">event</span><span class="o">-</span><span class="n">td_back</span><span class="p">:</span><span class="n">event</span><span class="o">+</span><span class="n">td_forward</span><span class="p">]</span>\
<span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="n">horizon</span><span class="o">*</span><span class="mi">2</span><span class="o">+</span><span class="mi">1</span><span class="p">]</span>\
<span class="p">[</span><span class="s">&#39;Adjusted Close&#39;</span><span class="p">]</span>
<span class="p">[</span><span class="s1">&#39;Adjusted Close&#39;</span><span class="p">]</span>
<span class="k">for</span> <span class="n">event</span> <span class="ow">in</span> <span class="n">events</span><span class="p">]</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="n">event_data</span><span class="p">)</span>
<span class="c"># Generate a couple of random events</span>
<span class="c1"># Generate a couple of random events</span>
<span class="n">event_dates</span> <span class="o">=</span> <span class="p">[</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">27</span><span class="p">)</span> <span class="o">-</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span> <span class="o">-</span> <span class="n">TradeDay</span><span class="p">(</span><span class="n">x</span><span class="o">*</span><span class="mi">20</span><span class="p">)</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">40</span><span class="p">)]</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">fetch_event_data</span><span class="p">(</span><span class="s">&#39;CELG&#39;</span><span class="p">,</span> <span class="n">event_dates</span><span class="p">)</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">fetch_event_data</span><span class="p">(</span><span class="s1">&#39;CELG&#39;</span><span class="p">,</span> <span class="n">event_dates</span><span class="p">)</span>
<span class="n">plot_study_small</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">3</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="mi">6</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="s">&#39;Mean price for days leading up to each event&#39;</span><span class="p">,</span>
<span class="n">plt</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="s1">&#39;Mean price for days leading up to each event&#39;</span><span class="p">,</span>
<span class="p">(</span><span class="o">-</span><span class="mi">5</span><span class="p">,</span> <span class="o">-.</span><span class="mi">01</span><span class="p">),</span> <span class="p">(</span><span class="o">-</span><span class="mf">4.5</span><span class="p">,</span> <span class="o">.</span><span class="mi">025</span><span class="p">),</span>
<span class="n">arrowprops</span><span class="o">=</span><span class="nb">dict</span><span class="p">(</span><span class="n">facecolor</span><span class="o">=</span><span class="s">&#39;black&#39;</span><span class="p">,</span> <span class="n">shrink</span><span class="o">=</span><span class="mf">0.05</span><span class="p">))</span>
<span class="n">plt</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="s">&#39;&#39;</span><span class="p">,</span> <span class="p">(</span><span class="o">-.</span><span class="mi">1</span><span class="p">,</span> <span class="o">.</span><span class="mi">005</span><span class="p">),</span> <span class="p">(</span><span class="o">-.</span><span class="mi">5</span><span class="p">,</span> <span class="o">.</span><span class="mi">02</span><span class="p">),</span>
<span class="n">arrowprops</span><span class="o">=</span><span class="p">{</span><span class="s">&#39;facecolor&#39;</span><span class="p">:</span> <span class="s">&#39;black&#39;</span><span class="p">,</span> <span class="s">&#39;shrink&#39;</span><span class="p">:</span> <span class="o">.</span><span class="mi">05</span><span class="p">})</span>
<span class="n">plt</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="s">&#39;$\pm$ 1 std. dev. each day&#39;</span><span class="p">,</span> <span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="o">.</span><span class="mi">055</span><span class="p">),</span> <span class="p">(</span><span class="mf">2.5</span><span class="p">,</span> <span class="o">.</span><span class="mi">085</span><span class="p">),</span>
<span class="n">arrowprops</span><span class="o">=</span><span class="p">{</span><span class="s">&#39;facecolor&#39;</span><span class="p">:</span> <span class="s">&#39;black&#39;</span><span class="p">,</span> <span class="s">&#39;shrink&#39;</span><span class="p">:</span> <span class="o">.</span><span class="mi">05</span><span class="p">})</span>
<span class="n">plt</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="s">&#39;Min/Max each day&#39;</span><span class="p">,</span> <span class="p">(</span><span class="o">.</span><span class="mi">9</span><span class="p">,</span> <span class="o">-.</span><span class="mi">07</span><span class="p">),</span> <span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="o">-.</span><span class="mi">1</span><span class="p">),</span>
<span class="n">arrowprops</span><span class="o">=</span><span class="p">{</span><span class="s">&#39;facecolor&#39;</span><span class="p">:</span> <span class="s">&#39;black&#39;</span><span class="p">,</span> <span class="s">&#39;shrink&#39;</span><span class="p">:</span> <span class="o">.</span><span class="mi">05</span><span class="p">});</span>
<span class="n">arrowprops</span><span class="o">=</span><span class="nb">dict</span><span class="p">(</span><span class="n">facecolor</span><span class="o">=</span><span class="s1">&#39;black&#39;</span><span class="p">,</span> <span class="n">shrink</span><span class="o">=</span><span class="mf">0.05</span><span class="p">))</span>
<span class="n">plt</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="s1">&#39;&#39;</span><span class="p">,</span> <span class="p">(</span><span class="o">-.</span><span class="mi">1</span><span class="p">,</span> <span class="o">.</span><span class="mi">005</span><span class="p">),</span> <span class="p">(</span><span class="o">-.</span><span class="mi">5</span><span class="p">,</span> <span class="o">.</span><span class="mi">02</span><span class="p">),</span>
<span class="n">arrowprops</span><span class="o">=</span><span class="p">{</span><span class="s1">&#39;facecolor&#39;</span><span class="p">:</span> <span class="s1">&#39;black&#39;</span><span class="p">,</span> <span class="s1">&#39;shrink&#39;</span><span class="p">:</span> <span class="o">.</span><span class="mi">05</span><span class="p">})</span>
<span class="n">plt</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="s1">&#39;$\pm$ 1 std. dev. each day&#39;</span><span class="p">,</span> <span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="o">.</span><span class="mi">055</span><span class="p">),</span> <span class="p">(</span><span class="mf">2.5</span><span class="p">,</span> <span class="o">.</span><span class="mi">085</span><span class="p">),</span>
<span class="n">arrowprops</span><span class="o">=</span><span class="p">{</span><span class="s1">&#39;facecolor&#39;</span><span class="p">:</span> <span class="s1">&#39;black&#39;</span><span class="p">,</span> <span class="s1">&#39;shrink&#39;</span><span class="p">:</span> <span class="o">.</span><span class="mi">05</span><span class="p">})</span>
<span class="n">plt</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="s1">&#39;Min/Max each day&#39;</span><span class="p">,</span> <span class="p">(</span><span class="o">.</span><span class="mi">9</span><span class="p">,</span> <span class="o">-.</span><span class="mi">07</span><span class="p">),</span> <span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="o">-.</span><span class="mi">1</span><span class="p">),</span>
<span class="n">arrowprops</span><span class="o">=</span><span class="p">{</span><span class="s1">&#39;facecolor&#39;</span><span class="p">:</span> <span class="s1">&#39;black&#39;</span><span class="p">,</span> <span class="s1">&#39;shrink&#39;</span><span class="p">:</span> <span class="o">.</span><span class="mi">05</span><span class="p">});</span>
</pre></div>
</div>
@ -1957,24 +1957,24 @@ PlISLSIiIiLiIyXRIiIiIiI+UhItIiIiIuIjJdEiIiIiIj76P+bhd3LjvwIqAAAAAElFTkSuQmCC
<span class="n">ohlc_dataframe</span><span class="p">(</span><span class="n">FB</span><span class="p">[</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">18</span><span class="p">):],</span> <span class="n">FB_plot</span><span class="p">)</span>
<span class="n">FB_truncated</span> <span class="o">=</span> <span class="n">FB</span><span class="p">[</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">18</span><span class="p">):</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">27</span><span class="p">)]</span>
<span class="n">midpoint</span> <span class="o">=</span> <span class="n">FB_truncated</span><span class="p">[</span><span class="s">&#39;Open&#39;</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span> <span class="o">+</span> <span class="n">FB_truncated</span><span class="p">[</span><span class="s">&#39;Close&#39;</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span>
<span class="n">FB_plot</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">FB_truncated</span><span class="o">.</span><span class="n">index</span><span class="p">,</span> <span class="n">midpoint</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&#39;Midpoint&#39;</span><span class="p">)</span>
<span class="n">midpoint</span> <span class="o">=</span> <span class="n">FB_truncated</span><span class="p">[</span><span class="s1">&#39;Open&#39;</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span> <span class="o">+</span> <span class="n">FB_truncated</span><span class="p">[</span><span class="s1">&#39;Close&#39;</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span>
<span class="n">FB_plot</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">FB_truncated</span><span class="o">.</span><span class="n">index</span><span class="p">,</span> <span class="n">midpoint</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">&#39;Midpoint&#39;</span><span class="p">)</span>
<span class="n">FB_plot</span><span class="o">.</span><span class="n">vlines</span><span class="p">(</span><span class="n">date2num</span><span class="p">(</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">27</span><span class="p">,</span> <span class="mi">12</span><span class="p">)),</span>
<span class="n">ax_fb</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">0</span><span class="p">],</span> <span class="n">ax_fb</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">1</span><span class="p">],</span>
<span class="n">color</span><span class="o">=</span><span class="s">&#39;g&#39;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&#39;Earnings Release&#39;</span><span class="p">)</span>
<span class="n">color</span><span class="o">=</span><span class="s1">&#39;g&#39;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">&#39;Earnings Release&#39;</span><span class="p">)</span>
<span class="n">FB_plot</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
<span class="n">FB_plot</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s">&#39;FB Midpoint Plot&#39;</span><span class="p">)</span>
<span class="n">FB_plot</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s1">&#39;FB Midpoint Plot&#39;</span><span class="p">)</span>
<span class="n">AAPL_plot</span> <span class="o">=</span> <span class="n">axarr</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="n">ohlc_dataframe</span><span class="p">(</span><span class="n">AAPL</span><span class="p">[</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">10</span><span class="p">):],</span> <span class="n">AAPL_plot</span><span class="p">)</span>
<span class="n">AAPL_truncated</span> <span class="o">=</span> <span class="n">AAPL</span><span class="p">[</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">10</span><span class="p">):</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">26</span><span class="p">)]</span>
<span class="n">midpoint</span> <span class="o">=</span> <span class="n">AAPL_truncated</span><span class="p">[</span><span class="s">&#39;Open&#39;</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span> <span class="o">+</span> <span class="n">AAPL_truncated</span><span class="p">[</span><span class="s">&#39;Close&#39;</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span>
<span class="n">AAPL_plot</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">AAPL_truncated</span><span class="o">.</span><span class="n">index</span><span class="p">,</span> <span class="n">midpoint</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&#39;Midpoint&#39;</span><span class="p">)</span>
<span class="n">midpoint</span> <span class="o">=</span> <span class="n">AAPL_truncated</span><span class="p">[</span><span class="s1">&#39;Open&#39;</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span> <span class="o">+</span> <span class="n">AAPL_truncated</span><span class="p">[</span><span class="s1">&#39;Close&#39;</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span>
<span class="n">AAPL_plot</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">AAPL_truncated</span><span class="o">.</span><span class="n">index</span><span class="p">,</span> <span class="n">midpoint</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">&#39;Midpoint&#39;</span><span class="p">)</span>
<span class="n">AAPL_plot</span><span class="o">.</span><span class="n">vlines</span><span class="p">(</span><span class="n">date2num</span><span class="p">(</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">26</span><span class="p">,</span> <span class="mi">12</span><span class="p">)),</span>
<span class="n">ax_aapl</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">0</span><span class="p">],</span> <span class="n">ax_aapl</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">1</span><span class="p">],</span>
<span class="n">color</span><span class="o">=</span><span class="s">&#39;g&#39;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&#39;Earnings Release&#39;</span><span class="p">)</span>
<span class="n">color</span><span class="o">=</span><span class="s1">&#39;g&#39;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">&#39;Earnings Release&#39;</span><span class="p">)</span>
<span class="n">AAPL_plot</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">3</span><span class="p">)</span>
<span class="n">AAPL_plot</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s">&#39;AAPL Midpoint Plot&#39;</span><span class="p">);</span>
<span class="n">AAPL_plot</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s1">&#39;AAPL Midpoint Plot&#39;</span><span class="p">);</span>
</pre></div>
</div>
@ -2453,74 +2453,74 @@ TkSuQmCC
<div class="prompt input_prompt">In&nbsp;[7]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="c"># Read in the events for each stock;</span>
<span class="c"># The file was created using the first code block in the Appendix</span>
<div class=" highlight hl-ipython3"><pre><span class="c1"># Read in the events for each stock;</span>
<span class="c1"># The file was created using the first code block in the Appendix</span>
<span class="kn">import</span> <span class="nn">yaml</span>
<span class="kn">from</span> <span class="nn">dateutil.parser</span> <span class="k">import</span> <span class="n">parse</span>
<span class="kn">from</span> <span class="nn">progressbar</span> <span class="k">import</span> <span class="n">ProgressBar</span>
<span class="n">data_str</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s">&#39;earnings_dates.yaml&#39;</span><span class="p">,</span> <span class="s">&#39;r&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
<span class="c"># Need to remove invalid lines</span>
<span class="n">filtered</span> <span class="o">=</span> <span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="s">&#39;{&#39;</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">x</span><span class="p">,</span> <span class="n">data_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s">&#39;</span><span class="se">\n</span><span class="s">&#39;</span><span class="p">))</span>
<span class="n">earnings_data</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s">&#39;</span><span class="se">\n</span><span class="s">&#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">filtered</span><span class="p">))</span>
<span class="n">data_str</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s1">&#39;earnings_dates.yaml&#39;</span><span class="p">,</span> <span class="s1">&#39;r&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
<span class="c1"># Need to remove invalid lines</span>
<span class="n">filtered</span> <span class="o">=</span> <span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="s1">&#39;{&#39;</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">x</span><span class="p">,</span> <span class="n">data_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;</span><span class="se">\n</span><span class="s1">&#39;</span><span class="p">))</span>
<span class="n">earnings_data</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s1">&#39;</span><span class="se">\n</span><span class="s1">&#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">filtered</span><span class="p">))</span>
<span class="c"># Convert our earnings data into a list of (ticker, date) pairs</span>
<span class="c"># to make it easy to work with.</span>
<span class="c"># This is horribly inefficient, but should get us what we need</span>
<span class="c1"># Convert our earnings data into a list of (ticker, date) pairs</span>
<span class="c1"># to make it easy to work with.</span>
<span class="c1"># This is horribly inefficient, but should get us what we need</span>
<span class="n">ticker_dates</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">ticker</span><span class="p">,</span> <span class="n">date_list</span> <span class="ow">in</span> <span class="n">earnings_data</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
<span class="k">for</span> <span class="n">iso_str</span> <span class="ow">in</span> <span class="n">date_list</span><span class="p">:</span>
<span class="n">ticker_dates</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">ticker</span><span class="p">,</span> <span class="n">parse</span><span class="p">(</span><span class="n">iso_str</span><span class="p">)))</span>
<span class="k">def</span> <span class="nf">does_trend_down</span><span class="p">(</span><span class="n">ticker</span><span class="p">,</span> <span class="n">event</span><span class="p">,</span> <span class="n">horizon</span><span class="p">):</span>
<span class="c"># Figure out if the `event` has a downtrend for</span>
<span class="c"># the `horizon` days preceding it</span>
<span class="c"># As an interpretation note: it is assumed that</span>
<span class="c"># the closing price of day `event` is the reference</span>
<span class="c"># point, and we want `horizon` days before that.</span>
<span class="c"># The price_data.hdf was created in the second appendix code block</span>
<span class="c1"># Figure out if the `event` has a downtrend for</span>
<span class="c1"># the `horizon` days preceding it</span>
<span class="c1"># As an interpretation note: it is assumed that</span>
<span class="c1"># the closing price of day `event` is the reference</span>
<span class="c1"># point, and we want `horizon` days before that.</span>
<span class="c1"># The price_data.hdf was created in the second appendix code block</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_hdf</span><span class="p">(</span><span class="s">&#39;price_data.hdf&#39;</span><span class="p">,</span> <span class="n">ticker</span><span class="p">)</span>
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_hdf</span><span class="p">(</span><span class="s1">&#39;price_data.hdf&#39;</span><span class="p">,</span> <span class="n">ticker</span><span class="p">)</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">ticker_data</span><span class="p">[</span><span class="n">event</span><span class="o">-</span><span class="n">TradeDay</span><span class="p">(</span><span class="n">horizon</span><span class="p">):</span><span class="n">event</span><span class="p">]</span>
<span class="n">midpoints</span> <span class="o">=</span> <span class="n">data</span><span class="p">[</span><span class="s">&#39;Open&#39;</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span> <span class="o">+</span> <span class="n">data</span><span class="p">[</span><span class="s">&#39;Close&#39;</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span>
<span class="n">midpoints</span> <span class="o">=</span> <span class="n">data</span><span class="p">[</span><span class="s1">&#39;Open&#39;</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span> <span class="o">+</span> <span class="n">data</span><span class="p">[</span><span class="s1">&#39;Close&#39;</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span>
<span class="c"># Shift dates one forward into the future and subtract</span>
<span class="c"># Effectively: do we trend down over all days?</span>
<span class="c1"># Shift dates one forward into the future and subtract</span>
<span class="c1"># Effectively: do we trend down over all days?</span>
<span class="n">elems</span> <span class="o">=</span> <span class="n">midpoints</span> <span class="o">-</span> <span class="n">midpoints</span><span class="o">.</span><span class="n">shift</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="n">elems</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="n">elems</span><span class="o">.</span><span class="n">dropna</span><span class="p">()[</span><span class="n">elems</span> <span class="o">&lt;=</span> <span class="mi">0</span><span class="p">])</span>
<span class="k">except</span> <span class="ne">KeyError</span><span class="p">:</span>
<span class="c"># If the stock doesn&#39;t exist, it doesn&#39;t qualify as trending down</span>
<span class="c"># Mostly this is here to make sure the entire analysis doesn&#39;t </span>
<span class="c"># blow up if there were issues in data retrieval</span>
<span class="k">return</span> <span class="k">False</span>
<span class="c1"># If the stock doesn&#39;t exist, it doesn&#39;t qualify as trending down</span>
<span class="c1"># Mostly this is here to make sure the entire analysis doesn&#39;t </span>
<span class="c1"># blow up if there were issues in data retrieval</span>
<span class="k">return</span> <span class="kc">False</span>
<span class="k">def</span> <span class="nf">study_trend</span><span class="p">(</span><span class="n">horizon</span><span class="p">,</span> <span class="n">trend_function</span><span class="p">):</span>
<span class="n">five_day_events</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">((</span><span class="mi">1</span><span class="p">,</span> <span class="n">horizon</span><span class="o">*</span><span class="mi">2</span> <span class="o">+</span> <span class="mi">1</span><span class="p">))</span>
<span class="n">invalid_events</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">ticker</span><span class="p">,</span> <span class="n">event</span> <span class="ow">in</span> <span class="n">ProgressBar</span><span class="p">()(</span><span class="n">ticker_dates</span><span class="p">):</span>
<span class="k">if</span> <span class="n">trend_function</span><span class="p">(</span><span class="n">ticker</span><span class="p">,</span> <span class="n">event</span><span class="p">,</span> <span class="n">horizon</span><span class="p">):</span>
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_hdf</span><span class="p">(</span><span class="s">&#39;price_data.hdf&#39;</span><span class="p">,</span> <span class="n">ticker</span><span class="p">)</span>
<span class="n">event_data</span> <span class="o">=</span> <span class="n">ticker_data</span><span class="p">[</span><span class="n">event</span><span class="o">-</span><span class="n">TradeDay</span><span class="p">(</span><span class="n">horizon</span><span class="p">):</span><span class="n">event</span><span class="o">+</span><span class="n">TradeDay</span><span class="p">(</span><span class="n">horizon</span><span class="p">)][</span><span class="s">&#39;Close&#39;</span><span class="p">]</span>
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_hdf</span><span class="p">(</span><span class="s1">&#39;price_data.hdf&#39;</span><span class="p">,</span> <span class="n">ticker</span><span class="p">)</span>
<span class="n">event_data</span> <span class="o">=</span> <span class="n">ticker_data</span><span class="p">[</span><span class="n">event</span><span class="o">-</span><span class="n">TradeDay</span><span class="p">(</span><span class="n">horizon</span><span class="p">):</span><span class="n">event</span><span class="o">+</span><span class="n">TradeDay</span><span class="p">(</span><span class="n">horizon</span><span class="p">)][</span><span class="s1">&#39;Close&#39;</span><span class="p">]</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">five_day_events</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">vstack</span><span class="p">([</span><span class="n">five_day_events</span><span class="p">,</span> <span class="n">event_data</span><span class="p">])</span>
<span class="k">except</span> <span class="ne">ValueError</span><span class="p">:</span>
<span class="c"># Sometimes we don&#39;t get exactly the right number of values due to calendar</span>
<span class="c"># issues. I&#39;ve fixed most everything I can, and the few issues that are left</span>
<span class="c"># I assume don&#39;t systemically bias the results (i.e. data could be missing</span>
<span class="c"># because it doesn&#39;t exist, etc.). After running through, ~1% of events get</span>
<span class="c"># discarded this way</span>
<span class="c1"># Sometimes we don&#39;t get exactly the right number of values due to calendar</span>
<span class="c1"># issues. I&#39;ve fixed most everything I can, and the few issues that are left</span>
<span class="c1"># I assume don&#39;t systemically bias the results (i.e. data could be missing</span>
<span class="c1"># because it doesn&#39;t exist, etc.). After running through, ~1% of events get</span>
<span class="c1"># discarded this way</span>
<span class="n">invalid_events</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">ticker</span><span class="p">,</span> <span class="n">event</span><span class="p">))</span>
<span class="c"># Remove our initial zero row</span>
<span class="c1"># Remove our initial zero row</span>
<span class="n">five_day_events</span> <span class="o">=</span> <span class="n">five_day_events</span><span class="p">[</span><span class="mi">1</span><span class="p">:,:]</span>
<span class="n">plot_study</span><span class="p">(</span><span class="n">five_day_events</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">&#39;Action over {} days: {} events&#39;</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s1">&#39;Action over {} days: {} events&#39;</span>
<span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">horizon</span><span class="p">,</span><span class="n">five_day_events</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]))</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
<span class="c"># Start with a 5 day study</span>
<span class="c1"># Start with a 5 day study</span>
<span class="n">study_trend</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="n">does_trend_down</span><span class="p">)</span>
</pre></div>
@ -3000,7 +3000,7 @@ ADBuCBoAAAAAAMC4+f+DLBbJVWQG2gAAAABJRU5ErkJggg==
<div class="prompt input_prompt">In&nbsp;[8]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="c"># 8 day study next</span>
<div class=" highlight hl-ipython3"><pre><span class="c1"># 8 day study next</span>
<span class="n">study_trend</span><span class="p">(</span><span class="mi">8</span><span class="p">,</span> <span class="n">does_trend_down</span><span class="p">)</span>
</pre></div>
@ -3479,7 +3479,7 @@ AAAAKMr/B9pAH9lY/+K6AAAAAElFTkSuQmCC
<div class="prompt input_prompt">In&nbsp;[9]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="c"># 3 day study after that</span>
<div class=" highlight hl-ipython3"><pre><span class="c1"># 3 day study after that</span>
<span class="n">study_trend</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="n">does_trend_down</span><span class="p">)</span>
</pre></div>
@ -3892,26 +3892,26 @@ STfmKmb+uQ4AAAAASUVORK5CYII=
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">does_trend_up</span><span class="p">(</span><span class="n">ticker</span><span class="p">,</span> <span class="n">event</span><span class="p">,</span> <span class="n">horizon</span><span class="p">):</span>
<span class="c"># Figure out if the `event` has an uptrend for</span>
<span class="c"># the `horizon` days preceding it</span>
<span class="c"># As an interpretation note: it is assumed that</span>
<span class="c"># the closing price of day `event` is the reference</span>
<span class="c"># point, and we want `horizon` days before that.</span>
<span class="c"># The price_data.hdf was created in the second appendix code block</span>
<span class="c1"># Figure out if the `event` has an uptrend for</span>
<span class="c1"># the `horizon` days preceding it</span>
<span class="c1"># As an interpretation note: it is assumed that</span>
<span class="c1"># the closing price of day `event` is the reference</span>
<span class="c1"># point, and we want `horizon` days before that.</span>
<span class="c1"># The price_data.hdf was created in the second appendix code block</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_hdf</span><span class="p">(</span><span class="s">&#39;price_data.hdf&#39;</span><span class="p">,</span> <span class="n">ticker</span><span class="p">)</span>
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_hdf</span><span class="p">(</span><span class="s1">&#39;price_data.hdf&#39;</span><span class="p">,</span> <span class="n">ticker</span><span class="p">)</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">ticker_data</span><span class="p">[</span><span class="n">event</span><span class="o">-</span><span class="n">TradeDay</span><span class="p">(</span><span class="n">horizon</span><span class="p">):</span><span class="n">event</span><span class="p">]</span>
<span class="n">midpoints</span> <span class="o">=</span> <span class="n">data</span><span class="p">[</span><span class="s">&#39;Open&#39;</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span> <span class="o">+</span> <span class="n">data</span><span class="p">[</span><span class="s">&#39;Close&#39;</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span>
<span class="n">midpoints</span> <span class="o">=</span> <span class="n">data</span><span class="p">[</span><span class="s1">&#39;Open&#39;</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span> <span class="o">+</span> <span class="n">data</span><span class="p">[</span><span class="s1">&#39;Close&#39;</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span>
<span class="c"># Shift dates one forward into the future and subtract</span>
<span class="c"># Effectively: do we trend down over all days?</span>
<span class="c1"># Shift dates one forward into the future and subtract</span>
<span class="c1"># Effectively: do we trend down over all days?</span>
<span class="n">elems</span> <span class="o">=</span> <span class="n">midpoints</span> <span class="o">-</span> <span class="n">midpoints</span><span class="o">.</span><span class="n">shift</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="n">elems</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="n">elems</span><span class="o">.</span><span class="n">dropna</span><span class="p">()[</span><span class="n">elems</span> <span class="o">&gt;=</span> <span class="mi">0</span><span class="p">])</span>
<span class="k">except</span> <span class="ne">KeyError</span><span class="p">:</span>
<span class="c"># If the stock doesn&#39;t exist, it doesn&#39;t qualify as trending down</span>
<span class="c"># Mostly this is here to make sure the entire analysis doesn&#39;t </span>
<span class="c"># blow up if there were issues in data retrieval</span>
<span class="k">return</span> <span class="k">False</span>
<span class="c1"># If the stock doesn&#39;t exist, it doesn&#39;t qualify as trending down</span>
<span class="c1"># Mostly this is here to make sure the entire analysis doesn&#39;t </span>
<span class="c1"># blow up if there were issues in data retrieval</span>
<span class="k">return</span> <span class="kc">False</span>
<span class="n">study_trend</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="n">does_trend_up</span><span class="p">)</span>
</pre></div>
@ -5192,38 +5192,38 @@ AAAAAGBWNBAAAAAAAMCsaCAAAAAAAIBZ0UAAAAAAAACz+n9WbX+0Yscg8AAAAABJRU5ErkJggg==
<span class="bp">self</span><span class="o">.</span><span class="n">dates</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">def</span> <span class="nf">handle_starttag</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">,</span> <span class="n">attrs</span><span class="p">):</span>
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s">&#39;table&#39;</span><span class="p">:</span>
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s1">&#39;table&#39;</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">store_dates</span> <span class="o">=</span> <span class="bp">True</span>
<span class="k">def</span> <span class="nf">handle_data</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">data</span><span class="p">):</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">store_dates</span><span class="p">:</span>
<span class="n">match</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s">r&#39;\d+/\d+/\d+&#39;</span><span class="p">,</span> <span class="n">data</span><span class="p">)</span>
<span class="n">match</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s1">r&#39;\d+/\d+/\d+&#39;</span><span class="p">,</span> <span class="n">data</span><span class="p">)</span>
<span class="k">if</span> <span class="n">match</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">dates</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">match</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">0</span><span class="p">))</span>
<span class="c"># If a company reports before the bell, record the earnings date</span>
<span class="c"># being at midnight the day before. Ex: WMT reports 5/19/2016,</span>
<span class="c"># but we want the reference point to be the closing price on 5/18/2016</span>
<span class="k">if</span> <span class="s">&#39;After Close&#39;</span> <span class="ow">in</span> <span class="n">data</span><span class="p">:</span>
<span class="c1"># If a company reports before the bell, record the earnings date</span>
<span class="c1"># being at midnight the day before. Ex: WMT reports 5/19/2016,</span>
<span class="c1"># but we want the reference point to be the closing price on 5/18/2016</span>
<span class="k">if</span> <span class="s1">&#39;After Close&#39;</span> <span class="ow">in</span> <span class="n">data</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">earnings_offset</span> <span class="o">=</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="k">elif</span> <span class="s">&#39;Before Open&#39;</span> <span class="ow">in</span> <span class="n">data</span><span class="p">:</span>
<span class="k">elif</span> <span class="s1">&#39;Before Open&#39;</span> <span class="ow">in</span> <span class="n">data</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">earnings_offset</span> <span class="o">=</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=-</span><span class="mi">1</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">handle_endtag</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">):</span>
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s">&#39;table&#39;</span><span class="p">:</span>
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s1">&#39;table&#39;</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">store_dates</span> <span class="o">=</span> <span class="bp">False</span>
<span class="k">def</span> <span class="nf">earnings_releases</span><span class="p">(</span><span class="n">ticker</span><span class="p">):</span>
<span class="c">#print(&quot;Looking up ticker {}&quot;.format(ticker))</span>
<span class="n">user_agent</span> <span class="o">=</span> <span class="s">&#39;Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) &#39;</span>\
<span class="s">&#39;Gecko/20100101 Firefox/46.0&#39;</span>
<span class="n">headers</span> <span class="o">=</span> <span class="p">{</span><span class="s">&#39;user-agent&#39;</span><span class="p">:</span> <span class="n">user_agent</span><span class="p">}</span>
<span class="n">base_url</span> <span class="o">=</span> <span class="s">&#39;http://www.streetinsider.com/ec_earnings.php?q={}&#39;</span>\
<span class="c1">#print(&quot;Looking up ticker {}&quot;.format(ticker))</span>
<span class="n">user_agent</span> <span class="o">=</span> <span class="s1">&#39;Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) &#39;</span>\
<span class="s1">&#39;Gecko/20100101 Firefox/46.0&#39;</span>
<span class="n">headers</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;user-agent&#39;</span><span class="p">:</span> <span class="n">user_agent</span><span class="p">}</span>
<span class="n">base_url</span> <span class="o">=</span> <span class="s1">&#39;http://www.streetinsider.com/ec_earnings.php?q={}&#39;</span>\
<span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">ticker</span><span class="p">)</span>
<span class="n">e</span> <span class="o">=</span> <span class="n">EarningsParser</span><span class="p">()</span>
<span class="n">s</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">Session</span><span class="p">()</span>
<span class="n">a</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">adapters</span><span class="o">.</span><span class="n">HTTPAdapter</span><span class="p">(</span><span class="n">max_retries</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="n">s</span><span class="o">.</span><span class="n">mount</span><span class="p">(</span><span class="s">&#39;http://&#39;</span><span class="p">,</span> <span class="n">a</span><span class="p">)</span>
<span class="n">s</span><span class="o">.</span><span class="n">mount</span><span class="p">(</span><span class="s1">&#39;http://&#39;</span><span class="p">,</span> <span class="n">a</span><span class="p">)</span>
<span class="n">e</span><span class="o">.</span><span class="n">feed</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">s</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">base_url</span><span class="p">,</span> <span class="n">headers</span><span class="o">=</span><span class="n">headers</span><span class="p">)</span><span class="o">.</span><span class="n">content</span><span class="p">))</span>
<span class="k">if</span> <span class="n">e</span><span class="o">.</span><span class="n">earnings_offset</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span><span class="p">:</span>
@ -5231,18 +5231,18 @@ AAAAAGBWNBAAAAAAAMCsaCAAAAAAAIBZ0UAAAAAAAACz+n9WbX+0Yscg8AAAAABJRU5ErkJggg==
<span class="n">past</span> <span class="o">=</span> <span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="o">&lt;</span> <span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">(),</span> <span class="n">dates</span><span class="p">)</span>
<span class="k">return</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">d</span><span class="p">:</span> <span class="n">d</span><span class="o">.</span><span class="n">isoformat</span><span class="p">(),</span> <span class="n">past</span><span class="p">))</span>
<span class="c"># Use a Russell-3000 ETF tracker (ticker IWV) to get a list of holdings</span>
<span class="n">r3000</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s">&#39;https://www.ishares.com/us/products/239714/&#39;</span>
<span class="s">&#39;ishares-russell-3000-etf/1449138789749.ajax?&#39;</span>
<span class="s">&#39;fileType=csv&amp;fileName=IWV_holdings&amp;dataType=fund&#39;</span><span class="p">,</span>
<span class="c1"># Use a Russell-3000 ETF tracker (ticker IWV) to get a list of holdings</span>
<span class="n">r3000</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s1">&#39;https://www.ishares.com/us/products/239714/&#39;</span>
<span class="s1">&#39;ishares-russell-3000-etf/1449138789749.ajax?&#39;</span>
<span class="s1">&#39;fileType=csv&amp;fileName=IWV_holdings&amp;dataType=fund&#39;</span><span class="p">,</span>
<span class="n">header</span><span class="o">=</span><span class="mi">10</span><span class="p">)</span>
<span class="n">r3000_equities</span> <span class="o">=</span> <span class="n">r3000</span><span class="p">[(</span><span class="n">r3000</span><span class="p">[</span><span class="s">&#39;Exchange&#39;</span><span class="p">]</span> <span class="o">==</span> <span class="s">&#39;NASDAQ&#39;</span><span class="p">)</span> <span class="o">|</span>
<span class="p">(</span><span class="n">r3000</span><span class="p">[</span><span class="s">&#39;Exchange&#39;</span><span class="p">]</span> <span class="o">==</span> <span class="s">&#39;New York Stock Exchange Inc.&#39;</span><span class="p">)]</span>
<span class="n">r3000_equities</span> <span class="o">=</span> <span class="n">r3000</span><span class="p">[(</span><span class="n">r3000</span><span class="p">[</span><span class="s1">&#39;Exchange&#39;</span><span class="p">]</span> <span class="o">==</span> <span class="s1">&#39;NASDAQ&#39;</span><span class="p">)</span> <span class="o">|</span>
<span class="p">(</span><span class="n">r3000</span><span class="p">[</span><span class="s1">&#39;Exchange&#39;</span><span class="p">]</span> <span class="o">==</span> <span class="s1">&#39;New York Stock Exchange Inc.&#39;</span><span class="p">)]</span>
<span class="n">dates_file</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s">&#39;earnings_dates.yaml&#39;</span><span class="p">,</span> <span class="s">&#39;w&#39;</span><span class="p">)</span>
<span class="n">dates_file</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s1">&#39;earnings_dates.yaml&#39;</span><span class="p">,</span> <span class="s1">&#39;w&#39;</span><span class="p">)</span>
<span class="k">with</span> <span class="n">futures</span><span class="o">.</span><span class="n">ThreadPoolExecutor</span><span class="p">(</span><span class="n">max_workers</span><span class="o">=</span><span class="mi">8</span><span class="p">)</span> <span class="k">as</span> <span class="n">pool</span><span class="p">:</span>
<span class="n">fs</span> <span class="o">=</span> <span class="p">{</span><span class="n">pool</span><span class="o">.</span><span class="n">submit</span><span class="p">(</span><span class="n">earnings_releases</span><span class="p">,</span> <span class="n">r3000_equities</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="n">t</span><span class="p">][</span><span class="s">&#39;Ticker&#39;</span><span class="p">]):</span> <span class="n">t</span>
<span class="n">fs</span> <span class="o">=</span> <span class="p">{</span><span class="n">pool</span><span class="o">.</span><span class="n">submit</span><span class="p">(</span><span class="n">earnings_releases</span><span class="p">,</span> <span class="n">r3000_equities</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="n">t</span><span class="p">][</span><span class="s1">&#39;Ticker&#39;</span><span class="p">]):</span> <span class="n">t</span>
<span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="n">r3000_equities</span><span class="o">.</span><span class="n">index</span><span class="p">}</span>
<span class="n">pbar</span> <span class="o">=</span> <span class="n">progressbar</span><span class="o">.</span><span class="n">ProgressBar</span><span class="p">(</span><span class="n">term_width</span><span class="o">=</span><span class="mi">80</span><span class="p">,</span>
<span class="n">max_value</span><span class="o">=</span><span class="n">r3000_equities</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">max</span><span class="p">())</span>
@ -5250,7 +5250,7 @@ AAAAAGBWNBAAAAAAAMCsaCAAAAAAAIBZ0UAAAAAAAACz+n9WbX+0Yscg8AAAAABJRU5ErkJggg==
<span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="o">.</span><span class="n">as_completed</span><span class="p">(</span><span class="n">fs</span><span class="p">):</span>
<span class="n">i</span> <span class="o">=</span> <span class="n">fs</span><span class="p">[</span><span class="n">future</span><span class="p">]</span>
<span class="n">pbar</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">i</span><span class="p">)</span>
<span class="n">dates_file</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">yaml</span><span class="o">.</span><span class="n">dump</span><span class="p">({</span><span class="n">r3000_equities</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s">&#39;Ticker&#39;</span><span class="p">]:</span>
<span class="n">dates_file</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">yaml</span><span class="o">.</span><span class="n">dump</span><span class="p">({</span><span class="n">r3000_equities</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s1">&#39;Ticker&#39;</span><span class="p">]:</span>
<span class="n">future</span><span class="o">.</span><span class="n">result</span><span class="p">()}))</span>
</pre></div>
<p>Downloading stock price data needed for the event studies:</p>
@ -5263,31 +5263,31 @@ AAAAAGBWNBAAAAAAAMCsaCAAAAAAAIBZ0UAAAAAAAACz+n9WbX+0Yscg8AAAAABJRU5ErkJggg==
<span class="kn">from</span> <span class="nn">progressbar</span> <span class="kn">import</span> <span class="n">ProgressBar</span>
<span class="k">def</span> <span class="nf">fetch_ticker</span><span class="p">(</span><span class="n">ticker</span><span class="p">,</span> <span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">):</span>
<span class="c"># Quandl is currently giving me issues with returning</span>
<span class="c"># the entire dataset and not slicing server-side.</span>
<span class="c"># So instead, we&#39;ll do it client-side!</span>
<span class="n">q_format</span> <span class="o">=</span> <span class="s">&#39;%Y-%m-</span><span class="si">%d</span><span class="s">&#39;</span>
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">&#39;YAHOO/&#39;</span> <span class="o">+</span> <span class="n">ticker</span><span class="p">,</span>
<span class="c1"># Quandl is currently giving me issues with returning</span>
<span class="c1"># the entire dataset and not slicing server-side.</span>
<span class="c1"># So instead, we&#39;ll do it client-side!</span>
<span class="n">q_format</span> <span class="o">=</span> <span class="s1">&#39;%Y-%m-</span><span class="si">%d</span><span class="s1">&#39;</span>
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">&#39;YAHOO/&#39;</span> <span class="o">+</span> <span class="n">ticker</span><span class="p">,</span>
<span class="n">start_date</span><span class="o">=</span><span class="n">start</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="n">q_format</span><span class="p">),</span>
<span class="n">end_date</span><span class="o">=</span><span class="n">end</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="n">q_format</span><span class="p">),</span>
<span class="n">authtoken</span><span class="o">=</span><span class="n">QUANDL_KEY</span><span class="p">)</span>
<span class="k">return</span> <span class="n">ticker_data</span>
<span class="n">data_str</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s">&#39;earnings_dates.yaml&#39;</span><span class="p">,</span> <span class="s">&#39;r&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
<span class="c"># Need to remove invalid lines</span>
<span class="n">filtered</span> <span class="o">=</span> <span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="s">&#39;{&#39;</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">x</span><span class="p">,</span> <span class="n">data_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s">&#39;</span><span class="se">\n</span><span class="s">&#39;</span><span class="p">))</span>
<span class="n">earnings_data</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s">&#39;</span><span class="se">\n</span><span class="s">&#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">filtered</span><span class="p">))</span>
<span class="n">data_str</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s1">&#39;earnings_dates.yaml&#39;</span><span class="p">,</span> <span class="s1">&#39;r&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
<span class="c1"># Need to remove invalid lines</span>
<span class="n">filtered</span> <span class="o">=</span> <span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="s1">&#39;{&#39;</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">x</span><span class="p">,</span> <span class="n">data_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;</span><span class="se">\n</span><span class="s1">&#39;</span><span class="p">))</span>
<span class="n">earnings_data</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s1">&#39;</span><span class="se">\n</span><span class="s1">&#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">filtered</span><span class="p">))</span>
<span class="c"># Get the first 1500 keys - split up into two statements</span>
<span class="c"># because of Quandl rate limits</span>
<span class="c1"># Get the first 1500 keys - split up into two statements</span>
<span class="c1"># because of Quandl rate limits</span>
<span class="n">tickers</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">earnings_data</span><span class="o">.</span><span class="n">keys</span><span class="p">())</span>
<span class="n">price_dict</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">invalid_tickers</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">ticker</span> <span class="ow">in</span> <span class="n">ProgressBar</span><span class="p">()(</span><span class="n">tickers</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">1500</span><span class="p">]):</span>
<span class="k">try</span><span class="p">:</span>
<span class="c"># Replace &#39;.&#39; with &#39;-&#39; in name for some tickers</span>
<span class="n">fixed</span> <span class="o">=</span> <span class="n">ticker</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s">&#39;.&#39;</span><span class="p">,</span> <span class="s">&#39;-&#39;</span><span class="p">)</span>
<span class="c1"># Replace &#39;.&#39; with &#39;-&#39; in name for some tickers</span>
<span class="n">fixed</span> <span class="o">=</span> <span class="n">ticker</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">&#39;.&#39;</span><span class="p">,</span> <span class="s1">&#39;-&#39;</span><span class="p">)</span>
<span class="n">event_strs</span> <span class="o">=</span> <span class="n">earnings_data</span><span class="p">[</span><span class="n">ticker</span><span class="p">]</span>
<span class="n">events</span> <span class="o">=</span> <span class="p">[</span><span class="n">parse</span><span class="p">(</span><span class="n">event</span><span class="p">)</span> <span class="k">for</span> <span class="n">event</span> <span class="ow">in</span> <span class="n">event_strs</span><span class="p">]</span>
<span class="n">td</span> <span class="o">=</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">20</span><span class="p">)</span>
@ -5296,11 +5296,11 @@ AAAAAGBWNBAAAAAAAMCsaCAAAAAAAIBZ0UAAAAAAAACz+n9WbX+0Yscg8AAAAABJRU5ErkJggg==
<span class="k">except</span> <span class="n">quandl</span><span class="o">.</span><span class="n">NotFoundError</span><span class="p">:</span>
<span class="n">invalid_tickers</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">ticker</span><span class="p">)</span>
<span class="c"># Execute this after 10 minutes have passed</span>
<span class="c1"># Execute this after 10 minutes have passed</span>
<span class="k">for</span> <span class="n">ticker</span> <span class="ow">in</span> <span class="n">ProgressBar</span><span class="p">()(</span><span class="n">tickers</span><span class="p">[</span><span class="mi">1500</span><span class="p">:]):</span>
<span class="k">try</span><span class="p">:</span>
<span class="c"># Replace &#39;.&#39; with &#39;-&#39; in name for some tickers</span>
<span class="n">fixed</span> <span class="o">=</span> <span class="n">ticker</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s">&#39;.&#39;</span><span class="p">,</span> <span class="s">&#39;-&#39;</span><span class="p">)</span>
<span class="c1"># Replace &#39;.&#39; with &#39;-&#39; in name for some tickers</span>
<span class="n">fixed</span> <span class="o">=</span> <span class="n">ticker</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">&#39;.&#39;</span><span class="p">,</span> <span class="s1">&#39;-&#39;</span><span class="p">)</span>
<span class="n">event_strs</span> <span class="o">=</span> <span class="n">earnings_data</span><span class="p">[</span><span class="n">ticker</span><span class="p">]</span>
<span class="n">events</span> <span class="o">=</span> <span class="p">[</span><span class="n">parse</span><span class="p">(</span><span class="n">event</span><span class="p">)</span> <span class="k">for</span> <span class="n">event</span> <span class="ow">in</span> <span class="n">event_strs</span><span class="p">]</span>
<span class="n">td</span> <span class="o">=</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">20</span><span class="p">)</span>
@ -5309,7 +5309,7 @@ AAAAAGBWNBAAAAAAAMCsaCAAAAAAAIBZ0UAAAAAAAACz+n9WbX+0Yscg8AAAAABJRU5ErkJggg==
<span class="k">except</span> <span class="n">quandl</span><span class="o">.</span><span class="n">NotFoundError</span><span class="p">:</span>
<span class="n">invalid_tickers</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">ticker</span><span class="p">)</span>
<span class="n">prices_store</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">HDFStore</span><span class="p">(</span><span class="s">&#39;price_data.hdf&#39;</span><span class="p">)</span>
<span class="n">prices_store</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">HDFStore</span><span class="p">(</span><span class="s1">&#39;price_data.hdf&#39;</span><span class="p">)</span>
<span class="k">for</span> <span class="n">ticker</span><span class="p">,</span> <span class="n">prices</span> <span class="ow">in</span> <span class="n">price_dict</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
<span class="n">prices_store</span><span class="p">[</span><span class="n">ticker</span><span class="p">]</span> <span class="o">=</span> <span class="n">prices</span>
</pre></div>

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -83,6 +83,8 @@
<div class="container content archive">
<h2><a href="https://bspeice.github.io/index.html"> </a></h2>
<dl class="dl-horizontal">
<dt>Sat 22 October 2016</dt>
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
<dt>Wed 08 June 2016</dt>
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
<dt>Sun 15 May 2016</dt>
@ -101,8 +103,6 @@
<dd><a href="https://bspeice.github.io/cloudy-in-seattle.html">Cloudy in Seattle</a></dd>
<dt>Fri 01 January 2016</dt>
<dd><a href="https://bspeice.github.io/complaining-about-the-weather.html">Complaining about the Weather</a></dd>
<dt>Sat 26 December 2015</dt>
<dd><a href="https://bspeice.github.io/testing-cramer.html">Testing Cramer</a></dd>
</dl>
</div>
</div>

View File

@ -83,6 +83,8 @@
<div class="container content archive">
<h2><a href="https://bspeice.github.io/index2.html"> <small>- page 2</small></a></h2>
<dl class="dl-horizontal">
<dt>Sat 26 December 2015</dt>
<dd><a href="https://bspeice.github.io/testing-cramer.html">Testing Cramer</a></dd>
<dt>Fri 27 November 2015</dt>
<dd><a href="https://bspeice.github.io/autocallable-bonds.html">Autocallable Bonds</a></dd>
<dt>Thu 19 November 2015</dt>

View File

@ -108,13 +108,13 @@
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
<span class="o">%</span><span class="k">matplotlib</span> inline
<span class="c"># Record how long it takes to run the notebook - I&#39;m curious.</span>
<span class="c1"># Record how long it takes to run the notebook - I&#39;m curious.</span>
<span class="kn">from</span> <span class="nn">datetime</span> <span class="k">import</span> <span class="n">datetime</span>
<span class="n">start</span> <span class="o">=</span> <span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span>
<span class="n">dataset</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s">&#39;split_train.csv&#39;</span><span class="p">)</span>
<span class="n">dataset</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s1">&#39;split_train.csv&#39;</span><span class="p">)</span>
<span class="n">dataset</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">ID</span>
<span class="n">X</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">drop</span><span class="p">([</span><span class="s">&#39;TARGET&#39;</span><span class="p">,</span> <span class="s">&#39;ID&#39;</span><span class="p">,</span> <span class="s">&#39;ID.1&#39;</span><span class="p">],</span> <span class="mi">1</span><span class="p">)</span>
<span class="n">X</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">drop</span><span class="p">([</span><span class="s1">&#39;TARGET&#39;</span><span class="p">,</span> <span class="s1">&#39;ID&#39;</span><span class="p">,</span> <span class="s1">&#39;ID.1&#39;</span><span class="p">],</span> <span class="mi">1</span><span class="p">)</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">TARGET</span>
</pre></div>
@ -244,13 +244,13 @@
<div class="prompt input_prompt">In&nbsp;[5]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="c"># First we need to `binarize` the data to 0-1; some of the labels are {0, 1},</span>
<span class="c"># some are {0, 3}, etc.</span>
<div class=" highlight hl-ipython3"><pre><span class="c1"># First we need to `binarize` the data to 0-1; some of the labels are {0, 1},</span>
<span class="c1"># some are {0, 3}, etc.</span>
<span class="kn">from</span> <span class="nn">sklearn.preprocessing</span> <span class="k">import</span> <span class="n">binarize</span>
<span class="n">X_bin</span> <span class="o">=</span> <span class="n">binarize</span><span class="p">(</span><span class="n">X</span><span class="p">[</span><span class="n">b_class</span><span class="p">])</span>
<span class="n">accuracy</span> <span class="o">=</span> <span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">X_bin</span><span class="p">[:,</span><span class="n">i</span><span class="p">]</span> <span class="o">==</span> <span class="n">y</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">b_class</span><span class="p">))]</span>
<span class="n">acc_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">({</span><span class="s">&quot;Accuracy&quot;</span><span class="p">:</span> <span class="n">accuracy</span><span class="p">},</span> <span class="n">index</span><span class="o">=</span><span class="n">b_class</span><span class="p">)</span>
<span class="n">acc_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">({</span><span class="s2">&quot;Accuracy&quot;</span><span class="p">:</span> <span class="n">accuracy</span><span class="p">},</span> <span class="n">index</span><span class="o">=</span><span class="n">b_class</span><span class="p">)</span>
<span class="n">acc_df</span><span class="o">.</span><span class="n">describe</span><span class="p">()</span>
</pre></div>
@ -334,9 +334,9 @@
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">unsat</span> <span class="o">=</span> <span class="n">y</span><span class="p">[</span><span class="n">y</span> <span class="o">==</span> <span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">count</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s">&quot;Satisfied customers: {}; Unsatisfied customers: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">y</span><span class="p">)</span> <span class="o">-</span> <span class="n">unsat</span><span class="p">,</span> <span class="n">unsat</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Satisfied customers: {}; Unsatisfied customers: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">y</span><span class="p">)</span> <span class="o">-</span> <span class="n">unsat</span><span class="p">,</span> <span class="n">unsat</span><span class="p">))</span>
<span class="n">naive_guess</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">y</span> <span class="o">==</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">y</span><span class="p">)))</span>
<span class="nb">print</span><span class="p">(</span><span class="s">&quot;Naive guess accuracy: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Naive guess accuracy: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">))</span>
</pre></div>
</div>
@ -397,9 +397,9 @@ Naive guess accuracy: 0.9608561656706882
<span class="k">return</span> <span class="n">gnb</span><span class="o">.</span><span class="n">score</span><span class="p">(</span><span class="n">X_xform</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
<span class="n">dim_range</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">21</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">dim_range</span><span class="p">,</span> <span class="p">[</span><span class="n">evaluate_gnb</span><span class="p">(</span><span class="n">dim</span><span class="p">)</span> <span class="k">for</span> <span class="n">dim</span> <span class="ow">in</span> <span class="n">dim_range</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s">&quot;Gaussian NB Accuracy&quot;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&quot;Naive Guess&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s">&#39;k&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="mi">1</span> <span class="o">-</span> <span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&quot;Inverse Naive Guess&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s">&#39;k&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">dim_range</span><span class="p">,</span> <span class="p">[</span><span class="n">evaluate_gnb</span><span class="p">(</span><span class="n">dim</span><span class="p">)</span> <span class="k">for</span> <span class="n">dim</span> <span class="ow">in</span> <span class="n">dim_range</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;Gaussian NB Accuracy&quot;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;Naive Guess&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s1">&#39;k&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="mi">1</span> <span class="o">-</span> <span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;Inverse Naive Guess&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s1">&#39;k&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">();</span>
</pre></div>
@ -683,9 +683,9 @@ rkJggg==
<span class="k">return</span> <span class="n">gnb</span><span class="o">.</span><span class="n">score</span><span class="p">(</span><span class="n">X_xform</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
<span class="n">dim_range</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">21</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">dim_range</span><span class="p">,</span> <span class="p">[</span><span class="n">evaluate_gnb</span><span class="p">(</span><span class="n">dim</span><span class="p">)</span> <span class="k">for</span> <span class="n">dim</span> <span class="ow">in</span> <span class="n">dim_range</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s">&quot;Gaussian NB Accuracy&quot;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&quot;Naive Guess&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s">&#39;k&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="mi">1</span> <span class="o">-</span> <span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">&quot;Inverse Naive Guess&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s">&#39;k&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">dim_range</span><span class="p">,</span> <span class="p">[</span><span class="n">evaluate_gnb</span><span class="p">(</span><span class="n">dim</span><span class="p">)</span> <span class="k">for</span> <span class="n">dim</span> <span class="ow">in</span> <span class="n">dim_range</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;Gaussian NB Accuracy&quot;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;Naive Guess&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s1">&#39;k&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="mi">1</span> <span class="o">-</span> <span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;Inverse Naive Guess&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s1">&#39;k&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">();</span>
</pre></div>
@ -961,7 +961,7 @@ rkJggg==
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">end</span> <span class="o">=</span> <span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s">&quot;Running time: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">end</span> <span class="o">-</span> <span class="n">start</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Running time: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">end</span> <span class="o">-</span> <span class="n">start</span><span class="p">))</span>
</pre></div>
</div>
@ -990,14 +990,14 @@ rkJggg==
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Appendix">Appendix<a class="anchor-link" href="#Appendix">&#182;</a></h1><p>Code used to split the initial training data:</p>
<div class="highlight"><pre><span class="kn">from</span> <span class="nn">sklearn.cross_validation</span> <span class="kn">import</span> <span class="n">train_test_split</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s">&#39;train.csv&#39;</span><span class="p">)</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s1">&#39;train.csv&#39;</span><span class="p">)</span>
<span class="n">data</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">ID</span>
<span class="n">data_train</span><span class="p">,</span> <span class="n">data_validate</span> <span class="o">=</span> <span class="n">train_test_split</span><span class="p">(</span>
<span class="n">data</span><span class="p">,</span> <span class="n">train_size</span><span class="o">=.</span><span class="mi">7</span><span class="p">)</span>
<span class="n">data_train</span><span class="o">.</span><span class="n">to_csv</span><span class="p">(</span><span class="s">&#39;split_train.csv&#39;</span><span class="p">)</span>
<span class="n">data_validate</span><span class="o">.</span><span class="n">to_csv</span><span class="p">(</span><span class="s">&#39;split_validate.csv&#39;</span><span class="p">)</span>
<span class="n">data_train</span><span class="o">.</span><span class="n">to_csv</span><span class="p">(</span><span class="s1">&#39;split_train.csv&#39;</span><span class="p">)</span>
<span class="n">data_validate</span><span class="o">.</span><span class="n">to_csv</span><span class="p">(</span><span class="s1">&#39;split_validate.csv&#39;</span><span class="p">)</span>
</pre></div>
</div>

View File

@ -123,7 +123,7 @@
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
<span class="kn">from</span> <span class="nn">Quandl</span> <span class="k">import</span> <span class="n">get</span> <span class="k">as</span> <span class="n">qget</span>
<span class="o">%</span><span class="k">matplotlib</span> inline
<span class="n">api_key</span> <span class="o">=</span> <span class="s">&#39;&#39;</span>
<span class="n">api_key</span> <span class="o">=</span> <span class="s1">&#39;&#39;</span>
<span class="n">profitability</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">p</span><span class="p">,</span> <span class="n">i</span><span class="p">,</span> <span class="n">m</span><span class="p">,</span> <span class="n">d</span><span class="p">:</span> <span class="mi">1000</span><span class="o">*</span><span class="n">p</span> <span class="o">/</span> <span class="p">(</span><span class="n">m</span> <span class="o">+</span> <span class="n">i</span><span class="o">*</span><span class="n">d</span><span class="p">)</span>
@ -147,15 +147,15 @@
<span class="n">expected_close</span> <span class="o">=</span> <span class="n">open_vals</span><span class="p">[</span><span class="n">day</span><span class="p">]</span> <span class="o">*</span> <span class="n">expected</span>
<span class="n">todays_purchase</span> <span class="o">=</span> <span class="n">daily_investment</span><span class="p">(</span><span class="n">open_vals</span><span class="p">[</span><span class="n">day</span><span class="p">],</span> <span class="n">expected_close</span><span class="p">,</span> <span class="n">invested</span><span class="p">,</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="n">day</span><span class="p">])</span>
<span class="n">invested</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">invested</span><span class="p">,</span> <span class="n">todays_purchase</span> <span class="o">+</span> <span class="n">bias</span><span class="p">)</span>
<span class="c"># expected_profit = expected_close * (invested / open_vals[0:len(invested)]).sum() - invested.sum()</span>
<span class="c1"># expected_profit = expected_close * (invested / open_vals[0:len(invested)]).sum() - invested.sum()</span>
<span class="n">day</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="n">profitable</span> <span class="o">=</span> <span class="n">is_profitable</span><span class="p">(</span><span class="n">close_vals</span><span class="p">[</span><span class="n">day</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span> <span class="n">invested</span><span class="p">,</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="nb">len</span><span class="p">(</span><span class="n">invested</span><span class="p">)])</span> \
<span class="ow">or</span> <span class="n">is_profitable</span><span class="p">(</span><span class="n">open_vals</span><span class="p">[</span><span class="n">day</span><span class="p">],</span> <span class="n">invested</span><span class="p">,</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="nb">len</span><span class="p">(</span><span class="n">invested</span><span class="p">)])</span>
<span class="n">shares</span> <span class="o">=</span> <span class="p">(</span><span class="n">invested</span> <span class="o">/</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="nb">len</span><span class="p">(</span><span class="n">invested</span><span class="p">)])</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
<span class="c"># Make sure we can&#39;t see into the future - we know either today&#39;s close or tomorrow&#39;s open</span>
<span class="c"># will be profitable, but we need to check which one.</span>
<span class="c1"># Make sure we can&#39;t see into the future - we know either today&#39;s close or tomorrow&#39;s open</span>
<span class="c1"># will be profitable, but we need to check which one.</span>
<span class="k">if</span> <span class="n">is_profitable</span><span class="p">(</span><span class="n">close_vals</span><span class="p">[</span><span class="n">day</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span> <span class="n">invested</span><span class="p">,</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="nb">len</span><span class="p">(</span><span class="n">invested</span><span class="p">)]):</span>
<span class="n">ending_price</span> <span class="o">=</span> <span class="n">close_vals</span><span class="p">[</span><span class="n">day</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span>
@ -168,21 +168,21 @@
<span class="n">ticker_info</span> <span class="o">=</span> <span class="n">qget</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start</span><span class="p">,</span> <span class="n">api_key</span><span class="o">=</span><span class="n">api_key</span><span class="p">)</span>
<span class="n">evaluation_times</span> <span class="o">=</span> <span class="n">ticker_info</span><span class="p">[:</span><span class="n">end</span><span class="p">]</span><span class="o">.</span><span class="n">index</span>
<span class="c"># Handle Google vs. YFinance data</span>
<span class="k">if</span> <span class="s">&quot;Adjusted Close&quot;</span> <span class="ow">in</span> <span class="n">ticker_info</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
<span class="n">close_column</span> <span class="o">=</span> <span class="s">&quot;Adjusted Close&quot;</span>
<span class="c1"># Handle Google vs. YFinance data</span>
<span class="k">if</span> <span class="s2">&quot;Adjusted Close&quot;</span> <span class="ow">in</span> <span class="n">ticker_info</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
<span class="n">close_column</span> <span class="o">=</span> <span class="s2">&quot;Adjusted Close&quot;</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">close_column</span> <span class="o">=</span> <span class="s">&quot;Close&quot;</span>
<span class="n">close_column</span> <span class="o">=</span> <span class="s2">&quot;Close&quot;</span>
<span class="n">sim</span> <span class="o">=</span> <span class="p">{</span><span class="n">d</span><span class="p">:</span> <span class="n">simulate_day</span><span class="p">(</span><span class="n">ticker_info</span><span class="p">[</span><span class="n">d</span><span class="p">:][</span><span class="s">&quot;Open&quot;</span><span class="p">],</span> <span class="n">ticker_info</span><span class="p">[</span><span class="n">d</span><span class="p">:][</span><span class="n">close_column</span><span class="p">],</span>
<span class="n">sim</span> <span class="o">=</span> <span class="p">{</span><span class="n">d</span><span class="p">:</span> <span class="n">simulate_day</span><span class="p">(</span><span class="n">ticker_info</span><span class="p">[</span><span class="n">d</span><span class="p">:][</span><span class="s2">&quot;Open&quot;</span><span class="p">],</span> <span class="n">ticker_info</span><span class="p">[</span><span class="n">d</span><span class="p">:][</span><span class="n">close_column</span><span class="p">],</span>
<span class="mi">100</span><span class="p">,</span> <span class="mf">1.02</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">evaluation_times</span><span class="p">}</span>
<span class="n">sim_series</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">sim</span><span class="p">)</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">()</span>
<span class="n">result</span><span class="p">[</span><span class="s">&quot;profit&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">])</span>
<span class="n">result</span><span class="p">[</span><span class="s">&quot;max&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">max</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]))</span>
<span class="n">result</span><span class="p">[</span><span class="s">&quot;days&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]))</span>
<span class="n">result</span><span class="p">[</span><span class="s">&quot;score&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">profitability</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">0</span><span class="p">],</span> <span class="nb">max</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]),</span> <span class="nb">len</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">])))</span>
<span class="n">result</span><span class="p">[</span><span class="s">&quot;investments&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
<span class="n">result</span><span class="p">[</span><span class="s2">&quot;profit&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">])</span>
<span class="n">result</span><span class="p">[</span><span class="s2">&quot;max&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">max</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]))</span>
<span class="n">result</span><span class="p">[</span><span class="s2">&quot;days&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]))</span>
<span class="n">result</span><span class="p">[</span><span class="s2">&quot;score&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">profitability</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">0</span><span class="p">],</span> <span class="nb">max</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]),</span> <span class="nb">len</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">])))</span>
<span class="n">result</span><span class="p">[</span><span class="s2">&quot;investments&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
<span class="k">return</span> <span class="n">result</span>
@ -217,7 +217,7 @@
<div class="prompt input_prompt">In&nbsp;[7]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">fang_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s">&quot;YAHOO/FB&quot;</span><span class="p">,</span> <span class="s">&quot;YAHOO/AAPL&quot;</span><span class="p">,</span> <span class="s">&quot;YAHOO/NFLX&quot;</span><span class="p">,</span> <span class="s">&quot;YAHOO/GOOG&quot;</span><span class="p">])</span>
<div class=" highlight hl-ipython3"><pre><span class="n">fang_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s2">&quot;YAHOO/FB&quot;</span><span class="p">,</span> <span class="s2">&quot;YAHOO/AAPL&quot;</span><span class="p">,</span> <span class="s2">&quot;YAHOO/NFLX&quot;</span><span class="p">,</span> <span class="s2">&quot;YAHOO/GOOG&quot;</span><span class="p">])</span>
</pre></div>
</div>
@ -230,9 +230,9 @@
<div class="prompt input_prompt">In&nbsp;[8]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">fang_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">&#39;days&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span>
<div class=" highlight hl-ipython3"><pre><span class="n">fang_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">&#39;days&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">8</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">&quot;Distribution of Days Until Profitability&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">&quot;Distribution of Days Until Profitability&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
</pre></div>
</div>
@ -704,9 +704,9 @@ kiSpNScYJEmSJElSa04wSJIkSZKk1pxgkCRJkiRJrf1/uiw9D8O0d6EAAAAASUVORK5CYII=
<div class="prompt input_prompt">In&nbsp;[10]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">fang_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">&#39;score&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
<div class=" highlight hl-ipython3"><pre><span class="n">fang_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">&#39;score&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">&quot;Profitability score over time&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">&quot;Profitability score over time&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
</pre></div>
</div>
@ -2343,7 +2343,7 @@ EZIkSZIkSZIkSZIk6RIpIiRJkiRJkiRJkiRJ0iX+D91jLOlExGw8AAAAAElFTkSuQmCC
<div class="prompt input_prompt">In&nbsp;[13]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">cyclic_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s">&quot;YAHOO/X&quot;</span><span class="p">,</span> <span class="s">&quot;YAHOO/CAT&quot;</span><span class="p">,</span> <span class="s">&quot;YAHOO/NFLX&quot;</span><span class="p">,</span> <span class="s">&quot;YAHOO/GOOG&quot;</span><span class="p">])</span>
<div class=" highlight hl-ipython3"><pre><span class="n">cyclic_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s2">&quot;YAHOO/X&quot;</span><span class="p">,</span> <span class="s2">&quot;YAHOO/CAT&quot;</span><span class="p">,</span> <span class="s2">&quot;YAHOO/NFLX&quot;</span><span class="p">,</span> <span class="s2">&quot;YAHOO/GOOG&quot;</span><span class="p">])</span>
</pre></div>
</div>
@ -2356,9 +2356,9 @@ EZIkSZIkSZIkSZIk6RIpIiRJkiRJkiRJkiRJ0iX+D91jLOlExGw8AAAAAElFTkSuQmCC
<div class="prompt input_prompt">In&nbsp;[14]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">cyclic_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">&#39;days&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span>
<div class=" highlight hl-ipython3"><pre><span class="n">cyclic_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">&#39;days&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">8</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">&quot;Distribution of Days Until Profitability&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">&quot;Distribution of Days Until Profitability&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
</pre></div>
</div>
@ -2844,9 +2844,9 @@ JEmSJEnSwGwwSJIkSZKkgdlgkCRJkiRJA7PBIEmSJEmSBvb/AfKAew4+oiojAAAAAElFTkSuQmCC
<div class="prompt input_prompt">In&nbsp;[15]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">cyclic_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">&#39;score&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
<div class=" highlight hl-ipython3"><pre><span class="n">cyclic_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">&#39;score&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">&quot;Profitability score over time&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">&quot;Profitability score over time&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
</pre></div>
</div>
@ -4629,7 +4629,7 @@ hUKhUCgUpor/A9AnO0PoIItTAAAAAElFTkSuQmCC
<div class="prompt input_prompt">In&nbsp;[21]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">biotech_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s">&#39;YAHOO/REGN&#39;</span><span class="p">,</span> <span class="s">&#39;YAHOO/CELG&#39;</span><span class="p">,</span> <span class="s">&#39;GOOG/NASDAQ_BIB&#39;</span><span class="p">,</span> <span class="s">&#39;GOOG/NASDAQ_IBB&#39;</span><span class="p">])</span>
<div class=" highlight hl-ipython3"><pre><span class="n">biotech_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s1">&#39;YAHOO/REGN&#39;</span><span class="p">,</span> <span class="s1">&#39;YAHOO/CELG&#39;</span><span class="p">,</span> <span class="s1">&#39;GOOG/NASDAQ_BIB&#39;</span><span class="p">,</span> <span class="s1">&#39;GOOG/NASDAQ_IBB&#39;</span><span class="p">])</span>
</pre></div>
</div>
@ -4642,9 +4642,9 @@ hUKhUCgUpor/A9AnO0PoIItTAAAAAElFTkSuQmCC
<div class="prompt input_prompt">In&nbsp;[22]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">biotech_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">&#39;days&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span>
<div class=" highlight hl-ipython3"><pre><span class="n">biotech_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">&#39;days&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">8</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">&quot;Distribution of Days Until Profitability&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">&quot;Distribution of Days Until Profitability&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
</pre></div>
</div>
@ -5195,9 +5195,9 @@ WKBTQVcCAAAAAElFTkSuQmCC
<div class="prompt input_prompt">In&nbsp;[23]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">biotech_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">&#39;score&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
<div class=" highlight hl-ipython3"><pre><span class="n">biotech_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">&#39;score&#39;</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">&quot;Profitability score over time&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">&quot;Profitability score over time&quot;</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
</pre></div>
</div>

123
tag/nutone.html Normal file
View File

@ -0,0 +1,123 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content=" nutone">
<meta name="keywords" content="">
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
<title> nutone - Bradlee Speice</title>
<!-- Stylesheets -->
<link href="https://bspeice.github.io/theme/css/bootstrap.min.css" rel="stylesheet">
<link href="https://bspeice.github.io/theme/css/fonts.css" rel="stylesheet">
<link href="https://bspeice.github.io/theme/css/nest.css" rel="stylesheet">
<link href="https://bspeice.github.io/theme/css/pygment.css" rel="stylesheet">
<!-- /Stylesheets -->
<!-- RSS Feeds -->
<link href="https://bspeice.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate" title="Bradlee Speice Full Atom Feed" />
<!-- /RSS Feeds -->
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-74711362-1', 'auto');
ga('send', 'pageview');
</script>
<!-- /Google Analytics -->
</head>
<body>
<!-- Header -->
<div class="header-container gradient">
<!-- Static navbar -->
<div class="container">
<div class="header-nav">
<div class="header-logo">
<a class="pull-left" href="https://bspeice.github.io/"><img class="mr20" src="https://bspeice.github.io/images/logo.svg" alt="logo">Bradlee Speice</a>
</div>
<div class="nav pull-right">
</div>
</div>
</div>
<!-- /Static navbar -->
<!-- Header -->
<div class="container header-wrapper">
<div class="row">
<div class="col-lg-12">
<div class="header-content">
<h1 class="header-title text-uppercase"> : #nutone</h1>
<div class="header-underline"></div>
<p class="header-subtitle header-subtitle-homepage"> #nutone</p>
</div>
</div>
</div>
</div>
<!-- /Header -->
</div>
<!-- /Header -->
<!-- Content -->
<div class="archive-container">
<div class="container content archive">
<h2><a href="https://bspeice.github.io/tag/nutone.html">nutone</a></h2>
<dl class="dl-horizontal">
<dt>Sat 22 October 2016</dt>
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
</dl>
</div>
</div>
<!-- /Content -->
<!-- Footer -->
<div class="footer gradient-2">
<div class="container footer-container ">
<div class="row">
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
<div class="footer-title"></div>
<ul class="list-unstyled">
<li><a href="https://bspeice.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate"></a></li>
</ul>
</div>
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
<div class="footer-title"></div>
<ul class="list-unstyled">
<li><a href="https://github.com/bspeice" target="_blank">Github</a></li>
<li><a href="https://www.linkedin.com/in/bradleespeice" target="_blank">LinkedIn</a></li>
</ul>
</div>
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
</div>
<div class="col-xs-12 col-sm-3 col-md-3 col-lg-3">
<p class="pull-right text-right">
<small><em>Proudly powered by <a href="http://docs.getpelican.com/" target="_blank">pelican</a></em></small><br/>
<small><em>Theme and code by <a href="https://github.com/molivier" target="_blank">molivier</a></em></small><br/>
<small></small>
</p>
</div>
</div>
</div>
</div>
<!-- /Footer -->
</body>
</html>

123
tag/rust.html Normal file
View File

@ -0,0 +1,123 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content=" Rust">
<meta name="keywords" content="">
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
<title> Rust - Bradlee Speice</title>
<!-- Stylesheets -->
<link href="https://bspeice.github.io/theme/css/bootstrap.min.css" rel="stylesheet">
<link href="https://bspeice.github.io/theme/css/fonts.css" rel="stylesheet">
<link href="https://bspeice.github.io/theme/css/nest.css" rel="stylesheet">
<link href="https://bspeice.github.io/theme/css/pygment.css" rel="stylesheet">
<!-- /Stylesheets -->
<!-- RSS Feeds -->
<link href="https://bspeice.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate" title="Bradlee Speice Full Atom Feed" />
<!-- /RSS Feeds -->
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-74711362-1', 'auto');
ga('send', 'pageview');
</script>
<!-- /Google Analytics -->
</head>
<body>
<!-- Header -->
<div class="header-container gradient">
<!-- Static navbar -->
<div class="container">
<div class="header-nav">
<div class="header-logo">
<a class="pull-left" href="https://bspeice.github.io/"><img class="mr20" src="https://bspeice.github.io/images/logo.svg" alt="logo">Bradlee Speice</a>
</div>
<div class="nav pull-right">
</div>
</div>
</div>
<!-- /Static navbar -->
<!-- Header -->
<div class="container header-wrapper">
<div class="row">
<div class="col-lg-12">
<div class="header-content">
<h1 class="header-title text-uppercase"> : #Rust</h1>
<div class="header-underline"></div>
<p class="header-subtitle header-subtitle-homepage"> #Rust</p>
</div>
</div>
</div>
</div>
<!-- /Header -->
</div>
<!-- /Header -->
<!-- Content -->
<div class="archive-container">
<div class="container content archive">
<h2><a href="https://bspeice.github.io/tag/rust.html">Rust</a></h2>
<dl class="dl-horizontal">
<dt>Sat 22 October 2016</dt>
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
</dl>
</div>
</div>
<!-- /Content -->
<!-- Footer -->
<div class="footer gradient-2">
<div class="container footer-container ">
<div class="row">
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
<div class="footer-title"></div>
<ul class="list-unstyled">
<li><a href="https://bspeice.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate"></a></li>
</ul>
</div>
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
<div class="footer-title"></div>
<ul class="list-unstyled">
<li><a href="https://github.com/bspeice" target="_blank">Github</a></li>
<li><a href="https://www.linkedin.com/in/bradleespeice" target="_blank">LinkedIn</a></li>
</ul>
</div>
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
</div>
<div class="col-xs-12 col-sm-3 col-md-3 col-lg-3">
<p class="pull-right text-right">
<small><em>Proudly powered by <a href="http://docs.getpelican.com/" target="_blank">pelican</a></em></small><br/>
<small><em>Theme and code by <a href="https://github.com/molivier" target="_blank">molivier</a></em></small><br/>
<small></small>
</p>
</div>
</div>
</div>
</div>
<!-- /Footer -->
</body>
</html>

View File

@ -115,10 +115,14 @@
<dt><span class="label label-default">1</span> article </dt>
<dd><a href="https://bspeice.github.io/tag/monte-carlo.html">monte carlo</a></dd>
<dt><span class="label label-default">1</span> article </dt>
<dd><a href="https://bspeice.github.io/tag/nutone.html">nutone</a></dd>
<dt><span class="label label-default">1</span> article </dt>
<dd><a href="https://bspeice.github.io/tag/probability.html">probability</a></dd>
<dt><span class="label label-default">1</span> article </dt>
<dd><a href="https://bspeice.github.io/tag/python.html">python</a></dd>
<dt><span class="label label-default">1</span> article </dt>
<dd><a href="https://bspeice.github.io/tag/rust.html">Rust</a></dd>
<dt><span class="label label-default">1</span> article </dt>
<dd><a href="https://bspeice.github.io/tag/simulated-annealing.html">simulated annealing</a></dd>
<dt><span class="label label-default">1</span> article </dt>
<dd><a href="https://bspeice.github.io/tag/simulation.html">simulation</a></dd>

View File

@ -4,7 +4,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="In [1]: import requests import pandas as pd import numpy as np from dateutil import parser as dtparser from dateutil.relativedelta import relativedelta from datetime import datetime from html.parser ...">
<meta name="description" content="In [1]: import requests import pandas as pd import numpy as np from dateutil import parser as dtparser from dateutil.relativedelta import relativedelta from datetime import datetime from ...">
<meta name="keywords" content="data science, futures">
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
@ -133,20 +133,20 @@
<span class="n">articles</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">def</span> <span class="nf">handle_starttag</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">,</span> <span class="n">attrs</span><span class="p">):</span>
<span class="c">#if tag == &#39;div&#39; and (&quot;id&quot;, &quot;author_articles_wrapper&quot;) in attrs:</span>
<span class="c"># self.fetch_links = True</span>
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s">&#39;a&#39;</span> <span class="ow">and</span> <span class="p">(</span><span class="s">&#39;class&#39;</span><span class="p">,</span> <span class="s">&#39;dashboard_article_link&#39;</span><span class="p">)</span> <span class="ow">in</span> <span class="n">attrs</span><span class="p">:</span>
<span class="n">href</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s">&#39;href&#39;</span><span class="p">,</span> <span class="n">attrs</span><span class="p">))[</span><span class="mi">0</span><span class="p">][</span><span class="mi">1</span><span class="p">]</span>
<span class="c1">#if tag == &#39;div&#39; and (&quot;id&quot;, &quot;author_articles_wrapper&quot;) in attrs:</span>
<span class="c1"># self.fetch_links = True</span>
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s1">&#39;a&#39;</span> <span class="ow">and</span> <span class="p">(</span><span class="s1">&#39;class&#39;</span><span class="p">,</span> <span class="s1">&#39;dashboard_article_link&#39;</span><span class="p">)</span> <span class="ow">in</span> <span class="n">attrs</span><span class="p">:</span>
<span class="n">href</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s1">&#39;href&#39;</span><span class="p">,</span> <span class="n">attrs</span><span class="p">))[</span><span class="mi">0</span><span class="p">][</span><span class="mi">1</span><span class="p">]</span>
<span class="bp">self</span><span class="o">.</span><span class="n">articles</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">href</span><span class="p">)</span>
<span class="n">base_url</span> <span class="o">=</span> <span class="s">&quot;http://seekingalpha.com/author/wall-street-breakfast/articles&quot;</span>
<span class="n">article_page_urls</span> <span class="o">=</span> <span class="p">[</span><span class="n">base_url</span><span class="p">]</span> <span class="o">+</span> <span class="p">[</span><span class="n">base_url</span> <span class="o">+</span> <span class="s">&#39;/{}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">20</span><span class="p">)]</span>
<span class="n">base_url</span> <span class="o">=</span> <span class="s2">&quot;http://seekingalpha.com/author/wall-street-breakfast/articles&quot;</span>
<span class="n">article_page_urls</span> <span class="o">=</span> <span class="p">[</span><span class="n">base_url</span><span class="p">]</span> <span class="o">+</span> <span class="p">[</span><span class="n">base_url</span> <span class="o">+</span> <span class="s1">&#39;/{}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">20</span><span class="p">)]</span>
<span class="n">global_articles</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">page</span> <span class="ow">in</span> <span class="n">article_page_urls</span><span class="p">:</span>
<span class="c"># We need to switch the user agent, as SA blocks the standard requests agent</span>
<span class="c1"># We need to switch the user agent, as SA blocks the standard requests agent</span>
<span class="n">articles_html</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">page</span><span class="p">,</span>
<span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="s">&quot;User-Agent&quot;</span><span class="p">:</span> <span class="s">&quot;Wget/1.13.4&quot;</span><span class="p">})</span>
<span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="s2">&quot;User-Agent&quot;</span><span class="p">:</span> <span class="s2">&quot;Wget/1.13.4&quot;</span><span class="p">})</span>
<span class="n">parser</span> <span class="o">=</span> <span class="n">ArticleListParser</span><span class="p">()</span>
<span class="n">parser</span><span class="o">.</span><span class="n">feed</span><span class="p">(</span><span class="n">articles_html</span><span class="o">.</span><span class="n">text</span><span class="p">)</span>
<span class="n">global_articles</span> <span class="o">+=</span> <span class="p">(</span><span class="n">parser</span><span class="o">.</span><span class="n">articles</span><span class="p">)</span>
@ -163,24 +163,24 @@
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="k">class</span> <span class="nc">ArticleReturnParser</span><span class="p">(</span><span class="n">HTMLParser</span><span class="p">):</span>
<span class="s">&quot;Given an article, parse out the futures returns in it&quot;</span>
<span class="s2">&quot;Given an article, parse out the futures returns in it&quot;</span>
<span class="n">record_font_tags</span> <span class="o">=</span> <span class="k">False</span>
<span class="n">in_font_tag</span> <span class="o">=</span> <span class="k">False</span>
<span class="n">record_font_tags</span> <span class="o">=</span> <span class="kc">False</span>
<span class="n">in_font_tag</span> <span class="o">=</span> <span class="kc">False</span>
<span class="n">counter</span> <span class="o">=</span> <span class="mi">0</span>
<span class="c"># data = {} # See __init__</span>
<span class="c1"># data = {} # See __init__</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">__init__</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span> <span class="o">=</span> <span class="p">{}</span>
<span class="k">def</span> <span class="nf">handle_starttag</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">,</span> <span class="n">attrs</span><span class="p">):</span>
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s">&#39;span&#39;</span> <span class="ow">and</span> <span class="p">(</span><span class="s">&#39;itemprop&#39;</span><span class="p">,</span> <span class="s">&#39;datePublished&#39;</span><span class="p">)</span> <span class="ow">in</span> <span class="n">attrs</span><span class="p">:</span>
<span class="n">date_string</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s">&#39;content&#39;</span><span class="p">,</span> <span class="n">attrs</span><span class="p">))[</span><span class="mi">0</span><span class="p">][</span><span class="mi">1</span><span class="p">]</span>
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s1">&#39;span&#39;</span> <span class="ow">and</span> <span class="p">(</span><span class="s1">&#39;itemprop&#39;</span><span class="p">,</span> <span class="s1">&#39;datePublished&#39;</span><span class="p">)</span> <span class="ow">in</span> <span class="n">attrs</span><span class="p">:</span>
<span class="n">date_string</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s1">&#39;content&#39;</span><span class="p">,</span> <span class="n">attrs</span><span class="p">))[</span><span class="mi">0</span><span class="p">][</span><span class="mi">1</span><span class="p">]</span>
<span class="n">date</span> <span class="o">=</span> <span class="n">dtparser</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="n">date_string</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">&#39;date&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">date</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">&#39;date&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">date</span>
<span class="bp">self</span><span class="o">.</span><span class="n">in_font_tag</span> <span class="o">=</span> <span class="n">tag</span> <span class="o">==</span> <span class="s">&#39;font&#39;</span>
<span class="bp">self</span><span class="o">.</span><span class="n">in_font_tag</span> <span class="o">=</span> <span class="n">tag</span> <span class="o">==</span> <span class="s1">&#39;font&#39;</span>
<span class="k">def</span> <span class="nf">safe_float</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">string</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
@ -189,43 +189,43 @@
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span>
<span class="k">def</span> <span class="nf">handle_data</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">content</span><span class="p">):</span>
<span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">record_font_tags</span> <span class="ow">and</span> <span class="s">&quot;Futures at 6&quot;</span> <span class="ow">in</span> <span class="n">content</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">record_font_tags</span> <span class="o">=</span> <span class="k">True</span>
<span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">record_font_tags</span> <span class="ow">and</span> <span class="s2">&quot;Futures at 6&quot;</span> <span class="ow">in</span> <span class="n">content</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">record_font_tags</span> <span class="o">=</span> <span class="kc">True</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">record_font_tags</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">in_font_tag</span><span class="p">:</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">&#39;DOW&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">&#39;DOW&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">&#39;S&amp;P&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">&#39;S&amp;P&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">2</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">&#39;NASDAQ&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">&#39;NASDAQ&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">3</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">&#39;Crude&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">&#39;Crude&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">4</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">&#39;Gold&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">&#39;Gold&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">def</span> <span class="nf">handle_endtag</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">in_font_tag</span> <span class="o">=</span> <span class="k">False</span>
<span class="bp">self</span><span class="o">.</span><span class="n">in_font_tag</span> <span class="o">=</span> <span class="kc">False</span>
<span class="k">def</span> <span class="nf">retrieve_data</span><span class="p">(</span><span class="n">url</span><span class="p">):</span>
<span class="n">sa</span> <span class="o">=</span> <span class="s">&quot;http://seekingalpha.com&quot;</span>
<span class="n">sa</span> <span class="o">=</span> <span class="s2">&quot;http://seekingalpha.com&quot;</span>
<span class="n">article_html</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">sa</span> <span class="o">+</span> <span class="n">url</span><span class="p">,</span>
<span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="s">&quot;User-Agent&quot;</span><span class="p">:</span> <span class="s">&quot;Wget/1.13.4&quot;</span><span class="p">})</span>
<span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="s2">&quot;User-Agent&quot;</span><span class="p">:</span> <span class="s2">&quot;Wget/1.13.4&quot;</span><span class="p">})</span>
<span class="n">parser</span> <span class="o">=</span> <span class="n">ArticleReturnParser</span><span class="p">()</span>
<span class="n">parser</span><span class="o">.</span><span class="n">feed</span><span class="p">(</span><span class="n">article_html</span><span class="o">.</span><span class="n">text</span><span class="p">)</span>
<span class="n">parser</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">update</span><span class="p">({</span><span class="s">&quot;url&quot;</span><span class="p">:</span> <span class="n">url</span><span class="p">})</span>
<span class="n">parser</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">update</span><span class="p">({</span><span class="s">&quot;text&quot;</span><span class="p">:</span> <span class="n">article_html</span><span class="o">.</span><span class="n">text</span><span class="p">})</span>
<span class="n">parser</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">update</span><span class="p">({</span><span class="s2">&quot;url&quot;</span><span class="p">:</span> <span class="n">url</span><span class="p">})</span>
<span class="n">parser</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">update</span><span class="p">({</span><span class="s2">&quot;text&quot;</span><span class="p">:</span> <span class="n">article_html</span><span class="o">.</span><span class="n">text</span><span class="p">})</span>
<span class="k">return</span> <span class="n">parser</span><span class="o">.</span><span class="n">data</span>
<span class="c"># This copy **MUST** be in place. I&#39;m not sure why,</span>
<span class="c"># as you&#39;d think that the data being returned would already</span>
<span class="c"># represent a different memory location. Even so, it blows up</span>
<span class="c"># if you don&#39;t do this.</span>
<span class="c1"># This copy **MUST** be in place. I&#39;m not sure why,</span>
<span class="c1"># as you&#39;d think that the data being returned would already</span>
<span class="c1"># represent a different memory location. Even so, it blows up</span>
<span class="c1"># if you don&#39;t do this.</span>
<span class="n">article_list</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">global_articles</span><span class="p">))</span>
<span class="n">article_data</span> <span class="o">=</span> <span class="p">[</span><span class="n">copy</span><span class="p">(</span><span class="n">retrieve_data</span><span class="p">(</span><span class="n">url</span><span class="p">))</span> <span class="k">for</span> <span class="n">url</span> <span class="ow">in</span> <span class="n">article_list</span><span class="p">]</span>
<span class="c"># If there&#39;s an issue downloading the article, drop it.</span>
<span class="c1"># If there&#39;s an issue downloading the article, drop it.</span>
<span class="n">article_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">article_data</span><span class="p">)</span><span class="o">.</span><span class="n">dropna</span><span class="p">()</span>
</pre></div>
@ -249,12 +249,12 @@
<div class="prompt input_prompt">In&nbsp;[4]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="c"># article_df is sorted by date, so we get the first row.</span>
<span class="n">start_date</span> <span class="o">=</span> <span class="n">article_df</span><span class="o">.</span><span class="n">sort_values</span><span class="p">(</span><span class="n">by</span><span class="o">=</span><span class="s">&#39;date&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="s">&#39;date&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="n">relativedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="n">SPY</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">&quot;GOOG/NYSE_SPY&quot;</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
<span class="n">DJIA</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">&quot;GOOG/AMS_DIA&quot;</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
<span class="n">RUSS</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">&quot;GOOG/AMEX_IWM&quot;</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
<span class="n">NASDAQ</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">&quot;GOOG/EPA_QQQ&quot;</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
<div class=" highlight hl-ipython3"><pre><span class="c1"># article_df is sorted by date, so we get the first row.</span>
<span class="n">start_date</span> <span class="o">=</span> <span class="n">article_df</span><span class="o">.</span><span class="n">sort_values</span><span class="p">(</span><span class="n">by</span><span class="o">=</span><span class="s1">&#39;date&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="s1">&#39;date&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="n">relativedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="n">SPY</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;GOOG/NYSE_SPY&quot;</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
<span class="n">DJIA</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;GOOG/AMS_DIA&quot;</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
<span class="n">RUSS</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;GOOG/AMEX_IWM&quot;</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
<span class="n">NASDAQ</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;GOOG/EPA_QQQ&quot;</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
</pre></div>
</div>
@ -279,13 +279,13 @@
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">calculate_opening_ret</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
<span class="c"># I&#39;m not a huge fan of the appending for loop,</span>
<span class="c"># but it&#39;s a bit verbose for a comprehension</span>
<span class="c1"># I&#39;m not a huge fan of the appending for loop,</span>
<span class="c1"># but it&#39;s a bit verbose for a comprehension</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">{}</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">frame</span><span class="p">)):</span>
<span class="n">date</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
<span class="n">prior_close</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="o">-</span><span class="mi">1</span><span class="p">][</span><span class="s">&#39;Close&#39;</span><span class="p">]</span>
<span class="n">open_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s">&#39;Open&#39;</span><span class="p">]</span>
<span class="n">prior_close</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="o">-</span><span class="mi">1</span><span class="p">][</span><span class="s1">&#39;Close&#39;</span><span class="p">]</span>
<span class="n">open_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s1">&#39;Open&#39;</span><span class="p">]</span>
<span class="n">data</span><span class="p">[</span><span class="n">date</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">open_val</span> <span class="o">-</span> <span class="n">prior_close</span><span class="p">)</span> <span class="o">/</span> <span class="n">prior_close</span>
<span class="k">return</span> <span class="n">data</span>
@ -296,8 +296,8 @@
<span class="n">NASDAQ_open_ret</span> <span class="o">=</span> <span class="n">calculate_opening_ret</span><span class="p">(</span><span class="n">NASDAQ</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">signs_match</span><span class="p">(</span><span class="n">list_1</span><span class="p">,</span> <span class="n">list_2</span><span class="p">):</span>
<span class="c"># This is a surprisingly difficult task - we have to match</span>
<span class="c"># up the dates in order to check if opening returns actually match</span>
<span class="c1"># This is a surprisingly difficult task - we have to match</span>
<span class="c1"># up the dates in order to check if opening returns actually match</span>
<span class="n">index_dict_dt</span> <span class="o">=</span> <span class="p">{</span><span class="n">key</span><span class="o">.</span><span class="n">to_datetime</span><span class="p">():</span> <span class="n">list_2</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">list_2</span><span class="o">.</span><span class="n">keys</span><span class="p">()}</span>
<span class="n">matches</span> <span class="o">=</span> <span class="p">[]</span>
@ -313,26 +313,26 @@
<span class="n">matches</span> <span class="o">+=</span> <span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">matches</span> <span class="o">+=</span> <span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="c">#print(&quot;{}&quot;.format(list_2[index_dt]))</span>
<span class="c1">#print(&quot;{}&quot;.format(list_2[index_dt]))</span>
<span class="k">return</span> <span class="n">matches</span>
<span class="n">prediction_dict</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">matches_dict</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">count_dict</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">index_dict</span> <span class="o">=</span> <span class="p">{</span><span class="s">&quot;SPY&quot;</span><span class="p">:</span> <span class="n">SPY_open_ret</span><span class="p">,</span> <span class="s">&quot;DJIA&quot;</span><span class="p">:</span> <span class="n">DJIA_open_ret</span><span class="p">,</span> <span class="s">&quot;RUSS&quot;</span><span class="p">:</span> <span class="n">RUSS_open_ret</span><span class="p">,</span> <span class="s">&quot;NASDAQ&quot;</span><span class="p">:</span> <span class="n">NASDAQ_open_ret</span><span class="p">}</span>
<span class="n">indices</span> <span class="o">=</span> <span class="p">[</span><span class="s">&quot;SPY&quot;</span><span class="p">,</span> <span class="s">&quot;DJIA&quot;</span><span class="p">,</span> <span class="s">&quot;RUSS&quot;</span><span class="p">,</span> <span class="s">&quot;NASDAQ&quot;</span><span class="p">]</span>
<span class="n">futures</span> <span class="o">=</span> <span class="p">[</span><span class="s">&quot;Crude&quot;</span><span class="p">,</span> <span class="s">&quot;Gold&quot;</span><span class="p">,</span> <span class="s">&quot;DOW&quot;</span><span class="p">,</span> <span class="s">&quot;NASDAQ&quot;</span><span class="p">,</span> <span class="s">&quot;S&amp;P&quot;</span><span class="p">]</span>
<span class="n">index_dict</span> <span class="o">=</span> <span class="p">{</span><span class="s2">&quot;SPY&quot;</span><span class="p">:</span> <span class="n">SPY_open_ret</span><span class="p">,</span> <span class="s2">&quot;DJIA&quot;</span><span class="p">:</span> <span class="n">DJIA_open_ret</span><span class="p">,</span> <span class="s2">&quot;RUSS&quot;</span><span class="p">:</span> <span class="n">RUSS_open_ret</span><span class="p">,</span> <span class="s2">&quot;NASDAQ&quot;</span><span class="p">:</span> <span class="n">NASDAQ_open_ret</span><span class="p">}</span>
<span class="n">indices</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;SPY&quot;</span><span class="p">,</span> <span class="s2">&quot;DJIA&quot;</span><span class="p">,</span> <span class="s2">&quot;RUSS&quot;</span><span class="p">,</span> <span class="s2">&quot;NASDAQ&quot;</span><span class="p">]</span>
<span class="n">futures</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;Crude&quot;</span><span class="p">,</span> <span class="s2">&quot;Gold&quot;</span><span class="p">,</span> <span class="s2">&quot;DOW&quot;</span><span class="p">,</span> <span class="s2">&quot;NASDAQ&quot;</span><span class="p">,</span> <span class="s2">&quot;S&amp;P&quot;</span><span class="p">]</span>
<span class="k">for</span> <span class="n">index</span> <span class="ow">in</span> <span class="n">indices</span><span class="p">:</span>
<span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">signs_match</span><span class="p">(</span><span class="n">article_df</span><span class="p">[[</span><span class="n">future</span><span class="p">,</span> <span class="s">&#39;date&#39;</span><span class="p">]],</span>
<span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">signs_match</span><span class="p">(</span><span class="n">article_df</span><span class="p">[[</span><span class="n">future</span><span class="p">,</span> <span class="s1">&#39;date&#39;</span><span class="p">]],</span>
<span class="n">index_dict</span><span class="p">[</span><span class="n">index</span><span class="p">])</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
<span class="n">count_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">future</span><span class="p">])</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
<span class="n">prediction_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">future</span><span class="p">])</span>
<span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
<span class="nb">print</span><span class="p">(</span><span class="s">&quot;Articles Checked: &quot;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Articles Checked: &quot;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">count_dict</span><span class="p">))</span>
<span class="nb">print</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s">&quot;Prediction Accuracy:&quot;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Prediction Accuracy:&quot;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">prediction_dict</span><span class="p">))</span>
</pre></div>
@ -392,13 +392,13 @@ S&amp;P 0.604478 0.597015 0.811808 0.848708
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">calculate_closing_ret</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
<span class="c"># I&#39;m not a huge fan of the appending for loop,</span>
<span class="c"># but it&#39;s a bit verbose for a comprehension</span>
<span class="c1"># I&#39;m not a huge fan of the appending for loop,</span>
<span class="c1"># but it&#39;s a bit verbose for a comprehension</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">{}</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">frame</span><span class="p">)):</span>
<span class="n">date</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
<span class="n">open_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s">&#39;Open&#39;</span><span class="p">]</span>
<span class="n">close_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s">&#39;Close&#39;</span><span class="p">]</span>
<span class="n">open_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s1">&#39;Open&#39;</span><span class="p">]</span>
<span class="n">close_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s1">&#39;Close&#39;</span><span class="p">]</span>
<span class="n">data</span><span class="p">[</span><span class="n">date</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">close_val</span> <span class="o">-</span> <span class="n">open_val</span><span class="p">)</span> <span class="o">/</span> <span class="n">open_val</span>
<span class="k">return</span> <span class="n">data</span>
@ -409,8 +409,8 @@ S&amp;P 0.604478 0.597015 0.811808 0.848708
<span class="n">NASDAQ_close_ret</span> <span class="o">=</span> <span class="n">calculate_closing_ret</span><span class="p">(</span><span class="n">NASDAQ</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">signs_match</span><span class="p">(</span><span class="n">list_1</span><span class="p">,</span> <span class="n">list_2</span><span class="p">):</span>
<span class="c"># This is a surprisingly difficult task - we have to match</span>
<span class="c"># up the dates in order to check if opening returns actually match</span>
<span class="c1"># This is a surprisingly difficult task - we have to match</span>
<span class="c1"># up the dates in order to check if opening returns actually match</span>
<span class="n">index_dict_dt</span> <span class="o">=</span> <span class="p">{</span><span class="n">key</span><span class="o">.</span><span class="n">to_datetime</span><span class="p">():</span> <span class="n">list_2</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">list_2</span><span class="o">.</span><span class="n">keys</span><span class="p">()}</span>
<span class="n">matches</span> <span class="o">=</span> <span class="p">[]</span>
@ -426,28 +426,28 @@ S&amp;P 0.604478 0.597015 0.811808 0.848708
<span class="n">matches</span> <span class="o">+=</span> <span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">matches</span> <span class="o">+=</span> <span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="c">#print(&quot;{}&quot;.format(list_2[index_dt]))</span>
<span class="c1">#print(&quot;{}&quot;.format(list_2[index_dt]))</span>
<span class="k">return</span> <span class="n">matches</span>
<span class="n">matches_dict</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">count_dict</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">prediction_dict</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">index_dict</span> <span class="o">=</span> <span class="p">{</span><span class="s">&quot;SPY&quot;</span><span class="p">:</span> <span class="n">SPY_close_ret</span><span class="p">,</span> <span class="s">&quot;DJIA&quot;</span><span class="p">:</span> <span class="n">DJIA_close_ret</span><span class="p">,</span>
<span class="s">&quot;RUSS&quot;</span><span class="p">:</span> <span class="n">RUSS_close_ret</span><span class="p">,</span> <span class="s">&quot;NASDAQ&quot;</span><span class="p">:</span> <span class="n">NASDAQ_close_ret</span><span class="p">}</span>
<span class="n">indices</span> <span class="o">=</span> <span class="p">[</span><span class="s">&quot;SPY&quot;</span><span class="p">,</span> <span class="s">&quot;DJIA&quot;</span><span class="p">,</span> <span class="s">&quot;RUSS&quot;</span><span class="p">,</span> <span class="s">&quot;NASDAQ&quot;</span><span class="p">]</span>
<span class="n">futures</span> <span class="o">=</span> <span class="p">[</span><span class="s">&quot;Crude&quot;</span><span class="p">,</span> <span class="s">&quot;Gold&quot;</span><span class="p">,</span> <span class="s">&quot;DOW&quot;</span><span class="p">,</span> <span class="s">&quot;NASDAQ&quot;</span><span class="p">,</span> <span class="s">&quot;S&amp;P&quot;</span><span class="p">]</span>
<span class="n">index_dict</span> <span class="o">=</span> <span class="p">{</span><span class="s2">&quot;SPY&quot;</span><span class="p">:</span> <span class="n">SPY_close_ret</span><span class="p">,</span> <span class="s2">&quot;DJIA&quot;</span><span class="p">:</span> <span class="n">DJIA_close_ret</span><span class="p">,</span>
<span class="s2">&quot;RUSS&quot;</span><span class="p">:</span> <span class="n">RUSS_close_ret</span><span class="p">,</span> <span class="s2">&quot;NASDAQ&quot;</span><span class="p">:</span> <span class="n">NASDAQ_close_ret</span><span class="p">}</span>
<span class="n">indices</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;SPY&quot;</span><span class="p">,</span> <span class="s2">&quot;DJIA&quot;</span><span class="p">,</span> <span class="s2">&quot;RUSS&quot;</span><span class="p">,</span> <span class="s2">&quot;NASDAQ&quot;</span><span class="p">]</span>
<span class="n">futures</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;Crude&quot;</span><span class="p">,</span> <span class="s2">&quot;Gold&quot;</span><span class="p">,</span> <span class="s2">&quot;DOW&quot;</span><span class="p">,</span> <span class="s2">&quot;NASDAQ&quot;</span><span class="p">,</span> <span class="s2">&quot;S&amp;P&quot;</span><span class="p">]</span>
<span class="k">for</span> <span class="n">index</span> <span class="ow">in</span> <span class="n">indices</span><span class="p">:</span>
<span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">signs_match</span><span class="p">(</span><span class="n">article_df</span><span class="p">[[</span><span class="n">future</span><span class="p">,</span> <span class="s">&#39;date&#39;</span><span class="p">]],</span>
<span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">signs_match</span><span class="p">(</span><span class="n">article_df</span><span class="p">[[</span><span class="n">future</span><span class="p">,</span> <span class="s1">&#39;date&#39;</span><span class="p">]],</span>
<span class="n">index_dict</span><span class="p">[</span><span class="n">index</span><span class="p">])</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
<span class="n">count_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">future</span><span class="p">])</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
<span class="n">prediction_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">future</span><span class="p">])</span>
<span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
<span class="nb">print</span><span class="p">(</span><span class="s">&quot;Articles Checked:&quot;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Articles Checked:&quot;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">count_dict</span><span class="p">))</span>
<span class="nb">print</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s">&quot;Prediction Accuracy:&quot;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Prediction Accuracy:&quot;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">prediction_dict</span><span class="p">))</span>
</pre></div>

View File

@ -4,7 +4,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="Or, how to get thrown out of a casino because you&#39;re a mathematician. In the ongoing eternal cycle of mathematicians asking generally useless questions about probability, I dreamt up another one. The ...">
<meta name="description" content="Or, how to get thrown out of a casino because you&#39;re a mathematician. In the ongoing eternal cycle of mathematicians asking generally useless questions about probability, I dreamt up another one. ...">
<meta name="keywords" content="casino, em, machine learning, probability, simulated annealing">
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
@ -114,8 +114,8 @@
<span class="n">fair_1</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
<span class="n">fair_2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
<span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">fair_1</span> <span class="o">+</span> <span class="n">fair_2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s">&#39;hist&#39;</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">&#39;Fair Distribution&#39;</span><span class="p">);</span>
<span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">fair_1</span> <span class="o">+</span> <span class="n">fair_2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s1">&#39;hist&#39;</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">&#39;Fair Distribution&#39;</span><span class="p">);</span>
</pre></div>
</div>
@ -326,8 +326,8 @@ D_2 = \left\{
<span class="n">x1</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d1</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
<span class="n">x2</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d2</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
<span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">x1</span> <span class="o">+</span> <span class="n">x2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s">&#39;hist&#39;</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">&#39;$D_1$ biased low, $D_2$ biased high&#39;</span><span class="p">);</span>
<span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">x1</span> <span class="o">+</span> <span class="n">x2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s1">&#39;hist&#39;</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">);</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">&#39;$D_1$ biased low, $D_2$ biased high&#39;</span><span class="p">);</span>
</pre></div>
</div>
@ -525,8 +525,8 @@ D_2 = \left\{
<span class="n">x1</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d1</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
<span class="n">x2</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d2</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
<span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">x1</span> <span class="o">+</span> <span class="n">x2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s">&#39;hist&#39;</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">&quot;$D_1$ and $D_2$ biased to 1 and 6&quot;</span><span class="p">);</span>
<span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">x1</span> <span class="o">+</span> <span class="n">x2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s1">&#39;hist&#39;</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s2">&quot;$D_1$ and $D_2$ biased to 1 and 6&quot;</span><span class="p">);</span>
</pre></div>
</div>
@ -746,18 +746,18 @@ and continue the next iteration.</li>
<span class="kn">from</span> <span class="nn">numpy</span> <span class="k">import</span> <span class="n">polynomial</span>
<span class="k">def</span> <span class="nf">density_coef</span><span class="p">(</span><span class="n">d1_density</span><span class="p">,</span> <span class="n">d2_density</span><span class="p">):</span>
<span class="c"># Calculating the probabilities of each outcome was taken</span>
<span class="c"># from this author: http://math.stackexchange.com/a/1710392/320784</span>
<span class="c1"># Calculating the probabilities of each outcome was taken</span>
<span class="c1"># from this author: http://math.stackexchange.com/a/1710392/320784</span>
<span class="n">d1_p</span> <span class="o">=</span> <span class="n">polynomial</span><span class="o">.</span><span class="n">Polynomial</span><span class="p">(</span><span class="n">d1_density</span><span class="p">)</span>
<span class="n">d2_p</span> <span class="o">=</span> <span class="n">polynomial</span><span class="o">.</span><span class="n">Polynomial</span><span class="p">(</span><span class="n">d2_density</span><span class="p">)</span>
<span class="n">coefs</span> <span class="o">=</span> <span class="p">(</span><span class="n">d1_p</span> <span class="o">*</span> <span class="n">d2_p</span><span class="p">)</span><span class="o">.</span><span class="n">coef</span>
<span class="k">return</span> <span class="n">coefs</span>
<span class="k">def</span> <span class="nf">score</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">d1_density</span><span class="p">,</span> <span class="n">d2_density</span><span class="p">):</span>
<span class="c"># We&#39;ve now got the probabilities of each event, but we need</span>
<span class="c"># to shift the array a bit so we can use the x values to actually</span>
<span class="c"># index into it. This will allow us to do all the calculations</span>
<span class="c"># incredibly quickly</span>
<span class="c1"># We&#39;ve now got the probabilities of each event, but we need</span>
<span class="c1"># to shift the array a bit so we can use the x values to actually</span>
<span class="c1"># index into it. This will allow us to do all the calculations</span>
<span class="c1"># incredibly quickly</span>
<span class="n">coefs</span> <span class="o">=</span> <span class="n">density_coef</span><span class="p">(</span><span class="n">d1_density</span><span class="p">,</span> <span class="n">d2_density</span><span class="p">)</span>
<span class="n">coefs</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">hstack</span><span class="p">((</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="n">coefs</span><span class="p">))</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">coefs</span><span class="p">[</span><span class="n">x</span><span class="p">])</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
@ -784,23 +784,23 @@ and continue the next iteration.</li>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">permute</span><span class="p">(</span><span class="n">d1_density</span><span class="p">,</span> <span class="n">d2_density</span><span class="p">):</span>
<span class="c"># To ensure we have legitimate densities, we will randomly</span>
<span class="c"># increase one die face probability by `change`,</span>
<span class="c"># and decrease one by `change`.</span>
<span class="c"># This means there are something less than (1/`change`)^12 possibilities</span>
<span class="c"># we are trying to search over.</span>
<span class="c1"># To ensure we have legitimate densities, we will randomly</span>
<span class="c1"># increase one die face probability by `change`,</span>
<span class="c1"># and decrease one by `change`.</span>
<span class="c1"># This means there are something less than (1/`change`)^12 possibilities</span>
<span class="c1"># we are trying to search over.</span>
<span class="n">change</span> <span class="o">=</span> <span class="o">.</span><span class="mi">01</span>
<span class="n">d1_index1</span><span class="p">,</span> <span class="n">d1_index2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
<span class="n">d2_index1</span><span class="p">,</span> <span class="n">d2_index2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
<span class="c"># Also make sure to copy. I&#39;ve had some weird aliasing issues</span>
<span class="c"># in the past that made everything blow up.</span>
<span class="c1"># Also make sure to copy. I&#39;ve had some weird aliasing issues</span>
<span class="c1"># in the past that made everything blow up.</span>
<span class="n">new_d1</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">float64</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="n">d1_density</span><span class="p">))</span>
<span class="n">new_d2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">float64</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="n">d2_density</span><span class="p">))</span>
<span class="c"># While this doesn&#39;t account for the possibility that some</span>
<span class="c"># values go negative, in practice this never happens</span>
<span class="c1"># While this doesn&#39;t account for the possibility that some</span>
<span class="c1"># values go negative, in practice this never happens</span>
<span class="n">new_d1</span><span class="p">[</span><span class="n">d1_index1</span><span class="p">]</span> <span class="o">+=</span> <span class="n">change</span>
<span class="n">new_d1</span><span class="p">[</span><span class="n">d1_index2</span><span class="p">]</span> <span class="o">-=</span> <span class="n">change</span>
<span class="n">new_d2</span><span class="p">[</span><span class="n">d2_index1</span><span class="p">]</span> <span class="o">+=</span> <span class="n">change</span>
@ -833,24 +833,24 @@ and continue the next iteration.</li>
<span class="n">switch_failures</span> <span class="o">=</span> <span class="mi">0</span>
<span class="n">iter_count</span> <span class="o">=</span> <span class="mi">0</span>
<span class="c"># Start with guessing fair dice</span>
<span class="c1"># Start with guessing fair dice</span>
<span class="n">cur_d1</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="mi">1</span><span class="o">/</span><span class="mi">6</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
<span class="n">cur_d2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="mi">1</span><span class="o">/</span><span class="mi">6</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
<span class="n">cur_score</span> <span class="o">=</span> <span class="n">score</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">cur_d1</span><span class="p">,</span> <span class="n">cur_d2</span><span class="p">)</span>
<span class="c"># Keep track of our best guesses - may not be</span>
<span class="c"># what we end with</span>
<span class="c1"># Keep track of our best guesses - may not be</span>
<span class="c1"># what we end with</span>
<span class="n">max_score</span> <span class="o">=</span> <span class="n">cur_score</span>
<span class="n">max_d1</span> <span class="o">=</span> <span class="n">cur_d1</span>
<span class="n">max_d2</span> <span class="o">=</span> <span class="n">cur_d2</span>
<span class="c"># Optimization stops when we have failed to switch `conv_count`</span>
<span class="c"># times (presumably because we have a great guess), or we reach</span>
<span class="c"># the maximum number of iterations.</span>
<span class="c1"># Optimization stops when we have failed to switch `conv_count`</span>
<span class="c1"># times (presumably because we have a great guess), or we reach</span>
<span class="c1"># the maximum number of iterations.</span>
<span class="k">while</span> <span class="n">switch_failures</span> <span class="o">&lt;</span> <span class="n">conv_count</span> <span class="ow">and</span> <span class="n">iter_count</span> <span class="o">&lt;</span> <span class="n">max_iter</span><span class="p">:</span>
<span class="n">iter_count</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">if</span> <span class="n">iter_count</span> <span class="o">%</span> <span class="p">(</span><span class="n">max_iter</span> <span class="o">/</span> <span class="mi">10</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s">&#39;Iteration: {}; Current score (higher is better): {}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Iteration: {}; Current score (higher is better): {}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
<span class="n">iter_count</span><span class="p">,</span> <span class="n">cur_score</span><span class="p">))</span>
<span class="n">new_d1</span><span class="p">,</span> <span class="n">new_d2</span> <span class="o">=</span> <span class="n">permute</span><span class="p">(</span><span class="n">cur_d1</span><span class="p">,</span> <span class="n">cur_d2</span><span class="p">)</span>
@ -862,19 +862,19 @@ and continue the next iteration.</li>
<span class="n">max_d2</span> <span class="o">=</span> <span class="n">new_d2</span>
<span class="k">if</span> <span class="n">new_score</span> <span class="o">&gt;</span> <span class="n">cur_score</span><span class="p">:</span>
<span class="c"># If the new permutation beats the old one,</span>
<span class="c"># automatically select it.</span>
<span class="c1"># If the new permutation beats the old one,</span>
<span class="c1"># automatically select it.</span>
<span class="n">cur_score</span> <span class="o">=</span> <span class="n">new_score</span>
<span class="n">cur_d1</span> <span class="o">=</span> <span class="n">new_d1</span>
<span class="n">cur_d2</span> <span class="o">=</span> <span class="n">new_d2</span>
<span class="n">switch_failures</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">else</span><span class="p">:</span>
<span class="c"># We didn&#39;t beat the current score, but allow</span>
<span class="c"># for possibly switching anyways.</span>
<span class="c1"># We didn&#39;t beat the current score, but allow</span>
<span class="c1"># for possibly switching anyways.</span>
<span class="n">accept_prob</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="n">new_score</span> <span class="o">-</span> <span class="n">cur_score</span><span class="p">)</span>
<span class="n">coin_toss</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">()</span>
<span class="k">if</span> <span class="n">coin_toss</span> <span class="o">&lt;</span> <span class="n">accept_prob</span><span class="p">:</span>
<span class="c"># We randomly switch to the new distribution</span>
<span class="c1"># We randomly switch to the new distribution</span>
<span class="n">cur_score</span> <span class="o">=</span> <span class="n">new_score</span>
<span class="n">cur_d1</span> <span class="o">=</span> <span class="n">new_d1</span>
<span class="n">cur_d2</span> <span class="o">=</span> <span class="n">new_d2</span>
@ -882,7 +882,7 @@ and continue the next iteration.</li>
<span class="k">else</span><span class="p">:</span>
<span class="n">switch_failures</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="c"># Return both our best guess, and the ending guess</span>
<span class="c1"># Return both our best guess, and the ending guess</span>
<span class="k">return</span> <span class="n">max_d1</span><span class="p">,</span> <span class="n">max_d2</span><span class="p">,</span> <span class="n">cur_d1</span><span class="p">,</span> <span class="n">cur_d2</span>
</pre></div>
@ -916,20 +916,20 @@ and continue the next iteration.</li>
<span class="n">start</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">perf_counter</span><span class="p">()</span>
<span class="n">max_d1</span><span class="p">,</span> <span class="n">max_d2</span><span class="p">,</span> <span class="n">final_d1</span><span class="p">,</span> <span class="n">final_d2</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span><span class="n">d1_vals</span> <span class="o">+</span> <span class="n">d2_vals</span><span class="p">)</span>
<span class="n">end</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">perf_counter</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s">&quot;Simulated Annealing time: {:.02f}s&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">end</span> <span class="o">-</span> <span class="n">start</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Simulated Annealing time: {:.02f}s&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">end</span> <span class="o">-</span> <span class="n">start</span><span class="p">))</span>
<span class="n">coef_range</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">13</span><span class="p">)</span> <span class="o">-</span> <span class="o">.</span><span class="mi">5</span>
<span class="n">plt</span><span class="o">.</span><span class="n">subplot</span><span class="p">(</span><span class="mi">221</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">bar</span><span class="p">(</span><span class="n">coef_range</span><span class="p">,</span> <span class="n">density_coef</span><span class="p">(</span><span class="n">d1_dist</span><span class="p">,</span> <span class="n">d2_dist</span><span class="p">),</span> <span class="n">width</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">&#39;True Distribution&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">&#39;True Distribution&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">subplot</span><span class="p">(</span><span class="mi">222</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">hist</span><span class="p">(</span><span class="n">d1_vals</span> <span class="o">+</span> <span class="n">d2_vals</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">&#39;Empirical Distribution&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">&#39;Empirical Distribution&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">subplot</span><span class="p">(</span><span class="mi">223</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">bar</span><span class="p">(</span><span class="n">coef_range</span><span class="p">,</span> <span class="n">density_coef</span><span class="p">(</span><span class="n">max_d1</span><span class="p">,</span> <span class="n">max_d2</span><span class="p">),</span> <span class="n">width</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">&#39;Recovered Distribution&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">&#39;Recovered Distribution&#39;</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span>

View File

@ -137,7 +137,7 @@
<div class="prompt input_prompt">In&nbsp;[2]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="c"># Import all the OAuth secret information from a local file</span>
<div class=" highlight hl-ipython3"><pre><span class="c1"># Import all the OAuth secret information from a local file</span>
<span class="kn">from</span> <span class="nn">secrets</span> <span class="k">import</span> <span class="n">CLIENT_SECRET</span><span class="p">,</span> <span class="n">CLIENT_ID</span><span class="p">,</span> <span class="n">CALLBACK_URL</span>
</pre></div>
@ -178,17 +178,17 @@ Application: Thank you, now give me all the data.</code></pre>
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">urllib</span>
<span class="n">FITBIT_URI</span> <span class="o">=</span> <span class="s">&#39;https://www.fitbit.com/oauth2/authorize&#39;</span>
<span class="n">FITBIT_URI</span> <span class="o">=</span> <span class="s1">&#39;https://www.fitbit.com/oauth2/authorize&#39;</span>
<span class="n">params</span> <span class="o">=</span> <span class="p">{</span>
<span class="c"># If we need more than one scope, must be a CSV string</span>
<span class="s">&#39;scope&#39;</span><span class="p">:</span> <span class="s">&#39;heartrate&#39;</span><span class="p">,</span>
<span class="s">&#39;response_type&#39;</span><span class="p">:</span> <span class="s">&#39;token&#39;</span><span class="p">,</span>
<span class="s">&#39;expires_in&#39;</span><span class="p">:</span> <span class="mi">86400</span><span class="p">,</span> <span class="c"># 1 day</span>
<span class="s">&#39;redirect_uri&#39;</span><span class="p">:</span> <span class="n">CALLBACK_URL</span><span class="p">,</span>
<span class="s">&#39;client_id&#39;</span><span class="p">:</span> <span class="n">CLIENT_ID</span>
<span class="c1"># If we need more than one scope, must be a CSV string</span>
<span class="s1">&#39;scope&#39;</span><span class="p">:</span> <span class="s1">&#39;heartrate&#39;</span><span class="p">,</span>
<span class="s1">&#39;response_type&#39;</span><span class="p">:</span> <span class="s1">&#39;token&#39;</span><span class="p">,</span>
<span class="s1">&#39;expires_in&#39;</span><span class="p">:</span> <span class="mi">86400</span><span class="p">,</span> <span class="c1"># 1 day</span>
<span class="s1">&#39;redirect_uri&#39;</span><span class="p">:</span> <span class="n">CALLBACK_URL</span><span class="p">,</span>
<span class="s1">&#39;client_id&#39;</span><span class="p">:</span> <span class="n">CLIENT_ID</span>
<span class="p">}</span>
<span class="n">request_url</span> <span class="o">=</span> <span class="n">FITBIT_URI</span> <span class="o">+</span> <span class="s">&#39;?&#39;</span> <span class="o">+</span> <span class="n">urllib</span><span class="o">.</span><span class="n">parse</span><span class="o">.</span><span class="n">urlencode</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
<span class="n">request_url</span> <span class="o">=</span> <span class="n">FITBIT_URI</span> <span class="o">+</span> <span class="s1">&#39;?&#39;</span> <span class="o">+</span> <span class="n">urllib</span><span class="o">.</span><span class="n">parse</span><span class="o">.</span><span class="n">urlencode</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
</pre></div>
</div>
@ -211,10 +211,10 @@ Application: Thank you, now give me all the data.</code></pre>
<div class="prompt input_prompt">In&nbsp;[6]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="c"># The `response_url` variable contains the full URL that</span>
<span class="c"># FitBit sent back to us, but most importantly,</span>
<span class="c"># contains the token we need for authorization.</span>
<span class="n">access_token</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">urllib</span><span class="o">.</span><span class="n">parse</span><span class="o">.</span><span class="n">parse_qsl</span><span class="p">(</span><span class="n">response_url</span><span class="p">))[</span><span class="s">&#39;access_token&#39;</span><span class="p">]</span>
<div class=" highlight hl-ipython3"><pre><span class="c1"># The `response_url` variable contains the full URL that</span>
<span class="c1"># FitBit sent back to us, but most importantly,</span>
<span class="c1"># contains the token we need for authorization.</span>
<span class="n">access_token</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">urllib</span><span class="o">.</span><span class="n">parse</span><span class="o">.</span><span class="n">parse_qsl</span><span class="p">(</span><span class="n">response_url</span><span class="p">))[</span><span class="s1">&#39;access_token&#39;</span><span class="p">]</span>
</pre></div>
</div>
@ -243,41 +243,41 @@ Application: Thank you, now give me all the data.</code></pre>
<span class="kn">from</span> <span class="nn">datetime</span> <span class="k">import</span> <span class="n">datetime</span>
<span class="n">session</span> <span class="o">=</span> <span class="n">OAuth2Session</span><span class="p">(</span><span class="n">token</span><span class="o">=</span><span class="p">{</span>
<span class="s">&#39;access_token&#39;</span><span class="p">:</span> <span class="n">access_token</span><span class="p">,</span>
<span class="s">&#39;token_type&#39;</span><span class="p">:</span> <span class="s">&#39;Bearer&#39;</span>
<span class="s1">&#39;access_token&#39;</span><span class="p">:</span> <span class="n">access_token</span><span class="p">,</span>
<span class="s1">&#39;token_type&#39;</span><span class="p">:</span> <span class="s1">&#39;Bearer&#39;</span>
<span class="p">})</span>
<span class="n">format_str</span> <span class="o">=</span> <span class="s">&#39;%Y-%m-%d&#39;</span>
<span class="n">format_str</span> <span class="o">=</span> <span class="s1">&#39;%Y-%m-</span><span class="si">%d</span><span class="s1">&#39;</span>
<span class="n">start_date</span> <span class="o">=</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="n">end_date</span> <span class="o">=</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">31</span><span class="p">)</span>
<span class="n">dr</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">date_range</span><span class="p">(</span><span class="n">start_date</span><span class="p">,</span> <span class="n">end_date</span><span class="p">)</span>
<span class="n">url</span> <span class="o">=</span> <span class="s">&#39;https://api.fitbit.com/1/user/-/activities/heart/date/{0}/1d/1min.json&#39;</span>
<span class="n">url</span> <span class="o">=</span> <span class="s1">&#39;https://api.fitbit.com/1/user/-/activities/heart/date/{0}/1d/1min.json&#39;</span>
<span class="n">hr_responses</span> <span class="o">=</span> <span class="p">[</span><span class="n">session</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">d</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="n">format_str</span><span class="p">)))</span> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">dr</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">record_to_df</span><span class="p">(</span><span class="n">record</span><span class="p">):</span>
<span class="k">if</span> <span class="s">&#39;activities-heart&#39;</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">record</span><span class="p">:</span>
<span class="k">return</span> <span class="k">None</span>
<span class="n">date_str</span> <span class="o">=</span> <span class="n">record</span><span class="p">[</span><span class="s">&#39;activities-heart&#39;</span><span class="p">][</span><span class="mi">0</span><span class="p">][</span><span class="s">&#39;dateTime&#39;</span><span class="p">]</span>
<span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">record</span><span class="p">[</span><span class="s">&#39;activities-heart-intraday&#39;</span><span class="p">][</span><span class="s">&#39;dataset&#39;</span><span class="p">])</span>
<span class="k">if</span> <span class="s1">&#39;activities-heart&#39;</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">record</span><span class="p">:</span>
<span class="k">return</span> <span class="kc">None</span>
<span class="n">date_str</span> <span class="o">=</span> <span class="n">record</span><span class="p">[</span><span class="s1">&#39;activities-heart&#39;</span><span class="p">][</span><span class="mi">0</span><span class="p">][</span><span class="s1">&#39;dateTime&#39;</span><span class="p">]</span>
<span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">record</span><span class="p">[</span><span class="s1">&#39;activities-heart-intraday&#39;</span><span class="p">][</span><span class="s1">&#39;dataset&#39;</span><span class="p">])</span>
<span class="n">df</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="s">&#39;time&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span>
<span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">strptime</span><span class="p">(</span><span class="n">date_str</span> <span class="o">+</span> <span class="s">&#39; &#39;</span> <span class="o">+</span> <span class="n">x</span><span class="p">,</span> <span class="s">&#39;%Y-%m-%d %H:%M:%S&#39;</span><span class="p">))</span>
<span class="n">df</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="s1">&#39;time&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span>
<span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">strptime</span><span class="p">(</span><span class="n">date_str</span> <span class="o">+</span> <span class="s1">&#39; &#39;</span> <span class="o">+</span> <span class="n">x</span><span class="p">,</span> <span class="s1">&#39;%Y-%m-</span><span class="si">%d</span><span class="s1"> %H:%M:%S&#39;</span><span class="p">))</span>
<span class="k">return</span> <span class="n">df</span>
<span class="n">hr_dataframes</span> <span class="o">=</span> <span class="p">[</span><span class="n">record_to_df</span><span class="p">(</span><span class="n">record</span><span class="o">.</span><span class="n">json</span><span class="p">())</span> <span class="k">for</span> <span class="n">record</span> <span class="ow">in</span> <span class="n">hr_responses</span><span class="p">]</span>
<span class="n">hr_df_concat</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">concat</span><span class="p">(</span><span class="n">hr_dataframes</span><span class="p">)</span>
<span class="c"># There are some minutes with missing data, so we need to correct that</span>
<span class="c1"># There are some minutes with missing data, so we need to correct that</span>
<span class="n">full_daterange</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">date_range</span><span class="p">(</span><span class="n">hr_df_concat</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span>
<span class="n">hr_df_concat</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span>
<span class="n">freq</span><span class="o">=</span><span class="s">&#39;min&#39;</span><span class="p">)</span>
<span class="n">hr_df_full</span> <span class="o">=</span> <span class="n">hr_df_concat</span><span class="o">.</span><span class="n">reindex</span><span class="p">(</span><span class="n">full_daterange</span><span class="p">,</span> <span class="n">method</span><span class="o">=</span><span class="s">&#39;nearest&#39;</span><span class="p">)</span>
<span class="n">freq</span><span class="o">=</span><span class="s1">&#39;min&#39;</span><span class="p">)</span>
<span class="n">hr_df_full</span> <span class="o">=</span> <span class="n">hr_df_concat</span><span class="o">.</span><span class="n">reindex</span><span class="p">(</span><span class="n">full_daterange</span><span class="p">,</span> <span class="n">method</span><span class="o">=</span><span class="s1">&#39;nearest&#39;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s">&quot;Heartbeats from {} to {}: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Heartbeats from {} to {}: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span>
<span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span>
<span class="n">hr_df_full</span><span class="p">[</span><span class="s">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()))</span>
<span class="n">hr_df_full</span><span class="p">[</span><span class="s1">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()))</span>
</pre></div>
</div>
@ -325,9 +325,9 @@ hr_q \cdot n - hr_d \cdot (n-m)
<div class="prompt input_prompt">In&nbsp;[8]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">quarterly_count</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[</span><span class="s">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
<div class=" highlight hl-ipython3"><pre><span class="n">quarterly_count</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[</span><span class="s1">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
<span class="n">leap_day_count</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">month</span> <span class="o">==</span> <span class="mi">2</span><span class="p">)</span> <span class="o">&amp;</span>
<span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">day</span> <span class="o">==</span> <span class="mi">29</span><span class="p">)][</span><span class="s">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
<span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">day</span> <span class="o">==</span> <span class="mi">29</span><span class="p">)][</span><span class="s1">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
<span class="n">num_quarters</span> <span class="o">=</span> <span class="mi">96</span>
<span class="n">leap_days</span> <span class="o">=</span> <span class="mi">6</span>
@ -372,17 +372,17 @@ hr_q \cdot n - hr_d \cdot (n-m)
<div class="prompt input_prompt">In&nbsp;[9]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">minute_mean</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[</span><span class="s">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">mean</span><span class="p">()</span>
<span class="c"># Don&#39;t you wish you knew?</span>
<span class="c"># birthday_minutes = ???</span>
<div class=" highlight hl-ipython3"><pre><span class="n">minute_mean</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[</span><span class="s1">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">mean</span><span class="p">()</span>
<span class="c1"># Don&#39;t you wish you knew?</span>
<span class="c1"># birthday_minutes = ???</span>
<span class="n">birthday_heartbeats</span> <span class="o">=</span> <span class="n">birthday_minutes</span> <span class="o">*</span> <span class="n">minute_mean</span>
<span class="n">heartbeats_until_2016</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">birthday_heartbeats</span> <span class="o">+</span> <span class="n">jan_92_jan_16</span><span class="p">)</span>
<span class="n">remaining_2016</span> <span class="o">=</span> <span class="n">total_heartbeats</span> <span class="o">-</span> <span class="n">heartbeats_until_2016</span>
<span class="nb">print</span><span class="p">(</span><span class="s">&quot;Heartbeats so far: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">heartbeats_until_2016</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s">&quot;Remaining heartbeats: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_2016</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Heartbeats so far: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">heartbeats_until_2016</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Remaining heartbeats: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_2016</span><span class="p">))</span>
</pre></div>
</div>
@ -473,8 +473,8 @@ hr_q \cdot n - hr_d \cdot (n - m)
<div class=" highlight hl-ipython3"><pre><span class="n">four_year_periods</span> <span class="o">=</span> <span class="n">remaining_2016</span> <span class="o">//</span> <span class="n">heartbeats_4year</span>
<span class="n">remaining_4y</span> <span class="o">=</span> <span class="n">remaining_2016</span> <span class="o">-</span> <span class="n">four_year_periods</span> <span class="o">*</span> <span class="n">heartbeats_4year</span>
<span class="nb">print</span><span class="p">(</span><span class="s">&quot;Four year periods remaining: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">four_year_periods</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s">&quot;Remaining heartbeats after 4 year periods: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_4y</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Four year periods remaining: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">four_year_periods</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Remaining heartbeats after 4 year periods: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_4y</span><span class="p">))</span>
</pre></div>
</div>
@ -513,14 +513,14 @@ Remaining heartbeats after 4 year periods: 48041640
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">remaining_leap</span> <span class="o">=</span> <span class="n">remaining_4y</span> <span class="o">-</span> <span class="n">leap_day_count</span>
<span class="c"># Ignore leap day in the data set</span>
<span class="c1"># Ignore leap day in the data set</span>
<span class="n">heartbeats_quarter</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">month</span> <span class="o">!=</span> <span class="mi">2</span><span class="p">)</span> <span class="o">&amp;</span>
<span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">day</span> <span class="o">!=</span> <span class="mi">29</span><span class="p">)][</span><span class="s">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
<span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">day</span> <span class="o">!=</span> <span class="mi">29</span><span class="p">)][</span><span class="s1">&#39;value&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
<span class="n">quarters_left</span> <span class="o">=</span> <span class="n">remaining_leap</span> <span class="o">//</span> <span class="n">heartbeats_quarter</span>
<span class="n">remaining_year</span> <span class="o">=</span> <span class="n">remaining_leap</span> <span class="o">-</span> <span class="n">quarters_left</span> <span class="o">*</span> <span class="n">heartbeats_quarter</span>
<span class="nb">print</span><span class="p">(</span><span class="s">&quot;Quarters left starting 2068: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">quarters_left</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s">&quot;Remaining heartbeats after that: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_year</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Quarters left starting 2068: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">quarters_left</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Remaining heartbeats after that: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_year</span><span class="p">))</span>
</pre></div>
</div>

View File

@ -4,7 +4,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="An experiment in creating a robot that will imitate me on Twitter. So, I&#39;m taking a Machine Learning course this semester in school, and one of the topics we keep coming back to is natural language ...">
<meta name="description" content="An experiment in creating a robot that will imitate me on Twitter. So, I&#39;m taking a Machine Learning course this semester in school, and one of the topics we keep coming back to is natural ...">
<meta name="keywords" content="MCMC, twitter">
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
@ -128,11 +128,11 @@
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
<span class="n">tweets</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s">&#39;tweets.csv&#39;</span><span class="p">)</span>
<span class="n">tweets</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s1">&#39;tweets.csv&#39;</span><span class="p">)</span>
<span class="n">text</span> <span class="o">=</span> <span class="n">tweets</span><span class="o">.</span><span class="n">text</span>
<span class="c"># Don&#39;t include tweets in reply to or mentioning people</span>
<span class="n">replies</span> <span class="o">=</span> <span class="n">text</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">contains</span><span class="p">(</span><span class="s">&#39;@&#39;</span><span class="p">)</span>
<span class="c1"># Don&#39;t include tweets in reply to or mentioning people</span>
<span class="n">replies</span> <span class="o">=</span> <span class="n">text</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">contains</span><span class="p">(</span><span class="s1">&#39;@&#39;</span><span class="p">)</span>
<span class="n">text_norep</span> <span class="o">=</span> <span class="n">text</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="o">~</span><span class="n">replies</span><span class="p">]</span>
</pre></div>
@ -187,10 +187,10 @@
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="kn">from</span> <span class="nn">functools</span> <span class="k">import</span> <span class="n">reduce</span>
<span class="c"># Get all possible words</span>
<span class="c1"># Get all possible words</span>
<span class="n">all_words</span> <span class="o">=</span> <span class="n">reduce</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span> <span class="n">x</span><span class="o">+</span><span class="n">y</span><span class="p">,</span> <span class="n">tokens</span><span class="p">,</span> <span class="p">[])</span>
<span class="n">unique_words</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">all_words</span><span class="p">)</span>
<span class="n">actual_words</span> <span class="o">=</span> <span class="nb">set</span><span class="p">([</span><span class="n">x</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s">&#39;.&#39;</span> <span class="k">else</span> <span class="k">None</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">unique_words</span><span class="p">])</span>
<span class="n">actual_words</span> <span class="o">=</span> <span class="nb">set</span><span class="p">([</span><span class="n">x</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s1">&#39;.&#39;</span> <span class="k">else</span> <span class="kc">None</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">unique_words</span><span class="p">])</span>
<span class="n">word_dist</span> <span class="o">=</span> <span class="p">{}</span>
<span class="k">for</span> <span class="n">word</span> <span class="ow">in</span> <span class="nb">iter</span><span class="p">(</span><span class="n">actual_words</span><span class="p">):</span>
@ -222,9 +222,9 @@
<div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
<span class="o">%</span><span class="k">matplotlib</span> inline
<span class="n">hashtags</span> <span class="o">=</span> <span class="n">text_norep</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">count</span><span class="p">(</span><span class="s">&#39;#&#39;</span><span class="p">)</span>
<span class="n">hashtags</span> <span class="o">=</span> <span class="n">text_norep</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">count</span><span class="p">(</span><span class="s1">&#39;#&#39;</span><span class="p">)</span>
<span class="n">bins</span> <span class="o">=</span> <span class="n">hashtags</span><span class="o">.</span><span class="n">unique</span><span class="p">()</span><span class="o">.</span><span class="n">max</span><span class="p">()</span>
<span class="n">hashtags</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s">&#39;hist&#39;</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="n">bins</span><span class="p">)</span>
<span class="n">hashtags</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s1">&#39;hist&#39;</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="n">bins</span><span class="p">)</span>
</pre></div>
</div>
@ -396,12 +396,12 @@ AElFTkSuQmCC
<div class="prompt input_prompt">In&nbsp;[6]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span class="n">hashtags</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">all_words</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s">&#39;#&#39;</span><span class="p">]</span>
<div class=" highlight hl-ipython3"><pre><span class="n">hashtags</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">all_words</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s1">&#39;#&#39;</span><span class="p">]</span>
<span class="n">n_hashtags</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">hashtags</span><span class="p">)</span>
<span class="n">unique_hashtags</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">([</span><span class="n">x</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">unique_words</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s">&#39;#&#39;</span><span class="p">]))</span>
<span class="n">hashtag_dist</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">({</span><span class="s">&#39;hashtags&#39;</span><span class="p">:</span> <span class="n">unique_hashtags</span><span class="p">,</span>
<span class="s">&#39;prob&#39;</span><span class="p">:</span> <span class="p">[</span><span class="n">all_words</span><span class="o">.</span><span class="n">count</span><span class="p">(</span><span class="n">h</span><span class="p">)</span> <span class="o">/</span> <span class="n">n_hashtags</span>
<span class="n">unique_hashtags</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">([</span><span class="n">x</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">unique_words</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s1">&#39;#&#39;</span><span class="p">]))</span>
<span class="n">hashtag_dist</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">({</span><span class="s1">&#39;hashtags&#39;</span><span class="p">:</span> <span class="n">unique_hashtags</span><span class="p">,</span>
<span class="s1">&#39;prob&#39;</span><span class="p">:</span> <span class="p">[</span><span class="n">all_words</span><span class="o">.</span><span class="n">count</span><span class="p">(</span><span class="n">h</span><span class="p">)</span> <span class="o">/</span> <span class="n">n_hashtags</span>
<span class="k">for</span> <span class="n">h</span> <span class="ow">in</span> <span class="n">unique_hashtags</span><span class="p">]})</span>
<span class="nb">len</span><span class="p">(</span><span class="n">hashtag_dist</span><span class="p">)</span>
</pre></div>
@ -459,7 +459,7 @@ z: .17</code></pre>
<span class="k">def</span> <span class="nf">multinom_sim</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">vals</span><span class="p">,</span> <span class="n">probs</span><span class="p">):</span>
<span class="n">occurrences</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">multinomial</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">probs</span><span class="p">)</span>
<span class="n">results</span> <span class="o">=</span> <span class="n">occurrences</span> <span class="o">*</span> <span class="n">vals</span>
<span class="k">return</span> <span class="s">&#39; &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">results</span><span class="p">[</span><span class="n">results</span> <span class="o">!=</span> <span class="s">&#39;&#39;</span><span class="p">])</span>
<span class="k">return</span> <span class="s1">&#39; &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">results</span><span class="p">[</span><span class="n">results</span> <span class="o">!=</span> <span class="s1">&#39;&#39;</span><span class="p">])</span>
<span class="k">def</span> <span class="nf">sim_n_hashtags</span><span class="p">(</span><span class="n">hashtag_freq</span><span class="p">):</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">poisson</span><span class="p">(</span><span class="n">hashtag_freq</span><span class="p">)</span>
@ -469,7 +469,7 @@ z: .17</code></pre>
<span class="k">def</span> <span class="nf">sim_first_word</span><span class="p">(</span><span class="n">first_word_dist</span><span class="p">):</span>
<span class="n">probs</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">float64</span><span class="p">(</span><span class="n">first_word_dist</span><span class="o">.</span><span class="n">values</span><span class="p">)</span>
<span class="k">return</span> <span class="n">multinom_sim</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">first_word_dist</span><span class="o">.</span><span class="n">reset_index</span><span class="p">()[</span><span class="s">&#39;index&#39;</span><span class="p">],</span> <span class="n">probs</span><span class="p">)</span>
<span class="k">return</span> <span class="n">multinom_sim</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">first_word_dist</span><span class="o">.</span><span class="n">reset_index</span><span class="p">()[</span><span class="s1">&#39;index&#39;</span><span class="p">],</span> <span class="n">probs</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">sim_next_word</span><span class="p">(</span><span class="n">current</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">):</span>
<span class="n">dist</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">word_dist</span><span class="p">[</span><span class="n">current</span><span class="p">])</span>
@ -504,7 +504,7 @@ z: .17</code></pre>
<span class="n">fifth</span> <span class="o">=</span> <span class="n">sim_next_word</span><span class="p">(</span><span class="n">fourth</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">)</span>
<span class="n">hashtag</span> <span class="o">=</span> <span class="n">sim_hashtags</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">hashtag_dist</span><span class="p">)</span>
<span class="s">&#39; &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">((</span><span class="n">first</span><span class="p">,</span> <span class="n">second</span><span class="p">,</span> <span class="n">third</span><span class="p">,</span> <span class="n">fourth</span><span class="p">,</span> <span class="n">fifth</span><span class="p">,</span> <span class="n">hashtag</span><span class="p">))</span>
<span class="s1">&#39; &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">((</span><span class="n">first</span><span class="p">,</span> <span class="n">second</span><span class="p">,</span> <span class="n">third</span><span class="p">,</span> <span class="n">fourth</span><span class="p">,</span> <span class="n">fifth</span><span class="p">,</span> <span class="n">hashtag</span><span class="p">))</span>
</pre></div>
</div>
@ -553,13 +553,13 @@ z: .17</code></pre>
<span class="n">tweet</span> <span class="o">=</span> <span class="n">first</span>
<span class="n">current</span> <span class="o">=</span> <span class="n">first</span>
<span class="k">while</span> <span class="n">chars_remaining</span> <span class="o">&gt;</span> <span class="nb">len</span><span class="p">(</span><span class="n">tweet</span><span class="p">)</span> <span class="o">+</span> <span class="nb">len</span><span class="p">(</span><span class="n">hashtags</span><span class="p">)</span> <span class="ow">and</span> <span class="n">current</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s">&#39;.&#39;</span> <span class="ow">and</span> <span class="n">current</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s">&#39;!&#39;</span><span class="p">:</span>
<span class="k">while</span> <span class="n">chars_remaining</span> <span class="o">&gt;</span> <span class="nb">len</span><span class="p">(</span><span class="n">tweet</span><span class="p">)</span> <span class="o">+</span> <span class="nb">len</span><span class="p">(</span><span class="n">hashtags</span><span class="p">)</span> <span class="ow">and</span> <span class="n">current</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s1">&#39;.&#39;</span> <span class="ow">and</span> <span class="n">current</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s1">&#39;!&#39;</span><span class="p">:</span>
<span class="n">current</span> <span class="o">=</span> <span class="n">sim_next_word</span><span class="p">(</span><span class="n">current</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">)</span>
<span class="n">tweet</span> <span class="o">+=</span> <span class="s">&#39; &#39;</span> <span class="o">+</span> <span class="n">current</span>
<span class="n">tweet</span> <span class="o">+=</span> <span class="s1">&#39; &#39;</span> <span class="o">+</span> <span class="n">current</span>
<span class="n">tweet</span> <span class="o">=</span> <span class="n">tweet</span><span class="p">[:</span><span class="o">-</span><span class="mi">2</span><span class="p">]</span> <span class="o">+</span> <span class="n">tweet</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
<span class="k">return</span> <span class="s">&#39; &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">((</span><span class="n">tweet</span><span class="p">,</span> <span class="n">hashtags</span><span class="p">))</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
<span class="k">return</span> <span class="s1">&#39; &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">((</span><span class="n">tweet</span><span class="p">,</span> <span class="n">hashtags</span><span class="p">))</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
</pre></div>
</div>

View File

@ -4,7 +4,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="Hello! Glad to meet you. I&#39;m currently a student at Columbia University studying Financial Engineering, and want to give an overview of the projects I&#39;m working on! To start things off, Columbia has ...">
<meta name="description" content="Hello! Glad to meet you. I&#39;m currently a student at Columbia University studying Financial Engineering, and want to give an overview of the projects I&#39;m working on! To start things off, Columbia ...">
<meta name="keywords" content="introduction, trading">
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
@ -161,12 +161,12 @@ to make some tweaks over the coming weeks, and do another forward test in Januar
<span class="kn">import</span> <span class="nn">Quandl</span>
<span class="kn">from</span> <span class="nn">datetime</span> <span class="k">import</span> <span class="n">datetime</span><span class="p">,</span> <span class="n">timedelta</span>
<span class="n">tickers</span> <span class="o">=</span> <span class="p">[</span><span class="s">&#39;XOM&#39;</span><span class="p">,</span> <span class="s">&#39;CVX&#39;</span><span class="p">,</span> <span class="s">&#39;CLB&#39;</span><span class="p">,</span> <span class="s">&#39;OXY&#39;</span><span class="p">,</span> <span class="s">&#39;SLB&#39;</span><span class="p">]</span>
<span class="n">market_ticker</span> <span class="o">=</span> <span class="s">&#39;GOOG/NYSE_VOO&#39;</span>
<span class="n">tickers</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;XOM&#39;</span><span class="p">,</span> <span class="s1">&#39;CVX&#39;</span><span class="p">,</span> <span class="s1">&#39;CLB&#39;</span><span class="p">,</span> <span class="s1">&#39;OXY&#39;</span><span class="p">,</span> <span class="s1">&#39;SLB&#39;</span><span class="p">]</span>
<span class="n">market_ticker</span> <span class="o">=</span> <span class="s1">&#39;GOOG/NYSE_VOO&#39;</span>
<span class="n">lookback</span> <span class="o">=</span> <span class="mi">30</span>
<span class="n">d_col</span> <span class="o">=</span> <span class="s">&#39;Close&#39;</span>
<span class="n">d_col</span> <span class="o">=</span> <span class="s1">&#39;Close&#39;</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">{</span><span class="n">tick</span><span class="p">:</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">&#39;YAHOO/{}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">tick</span><span class="p">))[</span><span class="o">-</span><span class="n">lookback</span><span class="p">:]</span> <span class="k">for</span> <span class="n">tick</span> <span class="ow">in</span> <span class="n">tickers</span><span class="p">}</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">{</span><span class="n">tick</span><span class="p">:</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">&#39;YAHOO/{}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">tick</span><span class="p">))[</span><span class="o">-</span><span class="n">lookback</span><span class="p">:]</span> <span class="k">for</span> <span class="n">tick</span> <span class="ow">in</span> <span class="n">tickers</span><span class="p">}</span>
<span class="n">market</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">market_ticker</span><span class="p">)</span>
</pre></div>
@ -340,22 +340,22 @@ s.t.\ \ & \vec{1} \omega = 1\\
<div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">from</span> <span class="nn">scipy.optimize</span> <span class="k">import</span> <span class="n">minimize</span>
<span class="c">#sharpe_limit = .1</span>
<span class="c1">#sharpe_limit = .1</span>
<span class="n">drawdown_limit</span> <span class="o">=</span> <span class="o">.</span><span class="mi">05</span>
<span class="n">leverage</span> <span class="o">=</span> <span class="mi">250</span>
<span class="c"># Use the map so we can guarantee we maintain the correct order</span>
<span class="c"># sharpe_a = np.array(list(map(lambda tick: sharpes[tick], tickers))) * -1 # So we can write as upper-bound</span>
<span class="c1"># Use the map so we can guarantee we maintain the correct order</span>
<span class="c1"># sharpe_a = np.array(list(map(lambda tick: sharpes[tick], tickers))) * -1 # So we can write as upper-bound</span>
<span class="n">dd_a</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">tick</span><span class="p">:</span> <span class="n">drawdowns</span><span class="p">[</span><span class="n">tick</span><span class="p">],</span> <span class="n">tickers</span><span class="p">)))</span>
<span class="n">returns_a</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">tick</span><span class="p">:</span> <span class="n">returns</span><span class="p">[</span><span class="n">tick</span><span class="p">]</span><span class="o">.</span><span class="n">mean</span><span class="p">(),</span> <span class="n">tickers</span><span class="p">)))</span> <span class="c"># Because minimizing</span>
<span class="n">returns_a</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">tick</span><span class="p">:</span> <span class="n">returns</span><span class="p">[</span><span class="n">tick</span><span class="p">]</span><span class="o">.</span><span class="n">mean</span><span class="p">(),</span> <span class="n">tickers</span><span class="p">)))</span> <span class="c1"># Because minimizing</span>
<span class="n">meets_sharpe</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="o">*</span> <span class="n">sharpe_a</span><span class="p">)</span> <span class="o">-</span> <span class="n">sharpe_limit</span>
<span class="k">def</span> <span class="nf">meets_dd</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="n">portfolio</span> <span class="o">=</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">))</span>
<span class="k">if</span> <span class="n">portfolio</span> <span class="o">&lt;</span> <span class="o">.</span><span class="mi">1</span><span class="p">:</span>
<span class="c"># If there are no stocks in the portfolio,</span>
<span class="c"># we can accidentally induce division by 0,</span>
<span class="c"># or division by something small enough to cause infinity</span>
<span class="c1"># If there are no stocks in the portfolio,</span>
<span class="c1"># we can accidentally induce division by 0,</span>
<span class="c1"># or division by something small enough to cause infinity</span>
<span class="k">return</span> <span class="mi">0</span>
<span class="k">return</span> <span class="n">drawdown_limit</span> <span class="o">-</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="o">*</span> <span class="n">dd_a</span><span class="p">)</span> <span class="o">/</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">))</span>
@ -365,41 +365,41 @@ s.t.\ \ & \vec{1} \omega = 1\\
<span class="k">def</span> <span class="nf">within_leverage</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">leverage</span> <span class="o">-</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">))</span>
<span class="n">objective</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">sum</span><span class="p">(</span><span class="n">x</span> <span class="o">*</span> <span class="n">returns_a</span><span class="p">)</span> <span class="o">*</span> <span class="o">-</span><span class="mi">1</span> <span class="c"># Because we&#39;re minimizing</span>
<span class="n">bounds</span> <span class="o">=</span> <span class="p">((</span><span class="k">None</span><span class="p">,</span> <span class="k">None</span><span class="p">),)</span> <span class="o">*</span> <span class="nb">len</span><span class="p">(</span><span class="n">tickers</span><span class="p">)</span>
<span class="n">objective</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">sum</span><span class="p">(</span><span class="n">x</span> <span class="o">*</span> <span class="n">returns_a</span><span class="p">)</span> <span class="o">*</span> <span class="o">-</span><span class="mi">1</span> <span class="c1"># Because we&#39;re minimizing</span>
<span class="n">bounds</span> <span class="o">=</span> <span class="p">((</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">),)</span> <span class="o">*</span> <span class="nb">len</span><span class="p">(</span><span class="n">tickers</span><span class="p">)</span>
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">tickers</span><span class="p">))</span>
<span class="n">constraints</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">{</span>
<span class="s">&#39;type&#39;</span><span class="p">:</span> <span class="s">&#39;eq&#39;</span><span class="p">,</span>
<span class="s">&#39;fun&#39;</span><span class="p">:</span> <span class="n">is_portfolio</span>
<span class="s1">&#39;type&#39;</span><span class="p">:</span> <span class="s1">&#39;eq&#39;</span><span class="p">,</span>
<span class="s1">&#39;fun&#39;</span><span class="p">:</span> <span class="n">is_portfolio</span>
<span class="p">},</span> <span class="p">{</span>
<span class="s">&#39;type&#39;</span><span class="p">:</span> <span class="s">&#39;ineq&#39;</span><span class="p">,</span>
<span class="s">&#39;fun&#39;</span><span class="p">:</span> <span class="n">within_leverage</span>
<span class="c">#}, {</span>
<span class="c"># &#39;type&#39;: &#39;ineq&#39;,</span>
<span class="c"># &#39;fun&#39;: meets_sharpe</span>
<span class="s1">&#39;type&#39;</span><span class="p">:</span> <span class="s1">&#39;ineq&#39;</span><span class="p">,</span>
<span class="s1">&#39;fun&#39;</span><span class="p">:</span> <span class="n">within_leverage</span>
<span class="c1">#}, {</span>
<span class="c1"># &#39;type&#39;: &#39;ineq&#39;,</span>
<span class="c1"># &#39;fun&#39;: meets_sharpe</span>
<span class="p">},</span> <span class="p">{</span>
<span class="s">&#39;type&#39;</span><span class="p">:</span> <span class="s">&#39;ineq&#39;</span><span class="p">,</span>
<span class="s">&#39;fun&#39;</span><span class="p">:</span> <span class="n">meets_dd</span>
<span class="s1">&#39;type&#39;</span><span class="p">:</span> <span class="s1">&#39;ineq&#39;</span><span class="p">,</span>
<span class="s1">&#39;fun&#39;</span><span class="p">:</span> <span class="n">meets_dd</span>
<span class="p">}</span>
<span class="p">]</span>
<span class="n">optimal</span> <span class="o">=</span> <span class="n">minimize</span><span class="p">(</span><span class="n">objective</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">bounds</span><span class="o">=</span><span class="n">bounds</span><span class="p">,</span> <span class="n">constraints</span><span class="o">=</span><span class="n">constraints</span><span class="p">,</span>
<span class="n">options</span><span class="o">=</span><span class="p">{</span><span class="s">&#39;maxiter&#39;</span><span class="p">:</span> <span class="mi">500</span><span class="p">})</span>
<span class="n">options</span><span class="o">=</span><span class="p">{</span><span class="s1">&#39;maxiter&#39;</span><span class="p">:</span> <span class="mi">500</span><span class="p">})</span>
<span class="c"># Optimization time!</span>
<span class="c1"># Optimization time!</span>
<span class="n">display</span><span class="p">(</span><span class="n">optimal</span><span class="o">.</span><span class="n">message</span><span class="p">)</span>
<span class="n">display</span><span class="p">(</span><span class="s">&quot;Holdings: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">zip</span><span class="p">(</span><span class="n">tickers</span><span class="p">,</span> <span class="n">optimal</span><span class="o">.</span><span class="n">x</span><span class="p">))))</span>
<span class="n">display</span><span class="p">(</span><span class="s2">&quot;Holdings: {}&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">zip</span><span class="p">(</span><span class="n">tickers</span><span class="p">,</span> <span class="n">optimal</span><span class="o">.</span><span class="n">x</span><span class="p">))))</span>
<span class="n">expected_return</span> <span class="o">=</span> <span class="n">optimal</span><span class="o">.</span><span class="n">fun</span> <span class="o">*</span> <span class="o">-</span><span class="mi">100</span> <span class="c"># multiply by -100 to scale, and compensate for minimizing</span>
<span class="n">display</span><span class="p">(</span><span class="s">&quot;Expected Return: {:.3f}%&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">expected_return</span><span class="p">))</span>
<span class="n">expected_return</span> <span class="o">=</span> <span class="n">optimal</span><span class="o">.</span><span class="n">fun</span> <span class="o">*</span> <span class="o">-</span><span class="mi">100</span> <span class="c1"># multiply by -100 to scale, and compensate for minimizing</span>
<span class="n">display</span><span class="p">(</span><span class="s2">&quot;Expected Return: {:.3f}%&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">expected_return</span><span class="p">))</span>
<span class="n">expected_drawdown</span> <span class="o">=</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">optimal</span><span class="o">.</span><span class="n">x</span><span class="p">)</span> <span class="o">*</span> <span class="n">dd_a</span><span class="p">)</span> <span class="o">/</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">optimal</span><span class="o">.</span><span class="n">x</span><span class="p">))</span> <span class="o">*</span> <span class="mi">100</span>
<span class="n">display</span><span class="p">(</span><span class="s">&quot;Expected Max Drawdown: {0:.2f}%&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">expected_drawdown</span><span class="p">))</span>
<span class="n">display</span><span class="p">(</span><span class="s2">&quot;Expected Max Drawdown: {0:.2f}%&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">expected_drawdown</span><span class="p">))</span>
<span class="c"># TODO: Calculate expected Sharpe</span>
<span class="c1"># TODO: Calculate expected Sharpe</span>
</pre></div>
</div>