mirror of
https://github.com/bspeice/bspeice.github.io
synced 2024-12-30 10:18:11 -05:00
Add a new Rust article!
This commit is contained in:
Bradlee Speice
parent
f9e8c08491
commit
6910e70b66
408
a-rustic-re-podcasting-server-part-1.html
Normal file
408
a-rustic-re-podcasting-server-part-1.html
Normal file
@ -0,0 +1,408 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="description" content="I listen to a lot of Drum and Bass music, because it's beautiful music. And there's a particular site, Bassdrive.com that hosts a lot of great content. Specifically, the archives section of the ...">
|
||||
<meta name="keywords" content="nutone, Rust">
|
||||
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
|
||||
|
||||
<title>A Rustic Re-Podcasting Server (Part 1) - Bradlee Speice</title>
|
||||
|
||||
<!-- Stylesheets -->
|
||||
<link href="https://bspeice.github.io/theme/css/bootstrap.min.css" rel="stylesheet">
|
||||
<link href="https://bspeice.github.io/theme/css/fonts.css" rel="stylesheet">
|
||||
<link href="https://bspeice.github.io/theme/css/nest.css" rel="stylesheet">
|
||||
<link href="https://bspeice.github.io/theme/css/pygment.css" rel="stylesheet">
|
||||
<!-- /Stylesheets -->
|
||||
|
||||
<!-- RSS Feeds -->
|
||||
<link href="https://bspeice.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate" title="Bradlee Speice Full Atom Feed" />
|
||||
<link href="https://bspeice.github.io/feeds/blog.atom.xml" type="application/atom+xml" rel="alternate" title="Bradlee Speice Categories Atom Feed" />
|
||||
<!-- /RSS Feeds -->
|
||||
|
||||
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
|
||||
<!--[if lt IE 9]>
|
||||
<script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
|
||||
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
|
||||
<![endif]-->
|
||||
|
||||
<!-- Google Analytics -->
|
||||
<script>
|
||||
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
|
||||
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
|
||||
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
|
||||
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
|
||||
|
||||
ga('create', 'UA-74711362-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script>
|
||||
<!-- /Google Analytics -->
|
||||
|
||||
|
||||
</head>
|
||||
|
||||
<body>
|
||||
|
||||
<!-- Header -->
|
||||
<div class="header-container gradient">
|
||||
|
||||
<!-- Static navbar -->
|
||||
<div class="container">
|
||||
<div class="header-nav">
|
||||
<div class="header-logo">
|
||||
<a class="pull-left" href="https://bspeice.github.io/"><img class="mr20" src="https://bspeice.github.io/images/logo.svg" alt="logo">Bradlee Speice</a>
|
||||
</div>
|
||||
<div class="nav pull-right">
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- /Static navbar -->
|
||||
|
||||
<!-- Header -->
|
||||
<!-- Header -->
|
||||
<div class="container header-wrapper">
|
||||
<div class="row">
|
||||
<div class="col-lg-12">
|
||||
<div class="header-content">
|
||||
<h1 class="header-title">A Rustic Re-Podcasting Server (Part 1)</h1>
|
||||
<p class="header-date"> <a href="https://bspeice.github.io/author/bradlee-speice.html">Bradlee Speice</a>, Sat 22 October 2016, <a href="https://bspeice.github.io/category/blog.html">Blog</a></p>
|
||||
<div class="header-underline"></div>
|
||||
<div class="clearfix"></div>
|
||||
<p class="pull-right header-tags">
|
||||
<span class="glyphicon glyphicon-tags mr5" aria-hidden="true"></span>
|
||||
<a href="https://bspeice.github.io/tag/nutone.html">nutone</a>, <a href="https://bspeice.github.io/tag/rust.html">Rust</a> </p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- /Header -->
|
||||
<!-- /Header -->
|
||||
|
||||
</div>
|
||||
<!-- /Header -->
|
||||
|
||||
|
||||
<!-- Content -->
|
||||
<div class="container content">
|
||||
<p>I listen to a lot of Drum and Bass music, because it's beautiful music. And
|
||||
there's a particular site, <a href="http://bassdrive.com/">Bassdrive.com</a> that hosts
|
||||
a lot of great content. Specifically, the
|
||||
<a href="http://archives.bassdrivearchive.com/">archives</a> section of the site has a
|
||||
list of the past shows that you can download and listen to. The issue is, it's
|
||||
just a <a href="http://archives.bassdrivearchive.com/6%20-%20Saturday/Electronic%20Warfare%20-%20The%20Overfiend/">giant list of links to download</a>. I'd really like
|
||||
this in a podcast format to take with me on the road, etc.</p>
|
||||
<p>So I wrote the <a href="https://github.com/bspeice/elektricity">elektricity</a> web
|
||||
application to actually accomplish all that. Whenever you request a feed, it
|
||||
goes out to Bassdrive, processes all the links on a page, and serves up some
|
||||
fresh, tasty RSS to satisfy your ears. I hosted it on Heroku using the free
|
||||
tier because it's really not resource-intensive at all.</p>
|
||||
<p><strong>The issue so far</strong> is that I keep running out of free tier hours during a
|
||||
month because my podcasting application likes to have a server scan for new
|
||||
episodes constantly. Not sure why it's doing that, but I don't have a whole
|
||||
lot of control over it. It's a phenomenal application otherwise.</p>
|
||||
<p><strong>My (over-engineered) solution</strong>: Re-write the application using the
|
||||
<a href="https://www.rust-lang.org/en-US/">Rust</a> programming language. I'd like to run
|
||||
this on a small hacker board I own, and doing this in Rust would allow me to
|
||||
easily cross-compile it. Plus, I've been very interested in the Rust language
|
||||
for a while and this would be a great opportunity to really learn it well.
|
||||
The code is available <a href="https://github.com/bspeice/nutone">here</a> as development
|
||||
progresses.</p>
|
||||
<h1>The Setup</h1>
|
||||
<p>We'll be using the <a href="http://ironframework.io/">iron</a> library to handle the
|
||||
server, and <a href="http://hyper.rs/">hyper</a> to fetch the data we need from elsewhere
|
||||
on the interwebs. <a href="http://doc.servo.org/html5ever/index.html">HTML5Ever</a> allows
|
||||
us to ingest the content that will be coming from Bassdrive, and finally,
|
||||
output is done with <a href="http://sunng87.github.io/handlebars-rust/handlebars/index.html">handlebars-rust</a>.</p>
|
||||
<p>It will ultimately be interesting to see how much more work must be done to
|
||||
actually get this working over another language like Python. Coming from a
|
||||
dynamic state of mind it's super easy to just chain stuff together, ship it out,
|
||||
and call it a day. I think I'm going to end up getting much dirtier trying to
|
||||
write all of this out.</p>
|
||||
<h1>Issue 1: Strings</h1>
|
||||
<p>Strings in Rust are hard. I acknowledge Python can get away with some things
|
||||
that make strings super easy (and Python 3 has gotten better at cracking down
|
||||
on some bad cases, <code>str <-> bytes</code> specifically), but Rust is hard.</p>
|
||||
<p>Let's take for example the <code>404</code> error handler I'm trying to write. The result
|
||||
should be incredibly simple: All I want is to echo back
|
||||
<code>Didn't find URL: <url></code>. Shouldn't be that hard right? In Python I'd just do
|
||||
something like:</p>
|
||||
<div class="highlight"><pre><span class="k">def</span> <span class="nf">echo_handler</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
|
||||
<span class="k">return</span> <span class="s2">"You're visiting: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">request</span><span class="o">.</span><span class="n">uri</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
|
||||
|
||||
<p>And we'd call it a day. Rust isn't so simple. Let's start with the trivial
|
||||
examples people post online:</p>
|
||||
<div class="highlight"><pre><span class="k">fn</span><span class="w"> </span><span class="n">hello_world</span><span class="p">(</span><span class="n">req</span><span class="o">:</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="p">)</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">IronResult</span><span class="o"><</span><span class="n">Response</span><span class="o">></span><span class="w"> </span><span class="p">{</span><span class="w"></span>
|
||||
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">Response</span><span class="o">::</span><span class="n">with</span><span class="p">((</span><span class="n">status</span><span class="o">::</span><span class="nb">Ok</span><span class="p">,</span><span class="w"> </span><span class="s">"You found the server!"</span><span class="p">)))</span><span class="w"></span>
|
||||
<span class="p">}</span><span class="w"></span>
|
||||
</pre></div>
|
||||
|
||||
|
||||
<p>Doesn't look too bad right? In fact, it's essentially the same as the Python
|
||||
version! All we need to do is just send back a string of some form. So, we
|
||||
look up the documentation for <a href="http://ironframework.io/doc/iron/request/struct.Request.html"><code>Request</code></a> and see a <code>url</code> field that will contain
|
||||
what we want. Let's try the first iteration:</p>
|
||||
<div class="highlight"><pre><span class="k">fn</span><span class="w"> </span><span class="n">hello_world</span><span class="p">(</span><span class="n">req</span><span class="o">:</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="p">)</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">IronResult</span><span class="o"><</span><span class="n">Response</span><span class="o">></span><span class="w"> </span><span class="p">{</span><span class="w"></span>
|
||||
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">Response</span><span class="o">::</span><span class="n">with</span><span class="p">((</span><span class="n">status</span><span class="o">::</span><span class="nb">Ok</span><span class="p">,</span><span class="w"> </span><span class="s">"You found the URL: "</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">req</span><span class="p">.</span><span class="n">url</span><span class="p">)))</span><span class="w"></span>
|
||||
<span class="p">}</span><span class="w"></span>
|
||||
</pre></div>
|
||||
|
||||
|
||||
<p>Which yields the error:</p>
|
||||
<div class="highlight"><pre>error[E0369]: binary operation `+` cannot be applied to type `&'static str`
|
||||
</pre></div>
|
||||
|
||||
|
||||
<p>OK, what's going on here? Time to start Googling for <a href="https://www.google.com/#q=concatenate+strings+in+rust">"concatenate strings in Rust"</a>. That's what we
|
||||
want to do right? Concatenate a static string and the URL.</p>
|
||||
<p>After Googling, we come across a helpful <a href="https://doc.rust-lang.org/std/macro.concat!.html"><code>concat!</code></a> macro that looks really nice! Let's try that one:</p>
|
||||
<div class="highlight"><pre><span class="k">fn</span><span class="w"> </span><span class="n">hello_world</span><span class="p">(</span><span class="n">req</span><span class="o">:</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="p">)</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">IronResult</span><span class="o"><</span><span class="n">Response</span><span class="o">></span><span class="w"> </span><span class="p">{</span><span class="w"></span>
|
||||
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">Response</span><span class="o">::</span><span class="n">with</span><span class="p">((</span><span class="n">status</span><span class="o">::</span><span class="nb">Ok</span><span class="p">,</span><span class="w"> </span><span class="n">concat</span><span class="o">!</span><span class="p">(</span><span class="s">"You found the URL: "</span><span class="p">,</span><span class="w"> </span><span class="n">req</span><span class="p">.</span><span class="n">url</span><span class="p">))))</span><span class="w"></span>
|
||||
<span class="p">}</span><span class="w"></span>
|
||||
</pre></div>
|
||||
|
||||
|
||||
<p>And the error:</p>
|
||||
<p><code>error: expected a literal</code></p>
|
||||
<p>Turns out Rust actually blows up because the <code>concat!</code> macro expects us to know
|
||||
at compile time what <code>req.url</code> is. Which, in my outsider opinion, is a bit
|
||||
strange. <code>println!</code> and <code>format!</code>, etc., all handle values they don't know at
|
||||
compile time. Why can't <code>concat!</code>? By any means, we need a new plan of attack.
|
||||
How about we try formatting strings?</p>
|
||||
<div class="highlight"><pre><span class="k">fn</span><span class="w"> </span><span class="n">hello_world</span><span class="p">(</span><span class="n">req</span><span class="o">:</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="p">)</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">IronResult</span><span class="o"><</span><span class="n">Response</span><span class="o">></span><span class="w"> </span><span class="p">{</span><span class="w"></span>
|
||||
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">Response</span><span class="o">::</span><span class="n">with</span><span class="p">((</span><span class="n">status</span><span class="o">::</span><span class="nb">Ok</span><span class="p">,</span><span class="w"> </span><span class="n">format</span><span class="o">!</span><span class="p">(</span><span class="s">"You found the URL: {}"</span><span class="p">,</span><span class="w"> </span><span class="n">req</span><span class="p">.</span><span class="n">url</span><span class="p">))))</span><span class="w"></span>
|
||||
<span class="p">}</span><span class="w"></span>
|
||||
</pre></div>
|
||||
|
||||
|
||||
<p>And at long last, it works. Onwards!</p>
|
||||
<h1>Issue 2: Fighting with the borrow checker</h1>
|
||||
<p>Rust's single coolest feature is how the compiler can guarantee safety in your
|
||||
program. As long as you don't use <code>unsafe</code> pointers in Rust, you're guaranteed
|
||||
safety. And not having truly manual memory management is really cool; I'm
|
||||
totally OK with never having to write <code>malloc()</code> again.</p>
|
||||
<p>That said, even <a href="https://doc.rust-lang.org/book/ownership.html">the Rust documentation</a> makes a specific note:</p>
|
||||
<blockquote>
|
||||
<p>Many new users to Rust experience something we like to call
|
||||
‘fighting with the borrow checker’, where the Rust compiler refuses to
|
||||
compile a program that the author thinks is valid.</p>
|
||||
</blockquote>
|
||||
<p>If you have to put it in the documentation, it's not a helpful note:
|
||||
it's hazing.</p>
|
||||
<p>So now that we have a handler which works with information from the request, we
|
||||
want to start making something that looks like an actual web application.
|
||||
The router provided by <code>iron</code> isn't terribly difficult so I won't cover it.
|
||||
Instead, the thing that had me stumped for a couple hours was trying to
|
||||
dynamically create routes.</p>
|
||||
<p>The unfortunate thing with Rust (in my limited experience at the moment) is that
|
||||
there is a severe lack of non-trivial examples. Using the router is easy when
|
||||
you want to give an example of a static function. But how do you you start
|
||||
working on things that are a bit more complex?</p>
|
||||
<p>We're going to cover that here. Our first try: creating a function which returns
|
||||
other functions. This is a principle called <a href="http://stackoverflow.com/a/36321/1454178">currying</a>. We set up a function that allows us to keep some data in scope
|
||||
for another function to come later.</p>
|
||||
<div class="highlight"><pre><span class="k">fn</span><span class="w"> </span><span class="n">build_handler</span><span class="p">(</span><span class="n">message</span><span class="o">:</span><span class="w"> </span><span class="nb">String</span><span class="p">)</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="nb">Fn</span><span class="p">(</span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="p">)</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">IronResult</span><span class="o"><</span><span class="n">Response</span><span class="o">></span><span class="w"> </span><span class="p">{</span><span class="w"></span>
|
||||
<span class="w"> </span><span class="n">move</span><span class="w"> </span><span class="o">|</span><span class="n">_</span><span class="o">:</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="o">|</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
|
||||
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">Response</span><span class="o">::</span><span class="n">with</span><span class="p">((</span><span class="n">status</span><span class="o">::</span><span class="nb">Ok</span><span class="p">,</span><span class="w"> </span><span class="n">message</span><span class="p">)))</span><span class="w"></span>
|
||||
<span class="w"> </span><span class="p">}</span><span class="w"></span>
|
||||
<span class="p">}</span><span class="w"></span>
|
||||
</pre></div>
|
||||
|
||||
|
||||
<p>We've simply set up a function that returns another anonymous function with the
|
||||
<code>message</code> parameter scoped in. If you compile this, you get not 1, not 2, but 5
|
||||
new errors. 4 of them are the same though:</p>
|
||||
<div class="highlight"><pre>error[E0277]: the trait bound `for<'r, 'r, 'r> std::ops::Fn(&'r mut iron::Request<'r, 'r>) -> std::result::Result<iron::Response, iron::IronError> + 'static: std::marker::Sized` is not satisfied
|
||||
</pre></div>
|
||||
|
||||
|
||||
<p>...oookay. I for one, am not going to spend time trying to figure out what's
|
||||
going on there.</p>
|
||||
<p>And it is here that I will save the audience many hours of frustrated effort.
|
||||
At this point, I decided to switch from <code>iron</code> to pure <code>hyper</code> since using
|
||||
<code>hyper</code> would give me a much simpler API. All I would have to do is build a
|
||||
function that took two parameters as input, and we're done. That said, it
|
||||
ultimately posed many more issues because I started getting into a weird fight
|
||||
with the <code>'static</code> <a href="https://doc.rust-lang.org/book/lifetimes.html">lifetime</a>
|
||||
and being a Rust newbie I just gave up on trying to understand it.</p>
|
||||
<p>Instead, we will abandon (mostly) the curried function attempt, and instead
|
||||
take advantage of something Rust actually intends us to use: <code>struct</code> and
|
||||
<code>trait</code>.</p>
|
||||
<p>Remember when I talked about a lack of non-trivial examples on the Internet?
|
||||
This is what I was talking about. I could only find <em>one</em> example of this
|
||||
available online, and it was incredibly complex and contained code we honestly
|
||||
don't need or care about. There was no documentation of how to build routes that
|
||||
didn't use static functions, etc. But, I'm assuming you don't really care about
|
||||
my whining, so let's get to it.</p>
|
||||
<p>The <code>iron</code> documentation mentions the <a href="http://ironframework.io/doc/iron/middleware/trait.Handler.html"><code>Handler</code></a> trait as being something we can implement.
|
||||
Does the function signature for that <code>handle()</code> method look familiar? It's what
|
||||
we've been working with so far.</p>
|
||||
<p>The principle is that we need to define a new <code>struct</code> to hold our data, then
|
||||
implement that <code>handle()</code> method to return the result. Something that looks
|
||||
like this might do:</p>
|
||||
<div class="highlight"><pre><span class="k">struct</span><span class="w"> </span><span class="n">EchoHandler</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
|
||||
<span class="w"> </span><span class="n">message</span><span class="o">:</span><span class="w"> </span><span class="nb">String</span><span class="w"></span>
|
||||
<span class="p">}</span><span class="w"></span>
|
||||
|
||||
<span class="k">impl</span><span class="w"> </span><span class="n">Handler</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">EchoHandler</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
|
||||
<span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="n">handle</span><span class="p">(</span><span class="o">&</span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">_</span><span class="o">:</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="p">)</span><span class="w"> </span><span class="o">-></span><span class="w"> </span><span class="n">IronResult</span><span class="o"><</span><span class="n">Response</span><span class="o">></span><span class="w"> </span><span class="p">{</span><span class="w"></span>
|
||||
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">Response</span><span class="o">::</span><span class="n">with</span><span class="p">((</span><span class="n">status</span><span class="o">::</span><span class="nb">Ok</span><span class="p">,</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">message</span><span class="p">)))</span><span class="w"></span>
|
||||
<span class="w"> </span><span class="p">}</span><span class="w"></span>
|
||||
<span class="p">}</span><span class="w"></span>
|
||||
|
||||
<span class="c1">// Later in the code when we set up the router...</span>
|
||||
<span class="kd">let</span><span class="w"> </span><span class="n">echo</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">EchoHandler</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
|
||||
<span class="w"> </span><span class="n">message</span><span class="o">:</span><span class="w"> </span><span class="s">"Is it working yet?"</span><span class="w"></span>
|
||||
<span class="p">}</span><span class="w"></span>
|
||||
<span class="n">router</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">"/"</span><span class="p">,</span><span class="w"> </span><span class="n">echo</span><span class="p">.</span><span class="n">handle</span><span class="p">,</span><span class="w"> </span><span class="s">"index"</span><span class="p">);</span><span class="w"></span>
|
||||
</pre></div>
|
||||
|
||||
|
||||
<p>We attempt to build a struct, and give its <code>handle</code> method off to the router
|
||||
so the router knows what to do.</p>
|
||||
<p>You guessed it, more errors:</p>
|
||||
<div class="highlight"><pre><span class="n">error</span><span class="o">:</span> <span class="n">attempted</span> <span class="n">to</span> <span class="n">take</span> <span class="n">value</span> <span class="n">of</span> <span class="n">method</span> <span class="err">`</span><span class="n">handle</span><span class="err">`</span> <span class="n">on</span> <span class="n">type</span> <span class="err">`</span><span class="n">EchoHandler</span><span class="err">`</span>
|
||||
</pre></div>
|
||||
|
||||
|
||||
<p>Now, the Rust compiler is actually a really nice fellow, and offers us help:</p>
|
||||
<div class="highlight"><pre><span class="n">help</span><span class="o">:</span> <span class="n">maybe</span> <span class="n">a</span> <span class="err">`</span><span class="o">()</span><span class="err">`</span> <span class="n">to</span> <span class="n">call</span> <span class="n">it</span> <span class="k">is</span> <span class="n">missing</span><span class="o">?</span> <span class="n">If</span> <span class="n">not</span><span class="o">,</span> <span class="k">try</span> <span class="n">an</span> <span class="n">anonymous</span> <span class="kd">function</span>
|
||||
</pre></div>
|
||||
|
||||
|
||||
<p>We definitely don't want to call that function, so maybe try an anonymous
|
||||
function as it recommends?</p>
|
||||
<div class="highlight"><pre><span class="n">router</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">"/"</span><span class="p">,</span><span class="w"> </span><span class="o">|</span><span class="n">req</span><span class="o">:</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="o">|</span><span class="w"> </span><span class="n">echo</span><span class="p">.</span><span class="n">handle</span><span class="p">(</span><span class="n">req</span><span class="p">),</span><span class="w"> </span><span class="s">"index"</span><span class="p">);</span><span class="w"></span>
|
||||
</pre></div>
|
||||
|
||||
|
||||
<p>Another error:</p>
|
||||
<div class="highlight"><pre>error[E0373]: closure may outlive the current function, but it borrows `echo`, which is owned by the current function
|
||||
</pre></div>
|
||||
|
||||
|
||||
<p>Another helpful message:</p>
|
||||
<div class="highlight"><pre><span class="n">help</span><span class="o">:</span> <span class="n">to</span> <span class="n">force</span> <span class="n">the</span> <span class="n">closure</span> <span class="n">to</span> <span class="n">take</span> <span class="n">ownership</span> <span class="n">of</span> <span class="err">`</span><span class="n">echo</span><span class="err">`</span> <span class="o">(</span><span class="n">and</span> <span class="n">any</span> <span class="n">other</span> <span class="n">referenced</span> <span class="n">variables</span><span class="o">),</span> <span class="n">use</span> <span class="n">the</span> <span class="err">`</span><span class="n">move</span><span class="err">`</span> <span class="n">keyword</span>
|
||||
</pre></div>
|
||||
|
||||
|
||||
<p>We're getting closer though! Let's implement this change:</p>
|
||||
<div class="highlight"><pre><span class="n">router</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">"/"</span><span class="p">,</span><span class="w"> </span><span class="n">move</span><span class="w"> </span><span class="o">|</span><span class="n">req</span><span class="o">:</span><span class="w"> </span><span class="o">&</span><span class="k">mut</span><span class="w"> </span><span class="n">Request</span><span class="o">|</span><span class="w"> </span><span class="n">echo</span><span class="p">.</span><span class="n">handle</span><span class="p">(</span><span class="n">req</span><span class="p">),</span><span class="w"> </span><span class="s">"index"</span><span class="p">);</span><span class="w"></span>
|
||||
</pre></div>
|
||||
|
||||
|
||||
<p>And here's where things get strange:</p>
|
||||
<div class="highlight"><pre>error[E0507]: cannot move out of borrowed content
|
||||
--> src/main.rs:18:40
|
||||
|
|
||||
18 | Ok(Response::with((status::Ok, self.message)))
|
||||
| ^^^^ cannot move out of borrowed content
|
||||
</pre></div>
|
||||
|
||||
|
||||
<p>Now, this took me another couple hours to figure out. I'm going to explain it,
|
||||
but <strong>keep this in mind: Rust only allows one reference at a time</strong> (exceptions
|
||||
apply of course).</p>
|
||||
<p>When we attempt to use <code>self.message</code> as it has been created in the earlier
|
||||
<code>struct</code>, we essentially are trying to give it away to another piece of code.
|
||||
Rust's semantics then state that <em>we may no longer access it</em> unless it is
|
||||
returned to us (which <code>iron</code>'s code does not do). There are two ways to fix
|
||||
this:</p>
|
||||
<ol>
|
||||
<li>Only give away references (i.e. <code>&self.message</code> instead of <code>self.message</code>)
|
||||
instead of transferring ownership</li>
|
||||
<li>Make a copy of the underlying value which will be safe to give away</li>
|
||||
</ol>
|
||||
<p>I didn't know these were the two options originally, so I hope this helps the
|
||||
audience out. Because <code>iron</code> won't accept a reference, we are forced into the
|
||||
second option: making a copy. To do so, we just need to change the function
|
||||
to look like this:</p>
|
||||
<div class="highlight"><pre><span class="nb">Ok</span><span class="p">(</span><span class="n">Response</span><span class="o">::</span><span class="n">with</span><span class="p">((</span><span class="n">status</span><span class="o">::</span><span class="nb">Ok</span><span class="p">,</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">message</span><span class="p">.</span><span class="n">clone</span><span class="p">())))</span><span class="w"></span>
|
||||
</pre></div>
|
||||
|
||||
|
||||
<p>Not so bad, huh? My only complaint is that it took so long to figure out exactly
|
||||
what was going on.</p>
|
||||
<p>And now we have a small server that we can configure dynamically. At long last.</p>
|
||||
<blockquote>
|
||||
<p>Final sidenote: You can actually do this without anonymous functions. Just
|
||||
change the router line to:
|
||||
<code>router.get("/", echo, "index");</code></p>
|
||||
<p>Rust's type system seems to figure out that we want to use the <code>handle()</code> method.</p>
|
||||
</blockquote>
|
||||
<h1>Conclusion</h1>
|
||||
<p>After a good long days' work, we now have the routing functionality set up on
|
||||
our application. We should be able to scale this pretty well in the future:
|
||||
the RSS content we need to deliver in the future can be treated as a string, so
|
||||
the building blocks are in place.</p>
|
||||
<p>There are two important things I learned starting with Rust today:</p>
|
||||
<ol>
|
||||
<li>Rust is a new language, and while the code is high-quality, the mindshare is coming.</li>
|
||||
<li>I'm a terrible programmer.</li>
|
||||
</ol>
|
||||
<p>Number 1 is pretty obvious and not surprising to anyone. Number two caught me
|
||||
off guard. I've gotten used to having either a garbage collector (Java, Python,
|
||||
etc.) or playing a little fast and loose with scoping rules (C, C++). You don't
|
||||
have to worry about object lifetime there. With Rust, it's forcing me to fully
|
||||
understand and use well the memory in my applications. In the final mistake I
|
||||
fixed (using <code>.clone()</code>) I would have been fine in C++ to just give away that
|
||||
reference and never use it again. I wouldn't have run into a "use-after-free"
|
||||
error, but I would have potentially been leaking memory. Rust forced me to be
|
||||
incredibly precise about how I use it.</p>
|
||||
<p>All said I'm excited for using Rust more. I think it's super cool, it's just
|
||||
going to take me a lot longer to do this than I originally thought.</p>
|
||||
|
||||
|
||||
<div class="comments">
|
||||
<div id="disqus_thread"></div>
|
||||
<script type="text/javascript">
|
||||
var disqus_shortname = 'bradleespeice';
|
||||
var disqus_identifier = 'a-rustic-re-podcasting-server-part-1.html';
|
||||
var disqus_url = 'https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html';
|
||||
(function() {
|
||||
var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
|
||||
dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js';
|
||||
(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
|
||||
})();
|
||||
</script>
|
||||
<noscript>Please enable JavaScript to view the comments.</noscript>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
<!-- /Content -->
|
||||
|
||||
<!-- Footer -->
|
||||
<div class="footer gradient-2">
|
||||
<div class="container footer-container ">
|
||||
<div class="row">
|
||||
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
|
||||
<div class="footer-title"></div>
|
||||
<ul class="list-unstyled">
|
||||
<li><a href="https://bspeice.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate"></a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
|
||||
<div class="footer-title"></div>
|
||||
<ul class="list-unstyled">
|
||||
<li><a href="https://github.com/bspeice" target="_blank">Github</a></li>
|
||||
<li><a href="https://www.linkedin.com/in/bradleespeice" target="_blank">LinkedIn</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
|
||||
</div>
|
||||
<div class="col-xs-12 col-sm-3 col-md-3 col-lg-3">
|
||||
<p class="pull-right text-right">
|
||||
<small><em>Proudly powered by <a href="http://docs.getpelican.com/" target="_blank">pelican</a></em></small><br/>
|
||||
<small><em>Theme and code by <a href="https://github.com/molivier" target="_blank">molivier</a></em></small><br/>
|
||||
<small></small>
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- /Footer -->
|
||||
</body>
|
||||
</html>
|
||||
@ -82,6 +82,8 @@
|
||||
<div class="container content archive">
|
||||
<h2><a href="https://bspeice.github.io/archives.html"></a></h2>
|
||||
<dl class="dl-horizontal">
|
||||
<dt>Sat 22 October 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
|
||||
<dt>Wed 08 June 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
|
||||
<dt>Sun 15 May 2016</dt>
|
||||
|
||||
@ -82,6 +82,8 @@
|
||||
<div class="container content archive">
|
||||
<h2><a href="https://bspeice.github.io/author/bradlee-speice.html"></a></h2>
|
||||
<dl class="dl-horizontal">
|
||||
<dt>Sat 22 October 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
|
||||
<dt>Wed 08 June 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
|
||||
<dt>Sun 15 May 2016</dt>
|
||||
|
||||
@ -82,6 +82,8 @@
|
||||
<div class="container content archive">
|
||||
<h2><a href="https://bspeice.github.io/author/bradlee-speice.html"></a></h2>
|
||||
<dl class="dl-horizontal">
|
||||
<dt>Sat 22 October 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
|
||||
<dt>Wed 08 June 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
|
||||
<dt>Sun 15 May 2016</dt>
|
||||
|
||||
@ -82,6 +82,8 @@
|
||||
<div class="container content archive">
|
||||
<h2><a href="https://bspeice.github.io/author/bradlee-speice.html">Bradlee Speice</a></h2>
|
||||
<dl class="dl-horizontal">
|
||||
<dt>Sat 22 October 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
|
||||
<dt>Wed 08 June 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
|
||||
<dt>Sun 15 May 2016</dt>
|
||||
|
||||
@ -134,7 +134,7 @@ Because these are all very similar, we decided to demonstrate all 3 products at
|
||||
<div class=" highlight hl-julia"><pre><span class="n">S0</span> <span class="o">=</span> <span class="mf">102.2</span>
|
||||
<span class="n">nominal</span> <span class="o">=</span> <span class="mi">100</span>
|
||||
<span class="n">q</span> <span class="o">=</span> <span class="mf">2.84</span> <span class="o">/</span> <span class="mi">100</span>
|
||||
<span class="err">σ</span> <span class="o">=</span> <span class="mf">15.37</span> <span class="o">/</span> <span class="mi">100</span>
|
||||
<span class="n">σ</span> <span class="o">=</span> <span class="mf">15.37</span> <span class="o">/</span> <span class="mi">100</span>
|
||||
<span class="n">term</span> <span class="o">=</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="o">.</span><span class="mi">49</span><span class="p">,</span> <span class="o">.</span><span class="mi">9</span><span class="p">,</span> <span class="mf">1.21</span><span class="p">,</span> <span class="mf">1.45</span><span class="p">,</span> <span class="mf">1.69</span><span class="p">]</span> <span class="o">/</span> <span class="mi">100</span> <span class="o">+</span> <span class="mi">1</span>
|
||||
|
||||
<span class="c">###</span>
|
||||
@ -187,7 +187,7 @@ Because these are all very similar, we decided to demonstrate all 3 products at
|
||||
<div class="prompt input_prompt">In [3]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-julia"><pre><span class="n">simulate_gbm</span> <span class="o">=</span> <span class="n">function</span><span class="p">(</span><span class="n">S0</span><span class="p">,</span> <span class="err">μ</span><span class="p">,</span> <span class="err">σ</span><span class="p">,</span> <span class="n">T</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
|
||||
<div class=" highlight hl-julia"><pre><span class="n">simulate_gbm</span> <span class="o">=</span> <span class="n">function</span><span class="p">(</span><span class="n">S0</span><span class="p">,</span> <span class="n">μ</span><span class="p">,</span> <span class="n">σ</span><span class="p">,</span> <span class="n">T</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
|
||||
<span class="c"># Set the initial state</span>
|
||||
<span class="n">m</span> <span class="o">=</span> <span class="n">length</span><span class="p">(</span><span class="n">S0</span><span class="p">)</span>
|
||||
<span class="n">t</span> <span class="o">=</span> <span class="n">T</span> <span class="o">/</span> <span class="n">n</span>
|
||||
@ -196,7 +196,7 @@ Because these are all very similar, we decided to demonstrate all 3 products at
|
||||
|
||||
<span class="c"># Build out all states</span>
|
||||
<span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:(</span><span class="n">n</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>
|
||||
<span class="n">motion</span><span class="p">[:,</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="n">motion</span><span class="p">[:,</span><span class="n">i</span><span class="p">]</span> <span class="o">.*</span> <span class="n">exp</span><span class="p">((</span><span class="err">μ</span> <span class="o">-</span> <span class="err">σ</span><span class="o">^</span><span class="mi">2</span><span class="o">/</span><span class="mi">2</span><span class="p">)</span><span class="o">*</span><span class="n">t</span><span class="p">)</span> <span class="o">.*</span> <span class="n">exp</span><span class="p">(</span><span class="n">sqrt</span><span class="p">(</span><span class="n">t</span><span class="p">)</span> <span class="o">*</span> <span class="err">σ</span> <span class="o">.*</span> <span class="n">randn</span><span class="p">(</span><span class="n">m</span><span class="p">))</span>
|
||||
<span class="n">motion</span><span class="p">[:,</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="n">motion</span><span class="p">[:,</span><span class="n">i</span><span class="p">]</span> <span class="o">.*</span> <span class="n">exp</span><span class="p">((</span><span class="n">μ</span> <span class="o">-</span> <span class="n">σ</span><span class="o">^</span><span class="mi">2</span><span class="o">/</span><span class="mi">2</span><span class="p">)</span><span class="o">*</span><span class="n">t</span><span class="p">)</span> <span class="o">.*</span> <span class="n">exp</span><span class="p">(</span><span class="n">sqrt</span><span class="p">(</span><span class="n">t</span><span class="p">)</span> <span class="o">*</span> <span class="n">σ</span> <span class="o">.*</span> <span class="n">randn</span><span class="p">(</span><span class="n">m</span><span class="p">))</span>
|
||||
<span class="k">end</span>
|
||||
|
||||
<span class="k">return</span> <span class="n">motion</span>
|
||||
@ -260,7 +260,7 @@ Because these are all very similar, we decided to demonstrate all 3 products at
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-julia"><pre><span class="n">initial</span> <span class="o">=</span> <span class="n">ones</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span> <span class="o">*</span> <span class="n">S0</span>
|
||||
<span class="c"># Using μ=0, T=.25 for now, we'll use the proper values later</span>
|
||||
<span class="n">motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">initial</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="err">σ</span><span class="p">,</span> <span class="o">.</span><span class="mi">25</span><span class="p">,</span> <span class="mi">200</span><span class="p">)</span>
|
||||
<span class="n">motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">initial</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="n">σ</span><span class="p">,</span> <span class="o">.</span><span class="mi">25</span><span class="p">,</span> <span class="mi">200</span><span class="p">)</span>
|
||||
|
||||
<span class="n">display_motion</span><span class="p">(</span><span class="n">motion</span><span class="p">,</span> <span class="o">.</span><span class="mi">25</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
@ -1943,8 +1943,8 @@ fig.select("#fig-3a6dd25ad25c4037a166889ee51bb151-element-20")
|
||||
<div class=" highlight hl-julia"><pre><span class="n">full_motion</span> <span class="o">=</span> <span class="n">ones</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span> <span class="o">*</span> <span class="n">S0</span>
|
||||
<span class="n">full_term</span> <span class="o">=</span> <span class="n">vcat</span><span class="p">(</span><span class="n">term</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">forward_yield</span><span class="p">)</span>
|
||||
<span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">T</span>
|
||||
<span class="err">μ</span> <span class="o">=</span> <span class="p">(</span><span class="n">full_term</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span> <span class="o">-</span> <span class="n">q</span><span class="p">)</span>
|
||||
<span class="n">year_motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">full_motion</span><span class="p">[:,</span><span class="k">end</span><span class="p">],</span> <span class="err">μ</span><span class="p">,</span> <span class="err">σ</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
|
||||
<span class="n">μ</span> <span class="o">=</span> <span class="p">(</span><span class="n">full_term</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span> <span class="o">-</span> <span class="n">q</span><span class="p">)</span>
|
||||
<span class="n">year_motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">full_motion</span><span class="p">[:,</span><span class="k">end</span><span class="p">],</span> <span class="n">μ</span><span class="p">,</span> <span class="n">σ</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
|
||||
<span class="n">full_motion</span> <span class="o">=</span> <span class="n">hcat</span><span class="p">(</span><span class="n">full_motion</span><span class="p">,</span> <span class="n">year_motion</span><span class="p">)</span>
|
||||
<span class="k">end</span>
|
||||
|
||||
@ -3507,8 +3507,8 @@ fig.select("#fig-0378e04b897742b597befd2e8e1c169e-element-20")
|
||||
<span class="c"># And an S0 to kick things off.</span>
|
||||
<span class="n">final_motion</span> <span class="o">=</span> <span class="n">ones</span><span class="p">(</span><span class="n">m</span><span class="p">)</span> <span class="o">*</span> <span class="n">S0</span>
|
||||
<span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">T</span>
|
||||
<span class="err">μ</span> <span class="o">=</span> <span class="p">(</span><span class="n">forward</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span> <span class="o">-</span> <span class="n">q</span><span class="p">)</span>
|
||||
<span class="n">year_motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">final_motion</span><span class="p">[:,</span><span class="k">end</span><span class="p">],</span> <span class="err">μ</span><span class="p">,</span> <span class="err">σ</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
|
||||
<span class="n">μ</span> <span class="o">=</span> <span class="p">(</span><span class="n">forward</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span> <span class="o">-</span> <span class="n">q</span><span class="p">)</span>
|
||||
<span class="n">year_motion</span> <span class="o">=</span> <span class="n">simulate_gbm</span><span class="p">(</span><span class="n">final_motion</span><span class="p">[:,</span><span class="k">end</span><span class="p">],</span> <span class="n">μ</span><span class="p">,</span> <span class="n">σ</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span>
|
||||
<span class="n">final_motion</span> <span class="o">=</span> <span class="n">hcat</span><span class="p">(</span><span class="n">final_motion</span><span class="p">,</span> <span class="n">year_motion</span><span class="p">)</span>
|
||||
<span class="k">end</span>
|
||||
<span class="k">return</span> <span class="n">final_motion</span>
|
||||
@ -3576,7 +3576,7 @@ fig.select("#fig-0378e04b897742b597befd2e8e1c169e-element-20")
|
||||
|
||||
<span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">t</span>
|
||||
<span class="n">price</span> <span class="o">=</span> <span class="n">year_prices</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
|
||||
<span class="k">if</span> <span class="n">price</span> <span class="err">≥</span> <span class="n">call_barrier</span>
|
||||
<span class="k">if</span> <span class="n">price</span> <span class="n">≥</span> <span class="n">call_barrier</span>
|
||||
<span class="k">return</span> <span class="p">(</span><span class="n">nominal</span> <span class="o">+</span> <span class="n">coupon</span><span class="o">*</span><span class="n">i</span><span class="p">)</span> <span class="o">*</span> <span class="n">exp</span><span class="p">((</span><span class="n">prod</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">[</span><span class="n">i</span><span class="p">:</span><span class="k">end</span><span class="p">])</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">*</span><span class="p">(</span><span class="n">t</span><span class="o">-</span><span class="n">i</span><span class="p">))</span>
|
||||
<span class="k">end</span>
|
||||
<span class="k">end</span>
|
||||
@ -3686,9 +3686,9 @@ Present value of Athena note: $95.00, notional: $100.00</pre>
|
||||
|
||||
<span class="k">for</span> <span class="n">i</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span><span class="n">t</span>
|
||||
<span class="n">price</span> <span class="o">=</span> <span class="n">year_prices</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
|
||||
<span class="k">if</span> <span class="n">price</span> <span class="err">≥</span> <span class="n">call_barrier</span>
|
||||
<span class="k">if</span> <span class="n">price</span> <span class="n">≥</span> <span class="n">call_barrier</span>
|
||||
<span class="k">return</span> <span class="p">(</span><span class="n">nominal</span> <span class="o">+</span> <span class="n">coupon</span> <span class="o">+</span> <span class="n">total_coupons</span><span class="p">)</span><span class="o">*</span><span class="n">exp</span><span class="p">((</span><span class="n">prod</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">[</span><span class="n">i</span><span class="p">:</span><span class="k">end</span><span class="p">])</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">*</span><span class="p">(</span><span class="n">t</span><span class="o">-</span><span class="n">i</span><span class="p">))</span>
|
||||
<span class="k">elseif</span> <span class="n">price</span> <span class="err">≥</span> <span class="n">coupon_barrier</span>
|
||||
<span class="k">elseif</span> <span class="n">price</span> <span class="n">≥</span> <span class="n">coupon_barrier</span>
|
||||
<span class="n">total_coupons</span> <span class="o">=</span> <span class="n">total_coupons</span> <span class="o">*</span> <span class="n">exp</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span> <span class="o">+</span> <span class="n">coupon</span>
|
||||
<span class="k">else</span>
|
||||
<span class="n">total_coupons</span> <span class="o">*=</span> <span class="n">exp</span><span class="p">(</span><span class="n">forward_structure</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>
|
||||
|
||||
@ -82,6 +82,8 @@
|
||||
<div class="container content archive">
|
||||
<h2><a href="https://bspeice.github.io/category/blog.html">Blog</a></h2>
|
||||
<dl class="dl-horizontal">
|
||||
<dt>Sat 22 October 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
|
||||
<dt>Wed 08 June 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
|
||||
<dt>Sun 15 May 2016</dt>
|
||||
|
||||
@ -83,6 +83,8 @@
|
||||
<div class="container content archive">
|
||||
<h2><a href="https://bspeice.github.io/category/blog.html">Blog</a></h2>
|
||||
<dl class="dl-horizontal">
|
||||
<dt>Sat 22 October 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
|
||||
<dt>Wed 08 June 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
|
||||
<dt>Sun 15 May 2016</dt>
|
||||
|
||||
@ -83,6 +83,8 @@
|
||||
<div class="container content archive">
|
||||
<h2><a href="https://bspeice.github.io/category/blog.html">Blog</a></h2>
|
||||
<dl class="dl-horizontal">
|
||||
<dt>Sat 22 October 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
|
||||
<dt>Wed 08 June 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
|
||||
<dt>Sun 15 May 2016</dt>
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="description" content="In [1]: import pickle import pandas as pd import numpy as np from bokeh.plotting import output_notebook, figure, show from bokeh.palettes import RdBu4 as Palette from datetime import datetime import ...">
|
||||
<meta name="description" content="In [1]: import pickle import pandas as pd import numpy as np from bokeh.plotting import output_notebook, figure, show from bokeh.palettes import RdBu4 as Palette from datetime import datetime ...">
|
||||
<meta name="keywords" content="data science, weather">
|
||||
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
|
||||
|
||||
@ -359,7 +359,7 @@
|
||||
<div class="prompt input_prompt">In [2]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">city_forecasts</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="nb">open</span><span class="p">(</span><span class="s">'city_forecasts.p'</span><span class="p">,</span> <span class="s">'rb'</span><span class="p">))</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">city_forecasts</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="nb">open</span><span class="p">(</span><span class="s1">'city_forecasts.p'</span><span class="p">,</span> <span class="s1">'rb'</span><span class="p">))</span>
|
||||
<span class="n">forecasts_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">city_forecasts</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
|
||||
@ -373,24 +373,24 @@
|
||||
<div class="prompt input_prompt">In [3]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">cities</span> <span class="o">=</span> <span class="p">[</span><span class="s">'binghamton'</span><span class="p">,</span> <span class="s">'cary'</span><span class="p">,</span> <span class="s">'nyc'</span><span class="p">,</span> <span class="s">'seattle'</span><span class="p">]</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">cities</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'binghamton'</span><span class="p">,</span> <span class="s1">'cary'</span><span class="p">,</span> <span class="s1">'nyc'</span><span class="p">,</span> <span class="s1">'seattle'</span><span class="p">]</span>
|
||||
<span class="n">city_colors</span> <span class="o">=</span> <span class="p">{</span><span class="n">cities</span><span class="p">[</span><span class="n">i</span><span class="p">]:</span> <span class="n">Palette</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">4</span><span class="p">)}</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">safe_cover</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s">'cloudCover'</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s">'cloudCover'</span><span class="p">]</span>
|
||||
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s1">'cloudCover'</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s1">'cloudCover'</span><span class="p">]</span>
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">monthly_avg_cloudcover</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
|
||||
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
|
||||
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
|
||||
<span class="n">freq</span><span class="o">=</span><span class="s">'D'</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">'left'</span><span class="p">)</span>
|
||||
<span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">forecasts_df</span><span class="p">[</span><span class="n">city</span><span class="p">][</span><span class="n">x</span><span class="p">][</span><span class="s">'currently'</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
|
||||
<span class="n">freq</span><span class="o">=</span><span class="s1">'D'</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">'left'</span><span class="p">)</span>
|
||||
<span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">forecasts_df</span><span class="p">[</span><span class="n">city</span><span class="p">][</span><span class="n">x</span><span class="p">][</span><span class="s1">'currently'</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
|
||||
<span class="n">cloud_cover_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">cloud_cover_vals</span><span class="p">)))</span>
|
||||
<span class="c"># Ignore an issue with nanmean having all NaN values. We'll discuss the data issues below.</span>
|
||||
<span class="c1"># Ignore an issue with nanmean having all NaN values. We'll discuss the data issues below.</span>
|
||||
<span class="k">with</span> <span class="n">warnings</span><span class="o">.</span><span class="n">catch_warnings</span><span class="p">():</span>
|
||||
<span class="n">warnings</span><span class="o">.</span><span class="n">simplefilter</span><span class="p">(</span><span class="s">'ignore'</span><span class="p">)</span>
|
||||
<span class="n">warnings</span><span class="o">.</span><span class="n">simplefilter</span><span class="p">(</span><span class="s1">'ignore'</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">nanmean</span><span class="p">(</span><span class="n">cloud_cover_vals</span><span class="p">),</span> <span class="n">cloud_cover_samples</span>
|
||||
</pre></div>
|
||||
|
||||
@ -409,18 +409,18 @@
|
||||
<span class="k">return</span> <span class="p">[</span><span class="n">monthly_avg_cloudcover</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">month</span><span class="p">)</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span>
|
||||
|
||||
<span class="n">months</span> <span class="o">=</span> <span class="p">[</span>
|
||||
<span class="p">(</span><span class="s">'July'</span><span class="p">,</span> <span class="mi">7</span><span class="p">),</span>
|
||||
<span class="p">(</span><span class="s">'August'</span><span class="p">,</span> <span class="mi">8</span><span class="p">),</span>
|
||||
<span class="p">(</span><span class="s">'September'</span><span class="p">,</span> <span class="mi">9</span><span class="p">),</span>
|
||||
<span class="p">(</span><span class="s">'October'</span><span class="p">,</span> <span class="mi">10</span><span class="p">),</span>
|
||||
<span class="p">(</span><span class="s">'November'</span><span class="p">,</span> <span class="mi">11</span><span class="p">)</span>
|
||||
<span class="p">(</span><span class="s1">'July'</span><span class="p">,</span> <span class="mi">7</span><span class="p">),</span>
|
||||
<span class="p">(</span><span class="s1">'August'</span><span class="p">,</span> <span class="mi">8</span><span class="p">),</span>
|
||||
<span class="p">(</span><span class="s1">'September'</span><span class="p">,</span> <span class="mi">9</span><span class="p">),</span>
|
||||
<span class="p">(</span><span class="s1">'October'</span><span class="p">,</span> <span class="mi">10</span><span class="p">),</span>
|
||||
<span class="p">(</span><span class="s1">'November'</span><span class="p">,</span> <span class="mi">11</span><span class="p">)</span>
|
||||
<span class="p">]</span>
|
||||
|
||||
<span class="k">for</span> <span class="n">month</span><span class="p">,</span> <span class="n">month_id</span> <span class="ow">in</span> <span class="n">months</span><span class="p">:</span>
|
||||
<span class="n">month_averages</span> <span class="o">=</span> <span class="p">{</span><span class="n">city</span><span class="p">:</span> <span class="n">city_avg_cc</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">month_id</span><span class="p">)</span> <span class="k">for</span> <span class="n">city</span> <span class="ow">in</span> <span class="n">cities</span><span class="p">}</span>
|
||||
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">"{} Average Cloud Cover"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">month</span><span class="p">),</span>
|
||||
<span class="n">x_axis_label</span><span class="o">=</span><span class="s">'Year'</span><span class="p">,</span>
|
||||
<span class="n">y_axis_label</span><span class="o">=</span><span class="s">'Cloud Cover Percentage'</span><span class="p">)</span>
|
||||
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s2">"{} Average Cloud Cover"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">month</span><span class="p">),</span>
|
||||
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">'Year'</span><span class="p">,</span>
|
||||
<span class="n">y_axis_label</span><span class="o">=</span><span class="s1">'Cloud Cover Percentage'</span><span class="p">)</span>
|
||||
<span class="k">for</span> <span class="n">city</span> <span class="ow">in</span> <span class="n">cities</span><span class="p">:</span>
|
||||
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="p">[</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">month_averages</span><span class="p">[</span><span class="n">city</span><span class="p">]],</span>
|
||||
<span class="n">legend</span><span class="o">=</span><span class="n">city</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="n">city_colors</span><span class="p">[</span><span class="n">city</span><span class="p">])</span>
|
||||
@ -610,20 +610,20 @@
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">safe_precip</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s">'precipProbability'</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s">'precipProbability'</span><span class="p">]</span>
|
||||
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s1">'precipProbability'</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s1">'precipProbability'</span><span class="p">]</span>
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">monthly_avg_precip</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
|
||||
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
|
||||
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
|
||||
<span class="n">freq</span><span class="o">=</span><span class="s">'D'</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">'left'</span><span class="p">)</span>
|
||||
<span class="n">precip_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">forecasts_df</span><span class="p">[</span><span class="n">city</span><span class="p">][</span><span class="n">x</span><span class="p">][</span><span class="s">'currently'</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
|
||||
<span class="n">freq</span><span class="o">=</span><span class="s1">'D'</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">'left'</span><span class="p">)</span>
|
||||
<span class="n">precip_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">forecasts_df</span><span class="p">[</span><span class="n">city</span><span class="p">][</span><span class="n">x</span><span class="p">][</span><span class="s1">'currently'</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
|
||||
<span class="n">precip_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">precip_vals</span><span class="p">)))</span>
|
||||
<span class="c"># Ignore an issue with nanmean having all NaN values. We'll discuss the data issues below.</span>
|
||||
<span class="c1"># Ignore an issue with nanmean having all NaN values. We'll discuss the data issues below.</span>
|
||||
<span class="k">with</span> <span class="n">warnings</span><span class="o">.</span><span class="n">catch_warnings</span><span class="p">():</span>
|
||||
<span class="n">warnings</span><span class="o">.</span><span class="n">simplefilter</span><span class="p">(</span><span class="s">'ignore'</span><span class="p">)</span>
|
||||
<span class="n">warnings</span><span class="o">.</span><span class="n">simplefilter</span><span class="p">(</span><span class="s1">'ignore'</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">nanmean</span><span class="p">(</span><span class="n">precip_vals</span><span class="p">),</span> <span class="n">precip_samples</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">city_avg_precip</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
|
||||
@ -631,9 +631,9 @@
|
||||
|
||||
<span class="k">for</span> <span class="n">month</span><span class="p">,</span> <span class="n">month_id</span> <span class="ow">in</span> <span class="n">months</span><span class="p">:</span>
|
||||
<span class="n">month_averages</span> <span class="o">=</span> <span class="p">{</span><span class="n">city</span><span class="p">:</span> <span class="n">city_avg_cc</span><span class="p">(</span><span class="n">city</span><span class="p">,</span> <span class="n">month_id</span><span class="p">)</span> <span class="k">for</span> <span class="n">city</span> <span class="ow">in</span> <span class="n">cities</span><span class="p">}</span>
|
||||
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">"{} Average Precipitation Chance"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">month</span><span class="p">),</span>
|
||||
<span class="n">x_axis_label</span><span class="o">=</span><span class="s">'Year'</span><span class="p">,</span>
|
||||
<span class="n">y_axis_label</span><span class="o">=</span><span class="s">'Precipitation Chance Percentage'</span><span class="p">)</span>
|
||||
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s2">"{} Average Precipitation Chance"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">month</span><span class="p">),</span>
|
||||
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">'Year'</span><span class="p">,</span>
|
||||
<span class="n">y_axis_label</span><span class="o">=</span><span class="s1">'Precipitation Chance Percentage'</span><span class="p">)</span>
|
||||
<span class="k">for</span> <span class="n">city</span> <span class="ow">in</span> <span class="n">cities</span><span class="p">:</span>
|
||||
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="p">[</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">month_averages</span><span class="p">[</span><span class="n">city</span><span class="p">]],</span>
|
||||
<span class="n">legend</span><span class="o">=</span><span class="n">city</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="n">city_colors</span><span class="p">[</span><span class="n">city</span><span class="p">])</span>
|
||||
|
||||
@ -352,7 +352,7 @@
|
||||
<div class="prompt input_prompt">In [2]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">city_forecasts</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="nb">open</span><span class="p">(</span><span class="s">'city_forecasts.p'</span><span class="p">,</span> <span class="s">'rb'</span><span class="p">))</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">city_forecasts</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="nb">open</span><span class="p">(</span><span class="s1">'city_forecasts.p'</span><span class="p">,</span> <span class="s1">'rb'</span><span class="p">))</span>
|
||||
<span class="n">forecast_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">city_forecasts</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
|
||||
@ -366,31 +366,31 @@
|
||||
<div class="prompt input_prompt">In [3]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">cary_forecast</span> <span class="o">=</span> <span class="n">forecast_df</span><span class="p">[</span><span class="s">'cary'</span><span class="p">]</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">cary_forecast</span> <span class="o">=</span> <span class="n">forecast_df</span><span class="p">[</span><span class="s1">'cary'</span><span class="p">]</span>
|
||||
<span class="n">years</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2016</span><span class="p">)</span>
|
||||
<span class="n">months</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">7</span><span class="p">,</span> <span class="mi">12</span><span class="p">)</span>
|
||||
<span class="n">months_str</span> <span class="o">=</span> <span class="p">[</span><span class="s">'July'</span><span class="p">,</span> <span class="s">'August'</span><span class="p">,</span> <span class="s">'September'</span><span class="p">,</span> <span class="s">'October'</span><span class="p">,</span> <span class="s">'November'</span><span class="p">]</span>
|
||||
<span class="n">months_str</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'July'</span><span class="p">,</span> <span class="s1">'August'</span><span class="p">,</span> <span class="s1">'September'</span><span class="p">,</span> <span class="s1">'October'</span><span class="p">,</span> <span class="s1">'November'</span><span class="p">]</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">safe_cover</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s">'cloudCover'</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s">'cloudCover'</span><span class="p">]</span>
|
||||
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s1">'cloudCover'</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s1">'cloudCover'</span><span class="p">]</span>
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">monthly_avg_cloudcover</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
|
||||
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
|
||||
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
|
||||
<span class="n">freq</span><span class="o">=</span><span class="s">'D'</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">'left'</span><span class="p">)</span>
|
||||
<span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s">'currently'</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
|
||||
<span class="n">freq</span><span class="o">=</span><span class="s1">'D'</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">'left'</span><span class="p">)</span>
|
||||
<span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s1">'currently'</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
|
||||
<span class="n">cloud_cover_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">cloud_cover_vals</span><span class="p">)))</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">nanmean</span><span class="p">(</span><span class="n">cloud_cover_vals</span><span class="p">),</span> <span class="n">cloud_cover_samples</span>
|
||||
|
||||
|
||||
<span class="n">monthly_cover_vals</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_avg_cloudcover</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
|
||||
|
||||
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">'Monthly Average Cloud Cover'</span><span class="p">,</span>
|
||||
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">'Monthly Average Cloud Cover'</span><span class="p">,</span>
|
||||
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
|
||||
<span class="n">x_axis_label</span><span class="o">=</span><span class="s">'Year'</span><span class="p">)</span>
|
||||
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">'Year'</span><span class="p">)</span>
|
||||
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
|
||||
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_cover_vals</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
|
||||
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
|
||||
@ -457,8 +457,8 @@
|
||||
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">monthly_cloudy_days</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
|
||||
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
|
||||
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
|
||||
<span class="n">freq</span><span class="o">=</span><span class="s">'D'</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">'left'</span><span class="p">)</span>
|
||||
<span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s">'currently'</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
|
||||
<span class="n">freq</span><span class="o">=</span><span class="s1">'D'</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">'left'</span><span class="p">)</span>
|
||||
<span class="n">cloud_cover_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_cover</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s1">'currently'</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
|
||||
<span class="n">cloud_cover_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">cloud_cover_vals</span><span class="p">)))</span>
|
||||
<span class="n">cloudy_days</span> <span class="o">=</span> <span class="p">[</span><span class="n">cover</span> <span class="o">></span> <span class="o">.</span><span class="mi">5</span> <span class="k">for</span> <span class="n">cover</span> <span class="ow">in</span> <span class="n">cloud_cover_vals</span><span class="p">]</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">count_nonzero</span><span class="p">(</span><span class="n">cloudy_days</span><span class="p">),</span> <span class="n">cloud_cover_samples</span>
|
||||
@ -466,16 +466,16 @@
|
||||
<span class="n">monthly_days_vals</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_cloudy_days</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
|
||||
<span class="n">monthly_cover_samples</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_cloudy_days</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
|
||||
|
||||
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">'Monthly Cloudy Days'</span><span class="p">,</span>
|
||||
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">'Monthly Cloudy Days'</span><span class="p">,</span>
|
||||
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
|
||||
<span class="n">x_axis_label</span><span class="o">=</span><span class="s">'Year'</span><span class="p">)</span>
|
||||
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">'Year'</span><span class="p">)</span>
|
||||
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
|
||||
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_days_vals</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
|
||||
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
|
||||
|
||||
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">'Monthly Cloud Cover Samples'</span><span class="p">,</span>
|
||||
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">'Monthly Cloud Cover Samples'</span><span class="p">,</span>
|
||||
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
|
||||
<span class="n">x_axis_label</span><span class="o">=</span><span class="s">'Year'</span><span class="p">,</span>
|
||||
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">'Year'</span><span class="p">,</span>
|
||||
<span class="n">height</span><span class="o">=</span><span class="mi">300</span><span class="p">)</span>
|
||||
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
|
||||
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_cover_samples</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
|
||||
@ -571,24 +571,24 @@
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">safe_precip</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s">'precipProbability'</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s">'precipProbability'</span><span class="p">]</span>
|
||||
<span class="k">if</span> <span class="n">frame</span> <span class="ow">and</span> <span class="s1">'precipProbability'</span> <span class="ow">in</span> <span class="n">frame</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="n">frame</span><span class="p">[</span><span class="s1">'precipProbability'</span><span class="p">]</span>
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">monthly_avg_precip</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
|
||||
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
|
||||
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
|
||||
<span class="n">freq</span><span class="o">=</span><span class="s">'D'</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">'left'</span><span class="p">)</span>
|
||||
<span class="n">precip_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s">'currently'</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
|
||||
<span class="n">freq</span><span class="o">=</span><span class="s1">'D'</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">'left'</span><span class="p">)</span>
|
||||
<span class="n">precip_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s1">'currently'</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
|
||||
<span class="n">precip_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">precip_vals</span><span class="p">)))</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">nanmean</span><span class="p">(</span><span class="n">precip_vals</span><span class="p">),</span> <span class="n">precip_samples</span>
|
||||
|
||||
<span class="n">monthly_avg_precip_vals</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_avg_precip</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
|
||||
|
||||
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">'Monthly Average Precipitation Chance'</span><span class="p">,</span>
|
||||
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">'Monthly Average Precipitation Chance'</span><span class="p">,</span>
|
||||
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
|
||||
<span class="n">x_axis_label</span><span class="o">=</span><span class="s">'Year'</span><span class="p">)</span>
|
||||
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">'Year'</span><span class="p">)</span>
|
||||
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
|
||||
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_avg_precip_vals</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
|
||||
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
|
||||
@ -654,8 +654,8 @@
|
||||
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">monthly_rainy_days</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">):</span>
|
||||
<span class="n">dates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DatetimeIndex</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
|
||||
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="n">month</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
|
||||
<span class="n">freq</span><span class="o">=</span><span class="s">'D'</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s">'left'</span><span class="p">)</span>
|
||||
<span class="n">precip_prob_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s">'currently'</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
|
||||
<span class="n">freq</span><span class="o">=</span><span class="s1">'D'</span><span class="p">,</span> <span class="n">closed</span><span class="o">=</span><span class="s1">'left'</span><span class="p">)</span>
|
||||
<span class="n">precip_prob_vals</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">safe_precip</span><span class="p">(</span><span class="n">cary_forecast</span><span class="p">[</span><span class="n">x</span><span class="p">][</span><span class="s1">'currently'</span><span class="p">]),</span> <span class="n">dates</span><span class="p">))</span>
|
||||
<span class="n">precip_prob_samples</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span><span class="p">,</span> <span class="n">precip_prob_vals</span><span class="p">)))</span>
|
||||
<span class="n">precip_days</span> <span class="o">=</span> <span class="p">[</span><span class="n">prob</span> <span class="o">></span> <span class="o">.</span><span class="mi">25</span> <span class="k">for</span> <span class="n">prob</span> <span class="ow">in</span> <span class="n">precip_prob_vals</span><span class="p">]</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">count_nonzero</span><span class="p">(</span><span class="n">precip_days</span><span class="p">),</span> <span class="n">precip_prob_samples</span>
|
||||
@ -663,16 +663,16 @@
|
||||
<span class="n">monthly_precip_days_vals</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_rainy_days</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
|
||||
<span class="n">monthly_precip_samples</span> <span class="o">=</span> <span class="p">[[</span><span class="n">monthly_rainy_days</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">m</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span> <span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">months</span><span class="p">]</span>
|
||||
|
||||
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">'Monthly Rainy Days'</span><span class="p">,</span>
|
||||
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">'Monthly Rainy Days'</span><span class="p">,</span>
|
||||
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
|
||||
<span class="n">x_axis_label</span><span class="o">=</span><span class="s">'Year'</span><span class="p">)</span>
|
||||
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">'Year'</span><span class="p">)</span>
|
||||
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
|
||||
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_precip_days_vals</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
|
||||
<span class="n">show</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
|
||||
|
||||
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s">'Monthly Rainy Days Samples'</span><span class="p">,</span>
|
||||
<span class="n">f</span> <span class="o">=</span> <span class="n">figure</span><span class="p">(</span><span class="n">title</span><span class="o">=</span><span class="s1">'Monthly Rainy Days Samples'</span><span class="p">,</span>
|
||||
<span class="n">x_range</span><span class="o">=</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span> <span class="mi">2015</span><span class="p">),</span>
|
||||
<span class="n">x_axis_label</span><span class="o">=</span><span class="s">'Year'</span><span class="p">,</span>
|
||||
<span class="n">x_axis_label</span><span class="o">=</span><span class="s1">'Year'</span><span class="p">,</span>
|
||||
<span class="n">height</span><span class="o">=</span><span class="mi">300</span><span class="p">)</span>
|
||||
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">months</span><span class="p">)):</span>
|
||||
<span class="n">f</span><span class="o">.</span><span class="n">line</span><span class="p">(</span><span class="n">years</span><span class="p">,</span> <span class="n">monthly_precip_samples</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">legend</span><span class="o">=</span><span class="n">months_str</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">color</span><span class="o">=</span><span class="n">Palette</span><span class="p">[</span><span class="n">x</span><span class="p">])</span>
|
||||
@ -775,64 +775,64 @@
|
||||
<span class="kn">import</span> <span class="nn">requests</span>
|
||||
<span class="kn">from</span> <span class="nn">datetime</span> <span class="kn">import</span> <span class="n">datetime</span>
|
||||
|
||||
<span class="c"># Coordinate data from http://itouchmap.com/latlong.html</span>
|
||||
<span class="c1"># Coordinate data from http://itouchmap.com/latlong.html</span>
|
||||
<span class="n">cary_loc</span> <span class="o">=</span> <span class="p">(</span><span class="mf">35.79154</span><span class="p">,</span><span class="o">-</span><span class="mf">78.781117</span><span class="p">)</span>
|
||||
<span class="n">nyc_loc</span> <span class="o">=</span> <span class="p">(</span><span class="mf">40.78306</span><span class="p">,</span><span class="o">-</span><span class="mf">73.971249</span><span class="p">)</span>
|
||||
<span class="n">seattle_loc</span> <span class="o">=</span> <span class="p">(</span><span class="mf">47.60621</span><span class="p">,</span><span class="o">-</span><span class="mf">122.332071</span><span class="p">)</span>
|
||||
<span class="n">binghamton_loc</span> <span class="o">=</span> <span class="p">(</span><span class="mf">42.098687</span><span class="p">,</span><span class="o">-</span><span class="mf">75.917974</span><span class="p">)</span>
|
||||
<span class="n">cities</span> <span class="o">=</span> <span class="p">{</span>
|
||||
<span class="s">'cary'</span><span class="p">:</span> <span class="n">cary_loc</span><span class="p">,</span>
|
||||
<span class="s">'nyc'</span><span class="p">:</span> <span class="n">nyc_loc</span><span class="p">,</span>
|
||||
<span class="s">'seattle'</span><span class="p">:</span> <span class="n">seattle_loc</span><span class="p">,</span>
|
||||
<span class="s">'binghamton'</span><span class="p">:</span> <span class="n">binghamton_loc</span>
|
||||
<span class="s1">'cary'</span><span class="p">:</span> <span class="n">cary_loc</span><span class="p">,</span>
|
||||
<span class="s1">'nyc'</span><span class="p">:</span> <span class="n">nyc_loc</span><span class="p">,</span>
|
||||
<span class="s1">'seattle'</span><span class="p">:</span> <span class="n">seattle_loc</span><span class="p">,</span>
|
||||
<span class="s1">'binghamton'</span><span class="p">:</span> <span class="n">binghamton_loc</span>
|
||||
<span class="p">}</span>
|
||||
|
||||
<span class="n">apikey</span> <span class="o">=</span> <span class="s">''</span> <span class="c"># My super-secret API Key</span>
|
||||
<span class="n">apikey</span> <span class="o">=</span> <span class="s1">''</span> <span class="c1"># My super-secret API Key</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">get_forecast</span><span class="p">(</span><span class="n">lat</span><span class="p">,</span> <span class="nb">long</span><span class="p">,</span> <span class="n">date</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span>
|
||||
<span class="n">forecast_base</span> <span class="o">=</span> <span class="s">"https://api.forecast.io/forecast/"</span>
|
||||
<span class="n">forecast_base</span> <span class="o">=</span> <span class="s2">"https://api.forecast.io/forecast/"</span>
|
||||
<span class="k">if</span> <span class="n">date</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
|
||||
<span class="n">url</span> <span class="o">=</span> <span class="n">forecast_base</span> <span class="o">+</span> <span class="n">apikey</span> <span class="o">+</span> <span class="s">'/{},{}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">lat</span><span class="p">,</span> <span class="nb">long</span><span class="p">)</span>
|
||||
<span class="n">url</span> <span class="o">=</span> <span class="n">forecast_base</span> <span class="o">+</span> <span class="n">apikey</span> <span class="o">+</span> <span class="s1">'/{},{}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">lat</span><span class="p">,</span> <span class="nb">long</span><span class="p">)</span>
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
<span class="n">epoch</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">date</span><span class="o">.</span><span class="n">timestamp</span><span class="p">())</span>
|
||||
<span class="n">url</span> <span class="o">=</span> <span class="n">forecast_base</span> <span class="o">+</span> <span class="n">apikey</span> <span class="o">+</span> <span class="s">'/{},{},{}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">lat</span><span class="p">,</span> <span class="nb">long</span><span class="p">,</span> <span class="n">epoch</span><span class="p">)</span>
|
||||
<span class="n">url</span> <span class="o">=</span> <span class="n">forecast_base</span> <span class="o">+</span> <span class="n">apikey</span> <span class="o">+</span> <span class="s1">'/{},{},{}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">lat</span><span class="p">,</span> <span class="nb">long</span><span class="p">,</span> <span class="n">epoch</span><span class="p">)</span>
|
||||
|
||||
<span class="k">return</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span><span class="o">.</span><span class="n">json</span><span class="p">()</span>
|
||||
|
||||
<span class="n">years</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1990</span><span class="p">,</span><span class="mi">2016</span><span class="p">)</span>
|
||||
<span class="c"># For datetimes, the 12 is for getting the weather at noon.</span>
|
||||
<span class="c"># We're doing this over midnight because we're more concerned</span>
|
||||
<span class="c"># with what people see, and people don't typically see the weather</span>
|
||||
<span class="c"># at midnight.</span>
|
||||
<span class="c1"># For datetimes, the 12 is for getting the weather at noon.</span>
|
||||
<span class="c1"># We're doing this over midnight because we're more concerned</span>
|
||||
<span class="c1"># with what people see, and people don't typically see the weather</span>
|
||||
<span class="c1"># at midnight.</span>
|
||||
<span class="n">dt_indices</span> <span class="o">=</span> <span class="p">[</span><span class="n">pd</span><span class="o">.</span><span class="n">date_range</span><span class="p">(</span><span class="n">start</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
|
||||
<span class="n">end</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="n">year</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">30</span><span class="p">,</span> <span class="mi">12</span><span class="p">))</span>
|
||||
<span class="k">for</span> <span class="n">year</span> <span class="ow">in</span> <span class="n">years</span><span class="p">]</span>
|
||||
<span class="n">dt_merge</span> <span class="o">=</span> <span class="nb">reduce</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">union</span><span class="p">(</span><span class="n">y</span><span class="p">),</span> <span class="n">dt_indices</span><span class="p">)</span>
|
||||
|
||||
<span class="c"># Because we have to pay a little bit to use the API, we use for loops here</span>
|
||||
<span class="c"># instead of a comprehension - if something breaks, we want to preserve the</span>
|
||||
<span class="c"># data already retrieved</span>
|
||||
<span class="c1"># Because we have to pay a little bit to use the API, we use for loops here</span>
|
||||
<span class="c1"># instead of a comprehension - if something breaks, we want to preserve the</span>
|
||||
<span class="c1"># data already retrieved</span>
|
||||
<span class="n">city_forecasts</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
<span class="k">for</span> <span class="n">city</span><span class="p">,</span> <span class="n">loc</span> <span class="ow">in</span> <span class="n">cities</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
<span class="k">print</span><span class="p">(</span><span class="s">"Retrieving data for {} starting at {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">city</span><span class="p">,</span>
|
||||
<span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s">"%I:%M:%S %p"</span><span class="p">)))</span>
|
||||
<span class="k">print</span><span class="p">(</span><span class="s2">"Retrieving data for {} starting at {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">city</span><span class="p">,</span>
|
||||
<span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s2">"%I:%M:%S %p"</span><span class="p">)))</span>
|
||||
<span class="k">for</span> <span class="n">dt</span> <span class="ow">in</span> <span class="n">dt_merge</span><span class="p">:</span>
|
||||
<span class="k">try</span><span class="p">:</span>
|
||||
<span class="n">city_forecasts</span><span class="p">[(</span><span class="n">city</span><span class="p">,</span> <span class="n">dt</span><span class="p">)]</span> <span class="o">=</span> <span class="n">get_forecast</span><span class="p">(</span><span class="o">*</span><span class="n">loc</span><span class="p">,</span> <span class="n">dt</span><span class="p">)</span>
|
||||
<span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
|
||||
<span class="k">print</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
|
||||
<span class="n">city_forecasts</span><span class="p">[(</span><span class="n">city</span><span class="p">,</span> <span class="n">dt</span><span class="p">)]</span> <span class="o">=</span> <span class="bp">None</span>
|
||||
<span class="k">print</span><span class="p">(</span><span class="s">"End forecast retrieval: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s">"%I:%M:%S %p"</span><span class="p">)))</span>
|
||||
<span class="k">print</span><span class="p">(</span><span class="s2">"End forecast retrieval: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s2">"%I:%M:%S %p"</span><span class="p">)))</span>
|
||||
|
||||
<span class="kn">import</span> <span class="nn">pickle</span>
|
||||
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">city_forecasts</span><span class="p">,</span> <span class="nb">open</span><span class="p">(</span><span class="s">'city_forecasts.p'</span><span class="p">,</span> <span class="s">'wb'</span><span class="p">))</span>
|
||||
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">city_forecasts</span><span class="p">,</span> <span class="nb">open</span><span class="p">(</span><span class="s1">'city_forecasts.p'</span><span class="p">,</span> <span class="s1">'wb'</span><span class="p">))</span>
|
||||
|
||||
<span class="c">### Output:</span>
|
||||
<span class="c"># Retrieving data for binghamton starting at 05:13:42 PM</span>
|
||||
<span class="c"># Retrieving data for seattle starting at 05:30:51 PM</span>
|
||||
<span class="c"># Retrieving data for nyc starting at 05:48:30 PM</span>
|
||||
<span class="c"># Retrieving data for cary starting at 06:08:32 PM</span>
|
||||
<span class="c"># End forecast retrieval: 06:25:21 PM</span>
|
||||
<span class="c1">### Output:</span>
|
||||
<span class="c1"># Retrieving data for binghamton starting at 05:13:42 PM</span>
|
||||
<span class="c1"># Retrieving data for seattle starting at 05:30:51 PM</span>
|
||||
<span class="c1"># Retrieving data for nyc starting at 05:48:30 PM</span>
|
||||
<span class="c1"># Retrieving data for cary starting at 06:08:32 PM</span>
|
||||
<span class="c1"># End forecast retrieval: 06:25:21 PM</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="description" content="Or, being suspicious of market insiders. Use the button below to show the code I've used to generate this article. Because there is a significant amount more code involved than most other posts I've ...">
|
||||
<meta name="description" content="Or, being suspicious of market insiders. Use the button below to show the code I've used to generate this article. Because there is a significant amount more code involved than most other posts ...">
|
||||
<meta name="keywords" content="earnings, event study">
|
||||
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
|
||||
|
||||
@ -108,19 +108,19 @@
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="kn">from</span> <span class="nn">IPython.display</span> <span class="k">import</span> <span class="n">HTML</span>
|
||||
|
||||
<span class="n">HTML</span><span class="p">(</span><span class="s">'''<script></span>
|
||||
<span class="s">code_show=true; </span>
|
||||
<span class="s">function code_toggle() {</span>
|
||||
<span class="s"> if (code_show){</span>
|
||||
<span class="s"> $('div.input').hide();</span>
|
||||
<span class="s"> } else {</span>
|
||||
<span class="s"> $('div.input').show();</span>
|
||||
<span class="s"> }</span>
|
||||
<span class="s"> code_show = !code_show</span>
|
||||
<span class="s">} </span>
|
||||
<span class="s">$( document ).ready(code_toggle);</span>
|
||||
<span class="s"></script></span>
|
||||
<span class="s"><form action="javascript:code_toggle()"><input type="submit" value="Click here to toggle on/off the raw code."></form>'''</span><span class="p">)</span>
|
||||
<span class="n">HTML</span><span class="p">(</span><span class="s1">'''<script></span>
|
||||
<span class="s1">code_show=true; </span>
|
||||
<span class="s1">function code_toggle() {</span>
|
||||
<span class="s1"> if (code_show){</span>
|
||||
<span class="s1"> $('div.input').hide();</span>
|
||||
<span class="s1"> } else {</span>
|
||||
<span class="s1"> $('div.input').show();</span>
|
||||
<span class="s1"> }</span>
|
||||
<span class="s1"> code_show = !code_show</span>
|
||||
<span class="s1">} </span>
|
||||
<span class="s1">$( document ).ready(code_toggle);</span>
|
||||
<span class="s1"></script></span>
|
||||
<span class="s1"><form action="javascript:code_toggle()"><input type="submit" value="Click here to toggle on/off the raw code."></form>'''</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -182,20 +182,20 @@ $( document ).ready(code_toggle);
|
||||
<span class="o">%</span><span class="k">matplotlib</span> inline
|
||||
|
||||
<span class="k">def</span> <span class="nf">fetch_ticker</span><span class="p">(</span><span class="n">ticker</span><span class="p">,</span> <span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">):</span>
|
||||
<span class="c"># Quandl is currently giving me issues with returning</span>
|
||||
<span class="c"># the entire dataset and not slicing server-side.</span>
|
||||
<span class="c"># So instead, we'll do it client-side!</span>
|
||||
<span class="n">q_format</span> <span class="o">=</span> <span class="s">'%Y-%m-%d'</span>
|
||||
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'YAHOO/'</span> <span class="o">+</span> <span class="n">ticker</span><span class="p">,</span>
|
||||
<span class="c1"># Quandl is currently giving me issues with returning</span>
|
||||
<span class="c1"># the entire dataset and not slicing server-side.</span>
|
||||
<span class="c1"># So instead, we'll do it client-side!</span>
|
||||
<span class="n">q_format</span> <span class="o">=</span> <span class="s1">'%Y-%m-</span><span class="si">%d</span><span class="s1">'</span>
|
||||
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'YAHOO/'</span> <span class="o">+</span> <span class="n">ticker</span><span class="p">,</span>
|
||||
<span class="n">start_date</span><span class="o">=</span><span class="n">start</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="n">q_format</span><span class="p">),</span>
|
||||
<span class="n">end_date</span><span class="o">=</span><span class="n">end</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="n">q_format</span><span class="p">),</span>
|
||||
<span class="n">authtoken</span><span class="o">=</span><span class="n">QUANDL_KEY</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">ticker_data</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">ohlc_dataframe</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">ax</span><span class="o">=</span><span class="k">None</span><span class="p">):</span>
|
||||
<span class="c"># Much of this code re-used from:</span>
|
||||
<span class="c"># http://matplotlib.org/examples/pylab_examples/finance_demo.html</span>
|
||||
<span class="k">if</span> <span class="n">ax</span> <span class="ow">is</span> <span class="k">None</span><span class="p">:</span>
|
||||
<span class="k">def</span> <span class="nf">ohlc_dataframe</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">ax</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
||||
<span class="c1"># Much of this code re-used from:</span>
|
||||
<span class="c1"># http://matplotlib.org/examples/pylab_examples/finance_demo.html</span>
|
||||
<span class="k">if</span> <span class="n">ax</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
<span class="n">f</span><span class="p">,</span> <span class="n">ax</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">()</span>
|
||||
|
||||
<span class="n">vals</span> <span class="o">=</span> <span class="p">[(</span><span class="n">date2num</span><span class="p">(</span><span class="n">date</span><span class="p">),</span> <span class="o">*</span><span class="p">(</span><span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">date</span><span class="p">]))</span>
|
||||
@ -204,20 +204,20 @@ $( document ).ready(code_toggle);
|
||||
|
||||
<span class="n">mondays</span> <span class="o">=</span> <span class="n">WeekdayLocator</span><span class="p">(</span><span class="n">MONDAY</span><span class="p">)</span>
|
||||
<span class="n">alldays</span> <span class="o">=</span> <span class="n">DayLocator</span><span class="p">()</span>
|
||||
<span class="n">weekFormatter</span> <span class="o">=</span> <span class="n">DateFormatter</span><span class="p">(</span><span class="s">'%b %d'</span><span class="p">)</span>
|
||||
<span class="n">weekFormatter</span> <span class="o">=</span> <span class="n">DateFormatter</span><span class="p">(</span><span class="s1">'%b </span><span class="si">%d</span><span class="s1">'</span><span class="p">)</span>
|
||||
<span class="n">ax</span><span class="o">.</span><span class="n">xaxis</span><span class="o">.</span><span class="n">set_major_locator</span><span class="p">(</span><span class="n">mondays</span><span class="p">)</span>
|
||||
<span class="n">ax</span><span class="o">.</span><span class="n">xaxis</span><span class="o">.</span><span class="n">set_minor_locator</span><span class="p">(</span><span class="n">alldays</span><span class="p">)</span>
|
||||
<span class="n">ax</span><span class="o">.</span><span class="n">xaxis</span><span class="o">.</span><span class="n">set_major_formatter</span><span class="p">(</span><span class="n">weekFormatter</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">ax</span>
|
||||
|
||||
<span class="n">AAPL</span> <span class="o">=</span> <span class="n">fetch_ticker</span><span class="p">(</span><span class="s">'AAPL'</span><span class="p">,</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span>
|
||||
<span class="n">AAPL</span> <span class="o">=</span> <span class="n">fetch_ticker</span><span class="p">(</span><span class="s1">'AAPL'</span><span class="p">,</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span>
|
||||
<span class="n">ax</span> <span class="o">=</span> <span class="n">ohlc_dataframe</span><span class="p">(</span><span class="n">AAPL</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">vlines</span><span class="p">(</span><span class="n">date2num</span><span class="p">(</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">26</span><span class="p">,</span> <span class="mi">12</span><span class="p">)),</span>
|
||||
<span class="n">ax</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">0</span><span class="p">],</span> <span class="n">ax</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">1</span><span class="p">],</span>
|
||||
<span class="n">color</span><span class="o">=</span><span class="s">'b'</span><span class="p">,</span>
|
||||
<span class="n">label</span><span class="o">=</span><span class="s">'Earnings Release'</span><span class="p">)</span>
|
||||
<span class="n">color</span><span class="o">=</span><span class="s1">'b'</span><span class="p">,</span>
|
||||
<span class="n">label</span><span class="o">=</span><span class="s1">'Earnings Release'</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">3</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">"Apple Price 3/1/2016 - 5/1/2016"</span><span class="p">);</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s2">"Apple Price 3/1/2016 - 5/1/2016"</span><span class="p">);</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -439,12 +439,12 @@ rgAAAABJRU5ErkJggg==
|
||||
<div class="prompt input_prompt">In [3]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">FB</span> <span class="o">=</span> <span class="n">fetch_ticker</span><span class="p">(</span><span class="s">'FB'</span><span class="p">,</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">5</span><span class="p">))</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">FB</span> <span class="o">=</span> <span class="n">fetch_ticker</span><span class="p">(</span><span class="s1">'FB'</span><span class="p">,</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">5</span><span class="p">))</span>
|
||||
<span class="n">ax</span> <span class="o">=</span> <span class="n">ohlc_dataframe</span><span class="p">(</span><span class="n">FB</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">vlines</span><span class="p">(</span><span class="n">date2num</span><span class="p">(</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">27</span><span class="p">,</span> <span class="mi">12</span><span class="p">)),</span>
|
||||
<span class="n">ax</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">0</span><span class="p">],</span> <span class="n">ax</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">1</span><span class="p">],</span>
|
||||
<span class="n">color</span><span class="o">=</span><span class="s">'b'</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">'Earnings Release'</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">'Facebook Price 3/5/2016 - 5/5/2016'</span><span class="p">)</span>
|
||||
<span class="n">color</span><span class="o">=</span><span class="s1">'b'</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">'Earnings Release'</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">'Facebook Price 3/5/2016 - 5/5/2016'</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">2</span><span class="p">);</span>
|
||||
</pre></div>
|
||||
|
||||
@ -698,36 +698,36 @@ czHjid0552LGE7tzzsXM/wfgLII59KYqMQAAAABJRU5ErkJggg==
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">plot_hilo</span><span class="p">(</span><span class="n">ax</span><span class="p">,</span> <span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">data</span><span class="p">):</span>
|
||||
<span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">([</span><span class="n">date2num</span><span class="p">(</span><span class="n">start</span><span class="p">),</span> <span class="n">date2num</span><span class="p">(</span><span class="n">end</span><span class="p">)],</span>
|
||||
<span class="p">[</span><span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">start</span><span class="p">][</span><span class="s">'High'</span><span class="p">],</span> <span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">end</span><span class="p">][</span><span class="s">'High'</span><span class="p">]],</span>
|
||||
<span class="n">color</span><span class="o">=</span><span class="s">'b'</span><span class="p">)</span>
|
||||
<span class="p">[</span><span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">start</span><span class="p">][</span><span class="s1">'High'</span><span class="p">],</span> <span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">end</span><span class="p">][</span><span class="s1">'High'</span><span class="p">]],</span>
|
||||
<span class="n">color</span><span class="o">=</span><span class="s1">'b'</span><span class="p">)</span>
|
||||
<span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">([</span><span class="n">date2num</span><span class="p">(</span><span class="n">start</span><span class="p">),</span> <span class="n">date2num</span><span class="p">(</span><span class="n">end</span><span class="p">)],</span>
|
||||
<span class="p">[</span><span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">start</span><span class="p">][</span><span class="s">'Low'</span><span class="p">],</span> <span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">end</span><span class="p">][</span><span class="s">'Low'</span><span class="p">]],</span>
|
||||
<span class="n">color</span><span class="o">=</span><span class="s">'b'</span><span class="p">)</span>
|
||||
<span class="p">[</span><span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">start</span><span class="p">][</span><span class="s1">'Low'</span><span class="p">],</span> <span class="n">data</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="n">end</span><span class="p">][</span><span class="s1">'Low'</span><span class="p">]],</span>
|
||||
<span class="n">color</span><span class="o">=</span><span class="s1">'b'</span><span class="p">)</span>
|
||||
|
||||
<span class="n">f</span><span class="p">,</span> <span class="n">axarr</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
|
||||
|
||||
<span class="n">ax_aapl</span> <span class="o">=</span> <span class="n">axarr</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
||||
<span class="n">ax_fb</span> <span class="o">=</span> <span class="n">axarr</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
|
||||
|
||||
<span class="c"># Plot the AAPL trend up and down</span>
|
||||
<span class="c1"># Plot the AAPL trend up and down</span>
|
||||
<span class="n">ohlc_dataframe</span><span class="p">(</span><span class="n">AAPL</span><span class="p">,</span> <span class="n">ax</span><span class="o">=</span><span class="n">ax_aapl</span><span class="p">)</span>
|
||||
<span class="n">plot_hilo</span><span class="p">(</span><span class="n">ax_aapl</span><span class="p">,</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">15</span><span class="p">),</span> <span class="n">AAPL</span><span class="p">)</span>
|
||||
<span class="n">plot_hilo</span><span class="p">(</span><span class="n">ax_aapl</span><span class="p">,</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">18</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">26</span><span class="p">),</span> <span class="n">AAPL</span><span class="p">)</span>
|
||||
<span class="n">ax_aapl</span><span class="o">.</span><span class="n">vlines</span><span class="p">(</span><span class="n">date2num</span><span class="p">(</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">26</span><span class="p">,</span> <span class="mi">12</span><span class="p">)),</span>
|
||||
<span class="n">ax_aapl</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">0</span><span class="p">],</span> <span class="n">ax_aapl</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">1</span><span class="p">],</span>
|
||||
<span class="n">color</span><span class="o">=</span><span class="s">'g'</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">'Earnings Release'</span><span class="p">)</span>
|
||||
<span class="n">color</span><span class="o">=</span><span class="s1">'g'</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">'Earnings Release'</span><span class="p">)</span>
|
||||
<span class="n">ax_aapl</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
|
||||
<span class="n">ax_aapl</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s">'AAPL Price History'</span><span class="p">)</span>
|
||||
<span class="n">ax_aapl</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s1">'AAPL Price History'</span><span class="p">)</span>
|
||||
|
||||
<span class="c"># Plot the FB trend down and up</span>
|
||||
<span class="c1"># Plot the FB trend down and up</span>
|
||||
<span class="n">ohlc_dataframe</span><span class="p">(</span><span class="n">FB</span><span class="p">,</span> <span class="n">ax</span><span class="o">=</span><span class="n">ax_fb</span><span class="p">)</span>
|
||||
<span class="n">plot_hilo</span><span class="p">(</span><span class="n">ax_fb</span><span class="p">,</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">30</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">27</span><span class="p">),</span> <span class="n">FB</span><span class="p">)</span>
|
||||
<span class="n">plot_hilo</span><span class="p">(</span><span class="n">ax_fb</span><span class="p">,</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">28</span><span class="p">),</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">5</span><span class="p">),</span> <span class="n">FB</span><span class="p">)</span>
|
||||
<span class="n">ax_fb</span><span class="o">.</span><span class="n">vlines</span><span class="p">(</span><span class="n">date2num</span><span class="p">(</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">27</span><span class="p">,</span> <span class="mi">12</span><span class="p">)),</span>
|
||||
<span class="n">ax_fb</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">0</span><span class="p">],</span> <span class="n">ax_fb</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">1</span><span class="p">],</span>
|
||||
<span class="n">color</span><span class="o">=</span><span class="s">'g'</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">'Earnings Release'</span><span class="p">)</span>
|
||||
<span class="n">color</span><span class="o">=</span><span class="s1">'g'</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">'Earnings Release'</span><span class="p">)</span>
|
||||
<span class="n">ax_fb</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
|
||||
<span class="n">ax_fb</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s">'FB Price History'</span><span class="p">)</span>
|
||||
<span class="n">ax_fb</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s1">'FB Price History'</span><span class="p">)</span>
|
||||
|
||||
<span class="n">f</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
@ -1385,17 +1385,17 @@ cAw4PwAAAABJRU5ErkJggg==
|
||||
<span class="kn">from</span> <span class="nn">pandas.tseries.offsets</span> <span class="k">import</span> <span class="n">CustomBusinessDay</span>
|
||||
<span class="kn">from</span> <span class="nn">datetime</span> <span class="k">import</span> <span class="n">datetime</span><span class="p">,</span> <span class="n">timedelta</span>
|
||||
|
||||
<span class="c"># If you remove rules, it removes them from *all* calendars</span>
|
||||
<span class="c"># To ensure we don't pop rules we don't want to, first make</span>
|
||||
<span class="c"># sure to fully copy the object</span>
|
||||
<span class="c1"># If you remove rules, it removes them from *all* calendars</span>
|
||||
<span class="c1"># To ensure we don't pop rules we don't want to, first make</span>
|
||||
<span class="c1"># sure to fully copy the object</span>
|
||||
<span class="n">trade_calendar</span> <span class="o">=</span> <span class="n">USFederalHolidayCalendar</span><span class="p">()</span>
|
||||
<span class="n">trade_calendar</span><span class="o">.</span><span class="n">rules</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="mi">6</span><span class="p">)</span> <span class="c"># Remove Columbus day</span>
|
||||
<span class="n">trade_calendar</span><span class="o">.</span><span class="n">rules</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="mi">7</span><span class="p">)</span> <span class="c"># Remove Veteran's day</span>
|
||||
<span class="n">trade_calendar</span><span class="o">.</span><span class="n">rules</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="mi">6</span><span class="p">)</span> <span class="c1"># Remove Columbus day</span>
|
||||
<span class="n">trade_calendar</span><span class="o">.</span><span class="n">rules</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="mi">7</span><span class="p">)</span> <span class="c1"># Remove Veteran's day</span>
|
||||
<span class="n">TradeDay</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">days</span><span class="p">:</span> <span class="n">CustomBusinessDay</span><span class="p">(</span><span class="n">days</span><span class="p">,</span> <span class="n">calendar</span><span class="o">=</span><span class="n">trade_calendar</span><span class="p">)</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">plot_study</span><span class="p">(</span><span class="n">array</span><span class="p">):</span>
|
||||
<span class="c"># Given a 2-d array, we assume the event happens at index `lookback`,</span>
|
||||
<span class="c"># and create all of our summary statistics from there.</span>
|
||||
<span class="c1"># Given a 2-d array, we assume the event happens at index `lookback`,</span>
|
||||
<span class="c1"># and create all of our summary statistics from there.</span>
|
||||
<span class="n">lookback</span> <span class="o">=</span> <span class="nb">int</span><span class="p">((</span><span class="n">array</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">/</span> <span class="mi">2</span><span class="p">)</span>
|
||||
<span class="n">norm_factor</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="n">array</span><span class="p">[:,</span><span class="n">lookback</span><span class="p">]</span><span class="o">.</span><span class="n">reshape</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="n">array</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
|
||||
<span class="n">centered_data</span> <span class="o">=</span> <span class="n">array</span> <span class="o">/</span> <span class="n">norm_factor</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
@ -1414,17 +1414,17 @@ cAw4PwAAAABJRU5ErkJggg==
|
||||
<span class="n">axarr</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">fill_between</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">range_end</span><span class="p">),</span>
|
||||
<span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="p">:]</span> <span class="o">+</span> <span class="n">std_dev</span><span class="p">,</span>
|
||||
<span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="p">:]</span> <span class="o">-</span> <span class="n">std_dev</span><span class="p">,</span>
|
||||
<span class="n">alpha</span><span class="o">=.</span><span class="mi">5</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">"$\pm$ 1 s.d."</span><span class="p">)</span>
|
||||
<span class="n">alpha</span><span class="o">=.</span><span class="mi">5</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">"$\pm$ 1 s.d."</span><span class="p">)</span>
|
||||
<span class="n">axarr</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">fill_between</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">range_end</span><span class="p">),</span>
|
||||
<span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="p">:]</span> <span class="o">+</span> <span class="n">std_dev</span><span class="p">,</span>
|
||||
<span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="p">:]</span> <span class="o">-</span> <span class="n">std_dev</span><span class="p">,</span>
|
||||
<span class="n">alpha</span><span class="o">=.</span><span class="mi">5</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">"$\pm$ 1 s.d."</span><span class="p">)</span>
|
||||
<span class="n">alpha</span><span class="o">=.</span><span class="mi">5</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">"$\pm$ 1 s.d."</span><span class="p">)</span>
|
||||
|
||||
<span class="n">max_err</span> <span class="o">=</span> <span class="n">maxes</span> <span class="o">-</span> <span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="o">+</span><span class="mi">1</span><span class="p">:]</span>
|
||||
<span class="n">min_err</span> <span class="o">=</span> <span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="o">+</span><span class="mi">1</span><span class="p">:]</span> <span class="o">-</span> <span class="n">mins</span>
|
||||
<span class="n">axarr</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">errorbar</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">range_end</span><span class="p">),</span>
|
||||
<span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="o">+</span><span class="mi">1</span><span class="p">:],</span>
|
||||
<span class="n">yerr</span><span class="o">=</span><span class="p">[</span><span class="n">min_err</span><span class="p">,</span> <span class="n">max_err</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s">'Max & Min'</span><span class="p">)</span>
|
||||
<span class="n">yerr</span><span class="o">=</span><span class="p">[</span><span class="n">min_err</span><span class="p">,</span> <span class="n">max_err</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s1">'Max & Min'</span><span class="p">)</span>
|
||||
<span class="n">axarr</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
|
||||
<span class="n">axarr</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
|
||||
|
||||
@ -1432,8 +1432,8 @@ cAw4PwAAAABJRU5ErkJggg==
|
||||
<span class="n">axarr</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">set_xlim</span><span class="p">((</span><span class="o">-</span><span class="n">lookback</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="n">lookback</span><span class="o">+</span><span class="mi">1</span><span class="p">))</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">plot_study_small</span><span class="p">(</span><span class="n">array</span><span class="p">):</span>
|
||||
<span class="c"># Given a 2-d array, we assume the event happens at index `lookback`,</span>
|
||||
<span class="c"># and create all of our summary statistics from there.</span>
|
||||
<span class="c1"># Given a 2-d array, we assume the event happens at index `lookback`,</span>
|
||||
<span class="c1"># and create all of our summary statistics from there.</span>
|
||||
<span class="n">lookback</span> <span class="o">=</span> <span class="nb">int</span><span class="p">((</span><span class="n">array</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">/</span> <span class="mi">2</span><span class="p">)</span>
|
||||
<span class="n">norm_factor</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="n">array</span><span class="p">[:,</span><span class="n">lookback</span><span class="p">]</span><span class="o">.</span><span class="n">reshape</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="n">array</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
|
||||
<span class="n">centered_data</span> <span class="o">=</span> <span class="n">array</span> <span class="o">/</span> <span class="n">norm_factor</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
@ -1450,24 +1450,24 @@ cAw4PwAAAABJRU5ErkJggg==
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">fill_between</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">range_end</span><span class="p">),</span>
|
||||
<span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="p">:]</span> <span class="o">+</span> <span class="n">std_dev</span><span class="p">,</span>
|
||||
<span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="p">:]</span> <span class="o">-</span> <span class="n">std_dev</span><span class="p">,</span>
|
||||
<span class="n">alpha</span><span class="o">=.</span><span class="mi">5</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">"$\pm$ 1 s.d."</span><span class="p">)</span>
|
||||
<span class="n">alpha</span><span class="o">=.</span><span class="mi">5</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">"$\pm$ 1 s.d."</span><span class="p">)</span>
|
||||
|
||||
<span class="n">max_err</span> <span class="o">=</span> <span class="n">maxes</span> <span class="o">-</span> <span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="o">+</span><span class="mi">1</span><span class="p">:]</span>
|
||||
<span class="n">min_err</span> <span class="o">=</span> <span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="o">+</span><span class="mi">1</span><span class="p">:]</span> <span class="o">-</span> <span class="n">mins</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">errorbar</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">range_end</span><span class="p">),</span>
|
||||
<span class="n">means</span><span class="p">[</span><span class="o">-</span><span class="n">lookforward</span><span class="o">+</span><span class="mi">1</span><span class="p">:],</span>
|
||||
<span class="n">yerr</span><span class="o">=</span><span class="p">[</span><span class="n">min_err</span><span class="p">,</span> <span class="n">max_err</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s">'Max & Min'</span><span class="p">)</span>
|
||||
<span class="n">yerr</span><span class="o">=</span><span class="p">[</span><span class="n">min_err</span><span class="p">,</span> <span class="n">max_err</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s1">'Max & Min'</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">xlim</span><span class="p">((</span><span class="o">-</span><span class="n">lookback</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="n">lookback</span><span class="o">+</span><span class="mi">1</span><span class="p">))</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">fetch_event_data</span><span class="p">(</span><span class="n">ticker</span><span class="p">,</span> <span class="n">events</span><span class="p">,</span> <span class="n">horizon</span><span class="o">=</span><span class="mi">5</span><span class="p">):</span>
|
||||
<span class="c"># Use horizon+1 to account for including the day of the event,</span>
|
||||
<span class="c"># and half-open interval - that is, for a horizon of 5,</span>
|
||||
<span class="c"># we should be including 11 events. Additionally, using the</span>
|
||||
<span class="c"># CustomBusinessDay means we automatically handle issues if</span>
|
||||
<span class="c"># for example a company reports Friday afternoon - the date</span>
|
||||
<span class="c"># calculator will turn this into a "Saturday" release, but</span>
|
||||
<span class="c"># we effectively shift that to Monday with the logic below.</span>
|
||||
<span class="c1"># Use horizon+1 to account for including the day of the event,</span>
|
||||
<span class="c1"># and half-open interval - that is, for a horizon of 5,</span>
|
||||
<span class="c1"># we should be including 11 events. Additionally, using the</span>
|
||||
<span class="c1"># CustomBusinessDay means we automatically handle issues if</span>
|
||||
<span class="c1"># for example a company reports Friday afternoon - the date</span>
|
||||
<span class="c1"># calculator will turn this into a "Saturday" release, but</span>
|
||||
<span class="c1"># we effectively shift that to Monday with the logic below.</span>
|
||||
<span class="n">td_back</span> <span class="o">=</span> <span class="n">TradeDay</span><span class="p">(</span><span class="n">horizon</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span>
|
||||
<span class="n">td_forward</span> <span class="o">=</span> <span class="n">TradeDay</span><span class="p">(</span><span class="n">horizon</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span>
|
||||
|
||||
@ -1476,28 +1476,28 @@ cAw4PwAAAABJRU5ErkJggg==
|
||||
<span class="n">total_data</span> <span class="o">=</span> <span class="n">fetch_ticker</span><span class="p">(</span><span class="n">ticker</span><span class="p">,</span> <span class="n">start_date</span><span class="p">,</span> <span class="n">end_date</span><span class="p">)</span>
|
||||
<span class="n">event_data</span> <span class="o">=</span> <span class="p">[</span><span class="n">total_data</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="n">event</span><span class="o">-</span><span class="n">td_back</span><span class="p">:</span><span class="n">event</span><span class="o">+</span><span class="n">td_forward</span><span class="p">]</span>\
|
||||
<span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="n">horizon</span><span class="o">*</span><span class="mi">2</span><span class="o">+</span><span class="mi">1</span><span class="p">]</span>\
|
||||
<span class="p">[</span><span class="s">'Adjusted Close'</span><span class="p">]</span>
|
||||
<span class="p">[</span><span class="s1">'Adjusted Close'</span><span class="p">]</span>
|
||||
<span class="k">for</span> <span class="n">event</span> <span class="ow">in</span> <span class="n">events</span><span class="p">]</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="n">event_data</span><span class="p">)</span>
|
||||
|
||||
<span class="c"># Generate a couple of random events</span>
|
||||
<span class="c1"># Generate a couple of random events</span>
|
||||
|
||||
<span class="n">event_dates</span> <span class="o">=</span> <span class="p">[</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">27</span><span class="p">)</span> <span class="o">-</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span> <span class="o">-</span> <span class="n">TradeDay</span><span class="p">(</span><span class="n">x</span><span class="o">*</span><span class="mi">20</span><span class="p">)</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">40</span><span class="p">)]</span>
|
||||
<span class="n">data</span> <span class="o">=</span> <span class="n">fetch_event_data</span><span class="p">(</span><span class="s">'CELG'</span><span class="p">,</span> <span class="n">event_dates</span><span class="p">)</span>
|
||||
<span class="n">data</span> <span class="o">=</span> <span class="n">fetch_event_data</span><span class="p">(</span><span class="s1">'CELG'</span><span class="p">,</span> <span class="n">event_dates</span><span class="p">)</span>
|
||||
<span class="n">plot_study_small</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">3</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="mi">6</span><span class="p">);</span>
|
||||
|
||||
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="s">'Mean price for days leading up to each event'</span><span class="p">,</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="s1">'Mean price for days leading up to each event'</span><span class="p">,</span>
|
||||
<span class="p">(</span><span class="o">-</span><span class="mi">5</span><span class="p">,</span> <span class="o">-.</span><span class="mi">01</span><span class="p">),</span> <span class="p">(</span><span class="o">-</span><span class="mf">4.5</span><span class="p">,</span> <span class="o">.</span><span class="mi">025</span><span class="p">),</span>
|
||||
<span class="n">arrowprops</span><span class="o">=</span><span class="nb">dict</span><span class="p">(</span><span class="n">facecolor</span><span class="o">=</span><span class="s">'black'</span><span class="p">,</span> <span class="n">shrink</span><span class="o">=</span><span class="mf">0.05</span><span class="p">))</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="s">''</span><span class="p">,</span> <span class="p">(</span><span class="o">-.</span><span class="mi">1</span><span class="p">,</span> <span class="o">.</span><span class="mi">005</span><span class="p">),</span> <span class="p">(</span><span class="o">-.</span><span class="mi">5</span><span class="p">,</span> <span class="o">.</span><span class="mi">02</span><span class="p">),</span>
|
||||
<span class="n">arrowprops</span><span class="o">=</span><span class="p">{</span><span class="s">'facecolor'</span><span class="p">:</span> <span class="s">'black'</span><span class="p">,</span> <span class="s">'shrink'</span><span class="p">:</span> <span class="o">.</span><span class="mi">05</span><span class="p">})</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="s">'$\pm$ 1 std. dev. each day'</span><span class="p">,</span> <span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="o">.</span><span class="mi">055</span><span class="p">),</span> <span class="p">(</span><span class="mf">2.5</span><span class="p">,</span> <span class="o">.</span><span class="mi">085</span><span class="p">),</span>
|
||||
<span class="n">arrowprops</span><span class="o">=</span><span class="p">{</span><span class="s">'facecolor'</span><span class="p">:</span> <span class="s">'black'</span><span class="p">,</span> <span class="s">'shrink'</span><span class="p">:</span> <span class="o">.</span><span class="mi">05</span><span class="p">})</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="s">'Min/Max each day'</span><span class="p">,</span> <span class="p">(</span><span class="o">.</span><span class="mi">9</span><span class="p">,</span> <span class="o">-.</span><span class="mi">07</span><span class="p">),</span> <span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="o">-.</span><span class="mi">1</span><span class="p">),</span>
|
||||
<span class="n">arrowprops</span><span class="o">=</span><span class="p">{</span><span class="s">'facecolor'</span><span class="p">:</span> <span class="s">'black'</span><span class="p">,</span> <span class="s">'shrink'</span><span class="p">:</span> <span class="o">.</span><span class="mi">05</span><span class="p">});</span>
|
||||
<span class="n">arrowprops</span><span class="o">=</span><span class="nb">dict</span><span class="p">(</span><span class="n">facecolor</span><span class="o">=</span><span class="s1">'black'</span><span class="p">,</span> <span class="n">shrink</span><span class="o">=</span><span class="mf">0.05</span><span class="p">))</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="s1">''</span><span class="p">,</span> <span class="p">(</span><span class="o">-.</span><span class="mi">1</span><span class="p">,</span> <span class="o">.</span><span class="mi">005</span><span class="p">),</span> <span class="p">(</span><span class="o">-.</span><span class="mi">5</span><span class="p">,</span> <span class="o">.</span><span class="mi">02</span><span class="p">),</span>
|
||||
<span class="n">arrowprops</span><span class="o">=</span><span class="p">{</span><span class="s1">'facecolor'</span><span class="p">:</span> <span class="s1">'black'</span><span class="p">,</span> <span class="s1">'shrink'</span><span class="p">:</span> <span class="o">.</span><span class="mi">05</span><span class="p">})</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="s1">'$\pm$ 1 std. dev. each day'</span><span class="p">,</span> <span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="o">.</span><span class="mi">055</span><span class="p">),</span> <span class="p">(</span><span class="mf">2.5</span><span class="p">,</span> <span class="o">.</span><span class="mi">085</span><span class="p">),</span>
|
||||
<span class="n">arrowprops</span><span class="o">=</span><span class="p">{</span><span class="s1">'facecolor'</span><span class="p">:</span> <span class="s1">'black'</span><span class="p">,</span> <span class="s1">'shrink'</span><span class="p">:</span> <span class="o">.</span><span class="mi">05</span><span class="p">})</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="s1">'Min/Max each day'</span><span class="p">,</span> <span class="p">(</span><span class="o">.</span><span class="mi">9</span><span class="p">,</span> <span class="o">-.</span><span class="mi">07</span><span class="p">),</span> <span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="o">-.</span><span class="mi">1</span><span class="p">),</span>
|
||||
<span class="n">arrowprops</span><span class="o">=</span><span class="p">{</span><span class="s1">'facecolor'</span><span class="p">:</span> <span class="s1">'black'</span><span class="p">,</span> <span class="s1">'shrink'</span><span class="p">:</span> <span class="o">.</span><span class="mi">05</span><span class="p">});</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -1957,24 +1957,24 @@ PlISLSIiIiLiIyXRIiIiIiI+UhItIiIiIuIjJdEiIiIiIj76P+bhd3LjvwIqAAAAAElFTkSuQmCC
|
||||
<span class="n">ohlc_dataframe</span><span class="p">(</span><span class="n">FB</span><span class="p">[</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">18</span><span class="p">):],</span> <span class="n">FB_plot</span><span class="p">)</span>
|
||||
|
||||
<span class="n">FB_truncated</span> <span class="o">=</span> <span class="n">FB</span><span class="p">[</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">18</span><span class="p">):</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">27</span><span class="p">)]</span>
|
||||
<span class="n">midpoint</span> <span class="o">=</span> <span class="n">FB_truncated</span><span class="p">[</span><span class="s">'Open'</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span> <span class="o">+</span> <span class="n">FB_truncated</span><span class="p">[</span><span class="s">'Close'</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span>
|
||||
<span class="n">FB_plot</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">FB_truncated</span><span class="o">.</span><span class="n">index</span><span class="p">,</span> <span class="n">midpoint</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">'Midpoint'</span><span class="p">)</span>
|
||||
<span class="n">midpoint</span> <span class="o">=</span> <span class="n">FB_truncated</span><span class="p">[</span><span class="s1">'Open'</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span> <span class="o">+</span> <span class="n">FB_truncated</span><span class="p">[</span><span class="s1">'Close'</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span>
|
||||
<span class="n">FB_plot</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">FB_truncated</span><span class="o">.</span><span class="n">index</span><span class="p">,</span> <span class="n">midpoint</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">'Midpoint'</span><span class="p">)</span>
|
||||
<span class="n">FB_plot</span><span class="o">.</span><span class="n">vlines</span><span class="p">(</span><span class="n">date2num</span><span class="p">(</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">27</span><span class="p">,</span> <span class="mi">12</span><span class="p">)),</span>
|
||||
<span class="n">ax_fb</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">0</span><span class="p">],</span> <span class="n">ax_fb</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">1</span><span class="p">],</span>
|
||||
<span class="n">color</span><span class="o">=</span><span class="s">'g'</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">'Earnings Release'</span><span class="p">)</span>
|
||||
<span class="n">color</span><span class="o">=</span><span class="s1">'g'</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">'Earnings Release'</span><span class="p">)</span>
|
||||
<span class="n">FB_plot</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
|
||||
<span class="n">FB_plot</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s">'FB Midpoint Plot'</span><span class="p">)</span>
|
||||
<span class="n">FB_plot</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s1">'FB Midpoint Plot'</span><span class="p">)</span>
|
||||
|
||||
<span class="n">AAPL_plot</span> <span class="o">=</span> <span class="n">axarr</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
|
||||
<span class="n">ohlc_dataframe</span><span class="p">(</span><span class="n">AAPL</span><span class="p">[</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">10</span><span class="p">):],</span> <span class="n">AAPL_plot</span><span class="p">)</span>
|
||||
<span class="n">AAPL_truncated</span> <span class="o">=</span> <span class="n">AAPL</span><span class="p">[</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">10</span><span class="p">):</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">26</span><span class="p">)]</span>
|
||||
<span class="n">midpoint</span> <span class="o">=</span> <span class="n">AAPL_truncated</span><span class="p">[</span><span class="s">'Open'</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span> <span class="o">+</span> <span class="n">AAPL_truncated</span><span class="p">[</span><span class="s">'Close'</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span>
|
||||
<span class="n">AAPL_plot</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">AAPL_truncated</span><span class="o">.</span><span class="n">index</span><span class="p">,</span> <span class="n">midpoint</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">'Midpoint'</span><span class="p">)</span>
|
||||
<span class="n">midpoint</span> <span class="o">=</span> <span class="n">AAPL_truncated</span><span class="p">[</span><span class="s1">'Open'</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span> <span class="o">+</span> <span class="n">AAPL_truncated</span><span class="p">[</span><span class="s1">'Close'</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span>
|
||||
<span class="n">AAPL_plot</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">AAPL_truncated</span><span class="o">.</span><span class="n">index</span><span class="p">,</span> <span class="n">midpoint</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">'Midpoint'</span><span class="p">)</span>
|
||||
<span class="n">AAPL_plot</span><span class="o">.</span><span class="n">vlines</span><span class="p">(</span><span class="n">date2num</span><span class="p">(</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">26</span><span class="p">,</span> <span class="mi">12</span><span class="p">)),</span>
|
||||
<span class="n">ax_aapl</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">0</span><span class="p">],</span> <span class="n">ax_aapl</span><span class="o">.</span><span class="n">get_ylim</span><span class="p">()[</span><span class="mi">1</span><span class="p">],</span>
|
||||
<span class="n">color</span><span class="o">=</span><span class="s">'g'</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">'Earnings Release'</span><span class="p">)</span>
|
||||
<span class="n">color</span><span class="o">=</span><span class="s1">'g'</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">'Earnings Release'</span><span class="p">)</span>
|
||||
<span class="n">AAPL_plot</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">3</span><span class="p">)</span>
|
||||
<span class="n">AAPL_plot</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s">'AAPL Midpoint Plot'</span><span class="p">);</span>
|
||||
<span class="n">AAPL_plot</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s1">'AAPL Midpoint Plot'</span><span class="p">);</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -2453,74 +2453,74 @@ TkSuQmCC
|
||||
<div class="prompt input_prompt">In [7]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="c"># Read in the events for each stock;</span>
|
||||
<span class="c"># The file was created using the first code block in the Appendix</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="c1"># Read in the events for each stock;</span>
|
||||
<span class="c1"># The file was created using the first code block in the Appendix</span>
|
||||
<span class="kn">import</span> <span class="nn">yaml</span>
|
||||
<span class="kn">from</span> <span class="nn">dateutil.parser</span> <span class="k">import</span> <span class="n">parse</span>
|
||||
<span class="kn">from</span> <span class="nn">progressbar</span> <span class="k">import</span> <span class="n">ProgressBar</span>
|
||||
|
||||
<span class="n">data_str</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s">'earnings_dates.yaml'</span><span class="p">,</span> <span class="s">'r'</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
|
||||
<span class="c"># Need to remove invalid lines</span>
|
||||
<span class="n">filtered</span> <span class="o">=</span> <span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="s">'{'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">x</span><span class="p">,</span> <span class="n">data_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s">'</span><span class="se">\n</span><span class="s">'</span><span class="p">))</span>
|
||||
<span class="n">earnings_data</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s">'</span><span class="se">\n</span><span class="s">'</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">filtered</span><span class="p">))</span>
|
||||
<span class="n">data_str</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s1">'earnings_dates.yaml'</span><span class="p">,</span> <span class="s1">'r'</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
|
||||
<span class="c1"># Need to remove invalid lines</span>
|
||||
<span class="n">filtered</span> <span class="o">=</span> <span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="s1">'{'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">x</span><span class="p">,</span> <span class="n">data_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">'</span><span class="se">\n</span><span class="s1">'</span><span class="p">))</span>
|
||||
<span class="n">earnings_data</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s1">'</span><span class="se">\n</span><span class="s1">'</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">filtered</span><span class="p">))</span>
|
||||
|
||||
<span class="c"># Convert our earnings data into a list of (ticker, date) pairs</span>
|
||||
<span class="c"># to make it easy to work with.</span>
|
||||
<span class="c"># This is horribly inefficient, but should get us what we need</span>
|
||||
<span class="c1"># Convert our earnings data into a list of (ticker, date) pairs</span>
|
||||
<span class="c1"># to make it easy to work with.</span>
|
||||
<span class="c1"># This is horribly inefficient, but should get us what we need</span>
|
||||
<span class="n">ticker_dates</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
<span class="k">for</span> <span class="n">ticker</span><span class="p">,</span> <span class="n">date_list</span> <span class="ow">in</span> <span class="n">earnings_data</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
<span class="k">for</span> <span class="n">iso_str</span> <span class="ow">in</span> <span class="n">date_list</span><span class="p">:</span>
|
||||
<span class="n">ticker_dates</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">ticker</span><span class="p">,</span> <span class="n">parse</span><span class="p">(</span><span class="n">iso_str</span><span class="p">)))</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">does_trend_down</span><span class="p">(</span><span class="n">ticker</span><span class="p">,</span> <span class="n">event</span><span class="p">,</span> <span class="n">horizon</span><span class="p">):</span>
|
||||
<span class="c"># Figure out if the `event` has a downtrend for</span>
|
||||
<span class="c"># the `horizon` days preceding it</span>
|
||||
<span class="c"># As an interpretation note: it is assumed that</span>
|
||||
<span class="c"># the closing price of day `event` is the reference</span>
|
||||
<span class="c"># point, and we want `horizon` days before that.</span>
|
||||
<span class="c"># The price_data.hdf was created in the second appendix code block</span>
|
||||
<span class="c1"># Figure out if the `event` has a downtrend for</span>
|
||||
<span class="c1"># the `horizon` days preceding it</span>
|
||||
<span class="c1"># As an interpretation note: it is assumed that</span>
|
||||
<span class="c1"># the closing price of day `event` is the reference</span>
|
||||
<span class="c1"># point, and we want `horizon` days before that.</span>
|
||||
<span class="c1"># The price_data.hdf was created in the second appendix code block</span>
|
||||
<span class="k">try</span><span class="p">:</span>
|
||||
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_hdf</span><span class="p">(</span><span class="s">'price_data.hdf'</span><span class="p">,</span> <span class="n">ticker</span><span class="p">)</span>
|
||||
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_hdf</span><span class="p">(</span><span class="s1">'price_data.hdf'</span><span class="p">,</span> <span class="n">ticker</span><span class="p">)</span>
|
||||
<span class="n">data</span> <span class="o">=</span> <span class="n">ticker_data</span><span class="p">[</span><span class="n">event</span><span class="o">-</span><span class="n">TradeDay</span><span class="p">(</span><span class="n">horizon</span><span class="p">):</span><span class="n">event</span><span class="p">]</span>
|
||||
<span class="n">midpoints</span> <span class="o">=</span> <span class="n">data</span><span class="p">[</span><span class="s">'Open'</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span> <span class="o">+</span> <span class="n">data</span><span class="p">[</span><span class="s">'Close'</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span>
|
||||
<span class="n">midpoints</span> <span class="o">=</span> <span class="n">data</span><span class="p">[</span><span class="s1">'Open'</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span> <span class="o">+</span> <span class="n">data</span><span class="p">[</span><span class="s1">'Close'</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span>
|
||||
|
||||
<span class="c"># Shift dates one forward into the future and subtract</span>
|
||||
<span class="c"># Effectively: do we trend down over all days?</span>
|
||||
<span class="c1"># Shift dates one forward into the future and subtract</span>
|
||||
<span class="c1"># Effectively: do we trend down over all days?</span>
|
||||
<span class="n">elems</span> <span class="o">=</span> <span class="n">midpoints</span> <span class="o">-</span> <span class="n">midpoints</span><span class="o">.</span><span class="n">shift</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="n">elems</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="n">elems</span><span class="o">.</span><span class="n">dropna</span><span class="p">()[</span><span class="n">elems</span> <span class="o"><=</span> <span class="mi">0</span><span class="p">])</span>
|
||||
<span class="k">except</span> <span class="ne">KeyError</span><span class="p">:</span>
|
||||
<span class="c"># If the stock doesn't exist, it doesn't qualify as trending down</span>
|
||||
<span class="c"># Mostly this is here to make sure the entire analysis doesn't </span>
|
||||
<span class="c"># blow up if there were issues in data retrieval</span>
|
||||
<span class="k">return</span> <span class="k">False</span>
|
||||
<span class="c1"># If the stock doesn't exist, it doesn't qualify as trending down</span>
|
||||
<span class="c1"># Mostly this is here to make sure the entire analysis doesn't </span>
|
||||
<span class="c1"># blow up if there were issues in data retrieval</span>
|
||||
<span class="k">return</span> <span class="kc">False</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">study_trend</span><span class="p">(</span><span class="n">horizon</span><span class="p">,</span> <span class="n">trend_function</span><span class="p">):</span>
|
||||
<span class="n">five_day_events</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">((</span><span class="mi">1</span><span class="p">,</span> <span class="n">horizon</span><span class="o">*</span><span class="mi">2</span> <span class="o">+</span> <span class="mi">1</span><span class="p">))</span>
|
||||
<span class="n">invalid_events</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
<span class="k">for</span> <span class="n">ticker</span><span class="p">,</span> <span class="n">event</span> <span class="ow">in</span> <span class="n">ProgressBar</span><span class="p">()(</span><span class="n">ticker_dates</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">trend_function</span><span class="p">(</span><span class="n">ticker</span><span class="p">,</span> <span class="n">event</span><span class="p">,</span> <span class="n">horizon</span><span class="p">):</span>
|
||||
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_hdf</span><span class="p">(</span><span class="s">'price_data.hdf'</span><span class="p">,</span> <span class="n">ticker</span><span class="p">)</span>
|
||||
<span class="n">event_data</span> <span class="o">=</span> <span class="n">ticker_data</span><span class="p">[</span><span class="n">event</span><span class="o">-</span><span class="n">TradeDay</span><span class="p">(</span><span class="n">horizon</span><span class="p">):</span><span class="n">event</span><span class="o">+</span><span class="n">TradeDay</span><span class="p">(</span><span class="n">horizon</span><span class="p">)][</span><span class="s">'Close'</span><span class="p">]</span>
|
||||
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_hdf</span><span class="p">(</span><span class="s1">'price_data.hdf'</span><span class="p">,</span> <span class="n">ticker</span><span class="p">)</span>
|
||||
<span class="n">event_data</span> <span class="o">=</span> <span class="n">ticker_data</span><span class="p">[</span><span class="n">event</span><span class="o">-</span><span class="n">TradeDay</span><span class="p">(</span><span class="n">horizon</span><span class="p">):</span><span class="n">event</span><span class="o">+</span><span class="n">TradeDay</span><span class="p">(</span><span class="n">horizon</span><span class="p">)][</span><span class="s1">'Close'</span><span class="p">]</span>
|
||||
|
||||
<span class="k">try</span><span class="p">:</span>
|
||||
<span class="n">five_day_events</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">vstack</span><span class="p">([</span><span class="n">five_day_events</span><span class="p">,</span> <span class="n">event_data</span><span class="p">])</span>
|
||||
<span class="k">except</span> <span class="ne">ValueError</span><span class="p">:</span>
|
||||
<span class="c"># Sometimes we don't get exactly the right number of values due to calendar</span>
|
||||
<span class="c"># issues. I've fixed most everything I can, and the few issues that are left</span>
|
||||
<span class="c"># I assume don't systemically bias the results (i.e. data could be missing</span>
|
||||
<span class="c"># because it doesn't exist, etc.). After running through, ~1% of events get</span>
|
||||
<span class="c"># discarded this way</span>
|
||||
<span class="c1"># Sometimes we don't get exactly the right number of values due to calendar</span>
|
||||
<span class="c1"># issues. I've fixed most everything I can, and the few issues that are left</span>
|
||||
<span class="c1"># I assume don't systemically bias the results (i.e. data could be missing</span>
|
||||
<span class="c1"># because it doesn't exist, etc.). After running through, ~1% of events get</span>
|
||||
<span class="c1"># discarded this way</span>
|
||||
<span class="n">invalid_events</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">ticker</span><span class="p">,</span> <span class="n">event</span><span class="p">))</span>
|
||||
|
||||
|
||||
<span class="c"># Remove our initial zero row</span>
|
||||
<span class="c1"># Remove our initial zero row</span>
|
||||
<span class="n">five_day_events</span> <span class="o">=</span> <span class="n">five_day_events</span><span class="p">[</span><span class="mi">1</span><span class="p">:,:]</span>
|
||||
<span class="n">plot_study</span><span class="p">(</span><span class="n">five_day_events</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">'Action over {} days: {} events'</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s1">'Action over {} days: {} events'</span>
|
||||
<span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">horizon</span><span class="p">,</span><span class="n">five_day_events</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]))</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
|
||||
|
||||
<span class="c"># Start with a 5 day study</span>
|
||||
<span class="c1"># Start with a 5 day study</span>
|
||||
<span class="n">study_trend</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="n">does_trend_down</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
|
||||
@ -3000,7 +3000,7 @@ ADBuCBoAAAAAAMC4+f+DLBbJVWQG2gAAAABJRU5ErkJggg==
|
||||
<div class="prompt input_prompt">In [8]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="c"># 8 day study next</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="c1"># 8 day study next</span>
|
||||
<span class="n">study_trend</span><span class="p">(</span><span class="mi">8</span><span class="p">,</span> <span class="n">does_trend_down</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
|
||||
@ -3479,7 +3479,7 @@ AAAAKMr/B9pAH9lY/+K6AAAAAElFTkSuQmCC
|
||||
<div class="prompt input_prompt">In [9]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="c"># 3 day study after that</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="c1"># 3 day study after that</span>
|
||||
<span class="n">study_trend</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="n">does_trend_down</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
|
||||
@ -3892,26 +3892,26 @@ STfmKmb+uQ4AAAAASUVORK5CYII=
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">does_trend_up</span><span class="p">(</span><span class="n">ticker</span><span class="p">,</span> <span class="n">event</span><span class="p">,</span> <span class="n">horizon</span><span class="p">):</span>
|
||||
<span class="c"># Figure out if the `event` has an uptrend for</span>
|
||||
<span class="c"># the `horizon` days preceding it</span>
|
||||
<span class="c"># As an interpretation note: it is assumed that</span>
|
||||
<span class="c"># the closing price of day `event` is the reference</span>
|
||||
<span class="c"># point, and we want `horizon` days before that.</span>
|
||||
<span class="c"># The price_data.hdf was created in the second appendix code block</span>
|
||||
<span class="c1"># Figure out if the `event` has an uptrend for</span>
|
||||
<span class="c1"># the `horizon` days preceding it</span>
|
||||
<span class="c1"># As an interpretation note: it is assumed that</span>
|
||||
<span class="c1"># the closing price of day `event` is the reference</span>
|
||||
<span class="c1"># point, and we want `horizon` days before that.</span>
|
||||
<span class="c1"># The price_data.hdf was created in the second appendix code block</span>
|
||||
<span class="k">try</span><span class="p">:</span>
|
||||
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_hdf</span><span class="p">(</span><span class="s">'price_data.hdf'</span><span class="p">,</span> <span class="n">ticker</span><span class="p">)</span>
|
||||
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_hdf</span><span class="p">(</span><span class="s1">'price_data.hdf'</span><span class="p">,</span> <span class="n">ticker</span><span class="p">)</span>
|
||||
<span class="n">data</span> <span class="o">=</span> <span class="n">ticker_data</span><span class="p">[</span><span class="n">event</span><span class="o">-</span><span class="n">TradeDay</span><span class="p">(</span><span class="n">horizon</span><span class="p">):</span><span class="n">event</span><span class="p">]</span>
|
||||
<span class="n">midpoints</span> <span class="o">=</span> <span class="n">data</span><span class="p">[</span><span class="s">'Open'</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span> <span class="o">+</span> <span class="n">data</span><span class="p">[</span><span class="s">'Close'</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span>
|
||||
<span class="n">midpoints</span> <span class="o">=</span> <span class="n">data</span><span class="p">[</span><span class="s1">'Open'</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span> <span class="o">+</span> <span class="n">data</span><span class="p">[</span><span class="s1">'Close'</span><span class="p">]</span><span class="o">/</span><span class="mi">2</span>
|
||||
|
||||
<span class="c"># Shift dates one forward into the future and subtract</span>
|
||||
<span class="c"># Effectively: do we trend down over all days?</span>
|
||||
<span class="c1"># Shift dates one forward into the future and subtract</span>
|
||||
<span class="c1"># Effectively: do we trend down over all days?</span>
|
||||
<span class="n">elems</span> <span class="o">=</span> <span class="n">midpoints</span> <span class="o">-</span> <span class="n">midpoints</span><span class="o">.</span><span class="n">shift</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="n">elems</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="n">elems</span><span class="o">.</span><span class="n">dropna</span><span class="p">()[</span><span class="n">elems</span> <span class="o">>=</span> <span class="mi">0</span><span class="p">])</span>
|
||||
<span class="k">except</span> <span class="ne">KeyError</span><span class="p">:</span>
|
||||
<span class="c"># If the stock doesn't exist, it doesn't qualify as trending down</span>
|
||||
<span class="c"># Mostly this is here to make sure the entire analysis doesn't </span>
|
||||
<span class="c"># blow up if there were issues in data retrieval</span>
|
||||
<span class="k">return</span> <span class="k">False</span>
|
||||
<span class="c1"># If the stock doesn't exist, it doesn't qualify as trending down</span>
|
||||
<span class="c1"># Mostly this is here to make sure the entire analysis doesn't </span>
|
||||
<span class="c1"># blow up if there were issues in data retrieval</span>
|
||||
<span class="k">return</span> <span class="kc">False</span>
|
||||
|
||||
<span class="n">study_trend</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="n">does_trend_up</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
@ -5192,38 +5192,38 @@ AAAAAGBWNBAAAAAAAMCsaCAAAAAAAIBZ0UAAAAAAAACz+n9WbX+0Yscg8AAAAABJRU5ErkJggg==
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">dates</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">handle_starttag</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">,</span> <span class="n">attrs</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s">'table'</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s1">'table'</span><span class="p">:</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">store_dates</span> <span class="o">=</span> <span class="bp">True</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">handle_data</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">data</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">store_dates</span><span class="p">:</span>
|
||||
<span class="n">match</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s">r'\d+/\d+/\d+'</span><span class="p">,</span> <span class="n">data</span><span class="p">)</span>
|
||||
<span class="n">match</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s1">r'\d+/\d+/\d+'</span><span class="p">,</span> <span class="n">data</span><span class="p">)</span>
|
||||
<span class="k">if</span> <span class="n">match</span><span class="p">:</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">dates</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">match</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">0</span><span class="p">))</span>
|
||||
|
||||
<span class="c"># If a company reports before the bell, record the earnings date</span>
|
||||
<span class="c"># being at midnight the day before. Ex: WMT reports 5/19/2016,</span>
|
||||
<span class="c"># but we want the reference point to be the closing price on 5/18/2016</span>
|
||||
<span class="k">if</span> <span class="s">'After Close'</span> <span class="ow">in</span> <span class="n">data</span><span class="p">:</span>
|
||||
<span class="c1"># If a company reports before the bell, record the earnings date</span>
|
||||
<span class="c1"># being at midnight the day before. Ex: WMT reports 5/19/2016,</span>
|
||||
<span class="c1"># but we want the reference point to be the closing price on 5/18/2016</span>
|
||||
<span class="k">if</span> <span class="s1">'After Close'</span> <span class="ow">in</span> <span class="n">data</span><span class="p">:</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">earnings_offset</span> <span class="o">=</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
|
||||
<span class="k">elif</span> <span class="s">'Before Open'</span> <span class="ow">in</span> <span class="n">data</span><span class="p">:</span>
|
||||
<span class="k">elif</span> <span class="s1">'Before Open'</span> <span class="ow">in</span> <span class="n">data</span><span class="p">:</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">earnings_offset</span> <span class="o">=</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=-</span><span class="mi">1</span><span class="p">)</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">handle_endtag</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s">'table'</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s1">'table'</span><span class="p">:</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">store_dates</span> <span class="o">=</span> <span class="bp">False</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">earnings_releases</span><span class="p">(</span><span class="n">ticker</span><span class="p">):</span>
|
||||
<span class="c">#print("Looking up ticker {}".format(ticker))</span>
|
||||
<span class="n">user_agent</span> <span class="o">=</span> <span class="s">'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) '</span>\
|
||||
<span class="s">'Gecko/20100101 Firefox/46.0'</span>
|
||||
<span class="n">headers</span> <span class="o">=</span> <span class="p">{</span><span class="s">'user-agent'</span><span class="p">:</span> <span class="n">user_agent</span><span class="p">}</span>
|
||||
<span class="n">base_url</span> <span class="o">=</span> <span class="s">'http://www.streetinsider.com/ec_earnings.php?q={}'</span>\
|
||||
<span class="c1">#print("Looking up ticker {}".format(ticker))</span>
|
||||
<span class="n">user_agent</span> <span class="o">=</span> <span class="s1">'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) '</span>\
|
||||
<span class="s1">'Gecko/20100101 Firefox/46.0'</span>
|
||||
<span class="n">headers</span> <span class="o">=</span> <span class="p">{</span><span class="s1">'user-agent'</span><span class="p">:</span> <span class="n">user_agent</span><span class="p">}</span>
|
||||
<span class="n">base_url</span> <span class="o">=</span> <span class="s1">'http://www.streetinsider.com/ec_earnings.php?q={}'</span>\
|
||||
<span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">ticker</span><span class="p">)</span>
|
||||
<span class="n">e</span> <span class="o">=</span> <span class="n">EarningsParser</span><span class="p">()</span>
|
||||
<span class="n">s</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">Session</span><span class="p">()</span>
|
||||
<span class="n">a</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">adapters</span><span class="o">.</span><span class="n">HTTPAdapter</span><span class="p">(</span><span class="n">max_retries</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
|
||||
<span class="n">s</span><span class="o">.</span><span class="n">mount</span><span class="p">(</span><span class="s">'http://'</span><span class="p">,</span> <span class="n">a</span><span class="p">)</span>
|
||||
<span class="n">s</span><span class="o">.</span><span class="n">mount</span><span class="p">(</span><span class="s1">'http://'</span><span class="p">,</span> <span class="n">a</span><span class="p">)</span>
|
||||
<span class="n">e</span><span class="o">.</span><span class="n">feed</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">s</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">base_url</span><span class="p">,</span> <span class="n">headers</span><span class="o">=</span><span class="n">headers</span><span class="p">)</span><span class="o">.</span><span class="n">content</span><span class="p">))</span>
|
||||
|
||||
<span class="k">if</span> <span class="n">e</span><span class="o">.</span><span class="n">earnings_offset</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span><span class="p">:</span>
|
||||
@ -5231,18 +5231,18 @@ AAAAAGBWNBAAAAAAAMCsaCAAAAAAAIBZ0UAAAAAAAACz+n9WbX+0Yscg8AAAAABJRU5ErkJggg==
|
||||
<span class="n">past</span> <span class="o">=</span> <span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span> <span class="o"><</span> <span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">(),</span> <span class="n">dates</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">d</span><span class="p">:</span> <span class="n">d</span><span class="o">.</span><span class="n">isoformat</span><span class="p">(),</span> <span class="n">past</span><span class="p">))</span>
|
||||
|
||||
<span class="c"># Use a Russell-3000 ETF tracker (ticker IWV) to get a list of holdings</span>
|
||||
<span class="n">r3000</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s">'https://www.ishares.com/us/products/239714/'</span>
|
||||
<span class="s">'ishares-russell-3000-etf/1449138789749.ajax?'</span>
|
||||
<span class="s">'fileType=csv&fileName=IWV_holdings&dataType=fund'</span><span class="p">,</span>
|
||||
<span class="c1"># Use a Russell-3000 ETF tracker (ticker IWV) to get a list of holdings</span>
|
||||
<span class="n">r3000</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s1">'https://www.ishares.com/us/products/239714/'</span>
|
||||
<span class="s1">'ishares-russell-3000-etf/1449138789749.ajax?'</span>
|
||||
<span class="s1">'fileType=csv&fileName=IWV_holdings&dataType=fund'</span><span class="p">,</span>
|
||||
<span class="n">header</span><span class="o">=</span><span class="mi">10</span><span class="p">)</span>
|
||||
<span class="n">r3000_equities</span> <span class="o">=</span> <span class="n">r3000</span><span class="p">[(</span><span class="n">r3000</span><span class="p">[</span><span class="s">'Exchange'</span><span class="p">]</span> <span class="o">==</span> <span class="s">'NASDAQ'</span><span class="p">)</span> <span class="o">|</span>
|
||||
<span class="p">(</span><span class="n">r3000</span><span class="p">[</span><span class="s">'Exchange'</span><span class="p">]</span> <span class="o">==</span> <span class="s">'New York Stock Exchange Inc.'</span><span class="p">)]</span>
|
||||
<span class="n">r3000_equities</span> <span class="o">=</span> <span class="n">r3000</span><span class="p">[(</span><span class="n">r3000</span><span class="p">[</span><span class="s1">'Exchange'</span><span class="p">]</span> <span class="o">==</span> <span class="s1">'NASDAQ'</span><span class="p">)</span> <span class="o">|</span>
|
||||
<span class="p">(</span><span class="n">r3000</span><span class="p">[</span><span class="s1">'Exchange'</span><span class="p">]</span> <span class="o">==</span> <span class="s1">'New York Stock Exchange Inc.'</span><span class="p">)]</span>
|
||||
|
||||
<span class="n">dates_file</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s">'earnings_dates.yaml'</span><span class="p">,</span> <span class="s">'w'</span><span class="p">)</span>
|
||||
<span class="n">dates_file</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s1">'earnings_dates.yaml'</span><span class="p">,</span> <span class="s1">'w'</span><span class="p">)</span>
|
||||
|
||||
<span class="k">with</span> <span class="n">futures</span><span class="o">.</span><span class="n">ThreadPoolExecutor</span><span class="p">(</span><span class="n">max_workers</span><span class="o">=</span><span class="mi">8</span><span class="p">)</span> <span class="k">as</span> <span class="n">pool</span><span class="p">:</span>
|
||||
<span class="n">fs</span> <span class="o">=</span> <span class="p">{</span><span class="n">pool</span><span class="o">.</span><span class="n">submit</span><span class="p">(</span><span class="n">earnings_releases</span><span class="p">,</span> <span class="n">r3000_equities</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="n">t</span><span class="p">][</span><span class="s">'Ticker'</span><span class="p">]):</span> <span class="n">t</span>
|
||||
<span class="n">fs</span> <span class="o">=</span> <span class="p">{</span><span class="n">pool</span><span class="o">.</span><span class="n">submit</span><span class="p">(</span><span class="n">earnings_releases</span><span class="p">,</span> <span class="n">r3000_equities</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="n">t</span><span class="p">][</span><span class="s1">'Ticker'</span><span class="p">]):</span> <span class="n">t</span>
|
||||
<span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="n">r3000_equities</span><span class="o">.</span><span class="n">index</span><span class="p">}</span>
|
||||
<span class="n">pbar</span> <span class="o">=</span> <span class="n">progressbar</span><span class="o">.</span><span class="n">ProgressBar</span><span class="p">(</span><span class="n">term_width</span><span class="o">=</span><span class="mi">80</span><span class="p">,</span>
|
||||
<span class="n">max_value</span><span class="o">=</span><span class="n">r3000_equities</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">max</span><span class="p">())</span>
|
||||
@ -5250,7 +5250,7 @@ AAAAAGBWNBAAAAAAAMCsaCAAAAAAAIBZ0UAAAAAAAACz+n9WbX+0Yscg8AAAAABJRU5ErkJggg==
|
||||
<span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="o">.</span><span class="n">as_completed</span><span class="p">(</span><span class="n">fs</span><span class="p">):</span>
|
||||
<span class="n">i</span> <span class="o">=</span> <span class="n">fs</span><span class="p">[</span><span class="n">future</span><span class="p">]</span>
|
||||
<span class="n">pbar</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">i</span><span class="p">)</span>
|
||||
<span class="n">dates_file</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">yaml</span><span class="o">.</span><span class="n">dump</span><span class="p">({</span><span class="n">r3000_equities</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s">'Ticker'</span><span class="p">]:</span>
|
||||
<span class="n">dates_file</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">yaml</span><span class="o">.</span><span class="n">dump</span><span class="p">({</span><span class="n">r3000_equities</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s1">'Ticker'</span><span class="p">]:</span>
|
||||
<span class="n">future</span><span class="o">.</span><span class="n">result</span><span class="p">()}))</span>
|
||||
</pre></div>
|
||||
<p>Downloading stock price data needed for the event studies:</p>
|
||||
@ -5263,31 +5263,31 @@ AAAAAGBWNBAAAAAAAMCsaCAAAAAAAIBZ0UAAAAAAAACz+n9WbX+0Yscg8AAAAABJRU5ErkJggg==
|
||||
<span class="kn">from</span> <span class="nn">progressbar</span> <span class="kn">import</span> <span class="n">ProgressBar</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">fetch_ticker</span><span class="p">(</span><span class="n">ticker</span><span class="p">,</span> <span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">):</span>
|
||||
<span class="c"># Quandl is currently giving me issues with returning</span>
|
||||
<span class="c"># the entire dataset and not slicing server-side.</span>
|
||||
<span class="c"># So instead, we'll do it client-side!</span>
|
||||
<span class="n">q_format</span> <span class="o">=</span> <span class="s">'%Y-%m-</span><span class="si">%d</span><span class="s">'</span>
|
||||
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'YAHOO/'</span> <span class="o">+</span> <span class="n">ticker</span><span class="p">,</span>
|
||||
<span class="c1"># Quandl is currently giving me issues with returning</span>
|
||||
<span class="c1"># the entire dataset and not slicing server-side.</span>
|
||||
<span class="c1"># So instead, we'll do it client-side!</span>
|
||||
<span class="n">q_format</span> <span class="o">=</span> <span class="s1">'%Y-%m-</span><span class="si">%d</span><span class="s1">'</span>
|
||||
<span class="n">ticker_data</span> <span class="o">=</span> <span class="n">quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'YAHOO/'</span> <span class="o">+</span> <span class="n">ticker</span><span class="p">,</span>
|
||||
<span class="n">start_date</span><span class="o">=</span><span class="n">start</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="n">q_format</span><span class="p">),</span>
|
||||
<span class="n">end_date</span><span class="o">=</span><span class="n">end</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="n">q_format</span><span class="p">),</span>
|
||||
<span class="n">authtoken</span><span class="o">=</span><span class="n">QUANDL_KEY</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">ticker_data</span>
|
||||
|
||||
<span class="n">data_str</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s">'earnings_dates.yaml'</span><span class="p">,</span> <span class="s">'r'</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
|
||||
<span class="c"># Need to remove invalid lines</span>
|
||||
<span class="n">filtered</span> <span class="o">=</span> <span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="s">'{'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">x</span><span class="p">,</span> <span class="n">data_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s">'</span><span class="se">\n</span><span class="s">'</span><span class="p">))</span>
|
||||
<span class="n">earnings_data</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s">'</span><span class="se">\n</span><span class="s">'</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">filtered</span><span class="p">))</span>
|
||||
<span class="n">data_str</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s1">'earnings_dates.yaml'</span><span class="p">,</span> <span class="s1">'r'</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
|
||||
<span class="c1"># Need to remove invalid lines</span>
|
||||
<span class="n">filtered</span> <span class="o">=</span> <span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="s1">'{'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">x</span><span class="p">,</span> <span class="n">data_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">'</span><span class="se">\n</span><span class="s1">'</span><span class="p">))</span>
|
||||
<span class="n">earnings_data</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s1">'</span><span class="se">\n</span><span class="s1">'</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">filtered</span><span class="p">))</span>
|
||||
|
||||
<span class="c"># Get the first 1500 keys - split up into two statements</span>
|
||||
<span class="c"># because of Quandl rate limits</span>
|
||||
<span class="c1"># Get the first 1500 keys - split up into two statements</span>
|
||||
<span class="c1"># because of Quandl rate limits</span>
|
||||
<span class="n">tickers</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">earnings_data</span><span class="o">.</span><span class="n">keys</span><span class="p">())</span>
|
||||
|
||||
<span class="n">price_dict</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
<span class="n">invalid_tickers</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
<span class="k">for</span> <span class="n">ticker</span> <span class="ow">in</span> <span class="n">ProgressBar</span><span class="p">()(</span><span class="n">tickers</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">1500</span><span class="p">]):</span>
|
||||
<span class="k">try</span><span class="p">:</span>
|
||||
<span class="c"># Replace '.' with '-' in name for some tickers</span>
|
||||
<span class="n">fixed</span> <span class="o">=</span> <span class="n">ticker</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s">'.'</span><span class="p">,</span> <span class="s">'-'</span><span class="p">)</span>
|
||||
<span class="c1"># Replace '.' with '-' in name for some tickers</span>
|
||||
<span class="n">fixed</span> <span class="o">=</span> <span class="n">ticker</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">'.'</span><span class="p">,</span> <span class="s1">'-'</span><span class="p">)</span>
|
||||
<span class="n">event_strs</span> <span class="o">=</span> <span class="n">earnings_data</span><span class="p">[</span><span class="n">ticker</span><span class="p">]</span>
|
||||
<span class="n">events</span> <span class="o">=</span> <span class="p">[</span><span class="n">parse</span><span class="p">(</span><span class="n">event</span><span class="p">)</span> <span class="k">for</span> <span class="n">event</span> <span class="ow">in</span> <span class="n">event_strs</span><span class="p">]</span>
|
||||
<span class="n">td</span> <span class="o">=</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">20</span><span class="p">)</span>
|
||||
@ -5296,11 +5296,11 @@ AAAAAGBWNBAAAAAAAMCsaCAAAAAAAIBZ0UAAAAAAAACz+n9WbX+0Yscg8AAAAABJRU5ErkJggg==
|
||||
<span class="k">except</span> <span class="n">quandl</span><span class="o">.</span><span class="n">NotFoundError</span><span class="p">:</span>
|
||||
<span class="n">invalid_tickers</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">ticker</span><span class="p">)</span>
|
||||
|
||||
<span class="c"># Execute this after 10 minutes have passed</span>
|
||||
<span class="c1"># Execute this after 10 minutes have passed</span>
|
||||
<span class="k">for</span> <span class="n">ticker</span> <span class="ow">in</span> <span class="n">ProgressBar</span><span class="p">()(</span><span class="n">tickers</span><span class="p">[</span><span class="mi">1500</span><span class="p">:]):</span>
|
||||
<span class="k">try</span><span class="p">:</span>
|
||||
<span class="c"># Replace '.' with '-' in name for some tickers</span>
|
||||
<span class="n">fixed</span> <span class="o">=</span> <span class="n">ticker</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s">'.'</span><span class="p">,</span> <span class="s">'-'</span><span class="p">)</span>
|
||||
<span class="c1"># Replace '.' with '-' in name for some tickers</span>
|
||||
<span class="n">fixed</span> <span class="o">=</span> <span class="n">ticker</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">'.'</span><span class="p">,</span> <span class="s1">'-'</span><span class="p">)</span>
|
||||
<span class="n">event_strs</span> <span class="o">=</span> <span class="n">earnings_data</span><span class="p">[</span><span class="n">ticker</span><span class="p">]</span>
|
||||
<span class="n">events</span> <span class="o">=</span> <span class="p">[</span><span class="n">parse</span><span class="p">(</span><span class="n">event</span><span class="p">)</span> <span class="k">for</span> <span class="n">event</span> <span class="ow">in</span> <span class="n">event_strs</span><span class="p">]</span>
|
||||
<span class="n">td</span> <span class="o">=</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">20</span><span class="p">)</span>
|
||||
@ -5309,7 +5309,7 @@ AAAAAGBWNBAAAAAAAMCsaCAAAAAAAIBZ0UAAAAAAAACz+n9WbX+0Yscg8AAAAABJRU5ErkJggg==
|
||||
<span class="k">except</span> <span class="n">quandl</span><span class="o">.</span><span class="n">NotFoundError</span><span class="p">:</span>
|
||||
<span class="n">invalid_tickers</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">ticker</span><span class="p">)</span>
|
||||
|
||||
<span class="n">prices_store</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">HDFStore</span><span class="p">(</span><span class="s">'price_data.hdf'</span><span class="p">)</span>
|
||||
<span class="n">prices_store</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">HDFStore</span><span class="p">(</span><span class="s1">'price_data.hdf'</span><span class="p">)</span>
|
||||
<span class="k">for</span> <span class="n">ticker</span><span class="p">,</span> <span class="n">prices</span> <span class="ow">in</span> <span class="n">price_dict</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
<span class="n">prices_store</span><span class="p">[</span><span class="n">ticker</span><span class="p">]</span> <span class="o">=</span> <span class="n">prices</span>
|
||||
</pre></div>
|
||||
|
||||
1283
feeds/all.atom.xml
1283
feeds/all.atom.xml
File diff suppressed because it is too large
Load Diff
1283
feeds/blog.atom.xml
1283
feeds/blog.atom.xml
File diff suppressed because it is too large
Load Diff
@ -83,6 +83,8 @@
|
||||
<div class="container content archive">
|
||||
<h2><a href="https://bspeice.github.io/index.html"> </a></h2>
|
||||
<dl class="dl-horizontal">
|
||||
<dt>Sat 22 October 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
|
||||
<dt>Wed 08 June 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/event-studies-and-earnings-releases.html">Event Studies and Earnings Releases</a></dd>
|
||||
<dt>Sun 15 May 2016</dt>
|
||||
@ -101,8 +103,6 @@
|
||||
<dd><a href="https://bspeice.github.io/cloudy-in-seattle.html">Cloudy in Seattle</a></dd>
|
||||
<dt>Fri 01 January 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/complaining-about-the-weather.html">Complaining about the Weather</a></dd>
|
||||
<dt>Sat 26 December 2015</dt>
|
||||
<dd><a href="https://bspeice.github.io/testing-cramer.html">Testing Cramer</a></dd>
|
||||
</dl>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@ -83,6 +83,8 @@
|
||||
<div class="container content archive">
|
||||
<h2><a href="https://bspeice.github.io/index2.html"> <small>- page 2</small></a></h2>
|
||||
<dl class="dl-horizontal">
|
||||
<dt>Sat 26 December 2015</dt>
|
||||
<dd><a href="https://bspeice.github.io/testing-cramer.html">Testing Cramer</a></dd>
|
||||
<dt>Fri 27 November 2015</dt>
|
||||
<dd><a href="https://bspeice.github.io/autocallable-bonds.html">Autocallable Bonds</a></dd>
|
||||
<dt>Thu 19 November 2015</dt>
|
||||
|
||||
@ -108,13 +108,13 @@
|
||||
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
|
||||
<span class="o">%</span><span class="k">matplotlib</span> inline
|
||||
|
||||
<span class="c"># Record how long it takes to run the notebook - I'm curious.</span>
|
||||
<span class="c1"># Record how long it takes to run the notebook - I'm curious.</span>
|
||||
<span class="kn">from</span> <span class="nn">datetime</span> <span class="k">import</span> <span class="n">datetime</span>
|
||||
<span class="n">start</span> <span class="o">=</span> <span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span>
|
||||
|
||||
<span class="n">dataset</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s">'split_train.csv'</span><span class="p">)</span>
|
||||
<span class="n">dataset</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s1">'split_train.csv'</span><span class="p">)</span>
|
||||
<span class="n">dataset</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">ID</span>
|
||||
<span class="n">X</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">drop</span><span class="p">([</span><span class="s">'TARGET'</span><span class="p">,</span> <span class="s">'ID'</span><span class="p">,</span> <span class="s">'ID.1'</span><span class="p">],</span> <span class="mi">1</span><span class="p">)</span>
|
||||
<span class="n">X</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">drop</span><span class="p">([</span><span class="s1">'TARGET'</span><span class="p">,</span> <span class="s1">'ID'</span><span class="p">,</span> <span class="s1">'ID.1'</span><span class="p">],</span> <span class="mi">1</span><span class="p">)</span>
|
||||
<span class="n">y</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">TARGET</span>
|
||||
</pre></div>
|
||||
|
||||
@ -244,13 +244,13 @@
|
||||
<div class="prompt input_prompt">In [5]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="c"># First we need to `binarize` the data to 0-1; some of the labels are {0, 1},</span>
|
||||
<span class="c"># some are {0, 3}, etc.</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="c1"># First we need to `binarize` the data to 0-1; some of the labels are {0, 1},</span>
|
||||
<span class="c1"># some are {0, 3}, etc.</span>
|
||||
<span class="kn">from</span> <span class="nn">sklearn.preprocessing</span> <span class="k">import</span> <span class="n">binarize</span>
|
||||
<span class="n">X_bin</span> <span class="o">=</span> <span class="n">binarize</span><span class="p">(</span><span class="n">X</span><span class="p">[</span><span class="n">b_class</span><span class="p">])</span>
|
||||
|
||||
<span class="n">accuracy</span> <span class="o">=</span> <span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">X_bin</span><span class="p">[:,</span><span class="n">i</span><span class="p">]</span> <span class="o">==</span> <span class="n">y</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">b_class</span><span class="p">))]</span>
|
||||
<span class="n">acc_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">({</span><span class="s">"Accuracy"</span><span class="p">:</span> <span class="n">accuracy</span><span class="p">},</span> <span class="n">index</span><span class="o">=</span><span class="n">b_class</span><span class="p">)</span>
|
||||
<span class="n">acc_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">({</span><span class="s2">"Accuracy"</span><span class="p">:</span> <span class="n">accuracy</span><span class="p">},</span> <span class="n">index</span><span class="o">=</span><span class="n">b_class</span><span class="p">)</span>
|
||||
<span class="n">acc_df</span><span class="o">.</span><span class="n">describe</span><span class="p">()</span>
|
||||
</pre></div>
|
||||
|
||||
@ -334,9 +334,9 @@
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">unsat</span> <span class="o">=</span> <span class="n">y</span><span class="p">[</span><span class="n">y</span> <span class="o">==</span> <span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">count</span><span class="p">()</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s">"Satisfied customers: {}; Unsatisfied customers: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">y</span><span class="p">)</span> <span class="o">-</span> <span class="n">unsat</span><span class="p">,</span> <span class="n">unsat</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Satisfied customers: {}; Unsatisfied customers: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">y</span><span class="p">)</span> <span class="o">-</span> <span class="n">unsat</span><span class="p">,</span> <span class="n">unsat</span><span class="p">))</span>
|
||||
<span class="n">naive_guess</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">y</span> <span class="o">==</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">y</span><span class="p">)))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s">"Naive guess accuracy: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Naive guess accuracy: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -397,9 +397,9 @@ Naive guess accuracy: 0.9608561656706882
|
||||
<span class="k">return</span> <span class="n">gnb</span><span class="o">.</span><span class="n">score</span><span class="p">(</span><span class="n">X_xform</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
|
||||
|
||||
<span class="n">dim_range</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">21</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">dim_range</span><span class="p">,</span> <span class="p">[</span><span class="n">evaluate_gnb</span><span class="p">(</span><span class="n">dim</span><span class="p">)</span> <span class="k">for</span> <span class="n">dim</span> <span class="ow">in</span> <span class="n">dim_range</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s">"Gaussian NB Accuracy"</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">"Naive Guess"</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s">'k'</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="mi">1</span> <span class="o">-</span> <span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">"Inverse Naive Guess"</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s">'k'</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">dim_range</span><span class="p">,</span> <span class="p">[</span><span class="n">evaluate_gnb</span><span class="p">(</span><span class="n">dim</span><span class="p">)</span> <span class="k">for</span> <span class="n">dim</span> <span class="ow">in</span> <span class="n">dim_range</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s2">"Gaussian NB Accuracy"</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">"Naive Guess"</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s1">'k'</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="mi">1</span> <span class="o">-</span> <span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">"Inverse Naive Guess"</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s1">'k'</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">();</span>
|
||||
</pre></div>
|
||||
@ -683,9 +683,9 @@ rkJggg==
|
||||
<span class="k">return</span> <span class="n">gnb</span><span class="o">.</span><span class="n">score</span><span class="p">(</span><span class="n">X_xform</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
|
||||
|
||||
<span class="n">dim_range</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">21</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">dim_range</span><span class="p">,</span> <span class="p">[</span><span class="n">evaluate_gnb</span><span class="p">(</span><span class="n">dim</span><span class="p">)</span> <span class="k">for</span> <span class="n">dim</span> <span class="ow">in</span> <span class="n">dim_range</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s">"Gaussian NB Accuracy"</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">"Naive Guess"</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s">'k'</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="mi">1</span> <span class="o">-</span> <span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">"Inverse Naive Guess"</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s">'k'</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">dim_range</span><span class="p">,</span> <span class="p">[</span><span class="n">evaluate_gnb</span><span class="p">(</span><span class="n">dim</span><span class="p">)</span> <span class="k">for</span> <span class="n">dim</span> <span class="ow">in</span> <span class="n">dim_range</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s2">"Gaussian NB Accuracy"</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">"Naive Guess"</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s1">'k'</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="mi">1</span> <span class="o">-</span> <span class="n">naive_guess</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">"Inverse Naive Guess"</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s1">'k'</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">();</span>
|
||||
</pre></div>
|
||||
@ -961,7 +961,7 @@ rkJggg==
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">end</span> <span class="o">=</span> <span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s">"Running time: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">end</span> <span class="o">-</span> <span class="n">start</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Running time: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">end</span> <span class="o">-</span> <span class="n">start</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -990,14 +990,14 @@ rkJggg==
|
||||
<div class="text_cell_render border-box-sizing rendered_html">
|
||||
<h1 id="Appendix">Appendix<a class="anchor-link" href="#Appendix">¶</a></h1><p>Code used to split the initial training data:</p>
|
||||
<div class="highlight"><pre><span class="kn">from</span> <span class="nn">sklearn.cross_validation</span> <span class="kn">import</span> <span class="n">train_test_split</span>
|
||||
<span class="n">data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s">'train.csv'</span><span class="p">)</span>
|
||||
<span class="n">data</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s1">'train.csv'</span><span class="p">)</span>
|
||||
<span class="n">data</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">ID</span>
|
||||
|
||||
<span class="n">data_train</span><span class="p">,</span> <span class="n">data_validate</span> <span class="o">=</span> <span class="n">train_test_split</span><span class="p">(</span>
|
||||
<span class="n">data</span><span class="p">,</span> <span class="n">train_size</span><span class="o">=.</span><span class="mi">7</span><span class="p">)</span>
|
||||
|
||||
<span class="n">data_train</span><span class="o">.</span><span class="n">to_csv</span><span class="p">(</span><span class="s">'split_train.csv'</span><span class="p">)</span>
|
||||
<span class="n">data_validate</span><span class="o">.</span><span class="n">to_csv</span><span class="p">(</span><span class="s">'split_validate.csv'</span><span class="p">)</span>
|
||||
<span class="n">data_train</span><span class="o">.</span><span class="n">to_csv</span><span class="p">(</span><span class="s1">'split_train.csv'</span><span class="p">)</span>
|
||||
<span class="n">data_validate</span><span class="o">.</span><span class="n">to_csv</span><span class="p">(</span><span class="s1">'split_validate.csv'</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
|
||||
@ -123,7 +123,7 @@
|
||||
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
|
||||
<span class="kn">from</span> <span class="nn">Quandl</span> <span class="k">import</span> <span class="n">get</span> <span class="k">as</span> <span class="n">qget</span>
|
||||
<span class="o">%</span><span class="k">matplotlib</span> inline
|
||||
<span class="n">api_key</span> <span class="o">=</span> <span class="s">''</span>
|
||||
<span class="n">api_key</span> <span class="o">=</span> <span class="s1">''</span>
|
||||
|
||||
<span class="n">profitability</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">p</span><span class="p">,</span> <span class="n">i</span><span class="p">,</span> <span class="n">m</span><span class="p">,</span> <span class="n">d</span><span class="p">:</span> <span class="mi">1000</span><span class="o">*</span><span class="n">p</span> <span class="o">/</span> <span class="p">(</span><span class="n">m</span> <span class="o">+</span> <span class="n">i</span><span class="o">*</span><span class="n">d</span><span class="p">)</span>
|
||||
|
||||
@ -147,15 +147,15 @@
|
||||
<span class="n">expected_close</span> <span class="o">=</span> <span class="n">open_vals</span><span class="p">[</span><span class="n">day</span><span class="p">]</span> <span class="o">*</span> <span class="n">expected</span>
|
||||
<span class="n">todays_purchase</span> <span class="o">=</span> <span class="n">daily_investment</span><span class="p">(</span><span class="n">open_vals</span><span class="p">[</span><span class="n">day</span><span class="p">],</span> <span class="n">expected_close</span><span class="p">,</span> <span class="n">invested</span><span class="p">,</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="n">day</span><span class="p">])</span>
|
||||
<span class="n">invested</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">invested</span><span class="p">,</span> <span class="n">todays_purchase</span> <span class="o">+</span> <span class="n">bias</span><span class="p">)</span>
|
||||
<span class="c"># expected_profit = expected_close * (invested / open_vals[0:len(invested)]).sum() - invested.sum()</span>
|
||||
<span class="c1"># expected_profit = expected_close * (invested / open_vals[0:len(invested)]).sum() - invested.sum()</span>
|
||||
<span class="n">day</span> <span class="o">+=</span> <span class="mi">1</span>
|
||||
<span class="n">profitable</span> <span class="o">=</span> <span class="n">is_profitable</span><span class="p">(</span><span class="n">close_vals</span><span class="p">[</span><span class="n">day</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span> <span class="n">invested</span><span class="p">,</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="nb">len</span><span class="p">(</span><span class="n">invested</span><span class="p">)])</span> \
|
||||
<span class="ow">or</span> <span class="n">is_profitable</span><span class="p">(</span><span class="n">open_vals</span><span class="p">[</span><span class="n">day</span><span class="p">],</span> <span class="n">invested</span><span class="p">,</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="nb">len</span><span class="p">(</span><span class="n">invested</span><span class="p">)])</span>
|
||||
|
||||
<span class="n">shares</span> <span class="o">=</span> <span class="p">(</span><span class="n">invested</span> <span class="o">/</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="nb">len</span><span class="p">(</span><span class="n">invested</span><span class="p">)])</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
|
||||
|
||||
<span class="c"># Make sure we can't see into the future - we know either today's close or tomorrow's open</span>
|
||||
<span class="c"># will be profitable, but we need to check which one.</span>
|
||||
<span class="c1"># Make sure we can't see into the future - we know either today's close or tomorrow's open</span>
|
||||
<span class="c1"># will be profitable, but we need to check which one.</span>
|
||||
<span class="k">if</span> <span class="n">is_profitable</span><span class="p">(</span><span class="n">close_vals</span><span class="p">[</span><span class="n">day</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span> <span class="n">invested</span><span class="p">,</span> <span class="n">open_vals</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="nb">len</span><span class="p">(</span><span class="n">invested</span><span class="p">)]):</span>
|
||||
<span class="n">ending_price</span> <span class="o">=</span> <span class="n">close_vals</span><span class="p">[</span><span class="n">day</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
@ -168,21 +168,21 @@
|
||||
<span class="n">ticker_info</span> <span class="o">=</span> <span class="n">qget</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start</span><span class="p">,</span> <span class="n">api_key</span><span class="o">=</span><span class="n">api_key</span><span class="p">)</span>
|
||||
<span class="n">evaluation_times</span> <span class="o">=</span> <span class="n">ticker_info</span><span class="p">[:</span><span class="n">end</span><span class="p">]</span><span class="o">.</span><span class="n">index</span>
|
||||
|
||||
<span class="c"># Handle Google vs. YFinance data</span>
|
||||
<span class="k">if</span> <span class="s">"Adjusted Close"</span> <span class="ow">in</span> <span class="n">ticker_info</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
<span class="n">close_column</span> <span class="o">=</span> <span class="s">"Adjusted Close"</span>
|
||||
<span class="c1"># Handle Google vs. YFinance data</span>
|
||||
<span class="k">if</span> <span class="s2">"Adjusted Close"</span> <span class="ow">in</span> <span class="n">ticker_info</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
<span class="n">close_column</span> <span class="o">=</span> <span class="s2">"Adjusted Close"</span>
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
<span class="n">close_column</span> <span class="o">=</span> <span class="s">"Close"</span>
|
||||
<span class="n">close_column</span> <span class="o">=</span> <span class="s2">"Close"</span>
|
||||
|
||||
<span class="n">sim</span> <span class="o">=</span> <span class="p">{</span><span class="n">d</span><span class="p">:</span> <span class="n">simulate_day</span><span class="p">(</span><span class="n">ticker_info</span><span class="p">[</span><span class="n">d</span><span class="p">:][</span><span class="s">"Open"</span><span class="p">],</span> <span class="n">ticker_info</span><span class="p">[</span><span class="n">d</span><span class="p">:][</span><span class="n">close_column</span><span class="p">],</span>
|
||||
<span class="n">sim</span> <span class="o">=</span> <span class="p">{</span><span class="n">d</span><span class="p">:</span> <span class="n">simulate_day</span><span class="p">(</span><span class="n">ticker_info</span><span class="p">[</span><span class="n">d</span><span class="p">:][</span><span class="s2">"Open"</span><span class="p">],</span> <span class="n">ticker_info</span><span class="p">[</span><span class="n">d</span><span class="p">:][</span><span class="n">close_column</span><span class="p">],</span>
|
||||
<span class="mi">100</span><span class="p">,</span> <span class="mf">1.02</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">evaluation_times</span><span class="p">}</span>
|
||||
<span class="n">sim_series</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">sim</span><span class="p">)</span>
|
||||
<span class="n">result</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">()</span>
|
||||
<span class="n">result</span><span class="p">[</span><span class="s">"profit"</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">])</span>
|
||||
<span class="n">result</span><span class="p">[</span><span class="s">"max"</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">max</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]))</span>
|
||||
<span class="n">result</span><span class="p">[</span><span class="s">"days"</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]))</span>
|
||||
<span class="n">result</span><span class="p">[</span><span class="s">"score"</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">profitability</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">0</span><span class="p">],</span> <span class="nb">max</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]),</span> <span class="nb">len</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">])))</span>
|
||||
<span class="n">result</span><span class="p">[</span><span class="s">"investments"</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
|
||||
<span class="n">result</span><span class="p">[</span><span class="s2">"profit"</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">])</span>
|
||||
<span class="n">result</span><span class="p">[</span><span class="s2">"max"</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">max</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]))</span>
|
||||
<span class="n">result</span><span class="p">[</span><span class="s2">"days"</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]))</span>
|
||||
<span class="n">result</span><span class="p">[</span><span class="s2">"score"</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">profitability</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="mi">0</span><span class="p">],</span> <span class="nb">max</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]),</span> <span class="nb">len</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">])))</span>
|
||||
<span class="n">result</span><span class="p">[</span><span class="s2">"investments"</span><span class="p">]</span> <span class="o">=</span> <span class="n">sim_series</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
|
||||
|
||||
<span class="k">return</span> <span class="n">result</span>
|
||||
|
||||
@ -217,7 +217,7 @@
|
||||
<div class="prompt input_prompt">In [7]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">fang_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s">"YAHOO/FB"</span><span class="p">,</span> <span class="s">"YAHOO/AAPL"</span><span class="p">,</span> <span class="s">"YAHOO/NFLX"</span><span class="p">,</span> <span class="s">"YAHOO/GOOG"</span><span class="p">])</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">fang_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s2">"YAHOO/FB"</span><span class="p">,</span> <span class="s2">"YAHOO/AAPL"</span><span class="p">,</span> <span class="s2">"YAHOO/NFLX"</span><span class="p">,</span> <span class="s2">"YAHOO/GOOG"</span><span class="p">])</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -230,9 +230,9 @@
|
||||
<div class="prompt input_prompt">In [8]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">fang_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">'days'</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">fang_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">'days'</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">8</span><span class="p">);</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">"Distribution of Days Until Profitability"</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">"Distribution of Days Until Profitability"</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -704,9 +704,9 @@ kiSpNScYJEmSJElSa04wSJIkSZKk1pxgkCRJkiRJrf1/uiw9D8O0d6EAAAAASUVORK5CYII=
|
||||
<div class="prompt input_prompt">In [10]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">fang_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">'score'</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">fang_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">'score'</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">"Profitability score over time"</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">"Profitability score over time"</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -2343,7 +2343,7 @@ EZIkSZIkSZIkSZIk6RIpIiRJkiRJkiRJkiRJ0iX+D91jLOlExGw8AAAAAElFTkSuQmCC
|
||||
<div class="prompt input_prompt">In [13]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">cyclic_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s">"YAHOO/X"</span><span class="p">,</span> <span class="s">"YAHOO/CAT"</span><span class="p">,</span> <span class="s">"YAHOO/NFLX"</span><span class="p">,</span> <span class="s">"YAHOO/GOOG"</span><span class="p">])</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">cyclic_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s2">"YAHOO/X"</span><span class="p">,</span> <span class="s2">"YAHOO/CAT"</span><span class="p">,</span> <span class="s2">"YAHOO/NFLX"</span><span class="p">,</span> <span class="s2">"YAHOO/GOOG"</span><span class="p">])</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -2356,9 +2356,9 @@ EZIkSZIkSZIkSZIk6RIpIiRJkiRJkiRJkiRJ0iX+D91jLOlExGw8AAAAAElFTkSuQmCC
|
||||
<div class="prompt input_prompt">In [14]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">cyclic_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">'days'</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">cyclic_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">'days'</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">8</span><span class="p">);</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">"Distribution of Days Until Profitability"</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">"Distribution of Days Until Profitability"</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -2844,9 +2844,9 @@ JEmSJEnSwGwwSJIkSZKkgdlgkCRJkiRJA7PBIEmSJEmSBvb/AfKAew4+oiojAAAAAElFTkSuQmCC
|
||||
<div class="prompt input_prompt">In [15]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">cyclic_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">'score'</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">cyclic_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">'score'</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">"Profitability score over time"</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">"Profitability score over time"</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -4629,7 +4629,7 @@ hUKhUCgUpor/A9AnO0PoIItTAAAAAElFTkSuQmCC
|
||||
<div class="prompt input_prompt">In [21]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">biotech_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s">'YAHOO/REGN'</span><span class="p">,</span> <span class="s">'YAHOO/CELG'</span><span class="p">,</span> <span class="s">'GOOG/NASDAQ_BIB'</span><span class="p">,</span> <span class="s">'GOOG/NASDAQ_IBB'</span><span class="p">])</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">biotech_df</span> <span class="o">=</span> <span class="n">simulate_tickers</span><span class="p">([</span><span class="s1">'YAHOO/REGN'</span><span class="p">,</span> <span class="s1">'YAHOO/CELG'</span><span class="p">,</span> <span class="s1">'GOOG/NASDAQ_BIB'</span><span class="p">,</span> <span class="s1">'GOOG/NASDAQ_IBB'</span><span class="p">])</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -4642,9 +4642,9 @@ hUKhUCgUpor/A9AnO0PoIItTAAAAAElFTkSuQmCC
|
||||
<div class="prompt input_prompt">In [22]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">biotech_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">'days'</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">biotech_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">'days'</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">hist</span><span class="p">()</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">8</span><span class="p">);</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">"Distribution of Days Until Profitability"</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">"Distribution of Days Until Profitability"</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -5195,9 +5195,9 @@ WKBTQVcCAAAAAElFTkSuQmCC
|
||||
<div class="prompt input_prompt">In [23]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">biotech_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s">'score'</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">biotech_df</span><span class="o">.</span><span class="n">xs</span><span class="p">(</span><span class="s1">'score'</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">()</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s">"Profitability score over time"</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">suptitle</span><span class="p">(</span><span class="s2">"Profitability score over time"</span><span class="p">,</span> <span class="n">fontsize</span><span class="o">=</span><span class="mi">18</span><span class="p">);</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
|
||||
123
tag/nutone.html
Normal file
123
tag/nutone.html
Normal file
@ -0,0 +1,123 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="description" content=" nutone">
|
||||
<meta name="keywords" content="">
|
||||
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
|
||||
|
||||
<title> nutone - Bradlee Speice</title>
|
||||
|
||||
<!-- Stylesheets -->
|
||||
<link href="https://bspeice.github.io/theme/css/bootstrap.min.css" rel="stylesheet">
|
||||
<link href="https://bspeice.github.io/theme/css/fonts.css" rel="stylesheet">
|
||||
<link href="https://bspeice.github.io/theme/css/nest.css" rel="stylesheet">
|
||||
<link href="https://bspeice.github.io/theme/css/pygment.css" rel="stylesheet">
|
||||
<!-- /Stylesheets -->
|
||||
|
||||
<!-- RSS Feeds -->
|
||||
<link href="https://bspeice.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate" title="Bradlee Speice Full Atom Feed" />
|
||||
<!-- /RSS Feeds -->
|
||||
|
||||
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
|
||||
<!--[if lt IE 9]>
|
||||
<script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
|
||||
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
|
||||
<![endif]-->
|
||||
|
||||
<!-- Google Analytics -->
|
||||
<script>
|
||||
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
|
||||
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
|
||||
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
|
||||
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
|
||||
|
||||
ga('create', 'UA-74711362-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script>
|
||||
<!-- /Google Analytics -->
|
||||
|
||||
|
||||
</head>
|
||||
|
||||
<body>
|
||||
|
||||
<!-- Header -->
|
||||
<div class="header-container gradient">
|
||||
|
||||
<!-- Static navbar -->
|
||||
<div class="container">
|
||||
<div class="header-nav">
|
||||
<div class="header-logo">
|
||||
<a class="pull-left" href="https://bspeice.github.io/"><img class="mr20" src="https://bspeice.github.io/images/logo.svg" alt="logo">Bradlee Speice</a>
|
||||
</div>
|
||||
<div class="nav pull-right">
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- /Static navbar -->
|
||||
|
||||
<!-- Header -->
|
||||
<div class="container header-wrapper">
|
||||
<div class="row">
|
||||
<div class="col-lg-12">
|
||||
<div class="header-content">
|
||||
<h1 class="header-title text-uppercase"> : #nutone</h1>
|
||||
<div class="header-underline"></div>
|
||||
<p class="header-subtitle header-subtitle-homepage"> #nutone</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- /Header -->
|
||||
|
||||
</div>
|
||||
<!-- /Header -->
|
||||
|
||||
|
||||
<!-- Content -->
|
||||
<div class="archive-container">
|
||||
<div class="container content archive">
|
||||
<h2><a href="https://bspeice.github.io/tag/nutone.html">nutone</a></h2>
|
||||
<dl class="dl-horizontal">
|
||||
<dt>Sat 22 October 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
|
||||
</dl>
|
||||
</div>
|
||||
</div>
|
||||
<!-- /Content -->
|
||||
|
||||
<!-- Footer -->
|
||||
<div class="footer gradient-2">
|
||||
<div class="container footer-container ">
|
||||
<div class="row">
|
||||
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
|
||||
<div class="footer-title"></div>
|
||||
<ul class="list-unstyled">
|
||||
<li><a href="https://bspeice.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate"></a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
|
||||
<div class="footer-title"></div>
|
||||
<ul class="list-unstyled">
|
||||
<li><a href="https://github.com/bspeice" target="_blank">Github</a></li>
|
||||
<li><a href="https://www.linkedin.com/in/bradleespeice" target="_blank">LinkedIn</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
|
||||
</div>
|
||||
<div class="col-xs-12 col-sm-3 col-md-3 col-lg-3">
|
||||
<p class="pull-right text-right">
|
||||
<small><em>Proudly powered by <a href="http://docs.getpelican.com/" target="_blank">pelican</a></em></small><br/>
|
||||
<small><em>Theme and code by <a href="https://github.com/molivier" target="_blank">molivier</a></em></small><br/>
|
||||
<small></small>
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- /Footer -->
|
||||
</body>
|
||||
</html>
|
||||
123
tag/rust.html
Normal file
123
tag/rust.html
Normal file
@ -0,0 +1,123 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="description" content=" Rust">
|
||||
<meta name="keywords" content="">
|
||||
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
|
||||
|
||||
<title> Rust - Bradlee Speice</title>
|
||||
|
||||
<!-- Stylesheets -->
|
||||
<link href="https://bspeice.github.io/theme/css/bootstrap.min.css" rel="stylesheet">
|
||||
<link href="https://bspeice.github.io/theme/css/fonts.css" rel="stylesheet">
|
||||
<link href="https://bspeice.github.io/theme/css/nest.css" rel="stylesheet">
|
||||
<link href="https://bspeice.github.io/theme/css/pygment.css" rel="stylesheet">
|
||||
<!-- /Stylesheets -->
|
||||
|
||||
<!-- RSS Feeds -->
|
||||
<link href="https://bspeice.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate" title="Bradlee Speice Full Atom Feed" />
|
||||
<!-- /RSS Feeds -->
|
||||
|
||||
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
|
||||
<!--[if lt IE 9]>
|
||||
<script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
|
||||
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
|
||||
<![endif]-->
|
||||
|
||||
<!-- Google Analytics -->
|
||||
<script>
|
||||
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
|
||||
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
|
||||
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
|
||||
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
|
||||
|
||||
ga('create', 'UA-74711362-1', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script>
|
||||
<!-- /Google Analytics -->
|
||||
|
||||
|
||||
</head>
|
||||
|
||||
<body>
|
||||
|
||||
<!-- Header -->
|
||||
<div class="header-container gradient">
|
||||
|
||||
<!-- Static navbar -->
|
||||
<div class="container">
|
||||
<div class="header-nav">
|
||||
<div class="header-logo">
|
||||
<a class="pull-left" href="https://bspeice.github.io/"><img class="mr20" src="https://bspeice.github.io/images/logo.svg" alt="logo">Bradlee Speice</a>
|
||||
</div>
|
||||
<div class="nav pull-right">
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- /Static navbar -->
|
||||
|
||||
<!-- Header -->
|
||||
<div class="container header-wrapper">
|
||||
<div class="row">
|
||||
<div class="col-lg-12">
|
||||
<div class="header-content">
|
||||
<h1 class="header-title text-uppercase"> : #Rust</h1>
|
||||
<div class="header-underline"></div>
|
||||
<p class="header-subtitle header-subtitle-homepage"> #Rust</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- /Header -->
|
||||
|
||||
</div>
|
||||
<!-- /Header -->
|
||||
|
||||
|
||||
<!-- Content -->
|
||||
<div class="archive-container">
|
||||
<div class="container content archive">
|
||||
<h2><a href="https://bspeice.github.io/tag/rust.html">Rust</a></h2>
|
||||
<dl class="dl-horizontal">
|
||||
<dt>Sat 22 October 2016</dt>
|
||||
<dd><a href="https://bspeice.github.io/a-rustic-re-podcasting-server-part-1.html">A Rustic Re-Podcasting Server (Part 1)</a></dd>
|
||||
</dl>
|
||||
</div>
|
||||
</div>
|
||||
<!-- /Content -->
|
||||
|
||||
<!-- Footer -->
|
||||
<div class="footer gradient-2">
|
||||
<div class="container footer-container ">
|
||||
<div class="row">
|
||||
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
|
||||
<div class="footer-title"></div>
|
||||
<ul class="list-unstyled">
|
||||
<li><a href="https://bspeice.github.io/feeds/all.atom.xml" type="application/atom+xml" rel="alternate"></a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
|
||||
<div class="footer-title"></div>
|
||||
<ul class="list-unstyled">
|
||||
<li><a href="https://github.com/bspeice" target="_blank">Github</a></li>
|
||||
<li><a href="https://www.linkedin.com/in/bradleespeice" target="_blank">LinkedIn</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="col-xs-4 col-sm-3 col-md-3 col-lg-3">
|
||||
</div>
|
||||
<div class="col-xs-12 col-sm-3 col-md-3 col-lg-3">
|
||||
<p class="pull-right text-right">
|
||||
<small><em>Proudly powered by <a href="http://docs.getpelican.com/" target="_blank">pelican</a></em></small><br/>
|
||||
<small><em>Theme and code by <a href="https://github.com/molivier" target="_blank">molivier</a></em></small><br/>
|
||||
<small></small>
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- /Footer -->
|
||||
</body>
|
||||
</html>
|
||||
@ -115,10 +115,14 @@
|
||||
<dt><span class="label label-default">1</span> article </dt>
|
||||
<dd><a href="https://bspeice.github.io/tag/monte-carlo.html">monte carlo</a></dd>
|
||||
<dt><span class="label label-default">1</span> article </dt>
|
||||
<dd><a href="https://bspeice.github.io/tag/nutone.html">nutone</a></dd>
|
||||
<dt><span class="label label-default">1</span> article </dt>
|
||||
<dd><a href="https://bspeice.github.io/tag/probability.html">probability</a></dd>
|
||||
<dt><span class="label label-default">1</span> article </dt>
|
||||
<dd><a href="https://bspeice.github.io/tag/python.html">python</a></dd>
|
||||
<dt><span class="label label-default">1</span> article </dt>
|
||||
<dd><a href="https://bspeice.github.io/tag/rust.html">Rust</a></dd>
|
||||
<dt><span class="label label-default">1</span> article </dt>
|
||||
<dd><a href="https://bspeice.github.io/tag/simulated-annealing.html">simulated annealing</a></dd>
|
||||
<dt><span class="label label-default">1</span> article </dt>
|
||||
<dd><a href="https://bspeice.github.io/tag/simulation.html">simulation</a></dd>
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="description" content="In [1]: import requests import pandas as pd import numpy as np from dateutil import parser as dtparser from dateutil.relativedelta import relativedelta from datetime import datetime from html.parser ...">
|
||||
<meta name="description" content="In [1]: import requests import pandas as pd import numpy as np from dateutil import parser as dtparser from dateutil.relativedelta import relativedelta from datetime import datetime from ...">
|
||||
<meta name="keywords" content="data science, futures">
|
||||
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
|
||||
|
||||
@ -133,20 +133,20 @@
|
||||
<span class="n">articles</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">handle_starttag</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">,</span> <span class="n">attrs</span><span class="p">):</span>
|
||||
<span class="c">#if tag == 'div' and ("id", "author_articles_wrapper") in attrs:</span>
|
||||
<span class="c"># self.fetch_links = True</span>
|
||||
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s">'a'</span> <span class="ow">and</span> <span class="p">(</span><span class="s">'class'</span><span class="p">,</span> <span class="s">'dashboard_article_link'</span><span class="p">)</span> <span class="ow">in</span> <span class="n">attrs</span><span class="p">:</span>
|
||||
<span class="n">href</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s">'href'</span><span class="p">,</span> <span class="n">attrs</span><span class="p">))[</span><span class="mi">0</span><span class="p">][</span><span class="mi">1</span><span class="p">]</span>
|
||||
<span class="c1">#if tag == 'div' and ("id", "author_articles_wrapper") in attrs:</span>
|
||||
<span class="c1"># self.fetch_links = True</span>
|
||||
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s1">'a'</span> <span class="ow">and</span> <span class="p">(</span><span class="s1">'class'</span><span class="p">,</span> <span class="s1">'dashboard_article_link'</span><span class="p">)</span> <span class="ow">in</span> <span class="n">attrs</span><span class="p">:</span>
|
||||
<span class="n">href</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s1">'href'</span><span class="p">,</span> <span class="n">attrs</span><span class="p">))[</span><span class="mi">0</span><span class="p">][</span><span class="mi">1</span><span class="p">]</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">articles</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">href</span><span class="p">)</span>
|
||||
|
||||
<span class="n">base_url</span> <span class="o">=</span> <span class="s">"http://seekingalpha.com/author/wall-street-breakfast/articles"</span>
|
||||
<span class="n">article_page_urls</span> <span class="o">=</span> <span class="p">[</span><span class="n">base_url</span><span class="p">]</span> <span class="o">+</span> <span class="p">[</span><span class="n">base_url</span> <span class="o">+</span> <span class="s">'/{}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">20</span><span class="p">)]</span>
|
||||
<span class="n">base_url</span> <span class="o">=</span> <span class="s2">"http://seekingalpha.com/author/wall-street-breakfast/articles"</span>
|
||||
<span class="n">article_page_urls</span> <span class="o">=</span> <span class="p">[</span><span class="n">base_url</span><span class="p">]</span> <span class="o">+</span> <span class="p">[</span><span class="n">base_url</span> <span class="o">+</span> <span class="s1">'/{}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">20</span><span class="p">)]</span>
|
||||
|
||||
<span class="n">global_articles</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
<span class="k">for</span> <span class="n">page</span> <span class="ow">in</span> <span class="n">article_page_urls</span><span class="p">:</span>
|
||||
<span class="c"># We need to switch the user agent, as SA blocks the standard requests agent</span>
|
||||
<span class="c1"># We need to switch the user agent, as SA blocks the standard requests agent</span>
|
||||
<span class="n">articles_html</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">page</span><span class="p">,</span>
|
||||
<span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="s">"User-Agent"</span><span class="p">:</span> <span class="s">"Wget/1.13.4"</span><span class="p">})</span>
|
||||
<span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="s2">"User-Agent"</span><span class="p">:</span> <span class="s2">"Wget/1.13.4"</span><span class="p">})</span>
|
||||
<span class="n">parser</span> <span class="o">=</span> <span class="n">ArticleListParser</span><span class="p">()</span>
|
||||
<span class="n">parser</span><span class="o">.</span><span class="n">feed</span><span class="p">(</span><span class="n">articles_html</span><span class="o">.</span><span class="n">text</span><span class="p">)</span>
|
||||
<span class="n">global_articles</span> <span class="o">+=</span> <span class="p">(</span><span class="n">parser</span><span class="o">.</span><span class="n">articles</span><span class="p">)</span>
|
||||
@ -163,24 +163,24 @@
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="k">class</span> <span class="nc">ArticleReturnParser</span><span class="p">(</span><span class="n">HTMLParser</span><span class="p">):</span>
|
||||
<span class="s">"Given an article, parse out the futures returns in it"</span>
|
||||
<span class="s2">"Given an article, parse out the futures returns in it"</span>
|
||||
|
||||
<span class="n">record_font_tags</span> <span class="o">=</span> <span class="k">False</span>
|
||||
<span class="n">in_font_tag</span> <span class="o">=</span> <span class="k">False</span>
|
||||
<span class="n">record_font_tags</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
<span class="n">in_font_tag</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
<span class="n">counter</span> <span class="o">=</span> <span class="mi">0</span>
|
||||
<span class="c"># data = {} # See __init__</span>
|
||||
<span class="c1"># data = {} # See __init__</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
||||
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">__init__</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">data</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">handle_starttag</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">,</span> <span class="n">attrs</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s">'span'</span> <span class="ow">and</span> <span class="p">(</span><span class="s">'itemprop'</span><span class="p">,</span> <span class="s">'datePublished'</span><span class="p">)</span> <span class="ow">in</span> <span class="n">attrs</span><span class="p">:</span>
|
||||
<span class="n">date_string</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s">'content'</span><span class="p">,</span> <span class="n">attrs</span><span class="p">))[</span><span class="mi">0</span><span class="p">][</span><span class="mi">1</span><span class="p">]</span>
|
||||
<span class="k">if</span> <span class="n">tag</span> <span class="o">==</span> <span class="s1">'span'</span> <span class="ow">and</span> <span class="p">(</span><span class="s1">'itemprop'</span><span class="p">,</span> <span class="s1">'datePublished'</span><span class="p">)</span> <span class="ow">in</span> <span class="n">attrs</span><span class="p">:</span>
|
||||
<span class="n">date_string</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s1">'content'</span><span class="p">,</span> <span class="n">attrs</span><span class="p">))[</span><span class="mi">0</span><span class="p">][</span><span class="mi">1</span><span class="p">]</span>
|
||||
<span class="n">date</span> <span class="o">=</span> <span class="n">dtparser</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="n">date_string</span><span class="p">)</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">'date'</span><span class="p">]</span> <span class="o">=</span> <span class="n">date</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">'date'</span><span class="p">]</span> <span class="o">=</span> <span class="n">date</span>
|
||||
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">in_font_tag</span> <span class="o">=</span> <span class="n">tag</span> <span class="o">==</span> <span class="s">'font'</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">in_font_tag</span> <span class="o">=</span> <span class="n">tag</span> <span class="o">==</span> <span class="s1">'font'</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">safe_float</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">string</span><span class="p">):</span>
|
||||
<span class="k">try</span><span class="p">:</span>
|
||||
@ -189,43 +189,43 @@
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">NaN</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">handle_data</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">content</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">record_font_tags</span> <span class="ow">and</span> <span class="s">"Futures at 6"</span> <span class="ow">in</span> <span class="n">content</span><span class="p">:</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">record_font_tags</span> <span class="o">=</span> <span class="k">True</span>
|
||||
<span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">record_font_tags</span> <span class="ow">and</span> <span class="s2">"Futures at 6"</span> <span class="ow">in</span> <span class="n">content</span><span class="p">:</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">record_font_tags</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
|
||||
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">record_font_tags</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">in_font_tag</span><span class="p">:</span>
|
||||
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">'DOW'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">'DOW'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
|
||||
<span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">'S&P'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">'S&P'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
|
||||
<span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">2</span><span class="p">:</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">'NASDAQ'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">'NASDAQ'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
|
||||
<span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">3</span><span class="p">:</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">'Crude'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">'Crude'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
|
||||
<span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">==</span> <span class="mi">4</span><span class="p">:</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">'Gold'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s1">'Gold'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">safe_float</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
|
||||
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">+=</span> <span class="mi">1</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">handle_endtag</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">):</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">in_font_tag</span> <span class="o">=</span> <span class="k">False</span>
|
||||
<span class="bp">self</span><span class="o">.</span><span class="n">in_font_tag</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">retrieve_data</span><span class="p">(</span><span class="n">url</span><span class="p">):</span>
|
||||
<span class="n">sa</span> <span class="o">=</span> <span class="s">"http://seekingalpha.com"</span>
|
||||
<span class="n">sa</span> <span class="o">=</span> <span class="s2">"http://seekingalpha.com"</span>
|
||||
<span class="n">article_html</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">sa</span> <span class="o">+</span> <span class="n">url</span><span class="p">,</span>
|
||||
<span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="s">"User-Agent"</span><span class="p">:</span> <span class="s">"Wget/1.13.4"</span><span class="p">})</span>
|
||||
<span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="s2">"User-Agent"</span><span class="p">:</span> <span class="s2">"Wget/1.13.4"</span><span class="p">})</span>
|
||||
<span class="n">parser</span> <span class="o">=</span> <span class="n">ArticleReturnParser</span><span class="p">()</span>
|
||||
<span class="n">parser</span><span class="o">.</span><span class="n">feed</span><span class="p">(</span><span class="n">article_html</span><span class="o">.</span><span class="n">text</span><span class="p">)</span>
|
||||
<span class="n">parser</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">update</span><span class="p">({</span><span class="s">"url"</span><span class="p">:</span> <span class="n">url</span><span class="p">})</span>
|
||||
<span class="n">parser</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">update</span><span class="p">({</span><span class="s">"text"</span><span class="p">:</span> <span class="n">article_html</span><span class="o">.</span><span class="n">text</span><span class="p">})</span>
|
||||
<span class="n">parser</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">update</span><span class="p">({</span><span class="s2">"url"</span><span class="p">:</span> <span class="n">url</span><span class="p">})</span>
|
||||
<span class="n">parser</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">update</span><span class="p">({</span><span class="s2">"text"</span><span class="p">:</span> <span class="n">article_html</span><span class="o">.</span><span class="n">text</span><span class="p">})</span>
|
||||
<span class="k">return</span> <span class="n">parser</span><span class="o">.</span><span class="n">data</span>
|
||||
|
||||
<span class="c"># This copy **MUST** be in place. I'm not sure why,</span>
|
||||
<span class="c"># as you'd think that the data being returned would already</span>
|
||||
<span class="c"># represent a different memory location. Even so, it blows up</span>
|
||||
<span class="c"># if you don't do this.</span>
|
||||
<span class="c1"># This copy **MUST** be in place. I'm not sure why,</span>
|
||||
<span class="c1"># as you'd think that the data being returned would already</span>
|
||||
<span class="c1"># represent a different memory location. Even so, it blows up</span>
|
||||
<span class="c1"># if you don't do this.</span>
|
||||
<span class="n">article_list</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">global_articles</span><span class="p">))</span>
|
||||
<span class="n">article_data</span> <span class="o">=</span> <span class="p">[</span><span class="n">copy</span><span class="p">(</span><span class="n">retrieve_data</span><span class="p">(</span><span class="n">url</span><span class="p">))</span> <span class="k">for</span> <span class="n">url</span> <span class="ow">in</span> <span class="n">article_list</span><span class="p">]</span>
|
||||
<span class="c"># If there's an issue downloading the article, drop it.</span>
|
||||
<span class="c1"># If there's an issue downloading the article, drop it.</span>
|
||||
<span class="n">article_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">article_data</span><span class="p">)</span><span class="o">.</span><span class="n">dropna</span><span class="p">()</span>
|
||||
</pre></div>
|
||||
|
||||
@ -249,12 +249,12 @@
|
||||
<div class="prompt input_prompt">In [4]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="c"># article_df is sorted by date, so we get the first row.</span>
|
||||
<span class="n">start_date</span> <span class="o">=</span> <span class="n">article_df</span><span class="o">.</span><span class="n">sort_values</span><span class="p">(</span><span class="n">by</span><span class="o">=</span><span class="s">'date'</span><span class="p">)</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="s">'date'</span><span class="p">]</span> <span class="o">-</span> <span class="n">relativedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
|
||||
<span class="n">SPY</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"GOOG/NYSE_SPY"</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
|
||||
<span class="n">DJIA</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"GOOG/AMS_DIA"</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
|
||||
<span class="n">RUSS</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"GOOG/AMEX_IWM"</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
|
||||
<span class="n">NASDAQ</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"GOOG/EPA_QQQ"</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="c1"># article_df is sorted by date, so we get the first row.</span>
|
||||
<span class="n">start_date</span> <span class="o">=</span> <span class="n">article_df</span><span class="o">.</span><span class="n">sort_values</span><span class="p">(</span><span class="n">by</span><span class="o">=</span><span class="s1">'date'</span><span class="p">)</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="s1">'date'</span><span class="p">]</span> <span class="o">-</span> <span class="n">relativedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
|
||||
<span class="n">SPY</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"GOOG/NYSE_SPY"</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
|
||||
<span class="n">DJIA</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"GOOG/AMS_DIA"</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
|
||||
<span class="n">RUSS</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"GOOG/AMEX_IWM"</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
|
||||
<span class="n">NASDAQ</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"GOOG/EPA_QQQ"</span><span class="p">,</span> <span class="n">trim_start</span><span class="o">=</span><span class="n">start_date</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -279,13 +279,13 @@
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">calculate_opening_ret</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
|
||||
<span class="c"># I'm not a huge fan of the appending for loop,</span>
|
||||
<span class="c"># but it's a bit verbose for a comprehension</span>
|
||||
<span class="c1"># I'm not a huge fan of the appending for loop,</span>
|
||||
<span class="c1"># but it's a bit verbose for a comprehension</span>
|
||||
<span class="n">data</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">frame</span><span class="p">)):</span>
|
||||
<span class="n">date</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||||
<span class="n">prior_close</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="o">-</span><span class="mi">1</span><span class="p">][</span><span class="s">'Close'</span><span class="p">]</span>
|
||||
<span class="n">open_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s">'Open'</span><span class="p">]</span>
|
||||
<span class="n">prior_close</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="o">-</span><span class="mi">1</span><span class="p">][</span><span class="s1">'Close'</span><span class="p">]</span>
|
||||
<span class="n">open_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s1">'Open'</span><span class="p">]</span>
|
||||
<span class="n">data</span><span class="p">[</span><span class="n">date</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">open_val</span> <span class="o">-</span> <span class="n">prior_close</span><span class="p">)</span> <span class="o">/</span> <span class="n">prior_close</span>
|
||||
|
||||
<span class="k">return</span> <span class="n">data</span>
|
||||
@ -296,8 +296,8 @@
|
||||
<span class="n">NASDAQ_open_ret</span> <span class="o">=</span> <span class="n">calculate_opening_ret</span><span class="p">(</span><span class="n">NASDAQ</span><span class="p">)</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">signs_match</span><span class="p">(</span><span class="n">list_1</span><span class="p">,</span> <span class="n">list_2</span><span class="p">):</span>
|
||||
<span class="c"># This is a surprisingly difficult task - we have to match</span>
|
||||
<span class="c"># up the dates in order to check if opening returns actually match</span>
|
||||
<span class="c1"># This is a surprisingly difficult task - we have to match</span>
|
||||
<span class="c1"># up the dates in order to check if opening returns actually match</span>
|
||||
<span class="n">index_dict_dt</span> <span class="o">=</span> <span class="p">{</span><span class="n">key</span><span class="o">.</span><span class="n">to_datetime</span><span class="p">():</span> <span class="n">list_2</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">list_2</span><span class="o">.</span><span class="n">keys</span><span class="p">()}</span>
|
||||
|
||||
<span class="n">matches</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
@ -313,26 +313,26 @@
|
||||
<span class="n">matches</span> <span class="o">+=</span> <span class="p">[</span><span class="mi">1</span><span class="p">]</span>
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
<span class="n">matches</span> <span class="o">+=</span> <span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
||||
<span class="c">#print("{}".format(list_2[index_dt]))</span>
|
||||
<span class="c1">#print("{}".format(list_2[index_dt]))</span>
|
||||
<span class="k">return</span> <span class="n">matches</span>
|
||||
|
||||
|
||||
<span class="n">prediction_dict</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
<span class="n">matches_dict</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
<span class="n">count_dict</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
<span class="n">index_dict</span> <span class="o">=</span> <span class="p">{</span><span class="s">"SPY"</span><span class="p">:</span> <span class="n">SPY_open_ret</span><span class="p">,</span> <span class="s">"DJIA"</span><span class="p">:</span> <span class="n">DJIA_open_ret</span><span class="p">,</span> <span class="s">"RUSS"</span><span class="p">:</span> <span class="n">RUSS_open_ret</span><span class="p">,</span> <span class="s">"NASDAQ"</span><span class="p">:</span> <span class="n">NASDAQ_open_ret</span><span class="p">}</span>
|
||||
<span class="n">indices</span> <span class="o">=</span> <span class="p">[</span><span class="s">"SPY"</span><span class="p">,</span> <span class="s">"DJIA"</span><span class="p">,</span> <span class="s">"RUSS"</span><span class="p">,</span> <span class="s">"NASDAQ"</span><span class="p">]</span>
|
||||
<span class="n">futures</span> <span class="o">=</span> <span class="p">[</span><span class="s">"Crude"</span><span class="p">,</span> <span class="s">"Gold"</span><span class="p">,</span> <span class="s">"DOW"</span><span class="p">,</span> <span class="s">"NASDAQ"</span><span class="p">,</span> <span class="s">"S&P"</span><span class="p">]</span>
|
||||
<span class="n">index_dict</span> <span class="o">=</span> <span class="p">{</span><span class="s2">"SPY"</span><span class="p">:</span> <span class="n">SPY_open_ret</span><span class="p">,</span> <span class="s2">"DJIA"</span><span class="p">:</span> <span class="n">DJIA_open_ret</span><span class="p">,</span> <span class="s2">"RUSS"</span><span class="p">:</span> <span class="n">RUSS_open_ret</span><span class="p">,</span> <span class="s2">"NASDAQ"</span><span class="p">:</span> <span class="n">NASDAQ_open_ret</span><span class="p">}</span>
|
||||
<span class="n">indices</span> <span class="o">=</span> <span class="p">[</span><span class="s2">"SPY"</span><span class="p">,</span> <span class="s2">"DJIA"</span><span class="p">,</span> <span class="s2">"RUSS"</span><span class="p">,</span> <span class="s2">"NASDAQ"</span><span class="p">]</span>
|
||||
<span class="n">futures</span> <span class="o">=</span> <span class="p">[</span><span class="s2">"Crude"</span><span class="p">,</span> <span class="s2">"Gold"</span><span class="p">,</span> <span class="s2">"DOW"</span><span class="p">,</span> <span class="s2">"NASDAQ"</span><span class="p">,</span> <span class="s2">"S&P"</span><span class="p">]</span>
|
||||
<span class="k">for</span> <span class="n">index</span> <span class="ow">in</span> <span class="n">indices</span><span class="p">:</span>
|
||||
<span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">signs_match</span><span class="p">(</span><span class="n">article_df</span><span class="p">[[</span><span class="n">future</span><span class="p">,</span> <span class="s">'date'</span><span class="p">]],</span>
|
||||
<span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">signs_match</span><span class="p">(</span><span class="n">article_df</span><span class="p">[[</span><span class="n">future</span><span class="p">,</span> <span class="s1">'date'</span><span class="p">]],</span>
|
||||
<span class="n">index_dict</span><span class="p">[</span><span class="n">index</span><span class="p">])</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
|
||||
<span class="n">count_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">future</span><span class="p">])</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
|
||||
<span class="n">prediction_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">future</span><span class="p">])</span>
|
||||
<span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s">"Articles Checked: "</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Articles Checked: "</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">count_dict</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">()</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s">"Prediction Accuracy:"</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Prediction Accuracy:"</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">prediction_dict</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
|
||||
@ -392,13 +392,13 @@ S&P 0.604478 0.597015 0.811808 0.848708
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">calculate_closing_ret</span><span class="p">(</span><span class="n">frame</span><span class="p">):</span>
|
||||
<span class="c"># I'm not a huge fan of the appending for loop,</span>
|
||||
<span class="c"># but it's a bit verbose for a comprehension</span>
|
||||
<span class="c1"># I'm not a huge fan of the appending for loop,</span>
|
||||
<span class="c1"># but it's a bit verbose for a comprehension</span>
|
||||
<span class="n">data</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">frame</span><span class="p">)):</span>
|
||||
<span class="n">date</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||||
<span class="n">open_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s">'Open'</span><span class="p">]</span>
|
||||
<span class="n">close_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s">'Close'</span><span class="p">]</span>
|
||||
<span class="n">open_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s1">'Open'</span><span class="p">]</span>
|
||||
<span class="n">close_val</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">iloc</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s1">'Close'</span><span class="p">]</span>
|
||||
<span class="n">data</span><span class="p">[</span><span class="n">date</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">close_val</span> <span class="o">-</span> <span class="n">open_val</span><span class="p">)</span> <span class="o">/</span> <span class="n">open_val</span>
|
||||
|
||||
<span class="k">return</span> <span class="n">data</span>
|
||||
@ -409,8 +409,8 @@ S&P 0.604478 0.597015 0.811808 0.848708
|
||||
<span class="n">NASDAQ_close_ret</span> <span class="o">=</span> <span class="n">calculate_closing_ret</span><span class="p">(</span><span class="n">NASDAQ</span><span class="p">)</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">signs_match</span><span class="p">(</span><span class="n">list_1</span><span class="p">,</span> <span class="n">list_2</span><span class="p">):</span>
|
||||
<span class="c"># This is a surprisingly difficult task - we have to match</span>
|
||||
<span class="c"># up the dates in order to check if opening returns actually match</span>
|
||||
<span class="c1"># This is a surprisingly difficult task - we have to match</span>
|
||||
<span class="c1"># up the dates in order to check if opening returns actually match</span>
|
||||
<span class="n">index_dict_dt</span> <span class="o">=</span> <span class="p">{</span><span class="n">key</span><span class="o">.</span><span class="n">to_datetime</span><span class="p">():</span> <span class="n">list_2</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">list_2</span><span class="o">.</span><span class="n">keys</span><span class="p">()}</span>
|
||||
|
||||
<span class="n">matches</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
@ -426,28 +426,28 @@ S&P 0.604478 0.597015 0.811808 0.848708
|
||||
<span class="n">matches</span> <span class="o">+=</span> <span class="p">[</span><span class="mi">1</span><span class="p">]</span>
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
<span class="n">matches</span> <span class="o">+=</span> <span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
||||
<span class="c">#print("{}".format(list_2[index_dt]))</span>
|
||||
<span class="c1">#print("{}".format(list_2[index_dt]))</span>
|
||||
<span class="k">return</span> <span class="n">matches</span>
|
||||
|
||||
|
||||
<span class="n">matches_dict</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
<span class="n">count_dict</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
<span class="n">prediction_dict</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
<span class="n">index_dict</span> <span class="o">=</span> <span class="p">{</span><span class="s">"SPY"</span><span class="p">:</span> <span class="n">SPY_close_ret</span><span class="p">,</span> <span class="s">"DJIA"</span><span class="p">:</span> <span class="n">DJIA_close_ret</span><span class="p">,</span>
|
||||
<span class="s">"RUSS"</span><span class="p">:</span> <span class="n">RUSS_close_ret</span><span class="p">,</span> <span class="s">"NASDAQ"</span><span class="p">:</span> <span class="n">NASDAQ_close_ret</span><span class="p">}</span>
|
||||
<span class="n">indices</span> <span class="o">=</span> <span class="p">[</span><span class="s">"SPY"</span><span class="p">,</span> <span class="s">"DJIA"</span><span class="p">,</span> <span class="s">"RUSS"</span><span class="p">,</span> <span class="s">"NASDAQ"</span><span class="p">]</span>
|
||||
<span class="n">futures</span> <span class="o">=</span> <span class="p">[</span><span class="s">"Crude"</span><span class="p">,</span> <span class="s">"Gold"</span><span class="p">,</span> <span class="s">"DOW"</span><span class="p">,</span> <span class="s">"NASDAQ"</span><span class="p">,</span> <span class="s">"S&P"</span><span class="p">]</span>
|
||||
<span class="n">index_dict</span> <span class="o">=</span> <span class="p">{</span><span class="s2">"SPY"</span><span class="p">:</span> <span class="n">SPY_close_ret</span><span class="p">,</span> <span class="s2">"DJIA"</span><span class="p">:</span> <span class="n">DJIA_close_ret</span><span class="p">,</span>
|
||||
<span class="s2">"RUSS"</span><span class="p">:</span> <span class="n">RUSS_close_ret</span><span class="p">,</span> <span class="s2">"NASDAQ"</span><span class="p">:</span> <span class="n">NASDAQ_close_ret</span><span class="p">}</span>
|
||||
<span class="n">indices</span> <span class="o">=</span> <span class="p">[</span><span class="s2">"SPY"</span><span class="p">,</span> <span class="s2">"DJIA"</span><span class="p">,</span> <span class="s2">"RUSS"</span><span class="p">,</span> <span class="s2">"NASDAQ"</span><span class="p">]</span>
|
||||
<span class="n">futures</span> <span class="o">=</span> <span class="p">[</span><span class="s2">"Crude"</span><span class="p">,</span> <span class="s2">"Gold"</span><span class="p">,</span> <span class="s2">"DOW"</span><span class="p">,</span> <span class="s2">"NASDAQ"</span><span class="p">,</span> <span class="s2">"S&P"</span><span class="p">]</span>
|
||||
<span class="k">for</span> <span class="n">index</span> <span class="ow">in</span> <span class="n">indices</span><span class="p">:</span>
|
||||
<span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">signs_match</span><span class="p">(</span><span class="n">article_df</span><span class="p">[[</span><span class="n">future</span><span class="p">,</span> <span class="s">'date'</span><span class="p">]],</span>
|
||||
<span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">signs_match</span><span class="p">(</span><span class="n">article_df</span><span class="p">[[</span><span class="n">future</span><span class="p">,</span> <span class="s1">'date'</span><span class="p">]],</span>
|
||||
<span class="n">index_dict</span><span class="p">[</span><span class="n">index</span><span class="p">])</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
|
||||
<span class="n">count_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">future</span><span class="p">])</span> <span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
|
||||
<span class="n">prediction_dict</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span><span class="n">future</span><span class="p">:</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">matches_dict</span><span class="p">[</span><span class="n">index</span><span class="p">][</span><span class="n">future</span><span class="p">])</span>
|
||||
<span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">futures</span><span class="p">}</span>
|
||||
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s">"Articles Checked:"</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Articles Checked:"</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">count_dict</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">()</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s">"Prediction Accuracy:"</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Prediction Accuracy:"</span><span class="p">)</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="o">.</span><span class="n">from_dict</span><span class="p">(</span><span class="n">prediction_dict</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="description" content="Or, how to get thrown out of a casino because you're a mathematician. In the ongoing eternal cycle of mathematicians asking generally useless questions about probability, I dreamt up another one. The ...">
|
||||
<meta name="description" content="Or, how to get thrown out of a casino because you're a mathematician. In the ongoing eternal cycle of mathematicians asking generally useless questions about probability, I dreamt up another one. ...">
|
||||
<meta name="keywords" content="casino, em, machine learning, probability, simulated annealing">
|
||||
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
|
||||
|
||||
@ -114,8 +114,8 @@
|
||||
<span class="n">fair_1</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
|
||||
<span class="n">fair_2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
|
||||
|
||||
<span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">fair_1</span> <span class="o">+</span> <span class="n">fair_2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s">'hist'</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">);</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">'Fair Distribution'</span><span class="p">);</span>
|
||||
<span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">fair_1</span> <span class="o">+</span> <span class="n">fair_2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s1">'hist'</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">);</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">'Fair Distribution'</span><span class="p">);</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -326,8 +326,8 @@ D_2 = \left\{
|
||||
<span class="n">x1</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d1</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
|
||||
<span class="n">x2</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d2</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
|
||||
|
||||
<span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">x1</span> <span class="o">+</span> <span class="n">x2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s">'hist'</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">);</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">'$D_1$ biased low, $D_2$ biased high'</span><span class="p">);</span>
|
||||
<span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">x1</span> <span class="o">+</span> <span class="n">x2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s1">'hist'</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">);</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">'$D_1$ biased low, $D_2$ biased high'</span><span class="p">);</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -525,8 +525,8 @@ D_2 = \left\{
|
||||
<span class="n">x1</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d1</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
|
||||
<span class="n">x2</span> <span class="o">=</span> <span class="n">unfair_die</span><span class="p">(</span><span class="n">d2</span><span class="p">,</span> <span class="mi">10000</span><span class="p">)</span>
|
||||
|
||||
<span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">x1</span> <span class="o">+</span> <span class="n">x2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s">'hist'</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">"$D_1$ and $D_2$ biased to 1 and 6"</span><span class="p">);</span>
|
||||
<span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">x1</span> <span class="o">+</span> <span class="n">x2</span><span class="p">)</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s1">'hist'</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s2">"$D_1$ and $D_2$ biased to 1 and 6"</span><span class="p">);</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -746,18 +746,18 @@ and continue the next iteration.</li>
|
||||
<span class="kn">from</span> <span class="nn">numpy</span> <span class="k">import</span> <span class="n">polynomial</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">density_coef</span><span class="p">(</span><span class="n">d1_density</span><span class="p">,</span> <span class="n">d2_density</span><span class="p">):</span>
|
||||
<span class="c"># Calculating the probabilities of each outcome was taken</span>
|
||||
<span class="c"># from this author: http://math.stackexchange.com/a/1710392/320784</span>
|
||||
<span class="c1"># Calculating the probabilities of each outcome was taken</span>
|
||||
<span class="c1"># from this author: http://math.stackexchange.com/a/1710392/320784</span>
|
||||
<span class="n">d1_p</span> <span class="o">=</span> <span class="n">polynomial</span><span class="o">.</span><span class="n">Polynomial</span><span class="p">(</span><span class="n">d1_density</span><span class="p">)</span>
|
||||
<span class="n">d2_p</span> <span class="o">=</span> <span class="n">polynomial</span><span class="o">.</span><span class="n">Polynomial</span><span class="p">(</span><span class="n">d2_density</span><span class="p">)</span>
|
||||
<span class="n">coefs</span> <span class="o">=</span> <span class="p">(</span><span class="n">d1_p</span> <span class="o">*</span> <span class="n">d2_p</span><span class="p">)</span><span class="o">.</span><span class="n">coef</span>
|
||||
<span class="k">return</span> <span class="n">coefs</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">score</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">d1_density</span><span class="p">,</span> <span class="n">d2_density</span><span class="p">):</span>
|
||||
<span class="c"># We've now got the probabilities of each event, but we need</span>
|
||||
<span class="c"># to shift the array a bit so we can use the x values to actually</span>
|
||||
<span class="c"># index into it. This will allow us to do all the calculations</span>
|
||||
<span class="c"># incredibly quickly</span>
|
||||
<span class="c1"># We've now got the probabilities of each event, but we need</span>
|
||||
<span class="c1"># to shift the array a bit so we can use the x values to actually</span>
|
||||
<span class="c1"># index into it. This will allow us to do all the calculations</span>
|
||||
<span class="c1"># incredibly quickly</span>
|
||||
<span class="n">coefs</span> <span class="o">=</span> <span class="n">density_coef</span><span class="p">(</span><span class="n">d1_density</span><span class="p">,</span> <span class="n">d2_density</span><span class="p">)</span>
|
||||
<span class="n">coefs</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">hstack</span><span class="p">((</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="n">coefs</span><span class="p">))</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">coefs</span><span class="p">[</span><span class="n">x</span><span class="p">])</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
|
||||
@ -784,23 +784,23 @@ and continue the next iteration.</li>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="k">def</span> <span class="nf">permute</span><span class="p">(</span><span class="n">d1_density</span><span class="p">,</span> <span class="n">d2_density</span><span class="p">):</span>
|
||||
<span class="c"># To ensure we have legitimate densities, we will randomly</span>
|
||||
<span class="c"># increase one die face probability by `change`,</span>
|
||||
<span class="c"># and decrease one by `change`.</span>
|
||||
<span class="c"># This means there are something less than (1/`change`)^12 possibilities</span>
|
||||
<span class="c"># we are trying to search over.</span>
|
||||
<span class="c1"># To ensure we have legitimate densities, we will randomly</span>
|
||||
<span class="c1"># increase one die face probability by `change`,</span>
|
||||
<span class="c1"># and decrease one by `change`.</span>
|
||||
<span class="c1"># This means there are something less than (1/`change`)^12 possibilities</span>
|
||||
<span class="c1"># we are trying to search over.</span>
|
||||
<span class="n">change</span> <span class="o">=</span> <span class="o">.</span><span class="mi">01</span>
|
||||
|
||||
<span class="n">d1_index1</span><span class="p">,</span> <span class="n">d1_index2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
|
||||
<span class="n">d2_index1</span><span class="p">,</span> <span class="n">d2_index2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
|
||||
|
||||
<span class="c"># Also make sure to copy. I've had some weird aliasing issues</span>
|
||||
<span class="c"># in the past that made everything blow up.</span>
|
||||
<span class="c1"># Also make sure to copy. I've had some weird aliasing issues</span>
|
||||
<span class="c1"># in the past that made everything blow up.</span>
|
||||
<span class="n">new_d1</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">float64</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="n">d1_density</span><span class="p">))</span>
|
||||
<span class="n">new_d2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">float64</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="n">d2_density</span><span class="p">))</span>
|
||||
|
||||
<span class="c"># While this doesn't account for the possibility that some</span>
|
||||
<span class="c"># values go negative, in practice this never happens</span>
|
||||
<span class="c1"># While this doesn't account for the possibility that some</span>
|
||||
<span class="c1"># values go negative, in practice this never happens</span>
|
||||
<span class="n">new_d1</span><span class="p">[</span><span class="n">d1_index1</span><span class="p">]</span> <span class="o">+=</span> <span class="n">change</span>
|
||||
<span class="n">new_d1</span><span class="p">[</span><span class="n">d1_index2</span><span class="p">]</span> <span class="o">-=</span> <span class="n">change</span>
|
||||
<span class="n">new_d2</span><span class="p">[</span><span class="n">d2_index1</span><span class="p">]</span> <span class="o">+=</span> <span class="n">change</span>
|
||||
@ -833,24 +833,24 @@ and continue the next iteration.</li>
|
||||
<span class="n">switch_failures</span> <span class="o">=</span> <span class="mi">0</span>
|
||||
<span class="n">iter_count</span> <span class="o">=</span> <span class="mi">0</span>
|
||||
|
||||
<span class="c"># Start with guessing fair dice</span>
|
||||
<span class="c1"># Start with guessing fair dice</span>
|
||||
<span class="n">cur_d1</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="mi">1</span><span class="o">/</span><span class="mi">6</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
|
||||
<span class="n">cur_d2</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="mi">1</span><span class="o">/</span><span class="mi">6</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
|
||||
<span class="n">cur_score</span> <span class="o">=</span> <span class="n">score</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">cur_d1</span><span class="p">,</span> <span class="n">cur_d2</span><span class="p">)</span>
|
||||
|
||||
<span class="c"># Keep track of our best guesses - may not be</span>
|
||||
<span class="c"># what we end with</span>
|
||||
<span class="c1"># Keep track of our best guesses - may not be</span>
|
||||
<span class="c1"># what we end with</span>
|
||||
<span class="n">max_score</span> <span class="o">=</span> <span class="n">cur_score</span>
|
||||
<span class="n">max_d1</span> <span class="o">=</span> <span class="n">cur_d1</span>
|
||||
<span class="n">max_d2</span> <span class="o">=</span> <span class="n">cur_d2</span>
|
||||
|
||||
<span class="c"># Optimization stops when we have failed to switch `conv_count`</span>
|
||||
<span class="c"># times (presumably because we have a great guess), or we reach</span>
|
||||
<span class="c"># the maximum number of iterations.</span>
|
||||
<span class="c1"># Optimization stops when we have failed to switch `conv_count`</span>
|
||||
<span class="c1"># times (presumably because we have a great guess), or we reach</span>
|
||||
<span class="c1"># the maximum number of iterations.</span>
|
||||
<span class="k">while</span> <span class="n">switch_failures</span> <span class="o"><</span> <span class="n">conv_count</span> <span class="ow">and</span> <span class="n">iter_count</span> <span class="o"><</span> <span class="n">max_iter</span><span class="p">:</span>
|
||||
<span class="n">iter_count</span> <span class="o">+=</span> <span class="mi">1</span>
|
||||
<span class="k">if</span> <span class="n">iter_count</span> <span class="o">%</span> <span class="p">(</span><span class="n">max_iter</span> <span class="o">/</span> <span class="mi">10</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s">'Iteration: {}; Current score (higher is better): {}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s1">'Iteration: {}; Current score (higher is better): {}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
|
||||
<span class="n">iter_count</span><span class="p">,</span> <span class="n">cur_score</span><span class="p">))</span>
|
||||
|
||||
<span class="n">new_d1</span><span class="p">,</span> <span class="n">new_d2</span> <span class="o">=</span> <span class="n">permute</span><span class="p">(</span><span class="n">cur_d1</span><span class="p">,</span> <span class="n">cur_d2</span><span class="p">)</span>
|
||||
@ -862,19 +862,19 @@ and continue the next iteration.</li>
|
||||
<span class="n">max_d2</span> <span class="o">=</span> <span class="n">new_d2</span>
|
||||
|
||||
<span class="k">if</span> <span class="n">new_score</span> <span class="o">></span> <span class="n">cur_score</span><span class="p">:</span>
|
||||
<span class="c"># If the new permutation beats the old one,</span>
|
||||
<span class="c"># automatically select it.</span>
|
||||
<span class="c1"># If the new permutation beats the old one,</span>
|
||||
<span class="c1"># automatically select it.</span>
|
||||
<span class="n">cur_score</span> <span class="o">=</span> <span class="n">new_score</span>
|
||||
<span class="n">cur_d1</span> <span class="o">=</span> <span class="n">new_d1</span>
|
||||
<span class="n">cur_d2</span> <span class="o">=</span> <span class="n">new_d2</span>
|
||||
<span class="n">switch_failures</span> <span class="o">=</span> <span class="mi">0</span>
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
<span class="c"># We didn't beat the current score, but allow</span>
|
||||
<span class="c"># for possibly switching anyways.</span>
|
||||
<span class="c1"># We didn't beat the current score, but allow</span>
|
||||
<span class="c1"># for possibly switching anyways.</span>
|
||||
<span class="n">accept_prob</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="n">new_score</span> <span class="o">-</span> <span class="n">cur_score</span><span class="p">)</span>
|
||||
<span class="n">coin_toss</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">()</span>
|
||||
<span class="k">if</span> <span class="n">coin_toss</span> <span class="o"><</span> <span class="n">accept_prob</span><span class="p">:</span>
|
||||
<span class="c"># We randomly switch to the new distribution</span>
|
||||
<span class="c1"># We randomly switch to the new distribution</span>
|
||||
<span class="n">cur_score</span> <span class="o">=</span> <span class="n">new_score</span>
|
||||
<span class="n">cur_d1</span> <span class="o">=</span> <span class="n">new_d1</span>
|
||||
<span class="n">cur_d2</span> <span class="o">=</span> <span class="n">new_d2</span>
|
||||
@ -882,7 +882,7 @@ and continue the next iteration.</li>
|
||||
<span class="k">else</span><span class="p">:</span>
|
||||
<span class="n">switch_failures</span> <span class="o">+=</span> <span class="mi">1</span>
|
||||
|
||||
<span class="c"># Return both our best guess, and the ending guess</span>
|
||||
<span class="c1"># Return both our best guess, and the ending guess</span>
|
||||
<span class="k">return</span> <span class="n">max_d1</span><span class="p">,</span> <span class="n">max_d2</span><span class="p">,</span> <span class="n">cur_d1</span><span class="p">,</span> <span class="n">cur_d2</span>
|
||||
</pre></div>
|
||||
|
||||
@ -916,20 +916,20 @@ and continue the next iteration.</li>
|
||||
<span class="n">start</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">perf_counter</span><span class="p">()</span>
|
||||
<span class="n">max_d1</span><span class="p">,</span> <span class="n">max_d2</span><span class="p">,</span> <span class="n">final_d1</span><span class="p">,</span> <span class="n">final_d2</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span><span class="n">d1_vals</span> <span class="o">+</span> <span class="n">d2_vals</span><span class="p">)</span>
|
||||
<span class="n">end</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">perf_counter</span><span class="p">()</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s">"Simulated Annealing time: {:.02f}s"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">end</span> <span class="o">-</span> <span class="n">start</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Simulated Annealing time: {:.02f}s"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">end</span> <span class="o">-</span> <span class="n">start</span><span class="p">))</span>
|
||||
|
||||
<span class="n">coef_range</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">13</span><span class="p">)</span> <span class="o">-</span> <span class="o">.</span><span class="mi">5</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">subplot</span><span class="p">(</span><span class="mi">221</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">bar</span><span class="p">(</span><span class="n">coef_range</span><span class="p">,</span> <span class="n">density_coef</span><span class="p">(</span><span class="n">d1_dist</span><span class="p">,</span> <span class="n">d2_dist</span><span class="p">),</span> <span class="n">width</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">'True Distribution'</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">'True Distribution'</span><span class="p">)</span>
|
||||
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">subplot</span><span class="p">(</span><span class="mi">222</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">hist</span><span class="p">(</span><span class="n">d1_vals</span> <span class="o">+</span> <span class="n">d2_vals</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="mi">11</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">'Empirical Distribution'</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">'Empirical Distribution'</span><span class="p">)</span>
|
||||
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">subplot</span><span class="p">(</span><span class="mi">223</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">bar</span><span class="p">(</span><span class="n">coef_range</span><span class="p">,</span> <span class="n">density_coef</span><span class="p">(</span><span class="n">max_d1</span><span class="p">,</span> <span class="n">max_d2</span><span class="p">),</span> <span class="n">width</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s">'Recovered Distribution'</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s1">'Recovered Distribution'</span><span class="p">)</span>
|
||||
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">gcf</span><span class="p">()</span><span class="o">.</span><span class="n">set_size_inches</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span>
|
||||
|
||||
|
||||
@ -137,7 +137,7 @@
|
||||
<div class="prompt input_prompt">In [2]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="c"># Import all the OAuth secret information from a local file</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="c1"># Import all the OAuth secret information from a local file</span>
|
||||
<span class="kn">from</span> <span class="nn">secrets</span> <span class="k">import</span> <span class="n">CLIENT_SECRET</span><span class="p">,</span> <span class="n">CLIENT_ID</span><span class="p">,</span> <span class="n">CALLBACK_URL</span>
|
||||
</pre></div>
|
||||
|
||||
@ -178,17 +178,17 @@ Application: Thank you, now give me all the data.</code></pre>
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">urllib</span>
|
||||
|
||||
<span class="n">FITBIT_URI</span> <span class="o">=</span> <span class="s">'https://www.fitbit.com/oauth2/authorize'</span>
|
||||
<span class="n">FITBIT_URI</span> <span class="o">=</span> <span class="s1">'https://www.fitbit.com/oauth2/authorize'</span>
|
||||
<span class="n">params</span> <span class="o">=</span> <span class="p">{</span>
|
||||
<span class="c"># If we need more than one scope, must be a CSV string</span>
|
||||
<span class="s">'scope'</span><span class="p">:</span> <span class="s">'heartrate'</span><span class="p">,</span>
|
||||
<span class="s">'response_type'</span><span class="p">:</span> <span class="s">'token'</span><span class="p">,</span>
|
||||
<span class="s">'expires_in'</span><span class="p">:</span> <span class="mi">86400</span><span class="p">,</span> <span class="c"># 1 day</span>
|
||||
<span class="s">'redirect_uri'</span><span class="p">:</span> <span class="n">CALLBACK_URL</span><span class="p">,</span>
|
||||
<span class="s">'client_id'</span><span class="p">:</span> <span class="n">CLIENT_ID</span>
|
||||
<span class="c1"># If we need more than one scope, must be a CSV string</span>
|
||||
<span class="s1">'scope'</span><span class="p">:</span> <span class="s1">'heartrate'</span><span class="p">,</span>
|
||||
<span class="s1">'response_type'</span><span class="p">:</span> <span class="s1">'token'</span><span class="p">,</span>
|
||||
<span class="s1">'expires_in'</span><span class="p">:</span> <span class="mi">86400</span><span class="p">,</span> <span class="c1"># 1 day</span>
|
||||
<span class="s1">'redirect_uri'</span><span class="p">:</span> <span class="n">CALLBACK_URL</span><span class="p">,</span>
|
||||
<span class="s1">'client_id'</span><span class="p">:</span> <span class="n">CLIENT_ID</span>
|
||||
<span class="p">}</span>
|
||||
|
||||
<span class="n">request_url</span> <span class="o">=</span> <span class="n">FITBIT_URI</span> <span class="o">+</span> <span class="s">'?'</span> <span class="o">+</span> <span class="n">urllib</span><span class="o">.</span><span class="n">parse</span><span class="o">.</span><span class="n">urlencode</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
|
||||
<span class="n">request_url</span> <span class="o">=</span> <span class="n">FITBIT_URI</span> <span class="o">+</span> <span class="s1">'?'</span> <span class="o">+</span> <span class="n">urllib</span><span class="o">.</span><span class="n">parse</span><span class="o">.</span><span class="n">urlencode</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -211,10 +211,10 @@ Application: Thank you, now give me all the data.</code></pre>
|
||||
<div class="prompt input_prompt">In [6]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="c"># The `response_url` variable contains the full URL that</span>
|
||||
<span class="c"># FitBit sent back to us, but most importantly,</span>
|
||||
<span class="c"># contains the token we need for authorization.</span>
|
||||
<span class="n">access_token</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">urllib</span><span class="o">.</span><span class="n">parse</span><span class="o">.</span><span class="n">parse_qsl</span><span class="p">(</span><span class="n">response_url</span><span class="p">))[</span><span class="s">'access_token'</span><span class="p">]</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="c1"># The `response_url` variable contains the full URL that</span>
|
||||
<span class="c1"># FitBit sent back to us, but most importantly,</span>
|
||||
<span class="c1"># contains the token we need for authorization.</span>
|
||||
<span class="n">access_token</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="n">urllib</span><span class="o">.</span><span class="n">parse</span><span class="o">.</span><span class="n">parse_qsl</span><span class="p">(</span><span class="n">response_url</span><span class="p">))[</span><span class="s1">'access_token'</span><span class="p">]</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -243,41 +243,41 @@ Application: Thank you, now give me all the data.</code></pre>
|
||||
<span class="kn">from</span> <span class="nn">datetime</span> <span class="k">import</span> <span class="n">datetime</span>
|
||||
|
||||
<span class="n">session</span> <span class="o">=</span> <span class="n">OAuth2Session</span><span class="p">(</span><span class="n">token</span><span class="o">=</span><span class="p">{</span>
|
||||
<span class="s">'access_token'</span><span class="p">:</span> <span class="n">access_token</span><span class="p">,</span>
|
||||
<span class="s">'token_type'</span><span class="p">:</span> <span class="s">'Bearer'</span>
|
||||
<span class="s1">'access_token'</span><span class="p">:</span> <span class="n">access_token</span><span class="p">,</span>
|
||||
<span class="s1">'token_type'</span><span class="p">:</span> <span class="s1">'Bearer'</span>
|
||||
<span class="p">})</span>
|
||||
|
||||
<span class="n">format_str</span> <span class="o">=</span> <span class="s">'%Y-%m-%d'</span>
|
||||
<span class="n">format_str</span> <span class="o">=</span> <span class="s1">'%Y-%m-</span><span class="si">%d</span><span class="s1">'</span>
|
||||
<span class="n">start_date</span> <span class="o">=</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
|
||||
<span class="n">end_date</span> <span class="o">=</span> <span class="n">datetime</span><span class="p">(</span><span class="mi">2016</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">31</span><span class="p">)</span>
|
||||
<span class="n">dr</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">date_range</span><span class="p">(</span><span class="n">start_date</span><span class="p">,</span> <span class="n">end_date</span><span class="p">)</span>
|
||||
|
||||
<span class="n">url</span> <span class="o">=</span> <span class="s">'https://api.fitbit.com/1/user/-/activities/heart/date/{0}/1d/1min.json'</span>
|
||||
<span class="n">url</span> <span class="o">=</span> <span class="s1">'https://api.fitbit.com/1/user/-/activities/heart/date/{0}/1d/1min.json'</span>
|
||||
<span class="n">hr_responses</span> <span class="o">=</span> <span class="p">[</span><span class="n">session</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">d</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="n">format_str</span><span class="p">)))</span> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">dr</span><span class="p">]</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">record_to_df</span><span class="p">(</span><span class="n">record</span><span class="p">):</span>
|
||||
<span class="k">if</span> <span class="s">'activities-heart'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">record</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="k">None</span>
|
||||
<span class="n">date_str</span> <span class="o">=</span> <span class="n">record</span><span class="p">[</span><span class="s">'activities-heart'</span><span class="p">][</span><span class="mi">0</span><span class="p">][</span><span class="s">'dateTime'</span><span class="p">]</span>
|
||||
<span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">record</span><span class="p">[</span><span class="s">'activities-heart-intraday'</span><span class="p">][</span><span class="s">'dataset'</span><span class="p">])</span>
|
||||
<span class="k">if</span> <span class="s1">'activities-heart'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">record</span><span class="p">:</span>
|
||||
<span class="k">return</span> <span class="kc">None</span>
|
||||
<span class="n">date_str</span> <span class="o">=</span> <span class="n">record</span><span class="p">[</span><span class="s1">'activities-heart'</span><span class="p">][</span><span class="mi">0</span><span class="p">][</span><span class="s1">'dateTime'</span><span class="p">]</span>
|
||||
<span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">record</span><span class="p">[</span><span class="s1">'activities-heart-intraday'</span><span class="p">][</span><span class="s1">'dataset'</span><span class="p">])</span>
|
||||
|
||||
<span class="n">df</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="s">'time'</span><span class="p">]</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span>
|
||||
<span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">strptime</span><span class="p">(</span><span class="n">date_str</span> <span class="o">+</span> <span class="s">' '</span> <span class="o">+</span> <span class="n">x</span><span class="p">,</span> <span class="s">'%Y-%m-%d %H:%M:%S'</span><span class="p">))</span>
|
||||
<span class="n">df</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="s1">'time'</span><span class="p">]</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span>
|
||||
<span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">strptime</span><span class="p">(</span><span class="n">date_str</span> <span class="o">+</span> <span class="s1">' '</span> <span class="o">+</span> <span class="n">x</span><span class="p">,</span> <span class="s1">'%Y-%m-</span><span class="si">%d</span><span class="s1"> %H:%M:%S'</span><span class="p">))</span>
|
||||
<span class="k">return</span> <span class="n">df</span>
|
||||
|
||||
<span class="n">hr_dataframes</span> <span class="o">=</span> <span class="p">[</span><span class="n">record_to_df</span><span class="p">(</span><span class="n">record</span><span class="o">.</span><span class="n">json</span><span class="p">())</span> <span class="k">for</span> <span class="n">record</span> <span class="ow">in</span> <span class="n">hr_responses</span><span class="p">]</span>
|
||||
<span class="n">hr_df_concat</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">concat</span><span class="p">(</span><span class="n">hr_dataframes</span><span class="p">)</span>
|
||||
|
||||
|
||||
<span class="c"># There are some minutes with missing data, so we need to correct that</span>
|
||||
<span class="c1"># There are some minutes with missing data, so we need to correct that</span>
|
||||
<span class="n">full_daterange</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">date_range</span><span class="p">(</span><span class="n">hr_df_concat</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span>
|
||||
<span class="n">hr_df_concat</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span>
|
||||
<span class="n">freq</span><span class="o">=</span><span class="s">'min'</span><span class="p">)</span>
|
||||
<span class="n">hr_df_full</span> <span class="o">=</span> <span class="n">hr_df_concat</span><span class="o">.</span><span class="n">reindex</span><span class="p">(</span><span class="n">full_daterange</span><span class="p">,</span> <span class="n">method</span><span class="o">=</span><span class="s">'nearest'</span><span class="p">)</span>
|
||||
<span class="n">freq</span><span class="o">=</span><span class="s1">'min'</span><span class="p">)</span>
|
||||
<span class="n">hr_df_full</span> <span class="o">=</span> <span class="n">hr_df_concat</span><span class="o">.</span><span class="n">reindex</span><span class="p">(</span><span class="n">full_daterange</span><span class="p">,</span> <span class="n">method</span><span class="o">=</span><span class="s1">'nearest'</span><span class="p">)</span>
|
||||
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s">"Heartbeats from {} to {}: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Heartbeats from {} to {}: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span>
|
||||
<span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span>
|
||||
<span class="n">hr_df_full</span><span class="p">[</span><span class="s">'value'</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()))</span>
|
||||
<span class="n">hr_df_full</span><span class="p">[</span><span class="s1">'value'</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()))</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -325,9 +325,9 @@ hr_q \cdot n - hr_d \cdot (n-m)
|
||||
<div class="prompt input_prompt">In [8]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">quarterly_count</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[</span><span class="s">'value'</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">quarterly_count</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[</span><span class="s1">'value'</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
|
||||
<span class="n">leap_day_count</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">month</span> <span class="o">==</span> <span class="mi">2</span><span class="p">)</span> <span class="o">&</span>
|
||||
<span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">day</span> <span class="o">==</span> <span class="mi">29</span><span class="p">)][</span><span class="s">'value'</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
|
||||
<span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">day</span> <span class="o">==</span> <span class="mi">29</span><span class="p">)][</span><span class="s1">'value'</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
|
||||
<span class="n">num_quarters</span> <span class="o">=</span> <span class="mi">96</span>
|
||||
<span class="n">leap_days</span> <span class="o">=</span> <span class="mi">6</span>
|
||||
|
||||
@ -372,17 +372,17 @@ hr_q \cdot n - hr_d \cdot (n-m)
|
||||
<div class="prompt input_prompt">In [9]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">minute_mean</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[</span><span class="s">'value'</span><span class="p">]</span><span class="o">.</span><span class="n">mean</span><span class="p">()</span>
|
||||
<span class="c"># Don't you wish you knew?</span>
|
||||
<span class="c"># birthday_minutes = ???</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">minute_mean</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[</span><span class="s1">'value'</span><span class="p">]</span><span class="o">.</span><span class="n">mean</span><span class="p">()</span>
|
||||
<span class="c1"># Don't you wish you knew?</span>
|
||||
<span class="c1"># birthday_minutes = ???</span>
|
||||
|
||||
<span class="n">birthday_heartbeats</span> <span class="o">=</span> <span class="n">birthday_minutes</span> <span class="o">*</span> <span class="n">minute_mean</span>
|
||||
|
||||
<span class="n">heartbeats_until_2016</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">birthday_heartbeats</span> <span class="o">+</span> <span class="n">jan_92_jan_16</span><span class="p">)</span>
|
||||
<span class="n">remaining_2016</span> <span class="o">=</span> <span class="n">total_heartbeats</span> <span class="o">-</span> <span class="n">heartbeats_until_2016</span>
|
||||
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s">"Heartbeats so far: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">heartbeats_until_2016</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s">"Remaining heartbeats: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_2016</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Heartbeats so far: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">heartbeats_until_2016</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Remaining heartbeats: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_2016</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -473,8 +473,8 @@ hr_q \cdot n - hr_d \cdot (n - m)
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">four_year_periods</span> <span class="o">=</span> <span class="n">remaining_2016</span> <span class="o">//</span> <span class="n">heartbeats_4year</span>
|
||||
<span class="n">remaining_4y</span> <span class="o">=</span> <span class="n">remaining_2016</span> <span class="o">-</span> <span class="n">four_year_periods</span> <span class="o">*</span> <span class="n">heartbeats_4year</span>
|
||||
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s">"Four year periods remaining: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">four_year_periods</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s">"Remaining heartbeats after 4 year periods: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_4y</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Four year periods remaining: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">four_year_periods</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Remaining heartbeats after 4 year periods: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_4y</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -513,14 +513,14 @@ Remaining heartbeats after 4 year periods: 48041640
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">remaining_leap</span> <span class="o">=</span> <span class="n">remaining_4y</span> <span class="o">-</span> <span class="n">leap_day_count</span>
|
||||
<span class="c"># Ignore leap day in the data set</span>
|
||||
<span class="c1"># Ignore leap day in the data set</span>
|
||||
<span class="n">heartbeats_quarter</span> <span class="o">=</span> <span class="n">hr_df_full</span><span class="p">[(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">month</span> <span class="o">!=</span> <span class="mi">2</span><span class="p">)</span> <span class="o">&</span>
|
||||
<span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">day</span> <span class="o">!=</span> <span class="mi">29</span><span class="p">)][</span><span class="s">'value'</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
|
||||
<span class="p">(</span><span class="n">hr_df_full</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">day</span> <span class="o">!=</span> <span class="mi">29</span><span class="p">)][</span><span class="s1">'value'</span><span class="p">]</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
|
||||
<span class="n">quarters_left</span> <span class="o">=</span> <span class="n">remaining_leap</span> <span class="o">//</span> <span class="n">heartbeats_quarter</span>
|
||||
<span class="n">remaining_year</span> <span class="o">=</span> <span class="n">remaining_leap</span> <span class="o">-</span> <span class="n">quarters_left</span> <span class="o">*</span> <span class="n">heartbeats_quarter</span>
|
||||
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s">"Quarters left starting 2068: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">quarters_left</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s">"Remaining heartbeats after that: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_year</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Quarters left starting 2068: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">quarters_left</span><span class="p">))</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Remaining heartbeats after that: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">remaining_year</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="description" content="An experiment in creating a robot that will imitate me on Twitter. So, I'm taking a Machine Learning course this semester in school, and one of the topics we keep coming back to is natural language ...">
|
||||
<meta name="description" content="An experiment in creating a robot that will imitate me on Twitter. So, I'm taking a Machine Learning course this semester in school, and one of the topics we keep coming back to is natural ...">
|
||||
<meta name="keywords" content="MCMC, twitter">
|
||||
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
|
||||
|
||||
@ -128,11 +128,11 @@
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
|
||||
|
||||
<span class="n">tweets</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s">'tweets.csv'</span><span class="p">)</span>
|
||||
<span class="n">tweets</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s1">'tweets.csv'</span><span class="p">)</span>
|
||||
<span class="n">text</span> <span class="o">=</span> <span class="n">tweets</span><span class="o">.</span><span class="n">text</span>
|
||||
|
||||
<span class="c"># Don't include tweets in reply to or mentioning people</span>
|
||||
<span class="n">replies</span> <span class="o">=</span> <span class="n">text</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">contains</span><span class="p">(</span><span class="s">'@'</span><span class="p">)</span>
|
||||
<span class="c1"># Don't include tweets in reply to or mentioning people</span>
|
||||
<span class="n">replies</span> <span class="o">=</span> <span class="n">text</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">contains</span><span class="p">(</span><span class="s1">'@'</span><span class="p">)</span>
|
||||
<span class="n">text_norep</span> <span class="o">=</span> <span class="n">text</span><span class="o">.</span><span class="n">loc</span><span class="p">[</span><span class="o">~</span><span class="n">replies</span><span class="p">]</span>
|
||||
</pre></div>
|
||||
|
||||
@ -187,10 +187,10 @@
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="kn">from</span> <span class="nn">functools</span> <span class="k">import</span> <span class="n">reduce</span>
|
||||
|
||||
<span class="c"># Get all possible words</span>
|
||||
<span class="c1"># Get all possible words</span>
|
||||
<span class="n">all_words</span> <span class="o">=</span> <span class="n">reduce</span><span class="p">(</span><span class="k">lambda</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">:</span> <span class="n">x</span><span class="o">+</span><span class="n">y</span><span class="p">,</span> <span class="n">tokens</span><span class="p">,</span> <span class="p">[])</span>
|
||||
<span class="n">unique_words</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">all_words</span><span class="p">)</span>
|
||||
<span class="n">actual_words</span> <span class="o">=</span> <span class="nb">set</span><span class="p">([</span><span class="n">x</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s">'.'</span> <span class="k">else</span> <span class="k">None</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">unique_words</span><span class="p">])</span>
|
||||
<span class="n">actual_words</span> <span class="o">=</span> <span class="nb">set</span><span class="p">([</span><span class="n">x</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s1">'.'</span> <span class="k">else</span> <span class="kc">None</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">unique_words</span><span class="p">])</span>
|
||||
|
||||
<span class="n">word_dist</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
<span class="k">for</span> <span class="n">word</span> <span class="ow">in</span> <span class="nb">iter</span><span class="p">(</span><span class="n">actual_words</span><span class="p">):</span>
|
||||
@ -222,9 +222,9 @@
|
||||
<div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
|
||||
<span class="o">%</span><span class="k">matplotlib</span> inline
|
||||
|
||||
<span class="n">hashtags</span> <span class="o">=</span> <span class="n">text_norep</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">count</span><span class="p">(</span><span class="s">'#'</span><span class="p">)</span>
|
||||
<span class="n">hashtags</span> <span class="o">=</span> <span class="n">text_norep</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">count</span><span class="p">(</span><span class="s1">'#'</span><span class="p">)</span>
|
||||
<span class="n">bins</span> <span class="o">=</span> <span class="n">hashtags</span><span class="o">.</span><span class="n">unique</span><span class="p">()</span><span class="o">.</span><span class="n">max</span><span class="p">()</span>
|
||||
<span class="n">hashtags</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s">'hist'</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="n">bins</span><span class="p">)</span>
|
||||
<span class="n">hashtags</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">kind</span><span class="o">=</span><span class="s1">'hist'</span><span class="p">,</span> <span class="n">bins</span><span class="o">=</span><span class="n">bins</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -396,12 +396,12 @@ AElFTkSuQmCC
|
||||
<div class="prompt input_prompt">In [6]:</div>
|
||||
<div class="inner_cell">
|
||||
<div class="input_area">
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">hashtags</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">all_words</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s">'#'</span><span class="p">]</span>
|
||||
<div class=" highlight hl-ipython3"><pre><span class="n">hashtags</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">all_words</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s1">'#'</span><span class="p">]</span>
|
||||
<span class="n">n_hashtags</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">hashtags</span><span class="p">)</span>
|
||||
|
||||
<span class="n">unique_hashtags</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">([</span><span class="n">x</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">unique_words</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s">'#'</span><span class="p">]))</span>
|
||||
<span class="n">hashtag_dist</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">({</span><span class="s">'hashtags'</span><span class="p">:</span> <span class="n">unique_hashtags</span><span class="p">,</span>
|
||||
<span class="s">'prob'</span><span class="p">:</span> <span class="p">[</span><span class="n">all_words</span><span class="o">.</span><span class="n">count</span><span class="p">(</span><span class="n">h</span><span class="p">)</span> <span class="o">/</span> <span class="n">n_hashtags</span>
|
||||
<span class="n">unique_hashtags</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">([</span><span class="n">x</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">unique_words</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s1">'#'</span><span class="p">]))</span>
|
||||
<span class="n">hashtag_dist</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">({</span><span class="s1">'hashtags'</span><span class="p">:</span> <span class="n">unique_hashtags</span><span class="p">,</span>
|
||||
<span class="s1">'prob'</span><span class="p">:</span> <span class="p">[</span><span class="n">all_words</span><span class="o">.</span><span class="n">count</span><span class="p">(</span><span class="n">h</span><span class="p">)</span> <span class="o">/</span> <span class="n">n_hashtags</span>
|
||||
<span class="k">for</span> <span class="n">h</span> <span class="ow">in</span> <span class="n">unique_hashtags</span><span class="p">]})</span>
|
||||
<span class="nb">len</span><span class="p">(</span><span class="n">hashtag_dist</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
@ -459,7 +459,7 @@ z: .17</code></pre>
|
||||
<span class="k">def</span> <span class="nf">multinom_sim</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">vals</span><span class="p">,</span> <span class="n">probs</span><span class="p">):</span>
|
||||
<span class="n">occurrences</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">multinomial</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">probs</span><span class="p">)</span>
|
||||
<span class="n">results</span> <span class="o">=</span> <span class="n">occurrences</span> <span class="o">*</span> <span class="n">vals</span>
|
||||
<span class="k">return</span> <span class="s">' '</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">results</span><span class="p">[</span><span class="n">results</span> <span class="o">!=</span> <span class="s">''</span><span class="p">])</span>
|
||||
<span class="k">return</span> <span class="s1">' '</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">results</span><span class="p">[</span><span class="n">results</span> <span class="o">!=</span> <span class="s1">''</span><span class="p">])</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">sim_n_hashtags</span><span class="p">(</span><span class="n">hashtag_freq</span><span class="p">):</span>
|
||||
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">poisson</span><span class="p">(</span><span class="n">hashtag_freq</span><span class="p">)</span>
|
||||
@ -469,7 +469,7 @@ z: .17</code></pre>
|
||||
|
||||
<span class="k">def</span> <span class="nf">sim_first_word</span><span class="p">(</span><span class="n">first_word_dist</span><span class="p">):</span>
|
||||
<span class="n">probs</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">float64</span><span class="p">(</span><span class="n">first_word_dist</span><span class="o">.</span><span class="n">values</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">multinom_sim</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">first_word_dist</span><span class="o">.</span><span class="n">reset_index</span><span class="p">()[</span><span class="s">'index'</span><span class="p">],</span> <span class="n">probs</span><span class="p">)</span>
|
||||
<span class="k">return</span> <span class="n">multinom_sim</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">first_word_dist</span><span class="o">.</span><span class="n">reset_index</span><span class="p">()[</span><span class="s1">'index'</span><span class="p">],</span> <span class="n">probs</span><span class="p">)</span>
|
||||
|
||||
<span class="k">def</span> <span class="nf">sim_next_word</span><span class="p">(</span><span class="n">current</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">):</span>
|
||||
<span class="n">dist</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">(</span><span class="n">word_dist</span><span class="p">[</span><span class="n">current</span><span class="p">])</span>
|
||||
@ -504,7 +504,7 @@ z: .17</code></pre>
|
||||
<span class="n">fifth</span> <span class="o">=</span> <span class="n">sim_next_word</span><span class="p">(</span><span class="n">fourth</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">)</span>
|
||||
<span class="n">hashtag</span> <span class="o">=</span> <span class="n">sim_hashtags</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">hashtag_dist</span><span class="p">)</span>
|
||||
|
||||
<span class="s">' '</span><span class="o">.</span><span class="n">join</span><span class="p">((</span><span class="n">first</span><span class="p">,</span> <span class="n">second</span><span class="p">,</span> <span class="n">third</span><span class="p">,</span> <span class="n">fourth</span><span class="p">,</span> <span class="n">fifth</span><span class="p">,</span> <span class="n">hashtag</span><span class="p">))</span>
|
||||
<span class="s1">' '</span><span class="o">.</span><span class="n">join</span><span class="p">((</span><span class="n">first</span><span class="p">,</span> <span class="n">second</span><span class="p">,</span> <span class="n">third</span><span class="p">,</span> <span class="n">fourth</span><span class="p">,</span> <span class="n">fifth</span><span class="p">,</span> <span class="n">hashtag</span><span class="p">))</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
@ -553,13 +553,13 @@ z: .17</code></pre>
|
||||
|
||||
<span class="n">tweet</span> <span class="o">=</span> <span class="n">first</span>
|
||||
<span class="n">current</span> <span class="o">=</span> <span class="n">first</span>
|
||||
<span class="k">while</span> <span class="n">chars_remaining</span> <span class="o">></span> <span class="nb">len</span><span class="p">(</span><span class="n">tweet</span><span class="p">)</span> <span class="o">+</span> <span class="nb">len</span><span class="p">(</span><span class="n">hashtags</span><span class="p">)</span> <span class="ow">and</span> <span class="n">current</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s">'.'</span> <span class="ow">and</span> <span class="n">current</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s">'!'</span><span class="p">:</span>
|
||||
<span class="k">while</span> <span class="n">chars_remaining</span> <span class="o">></span> <span class="nb">len</span><span class="p">(</span><span class="n">tweet</span><span class="p">)</span> <span class="o">+</span> <span class="nb">len</span><span class="p">(</span><span class="n">hashtags</span><span class="p">)</span> <span class="ow">and</span> <span class="n">current</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s1">'.'</span> <span class="ow">and</span> <span class="n">current</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">!=</span> <span class="s1">'!'</span><span class="p">:</span>
|
||||
<span class="n">current</span> <span class="o">=</span> <span class="n">sim_next_word</span><span class="p">(</span><span class="n">current</span><span class="p">,</span> <span class="n">word_dist</span><span class="p">)</span>
|
||||
<span class="n">tweet</span> <span class="o">+=</span> <span class="s">' '</span> <span class="o">+</span> <span class="n">current</span>
|
||||
<span class="n">tweet</span> <span class="o">+=</span> <span class="s1">' '</span> <span class="o">+</span> <span class="n">current</span>
|
||||
|
||||
<span class="n">tweet</span> <span class="o">=</span> <span class="n">tweet</span><span class="p">[:</span><span class="o">-</span><span class="mi">2</span><span class="p">]</span> <span class="o">+</span> <span class="n">tweet</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
|
||||
|
||||
<span class="k">return</span> <span class="s">' '</span><span class="o">.</span><span class="n">join</span><span class="p">((</span><span class="n">tweet</span><span class="p">,</span> <span class="n">hashtags</span><span class="p">))</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
|
||||
<span class="k">return</span> <span class="s1">' '</span><span class="o">.</span><span class="n">join</span><span class="p">((</span><span class="n">tweet</span><span class="p">,</span> <span class="n">hashtags</span><span class="p">))</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
<meta charset="utf-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="description" content="Hello! Glad to meet you. I'm currently a student at Columbia University studying Financial Engineering, and want to give an overview of the projects I'm working on! To start things off, Columbia has ...">
|
||||
<meta name="description" content="Hello! Glad to meet you. I'm currently a student at Columbia University studying Financial Engineering, and want to give an overview of the projects I'm working on! To start things off, Columbia ...">
|
||||
<meta name="keywords" content="introduction, trading">
|
||||
<link rel="icon" href="https://bspeice.github.io/favicon.ico">
|
||||
|
||||
@ -161,12 +161,12 @@ to make some tweaks over the coming weeks, and do another forward test in Januar
|
||||
<span class="kn">import</span> <span class="nn">Quandl</span>
|
||||
<span class="kn">from</span> <span class="nn">datetime</span> <span class="k">import</span> <span class="n">datetime</span><span class="p">,</span> <span class="n">timedelta</span>
|
||||
|
||||
<span class="n">tickers</span> <span class="o">=</span> <span class="p">[</span><span class="s">'XOM'</span><span class="p">,</span> <span class="s">'CVX'</span><span class="p">,</span> <span class="s">'CLB'</span><span class="p">,</span> <span class="s">'OXY'</span><span class="p">,</span> <span class="s">'SLB'</span><span class="p">]</span>
|
||||
<span class="n">market_ticker</span> <span class="o">=</span> <span class="s">'GOOG/NYSE_VOO'</span>
|
||||
<span class="n">tickers</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'XOM'</span><span class="p">,</span> <span class="s1">'CVX'</span><span class="p">,</span> <span class="s1">'CLB'</span><span class="p">,</span> <span class="s1">'OXY'</span><span class="p">,</span> <span class="s1">'SLB'</span><span class="p">]</span>
|
||||
<span class="n">market_ticker</span> <span class="o">=</span> <span class="s1">'GOOG/NYSE_VOO'</span>
|
||||
<span class="n">lookback</span> <span class="o">=</span> <span class="mi">30</span>
|
||||
<span class="n">d_col</span> <span class="o">=</span> <span class="s">'Close'</span>
|
||||
<span class="n">d_col</span> <span class="o">=</span> <span class="s1">'Close'</span>
|
||||
|
||||
<span class="n">data</span> <span class="o">=</span> <span class="p">{</span><span class="n">tick</span><span class="p">:</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'YAHOO/{}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">tick</span><span class="p">))[</span><span class="o">-</span><span class="n">lookback</span><span class="p">:]</span> <span class="k">for</span> <span class="n">tick</span> <span class="ow">in</span> <span class="n">tickers</span><span class="p">}</span>
|
||||
<span class="n">data</span> <span class="o">=</span> <span class="p">{</span><span class="n">tick</span><span class="p">:</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'YAHOO/{}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">tick</span><span class="p">))[</span><span class="o">-</span><span class="n">lookback</span><span class="p">:]</span> <span class="k">for</span> <span class="n">tick</span> <span class="ow">in</span> <span class="n">tickers</span><span class="p">}</span>
|
||||
<span class="n">market</span> <span class="o">=</span> <span class="n">Quandl</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">market_ticker</span><span class="p">)</span>
|
||||
</pre></div>
|
||||
|
||||
@ -340,22 +340,22 @@ s.t.\ \ & \vec{1} \omega = 1\\
|
||||
<div class=" highlight hl-ipython3"><pre><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
|
||||
<span class="kn">from</span> <span class="nn">scipy.optimize</span> <span class="k">import</span> <span class="n">minimize</span>
|
||||
|
||||
<span class="c">#sharpe_limit = .1</span>
|
||||
<span class="c1">#sharpe_limit = .1</span>
|
||||
<span class="n">drawdown_limit</span> <span class="o">=</span> <span class="o">.</span><span class="mi">05</span>
|
||||
<span class="n">leverage</span> <span class="o">=</span> <span class="mi">250</span>
|
||||
|
||||
<span class="c"># Use the map so we can guarantee we maintain the correct order</span>
|
||||
<span class="c"># sharpe_a = np.array(list(map(lambda tick: sharpes[tick], tickers))) * -1 # So we can write as upper-bound</span>
|
||||
<span class="c1"># Use the map so we can guarantee we maintain the correct order</span>
|
||||
<span class="c1"># sharpe_a = np.array(list(map(lambda tick: sharpes[tick], tickers))) * -1 # So we can write as upper-bound</span>
|
||||
<span class="n">dd_a</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">tick</span><span class="p">:</span> <span class="n">drawdowns</span><span class="p">[</span><span class="n">tick</span><span class="p">],</span> <span class="n">tickers</span><span class="p">)))</span>
|
||||
<span class="n">returns_a</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">tick</span><span class="p">:</span> <span class="n">returns</span><span class="p">[</span><span class="n">tick</span><span class="p">]</span><span class="o">.</span><span class="n">mean</span><span class="p">(),</span> <span class="n">tickers</span><span class="p">)))</span> <span class="c"># Because minimizing</span>
|
||||
<span class="n">returns_a</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="k">lambda</span> <span class="n">tick</span><span class="p">:</span> <span class="n">returns</span><span class="p">[</span><span class="n">tick</span><span class="p">]</span><span class="o">.</span><span class="n">mean</span><span class="p">(),</span> <span class="n">tickers</span><span class="p">)))</span> <span class="c1"># Because minimizing</span>
|
||||
|
||||
<span class="n">meets_sharpe</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="o">*</span> <span class="n">sharpe_a</span><span class="p">)</span> <span class="o">-</span> <span class="n">sharpe_limit</span>
|
||||
<span class="k">def</span> <span class="nf">meets_dd</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
|
||||
<span class="n">portfolio</span> <span class="o">=</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">))</span>
|
||||
<span class="k">if</span> <span class="n">portfolio</span> <span class="o"><</span> <span class="o">.</span><span class="mi">1</span><span class="p">:</span>
|
||||
<span class="c"># If there are no stocks in the portfolio,</span>
|
||||
<span class="c"># we can accidentally induce division by 0,</span>
|
||||
<span class="c"># or division by something small enough to cause infinity</span>
|
||||
<span class="c1"># If there are no stocks in the portfolio,</span>
|
||||
<span class="c1"># we can accidentally induce division by 0,</span>
|
||||
<span class="c1"># or division by something small enough to cause infinity</span>
|
||||
<span class="k">return</span> <span class="mi">0</span>
|
||||
|
||||
<span class="k">return</span> <span class="n">drawdown_limit</span> <span class="o">-</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="o">*</span> <span class="n">dd_a</span><span class="p">)</span> <span class="o">/</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">))</span>
|
||||
@ -365,41 +365,41 @@ s.t.\ \ & \vec{1} \omega = 1\\
|
||||
<span class="k">def</span> <span class="nf">within_leverage</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
|
||||
<span class="k">return</span> <span class="n">leverage</span> <span class="o">-</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">x</span><span class="p">))</span>
|
||||
|
||||
<span class="n">objective</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">sum</span><span class="p">(</span><span class="n">x</span> <span class="o">*</span> <span class="n">returns_a</span><span class="p">)</span> <span class="o">*</span> <span class="o">-</span><span class="mi">1</span> <span class="c"># Because we're minimizing</span>
|
||||
<span class="n">bounds</span> <span class="o">=</span> <span class="p">((</span><span class="k">None</span><span class="p">,</span> <span class="k">None</span><span class="p">),)</span> <span class="o">*</span> <span class="nb">len</span><span class="p">(</span><span class="n">tickers</span><span class="p">)</span>
|
||||
<span class="n">objective</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="nb">sum</span><span class="p">(</span><span class="n">x</span> <span class="o">*</span> <span class="n">returns_a</span><span class="p">)</span> <span class="o">*</span> <span class="o">-</span><span class="mi">1</span> <span class="c1"># Because we're minimizing</span>
|
||||
<span class="n">bounds</span> <span class="o">=</span> <span class="p">((</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">),)</span> <span class="o">*</span> <span class="nb">len</span><span class="p">(</span><span class="n">tickers</span><span class="p">)</span>
|
||||
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">tickers</span><span class="p">))</span>
|
||||
|
||||
<span class="n">constraints</span> <span class="o">=</span> <span class="p">[</span>
|
||||
<span class="p">{</span>
|
||||
<span class="s">'type'</span><span class="p">:</span> <span class="s">'eq'</span><span class="p">,</span>
|
||||
<span class="s">'fun'</span><span class="p">:</span> <span class="n">is_portfolio</span>
|
||||
<span class="s1">'type'</span><span class="p">:</span> <span class="s1">'eq'</span><span class="p">,</span>
|
||||
<span class="s1">'fun'</span><span class="p">:</span> <span class="n">is_portfolio</span>
|
||||
<span class="p">},</span> <span class="p">{</span>
|
||||
<span class="s">'type'</span><span class="p">:</span> <span class="s">'ineq'</span><span class="p">,</span>
|
||||
<span class="s">'fun'</span><span class="p">:</span> <span class="n">within_leverage</span>
|
||||
<span class="c">#}, {</span>
|
||||
<span class="c"># 'type': 'ineq',</span>
|
||||
<span class="c"># 'fun': meets_sharpe</span>
|
||||
<span class="s1">'type'</span><span class="p">:</span> <span class="s1">'ineq'</span><span class="p">,</span>
|
||||
<span class="s1">'fun'</span><span class="p">:</span> <span class="n">within_leverage</span>
|
||||
<span class="c1">#}, {</span>
|
||||
<span class="c1"># 'type': 'ineq',</span>
|
||||
<span class="c1"># 'fun': meets_sharpe</span>
|
||||
<span class="p">},</span> <span class="p">{</span>
|
||||
<span class="s">'type'</span><span class="p">:</span> <span class="s">'ineq'</span><span class="p">,</span>
|
||||
<span class="s">'fun'</span><span class="p">:</span> <span class="n">meets_dd</span>
|
||||
<span class="s1">'type'</span><span class="p">:</span> <span class="s1">'ineq'</span><span class="p">,</span>
|
||||
<span class="s1">'fun'</span><span class="p">:</span> <span class="n">meets_dd</span>
|
||||
<span class="p">}</span>
|
||||
<span class="p">]</span>
|
||||
|
||||
<span class="n">optimal</span> <span class="o">=</span> <span class="n">minimize</span><span class="p">(</span><span class="n">objective</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">bounds</span><span class="o">=</span><span class="n">bounds</span><span class="p">,</span> <span class="n">constraints</span><span class="o">=</span><span class="n">constraints</span><span class="p">,</span>
|
||||
<span class="n">options</span><span class="o">=</span><span class="p">{</span><span class="s">'maxiter'</span><span class="p">:</span> <span class="mi">500</span><span class="p">})</span>
|
||||
<span class="n">options</span><span class="o">=</span><span class="p">{</span><span class="s1">'maxiter'</span><span class="p">:</span> <span class="mi">500</span><span class="p">})</span>
|
||||
|
||||
<span class="c"># Optimization time!</span>
|
||||
<span class="c1"># Optimization time!</span>
|
||||
<span class="n">display</span><span class="p">(</span><span class="n">optimal</span><span class="o">.</span><span class="n">message</span><span class="p">)</span>
|
||||
|
||||
<span class="n">display</span><span class="p">(</span><span class="s">"Holdings: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">zip</span><span class="p">(</span><span class="n">tickers</span><span class="p">,</span> <span class="n">optimal</span><span class="o">.</span><span class="n">x</span><span class="p">))))</span>
|
||||
<span class="n">display</span><span class="p">(</span><span class="s2">"Holdings: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">zip</span><span class="p">(</span><span class="n">tickers</span><span class="p">,</span> <span class="n">optimal</span><span class="o">.</span><span class="n">x</span><span class="p">))))</span>
|
||||
|
||||
<span class="n">expected_return</span> <span class="o">=</span> <span class="n">optimal</span><span class="o">.</span><span class="n">fun</span> <span class="o">*</span> <span class="o">-</span><span class="mi">100</span> <span class="c"># multiply by -100 to scale, and compensate for minimizing</span>
|
||||
<span class="n">display</span><span class="p">(</span><span class="s">"Expected Return: {:.3f}%"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">expected_return</span><span class="p">))</span>
|
||||
<span class="n">expected_return</span> <span class="o">=</span> <span class="n">optimal</span><span class="o">.</span><span class="n">fun</span> <span class="o">*</span> <span class="o">-</span><span class="mi">100</span> <span class="c1"># multiply by -100 to scale, and compensate for minimizing</span>
|
||||
<span class="n">display</span><span class="p">(</span><span class="s2">"Expected Return: {:.3f}%"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">expected_return</span><span class="p">))</span>
|
||||
|
||||
<span class="n">expected_drawdown</span> <span class="o">=</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">optimal</span><span class="o">.</span><span class="n">x</span><span class="p">)</span> <span class="o">*</span> <span class="n">dd_a</span><span class="p">)</span> <span class="o">/</span> <span class="nb">sum</span><span class="p">(</span><span class="nb">abs</span><span class="p">(</span><span class="n">optimal</span><span class="o">.</span><span class="n">x</span><span class="p">))</span> <span class="o">*</span> <span class="mi">100</span>
|
||||
<span class="n">display</span><span class="p">(</span><span class="s">"Expected Max Drawdown: {0:.2f}%"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">expected_drawdown</span><span class="p">))</span>
|
||||
<span class="n">display</span><span class="p">(</span><span class="s2">"Expected Max Drawdown: {0:.2f}%"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">expected_drawdown</span><span class="p">))</span>
|
||||
|
||||
<span class="c"># TODO: Calculate expected Sharpe</span>
|
||||
<span class="c1"># TODO: Calculate expected Sharpe</span>
|
||||
</pre></div>
|
||||
|
||||
</div>
|
||||
|
||||
Loading…
Reference in New Issue
Block a user