From 081d0fa0fe81c812fe4b1272d3444b9e8f18dfb6 Mon Sep 17 00:00:00 2001 From: Bradlee Speice Date: Sat, 2 Feb 2019 19:35:42 -0500 Subject: [PATCH] Add arrays and closures --- _drafts/understanding-allocations-in-rust.md | 145 +++++++++++++++++-- 1 file changed, 129 insertions(+), 16 deletions(-) diff --git a/_drafts/understanding-allocations-in-rust.md b/_drafts/understanding-allocations-in-rust.md index 3cdd50a..575ffbe 100644 --- a/_drafts/understanding-allocations-in-rust.md +++ b/_drafts/understanding-allocations-in-rust.md @@ -345,8 +345,7 @@ static MY_LOCK: RefCell = RefCell::new(0); ``` -- [Rust Playground](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=c76ef86e473d07117a1700e21fd45560) -It's likely that this will [change in the future](https://github.com/rust-lang/rfcs/blob/master/text/0911-const-fn.md) though, -so be on the lookout. +It's likely that this will [change in the future](https://github.com/rust-lang/rfcs/blob/master/text/0911-const-fn.md) though. Which leads well to the next point: static variable types must implement the [`Sync` marker](https://doc.rust-lang.org/std/marker/trait.Sync.html). @@ -442,8 +441,8 @@ is one) being the way to use heap memory. Old C++ has the [`new`](https://stacko keyword, though modern C++/C++11 is more complicated with [RAII](https://en.cppreference.com/w/cpp/language/raii). For Rust specifically, the principle is this: *stack allocation will be used for everything -that doesn't involve "smart pointers" and collections.* If we're interested in proving -it though, there are three things to watch for: +that doesn't involve "smart pointers" and collections.* If we're interested in dissecting it though, +there are three things we pay attention to: 1. Stack manipulation instructions (`push`, `pop`, and `add`/`sub` of the `rsp` register) indicate allocation of stack memory: @@ -459,7 +458,7 @@ it though, there are three things to watch for: -- [Compiler Explorer](https://godbolt.org/z/5WSgc9) 2. Tracking when exactly heap allocation calls happen is difficult. It's typically easier to - watch for `call core::ptr::drop_in_place`, and infer that a heap allocation happened + watch for `call core::ptr::real_drop_in_place`, and infer that a heap allocation happened in the recent past: ```rust pub fn heap_alloc(x: usize) -> usize { @@ -470,10 +469,10 @@ it though, there are three things to watch for: x } ``` - -- [Compiler Explorer](https://godbolt.org/z/epfgoQ) (`drop_in_place` happens on line 1317) + -- [Compiler Explorer](https://godbolt.org/z/epfgoQ) (`real_drop_in_place` happens on line 1317) Note: While the [`Drop` trait](https://doc.rust-lang.org/std/ops/trait.Drop.html) - is called for stack-allocated objects, the Rust standard library only defines `Drop` implementations - for types that involve heap allocation. + is [called for stack-allocated objects](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=87edf374d8983816eb3d8cfeac657b46), + the Rust standard library only defines `Drop` implementations for types that involve heap allocation. 3. If you don't want to inspect the assembly, use a custom allocator that's able to track and alert when heap allocations occur. As an unashamed plug, [qadapt](https://crates.io/crates/qadapt) @@ -481,13 +480,19 @@ it though, there are three things to watch for: With all that in mind, let's talk about situations in which we're guaranteed to use stack memory: -- Structs not wrapped by smart pointers are created on the stack. +- Structs are created on the stack. +- Function arguments are passed on the stack. - Enums and unions are stack-allocated. - [Arrays](https://doc.rust-lang.org/std/primitive.array.html) are always stack-allocated. - Using the [`#[inline]` attribute](https://doc.rust-lang.org/reference/attributes.html#inline-attribute) will not change the memory region used. +- Closures capture their arguments on the stack - Generics will use stack allocation, even with dynamic dispatch. +## Structs + + + ## Enums It's been a worry of mine that I'd manage to trigger a heap allocation because @@ -503,6 +508,121 @@ discriminator, the compiler can predict how much memory is used. If enums were sized as tightly as possible, heap allocations would be needed to handle the fact that enum variants were of dynamic size! +## Arrays + +The array type is guaranteed to be stack allocated, which is why the array size must +be declared. Interestingly enough, this can be used to cause safe Rust programs to crash: + +```rust +// 256 bytes +#[derive(Default)] +struct TwoFiftySix { + _a: [u64; 32] +} + +// 8 kilobytes +#[derive(Default)] +struct EightK { + _a: [TwoFiftySix; 32] +} + +// 256 kilobytes +#[derive(Default)] +struct TwoFiftySixK { + _a: [EightK; 32] +} + +// 8 megabytes - exceeds space typically provided for the stack, +// though the kernel can be instructed to allocate more. +// On Linux, you can check stack size using `ulimit -s` +#[derive(Default)] +struct EightM { + _a: [TwoFiftySixK; 32] +} + +fn main() { + // Because we already have things in stack memory + // (like the current function), allocating another + // eight megabytes of stack memory crashes the program + let _x = EightM::default(); +} +``` +-- [Rust Playground](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=137893e3ae05c2f32fe07d6f6f754709) + +There aren't any security implications of this (no memory corruption occurs, +just running out of memory), but it's good to note that the Rust compiler +won't move arrays into heap memory even if they can be reasonably expected +to overflow the stack. + +## **inline** attributes + +## Closures + +Rules for how anonymous functions capture their arguments are typically language-specific. +In Java, [Lambda Expressions](https://docs.oracle.com/javase/tutorial/java/javaOO/lambdaexpressions.html) +are actually objects created on the heap that capture local primitives by copying, and capture +local non-primitives as (`final`) references. +[Python](https://docs.python.org/3.7/reference/expressions.html#lambda) and +[JavaScript](https://javascriptweblog.wordpress.com/2010/10/25/understanding-javascript-closures/) +both bind *everything* by reference normally, but Python can also +[capture values](https://stackoverflow.com/a/235764/1454178) and JavaScript has +[Arrow functions](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Arrow_functions). + +In Rust, arguments to closures are the same as arguments to other functions; +closures are simply functions that don't have a declared name. Some weird ordering +of the stack may be required to handle them, but it's the compiler's responsiblity +to figure it out. + +Each example below has the same effect, but compile to very different programs. +In the simplest case, we immediately run a closure returned by another function. +Because we don't store a reference to the closure, the stack memory needed to +store the captured values is contiguous: + +```rust +fn my_func() -> impl FnOnce() { + let x = 24; + // Note that this closure in assembly looks exactly like + // any other function; you even use the `call` instruction + // to start running it. + move || { x; } +} + +pub fn immediate() { + my_func()(); + my_func()(); +} +``` +-- [Compiler Explorer](https://godbolt.org/z/mgJ2zl), 25 total assembly instructions + +If we store a reference to the bound closure though, the Rust compiler has to +work a bit harder to make sure everything is correctly laid out in stack memory: + +```rust +pub fn simple_reference() { + let x = my_func(); + let y = my_func(); + y(); + x(); +} +``` +-- [Compiler Explorer](https://godbolt.org/z/K_dj5n), 55 total assembly instructions + +In more complex cases, even things like variable order matter: + +```rust +pub fn complex() { + let x = my_func(); + let y = my_func(); + x(); + y(); +} +``` +-- [Compiler Explorer](https://godbolt.org/z/p37qFl), 70 total assembly instructions + +In every circumstance though, the compiler ensured that no heap allocations were necessary. + +## Generics + # A Heaping Helping: Rust and Dynamic Memory Opening question: How many allocations happen before `fn main()` is called? @@ -611,10 +731,3 @@ and [`String::new()`](https://doc.rust-lang.org/std/string/struct.String.html#me 1. Box<> getting inlined into stack allocations 2. Vec::push() === Vec::with_capacity() for fixed/predictable capacities 3. Inlining statics that don't change value - -# Appendix and Further Reading - -[embedonomicon]: https://docs.rust-embedded.org/embedonomicon/ -[libc]: CRATES.IO LINK -[winapi]: CRATES.IO LINK -[malloc]: MANPAGE LINK \ No newline at end of file