Deploy site
Lonami Exo totufals@hotmail.com
Sun, 28 Feb 2021 20:52:42 +0100
8 files changed,
639 insertions(+),
9 deletions(-)
M
blog/atom.xml
→
blog/atom.xml
@@ -4,9 +4,422 @@ <title>Lonami's Site - My Blog</title>
<link href="https://lonami.dev/blog/atom.xml" rel="self" type="application/atom+xml"/> <link href="https://lonami.dev/blog/"/> <generator uri="https://www.getzola.org/">Zola</generator> - <updated>2021-02-19T00:00:00+00:00</updated> + <updated>2021-02-22T00:00:00+00:00</updated> <id>https://lonami.dev/blog/atom.xml</id> <entry xml:lang="en"> + <title>Writing our own Cheat Engine: Floating points</title> + <published>2021-02-22T00:00:00+00:00</published> + <updated>2021-02-22T00:00:00+00:00</updated> + <link href="https://lonami.dev/blog/woce-4/" type="text/html"/> + <id>https://lonami.dev/blog/woce-4/</id> + <content type="html"><p>This is part 4 on the <em>Writing our own Cheat Engine</em> series:</p> +<ul> +<li><a href="/blog/woce-1">Part 1: Introduction</a> (start here if you're new to the series!)</li> +<li><a href="/blog/woce-2">Part 2: Exact Value scanning</a></li> +<li><a href="/blog/woce-3">Part 3: Unknown initial value</a></li> +<li>Part 4: Floating points</li> +</ul> +<p>In part 3 we did a fair amount of plumbing in order to support scan modes beyond the trivial &quot;exact value scan&quot;. As a result, we have abstracted away the <code>Scan</code>, <code>CandidateLocations</code> and <code>Value</code> types as a separate <code>enum</code> each. Scanning for changed memory regions in an opened process can now be achieved with three lines of code:</p> +<pre><code class="language-rust" data-lang="rust">let regions = process.memory_regions(); +let first_scan = process.scan_regions(&amp;regions, Scan::InRange(0, 500)); +let second_scan = process.rescan_regions(&amp;first_scan, Scan::DecreasedBy(7)); +</code></pre> +<p>How's that for programmability? No need to fire up Cheat Engine's GUI anymore!</p> +<p>The <code>first_scan</code> in the example above remembers all the found <code>Value</code> within the range specified by <code>Scan</code>. Up until now, we have only worked with <code>i32</code>, so that's the type the scans expect and what they work with.</p> +<p>Now it's time to introduce support for different types, like <code>f32</code>, <code>i64</code>, or even more atypical ones, like arbitrary sequences of bytes (think of strings) or even numbers in big-endian.</p> +<p>Tighten your belt, because this post is quite the ride. Let's get right into it!</p> +<h2 id="floating-points">Floating points</h2> +<details open><summary>Cheat Engine Tutorial: Step 4</summary> +<blockquote> +<p>In the previous tutorial we used bytes to scan, but some games store information in so called 'floating point' notations. +(probably to prevent simple memory scanners from finding it the easy way). A floating point is a value with some digits behind the point. (like 5.12 or 11321.1)</p> +<p>Below you see your health and ammo. Both are stored as Floating point notations, but health is stored as a float and ammo is stored as a double. +Click on hit me to lose some health, and on shoot to decrease your ammo with 0.5</p> +<p>You have to set BOTH values to 5000 or higher to proceed.</p> +<p>Exact value scan will work fine here, but you may want to experiment with other types too.</p> +<p>Hint: It is recommended to disable &quot;Fast Scan&quot; for type double</p> +</blockquote> +</details> +<h2 id="generic-values">Generic values</h2> +<p>The <code>Value</code> enumeration holds scanned values, and is currently hardcoded to store <code>i32</code>. The <code>Scan</code> type also holds a value, the value we want to scan for. Changing it to support other types is trivial:</p> +<pre><code class="language-rust" data-lang="rust">pub enum Scan&lt;T&gt; { + Exact(T), + Unknown, + Decreased, + // ...other variants... +} + +pub enum Value&lt;T&gt; { + Exact(T), + AnyWithin(Vec&lt;u8&gt;), +} +</code></pre> +<p><code>AnyWithin</code> is the raw memory, and <code>T</code> can be interpreted from any sequence of bytes thanks to our friend <a href="https://doc.rust-lang.org/stable/std/mem/fn.transmute.html"><code>mem::transmute</code></a>. This change alone is enough to store an arbitrary <code>T</code>! So we're done now? Not really, no.</p> +<p>First of all, we need to update all the places where <code>Scan</code> or <code>Value</code> are used. Our first stop is the scanned <code>Region</code>, which holds the found <code>Value</code>:</p> +<pre><code class="language-rust" data-lang="rust">pub struct Region&lt;T&gt; { + pub info: MEMORY_BASIC_INFORMATION, + pub locations: CandidateLocations, + pub value: Value&lt;T&gt;, +} +</code></pre> +<p>Then, we need to update everywhere <code>Region</code> is used, and on and on… All in all this process is just repeating <code>cargo check</code>, letting the compiler vent on you, and taking good care of it by fixing the errors. It's quite reassuring to know you will not miss a single place. Thank you, compiler!</p> +<p>But wait, how could scanning for a decreased value work for any <code>T</code>? The type is not <code>Ord</code>, we should add some trait bounds. And also, what happens if the type is not <code>Copy</code>? It could implement <code>Drop</code><sup class="footnote-reference"><a href="#1">1</a></sup>, and we will be transmuting from raw bytes, which would trigger the <code>Drop</code> implementation when we're done with the value! Not memory safe at all! And how could we possibly cast raw memory to the type without knowing its siz– oh nevermind, <a href="https://doc.rust-lang.org/stable/std/marker/trait.Sized.html"><code>T</code> is already <code>Sized</code> by default</a>. But anyway, we need the other bounds.</p> +<p>In order to not repeat ourselves, we will implement a new <code>trait</code>, let's say <code>Scannable</code>, which requires all other bounds:</p> +<pre><code class="language-rust" data-lang="rust">pub trait Scannable: Copy + PartialEq + PartialOrd {} + +impl&lt;T: Copy + PartialEq + PartialOrd&gt; Scannable for T {} +</code></pre> +<p>And fix our definitions:</p> +<pre><code class="language-rust" data-lang="rust">pub enum Scan&lt;T: Scannable&gt; { ... } +pub enum Value&lt;T: Scannable&gt; { ... } +pub struct Region&lt;T: Scannable&gt; { ... } + +// ...and the many other places referring to T +</code></pre> +<p>Every type which is <code>Copy</code>, <code>PartialEq</code> and <code>PartialOrd</code> can be scanned over<sup class="footnote-reference"><a href="#2">2</a></sup>, because we <code>impl Scan for T</code> where the bounds are met. Unfortunately, we cannot require <code>Eq</code> or <code>Ord</code> because the floating point types do not implement it.</p> +<h2 id="transmuting-memory">Transmuting memory</h2> +<p>Also known as reinterpreting a bunch of bytes as something else, or perhaps it stands for &quot;summoning the demon&quot;:</p> +<blockquote> +<p><code>transmute</code> is <strong>incredibly</strong> unsafe. There are a vast number of ways to cause <a href="https://doc.rust-lang.org/stable/reference/behavior-considered-undefined.html">undefined behavior</a> with this function. <code>transmute</code> should be the absolute last resort.</p> +</blockquote> +<p>Types like <code>i32</code> define methods such as <a href="https://doc.rust-lang.org/stable/std/primitive.i32.html#method.from_ne_bytes"><code>from_ne_bytes</code></a> and <a href="https://doc.rust-lang.org/stable/std/primitive.i32.html#method.to_ne_bytes"><code>to_ne_bytes</code></a> which convert raw bytes from and into its native representation. This is all really nice, but unfortunately, there's no standard trait in the Rust's standard library to &quot;interpret a type <code>T</code> as the byte sequence of its native representation&quot;. <code>transmute</code>, however, does exist, and similar to any other <code>unsafe</code> function, it's safe to call <strong>as long as we respect its invariants</strong>. What are these invariants<sup class="footnote-reference"><a href="#3">3</a></sup>?</p> +<blockquote> +<p>Both types must have the same size</p> +</blockquote> +<p>Okay, we can just assert that the window length matches the type's length. What else?</p> +<blockquote> +<p>Neither the original, nor the result, may be an <a href="https://doc.rust-lang.org/nomicon/what-unsafe-does.html">invalid value</a>.</p> +</blockquote> +<p>What's an invalid value?</p> +<blockquote> +<ul> +<li>a <code>bool</code> that isn't 0 or 1</li> +<li>an <code>enum</code> with an invalid discriminant</li> +<li>a null <code>fn</code> pointer</li> +<li>a <code>char</code> outside the ranges [0x0, 0xD7FF] and [0xE000, 0x10FFFF]</li> +<li>a <code>!</code> (all values are invalid for this type)</li> +<li>an integer (<code>i*</code>/<code>u*</code>), floating point value (<code>f*</code>), or raw pointer read from uninitialized memory, or uninitialized memory in a <code>str</code>.</li> +<li>a reference/<code>Box</code> that is dangling, unaligned, or points to an invalid value.</li> +<li>a wide reference, <code>Box</code>, or raw pointer that has invalid metadata: +<ul> +<li><code>dyn Trait</code> metadata is invalid if it is not a pointer to a vtable for <code>Trait</code> that matches the actual dynamic trait the pointer or reference points to</li> +<li>slice metadata is invalid if the length is not a valid <code>usize</code> (i.e., it must not be read from uninitialized memory)</li> +</ul> +</li> +<li>a type with custom invalid values that is one of those values, such as a <code>NonNull</code> that is null. (Requesting custom invalid values is an unstable feature, but some stable libstd types, like <code>NonNull</code>, make use of it.)</li> +</ul> +</blockquote> +<p>Okay, that's actually an awful lot. Types like <code>bool</code> implement all the trait bounds we defined, and it would be insta-UB to ever try to cast them from arbitrary bytes. The same goes for <code>char</code>, and all <code>enum</code> are out of our control, too. At least we're safe on the &quot;memory is initialized&quot; front.</p> +<p>Dang it, I really wanted to use <code>transmute</code>! But if we were to use it for arbitrary types, it would trigger undefined behaviour sooner than later.</p> +<p>We have several options here:</p> +<ul> +<li>Make it an <code>unsafe trait</code>. Implementors will be responsible for ensuring that the type they're implementing it for can be safely transmuted from and into.</li> +<li><a href="https://rust-lang.github.io/api-guidelines/future-proofing.html">Seal the <code>trait</code></a> and implement it only for types we know are safe<sup class="footnote-reference"><a href="#4">4</a></sup>, like <code>i32</code>.</li> +<li>Add methods to the <code>trait</code> definition that do the conversion of the type into its native representation.</li> +</ul> +<p>We will go with the first option<sup class="footnote-reference"><a href="#5">5</a></sup>, because I really want to use <code>transmute</code>, and I want users to be able to implement the trait on their own types.</p> +<p>In any case, we need to change our <code>impl</code> to something more specific, in order to prevent it from automatically implementing the trait for types for which their memory representation has invalid values. So we get rid of this:</p> +<pre><code class="language-rust" data-lang="rust">pub trait Scannable: Copy + PartialEq + PartialOrd {} + +impl&lt;T: Copy + PartialEq + PartialOrd&gt; Scannable for T {} +</code></pre> +<p>And replace it with this:</p> +<pre><code class="language-rust" data-lang="rust">pub unsafe trait Scannable: Copy + PartialEq + PartialOrd {} + +macro_rules! impl_many { + ( unsafe impl $trait:tt for $( $ty:ty ),* ) =&gt; { + $( unsafe impl $trait for $ty {} )* + }; +} + +// SAFETY: all these types respect `Scannable` invariants. +impl_many!(unsafe impl Scannable for i8, u8, i16, u16, i32, u32, i64, u64, f32, f64); +</code></pre> +<p>Making a small macro for things like these is super useful. You could of course write <code>unsafe impl Scannable for T</code> for all ten <code>T</code> as well, but that introduces even more <code>unsafe</code> to read. Last but not least, let's replace the hardcoded <code>i32::from_ne_bytes</code> and <code>i32::to_ne_bytes</code> with <code>mem::transmute</code>.</p> +<p>All the <code>windows(4)</code> need to be replaced with <code>windows(mem::size_of::&lt;T&gt;())</code> because the size may no longer be <code>4</code>. All the <code>i32::from_ne_bytes(...)</code> need to be replaced with <code>mem::transmute::&lt;_, T&gt;(...)</code>. We explicitly write out <code>T</code> to make sure the compiler doesn't accidentally infer something we didn't intend.</p> +<p>And… it doesn't work at all. We're working with byte slices of arbitrary length. We cannot transmute a <code>&amp;[]</code> type, which is 16 bytes (8 for the pointer and 8 for the length), to our <code>T</code>. My plan to use transmute can't possibly work here. Sigh.</p> +<h2 id="not-quite-transmuting-memory">Not quite transmuting memory</h2> +<p>Okay, we can't transmute, because we don't have a sized value, we only have a slice of bytes pointing somewhere else. What we <em>could</em> do is reinterpret the pointer to those bytes as a different type, and then dereference it! This is still a form of &quot;transmutation&quot;, just without using <code>transmute</code>.</p> +<pre><code class="language-rust" data-lang="rust">let value = unsafe { *(window.as_ptr() as *const T) }; +</code></pre> +<p>Woop! You can compile this and test it out on the step 2 and 3 of the tutorial, using <code>i32</code>, and it will still work! Something troubles me, though. Can you see what it is?</p> +<p>When we talked about invalid values, it had a note about unaligned references:</p> +<blockquote> +<p>a reference/<code>Box</code> that is dangling, unaligned, or points to an invalid value.</p> +</blockquote> +<p>Our <code>window</code> is essentially a reference to <code>T</code>. The only difference is we're working at the pointer level, but they're pretty much references. Let's see what the documentation for <a href="https://doc.rust-lang.org/std/primitive.pointer.html"><code>pointer</code></a> has to say as well, since we're dereferencing pointers:</p> +<blockquote> +<p>when a raw pointer is dereferenced (using the <code>*</code> operator), it must be non-null and aligned.</p> +</blockquote> +<p>It must be aligned. The only reason why our data is aligned is because we are also performing a &quot;fast scan&quot;, so we only look at aligned locations. This is a time bomb waiting to blow up. Is there any other way to <a href="https://doc.rust-lang.org/std/ptr/fn.read.html"><code>read</code></a> from a pointer which is safer?</p> +<blockquote> +<p><code>src</code> must be properly aligned. Use <a href="https://doc.rust-lang.org/std/ptr/fn.read_unaligned.html"><code>read_unaligned</code></a> if this is not the case.</p> +</blockquote> +<p>Bingo! Both <code>read</code> and <code>read_unaligned</code>, unlike dereferencing the pointer, will perform a copy, but if it can make the code less prone to blowing up, I'll take it<sup class="footnote-reference"><a href="#6">6</a></sup>. Let's change the code one more time:</p> +<pre><code class="language-rust" data-lang="rust">let current = unsafe { window.as_ptr().cast::&lt;T&gt;().read_unaligned() }; +</code></pre> +<p>I prefer to avoid type annotations in variables where possible, which is why I use the <a href="https://www.reddit.com/r/rust/comments/3fimgp/why_double_colon_rather_that_dot/ctozkd0/">turbofish</a> so often. You can get rid of the cast and use a type annotation instead, but make sure the type is known, otherwise it will think it's <code>u8</code> because <code>window</code> is a <code>&amp;[u8]</code>.</p> +<p>Now, this is all cool and good. You can replace <code>i32</code> with <code>f32</code> for <code>T</code> and you'll be able to get halfway done with the step 4 of Cheat Engine's tutorial. Unfortunately, as it is, this code is not enough to complete step 4 with exact scans<sup class="footnote-reference"><a href="#7">7</a></sup>. You see, comparing floating point values is not as simple as checking for bitwise equality. We were actually really lucky that the <code>f32</code> part works! But the values in the <code>f64</code> part are not as precise as our inputs, so our exact scan fails.</p> +<p>Using a fixed type parameter is pretty limiting as well. On the one hand, it is nice that, if you scan for <code>i32</code>, the compiler statically guarantees that subsequent scans will also happen on <code>i32</code> and thus be compatible. On the other, this requires us to know the type at compile time, which for an interactive program, is not possible. While we <em>could</em> create different methods for each supported type and, at runtime, decide to which we should jump, I am not satisfied with that solution. It also means we can't switch from scanning an <code>u32</code> to an <code>i32</code>, for whatever reason.</p> +<p>So we need to work around this once more.</p> +<h2 id="rethinking-the-scans">Rethinking the scans</h2> +<p>What does our scanning function need, really? It needs a way to compare two chunks of memory as being equal or not (as we have seen, this isn't trivial with types such as floating point numbers) and, for other types of scans, it needs to be able to produce an ordering, or calculate a difference.</p> +<p>Instead of having a our trait require the bounds <code>PartialEq</code> and <code>PartialOrd</code>, we can define our own methods to compare <code>Self</code> with <code>&amp;[u8]</code>. It still should be <code>Clone</code>, so we can pass it around without worrying about lifetimes:</p> +<pre><code class="language-rust" data-lang="rust">// Callers must `assert_eq!(memory.len(), mem::size_of::&lt;Self&gt;())`. +unsafe fn eq(&amp;self, memory: &amp;[u8]) -&gt; bool; +unsafe fn cmp(&amp;self, memory: &amp;[u8]) -&gt; Ordering; +</code></pre> +<p>This can be trivially implemented for all integer types:</p> +<pre><code class="language-rust" data-lang="rust">macro_rules! impl_scannable_for_int { + ( $( $ty:ty ),* ) =&gt; { + $( + // SAFETY: caller is responsible to `assert_eq!(memory.len(), mem::size_of::&lt;T&gt;())` + impl Scannable for $ty { + unsafe fn eq(&amp;self, memory: &amp;[u8]) -&gt; bool { + let other = unsafe { memory.as_ptr().cast::&lt;$ty&gt;().read_unaligned() }; + *self == other + } + + unsafe fn cmp(&amp;self, memory: &amp;[u8]) -&gt; Ordering { + let other = unsafe { memory.as_ptr().cast::&lt;$ty&gt;().read_unaligned() }; + &lt;$ty as Ord&gt;::cmp(self, &amp;other) + } + } + )* + }; +} + +impl_scannable_for_int!(i8, u8, i16, u16, i32, u32, i64, u64); +</code></pre> +<p>The funny <code>&lt;$ty as Ord&gt;</code> is because I decided to call the method <code>Scannable::cmp</code>, so I have to disambiguate between it and <code>Ord::cmp</code>. We can go ahead and update the code using <code>Scannable</code> to use these new functions instead.</p> +<p>Now, you may have noticed I only implemented it for the integer types. That's because floats need some extra care. Unfortunately, floating point types do not have any form of &quot;precision&quot; embedded in them, so we can't accurately say &quot;compare these floats to the precision level the user specified&quot;. What we can do, however, is drop a few bits from the mantissa, so &quot;relatively close&quot; quantities are considered equal. It's definitely not as good as comparing floats to the user's precision, but it will get the job done.</p> +<p>I'm going to arbitrarily say that we are okay comparing with &quot;half&quot; the precision. We can achieve that by masking half of the bits from the mantissa to zero:</p> +<pre><code class="language-rust" data-lang="rust"> +macro_rules! impl_scannable_for_float { + ( $( $ty:ty : $int_ty:ty ),* ) =&gt; { + $( + #[allow(unused_unsafe)] // mind you, it is necessary + impl Scannable for $ty { + unsafe fn eq(&amp;self, memory: &amp;[u8]) -&gt; bool { + const MASK: $int_ty = !((1 &lt;&lt; (&lt;$ty&gt;::MANTISSA_DIGITS / 2)) - 1); + + // SAFETY: caller is responsible to `assert_eq!(memory.len(), mem::size_of::&lt;T&gt;())` + let other = unsafe { memory.as_ptr().cast::&lt;$ty&gt;().read_unaligned() }; + let left = &lt;$ty&gt;::from_bits(self.to_bits() &amp; MASK); + let right = &lt;$ty&gt;::from_bits(other.to_bits() &amp; MASK); + left == right + } + + ... + } + )* + }; +} + +impl_scannable_for_float!(f32: u32, f64: u64); +</code></pre> +<p>You may be wondering what's up with that weird <code>MASK</code>. Let's visualize it with a <a href="https://en.wikipedia.org/wiki/Bfloat16_floating-point_format"><code>f16</code></a>. This type has 16 bits, 1 for sign, 5 for exponent, and 10 for the mantissa:</p> +<pre><code>S EEEEE MMMMMMMMMM +</code></pre> +<p>If we substitute the constant with the numeric value and operate:</p> +<pre><code class="language-rust" data-lang="rust">!((1 &lt;&lt; (10 / 2)) - 1) +!((1 &lt;&lt; 5) - 1) +!(0b00000000_00100000 - 1) +!(0b00000000_00011111) +0b11111111_11100000 +</code></pre> +<p>So effectively, half of the mantisssa bit will be masked to 0. For the <code>f16</code> example, this makes us lose 5 bits of precision. Comparing two floating point values with their last five bits truncated is equivalent to checking if they are &quot;roughly equal&quot;!</p> +<p>When Cheat Engine scans for floating point values, several additional settings show, and one such option is &quot;truncated&quot;. I do not know if it behaves like this, but it might.</p> +<p>Let's try this out:</p> +<pre><code class="language-rust" data-lang="rust">#[test] +fn f32_roughly_eq() { + let left = 0.25f32; + let right = 0.25000123f32; + let memory = unsafe { mem::transmute::&lt;_, [u8; 4]&gt;(right) }; + assert_ne!(left, right); + assert!(unsafe { Scannable::eq(&amp;left, &amp;memory) }); +} +</code></pre> +<pre><code>&gt;cargo test f32_roughly_eq + +running 1 test +test scan::candidate_location_tests::f32_roughly_eq ... ok +</code></pre> +<p>Huzzah! The <code>assert_ne!</code> makes sure that a normal comparision would fail, and then we <code>assert!</code> that our custom one passes the test. When the user performs an exact scan, the code will be more tolerant to the user's less precise inputs, which overall should result in a nicer experience.</p> +<h2 id="dynamically-sized-scans">Dynamically sized scans</h2> +<p>The second problem we need to solve is the possibility of the size not being known at compile time<sup class="footnote-reference"><a href="#8">8</a></sup>. While we can go as far as scanning over strings of a known length, this is rather limiting, because we need to know the length at compile time<sup class="footnote-reference"><a href="#9">9</a></sup>. Heap allocated objects are another problem, because we don't want to compare the memory representation of the stack object, but likely the memory where they point to (such as <code>String</code>).</p> +<p>Instead of using <code>mem::size_of</code>, we can add a new method to our <code>Scannable</code>, <code>size</code>, which will tell us the size required of the memory view we're comparing against:</p> +<pre><code class="language-rust" data-lang="rust">unsafe impl Scannable { + ... + + fn size(&amp;self) -&gt; usize; +} +</code></pre> +<p>It is <code>unsafe</code> to implement, because we are relying on the returned value to be truthful and unchanging. It should be safe to call, because it cannot have any invariants. Unfortunately, signaling &quot;unsafe to implement&quot; is done by marking the entire trait as <code>unsafe</code>, since &quot;unsafe to call&quot; is reserved for <code>unsafe fn</code>, and even though the rest of methods are not necessarily unsafe to implement, they're treated as such.</p> +<p>At the moment, <code>Scannable</code> cannot be made into a trait object because it is <a href="https://doc.rust-lang.org/stable/error-index.html#E0038">not object safe</a>. This is caused by the <code>Clone</code> requirement on all <code>Scannable</code> object, which in turn needs the types to be <code>Sized</code> because <code>clone</code> returns <code>Self</code>. Because of this, the size must be known.</p> +<p>However, we <em>can</em> move the <code>Clone</code> requirement to the methods that need it! This way, <code>Scannable</code> can remain object safe, enabling us to do the following:</p> +<pre><code class="language-rust" data-lang="rust">unsafe impl&lt;T: AsRef&lt;dyn Scannable&gt; + AsMut&lt;dyn Scannable&gt;&gt; Scannable for T { + unsafe fn eq(&amp;self, memory: &amp;[u8]) -&gt; bool { + self.as_ref().eq(memory) + } + + unsafe fn cmp(&amp;self, memory: &amp;[u8]) -&gt; Ordering { + self.as_ref().cmp(memory) + } + + fn mem_view(&amp;self) -&gt; &amp;[u8] { + self.as_ref().mem_view() + } + + fn size(&amp;self) -&gt; usize { + self.as_ref().size() + } +} +</code></pre> +<p>Any type which can be interpreted as a reference to <code>Scannable</code> is also a scannable! This enables us to perform scans over <code>Box&lt;dyn i32&gt;</code>, where the type is known at runtime! Or rather, it would, if <code>Box&lt;dyn T&gt;</code> implemented <code>Clone</code>, which it can't<sup class="footnote-reference"><a href="#10">10</a></sup> because that's what prompted this entire issue. Dang it! I can't catch a breath today!</p> +<p>Okay, let's step back. Why did we need our scannables to be clone in the first place? When we perform exact scans, we store the original value in the region, which we don't own, so we clone it. But what if we <em>did</em> own the value? Instead of taking the <code>Scan</code> by reference, which holds <code>T: Scannable</code>, we could take it by value. If we get rid of all the <code>Clone</code> bounds and update <code>Scan::run</code> to take <code>self</code>, along with updating all the things that take a <code>Region</code> to take them by value as well, it should all work out.</p> +<p>But it does not. If we take <code>Scan</code> by value, with it not being <code>Clone</code>, we simply can't use it to scan over multiple regions. After the first region, we have lost the <code>Scan</code>.</p> +<p>Let's take a second step back. We are scanning memory, and we want to compare memory, but we want to treat the memory with different semantics (for example, if we treat it as <code>f32</code>, we want to check for rough equality). Instead of storing the <em>value</em> itself, we could store its <em>memory representation</em>, and when we compare memory representations, we can do so under certain semantics.</p> +<p>First off, let's revert getting rid of all <code>Clone</code>. Wherever we stored a <code>T</code>, we will now store a <code>Vec&lt;u8&gt;</code>. We will still use a type parameter to represent the &quot;implementations of <code>Scannable</code>&quot;. For this to work, our definitions need to use <code>T</code> somewhere, or else the compiler refuses to compile the code with error <a href="https://doc.rust-lang.org/stable/error-index.html#E0392">E0392</a>. For this, I will stick a <a href="https://doc.rust-lang.org/stable/std/marker/struct.PhantomData.html"><code>PhantomData</code></a> in the <code>Exact</code> variant. It's a bit pointless to include it in all variants, and <code>Exact</code> seems the most appropriated:</p> +<pre><code class="language-rust" data-lang="rust">pub enum Scan&lt;T: Scannable&gt; { + Exact(Vec&lt;u8&gt;, PhantomData&lt;T&gt;), + Unknown, + ... +} +</code></pre> +<p>This keeps in line with <code>Value</code>:</p> +<pre><code class="language-rust" data-lang="rust">pub enum Value&lt;T: Scannable&gt; { + Exact(Vec&lt;u8&gt;, PhantomData&lt;T&gt;), + ... +} +</code></pre> +<p>Our <code>Scannable</code> will no longer work on <code>T</code> and <code>&amp;[u8]</code>. Instead, it will work on two <code>&amp;[u8]</code>. We will also need a way to interpret a <code>T</code> as <code>&amp;[u8]</code>, which we can achieve with a new method, <code>mem_view</code>. This method interprets the raw memory representation of <code>self</code> as its raw bytes. It also lets us get rid of <code>size</code>, because we can simply do <code>mem_view().len()</code>. It's still <code>unsafe</code> to implement, because it should return the same length every time:</p> +<pre><code class="language-rust" data-lang="rust">pub unsafe trait Scannable { + // Callers must `assert_eq!(left.len(), right.len(), self.mem_view().len())`. + unsafe fn eq(left: &amp;[u8], right: &amp;[u8]) -&gt; bool; + unsafe fn cmp(left: &amp;[u8], right: &amp;[u8]) -&gt; Ordering; + fn mem_view(&amp;self) -&gt; &amp;[u8]; +} +</code></pre> +<p>But now we can't use it in trait object, so the following no longer works:</p> +<pre><code class="language-rust" data-lang="rust">unsafe impl&lt;T: AsRef&lt;dyn Scannable&gt; + AsMut&lt;dyn Scannable&gt;&gt; Scannable for T { + ... +} +</code></pre> +<p>Ugh! Well, to be fair, we no longer have a &quot;scannable&quot; at this point. It's more like a scan mode that tells us how memory should be compared according to a certain type. Let's split the trait into two: one for the scan mode, and other for &quot;things which are scannable&quot;:</p> +<pre><code class="language-rust" data-lang="rust">pub trait ScanMode { + unsafe fn eq(left: &amp;[u8], right: &amp;[u8]) -&gt; bool; + unsafe fn cmp(left: &amp;[u8], right: &amp;[u8]) -&gt; Ordering; +} + +pub unsafe trait Scannable { + type Mode: ScanMode; + + fn mem_view(&amp;self) -&gt; &amp;[u8]; +} +</code></pre> +<p>Note that we have an associated <code>type Mode</code> which contains the corresponding <code>ScanMode</code>. If we used a trait bound such as <code>Scannable: ScanMode</code>, we'd be back to square one: it would inherit the method definitions that don't use <code>&amp;self</code> and thus cannot be used as trait objects.</p> +<p>With these changes, it is possible to implement <code>Scannable</code> for any <code>dyn Scannable</code>:</p> +<pre><code class="language-rust" data-lang="rust">unsafe impl&lt;T: ScanMode + AsRef&lt;dyn Scannable&lt;Mode = Self&gt;&gt;&gt; Scannable for T { + type Mode = Self; + + fn mem_view(&amp;self) -&gt; &amp;[u8] { + self.as_ref().mem_view() + } +} +</code></pre> +<p>We do have to adjust a few places of the code to account for both <code>Scannable</code> and <code>ScanMode</code>, but all in all, it's pretty straightforward. Things like <code>Value</code> don't need to store the <code>Scannable</code> anymore, just a <code>Vec&lt;u8&gt;</code>. It also doesn't need the <code>ScanMode</code>, because it's not going to be scanning anything on its own. This applies transitively to <code>Region</code> which was holding a <code>Value</code>.</p> +<p><code>Value</code> <em>does</em> need to be updated to store the size of the region we are scanning for, however, because we need that information when running a subsequent scan. For all <code>Scan</code> that don't have a explicit thing to scan for (like <code>Decreased</code>), the <code>size</code> also needs to be stored in them.</p> +<p>Despite all our efforts, we're still unable to return an <code>Scannable</code> chosen at runtime.</p> +<pre><code class="language-rust" data-lang="rust">fn prompt_user_for_scan() -&gt; Scan&lt;Box&lt;dyn Scannable&lt;Mode = ???&gt;&gt;&gt; { + todo!() +} +</code></pre> +<p>As far as I can tell, there's simply no way to specify that type. We want to return a type which is scannable, which has itself (which is also a <code>ScanMode</code>) as the corresponding mode. Even if we just tried to return the mode, we simply can't, because it's not object-safe. Is this the end of the road?</p> +<h2 id="specifying-the-scan-mode">Specifying the scan mode</h2> +<p>We need a way to pass an arbitrary scan mode to our <code>Scan</code>. This scan mode should go in tandem with <code>Scannable</code> types, because it would be unsafe otherwise. We've seen that using a type just doesn't cut it. What else can we do?</p> +<p>Using an enumeration is a no-go, because I want users to be able to extend it further. I also would like to avoid having to update the <code>enum</code> and all the matches every time I come up with a different type combination. And it could get pretty complicated if I ever built something dynamically, such as letting the user combine different scans in one pass.</p> +<p>So what if we make <code>Scannable</code> return a value that implements the functions we need?</p> +<pre><code class="language-rust" data-lang="rust">pub struct ScanMode { + eq: unsafe fn(left: &amp;[u8], right: &amp;[u8]) -&gt; bool, + cmp: unsafe fn(left: &amp;[u8], right: &amp;[u8]) -&gt; Ordering, +} +</code></pre> +<p>It's definitely… non-conventional. But hey, now we're left with the <code>Scannable</code> trait, which is object-safe, and does not have any type parameters!</p> +<pre><code class="language-rust" data-lang="rust">pub unsafe trait Scannable { + fn mem_view(&amp;self) -&gt; &amp;[u8]; + fn scan_mode(&amp;self) -&gt; ScanMode; +} +</code></pre> +<p>It is a bit weird, but defining local functions and using those in the returned value is a nice way to keep things properly scoped:</p> +<pre><code class="language-rust" data-lang="rust">macro_rules! impl_scannable_for_int { + ( $( $ty:ty ),* ) =&gt; { + $( + unsafe impl Scannable for $ty { + fn mem_view(&amp;self) -&gt; &amp;[u8] { + unsafe { std::slice::from_raw_parts(self as *const _ as *const u8, mem::size_of::&lt;$ty&gt;()) } + } + + fn scan_mode(&amp;self) -&gt; ScanMode { + unsafe fn eq(left: &amp;[u8], right: &amp;[u8]) -&gt; bool { + ... + } + + unsafe fn cmp(left: &amp;[u8], right: &amp;[u8]) -&gt; Ordering { + ... + } + + ScanMode { eq, cmp } + } + } + )* + }; +} +</code></pre> +<p>Our <code>Scan</code> needs to store the <code>Scannable</code> type, and not just the memory, once again. For variants that don't need any value, they can store the <code>ScanMode</code> and size instead.</p> +<p>Does this solution work? Yes! It's possible to return a <code>Box&lt;dyn Scannable&gt;</code> from a function, and underneath, it may be using any type which is <code>Scannable</code>. Is this the best solution? Well, that's hard to say. This is <em>one</em> of the possible solutions.</p> +<p>We have been going around in circles for quite some time now, so I'll leave it there. It's a solution, which may not be pretty, but it works. With these changes, the code is capable of completing all of the steps in the Cheat Engine tutorial up until point!</p> +<h2 id="finale">Finale</h2> +<p>If there's one lesson to learn from this post, it's that there is often no single correct solution to a problem. We could have approached the scan types in many, many ways (and we tried quite a few!), but in the end, choosing one option or the other comes down to your (sometimes self-imposed) requirements.</p> +<p>You may <a href="https://github.com/lonami/memo">obtain the code for this post</a> over at my GitHub. You can run <code>git checkout step4</code> after cloning the repository to get the right version of the code. The code has gone through a lot of iterations, and I'd still like to polish it a bit more, so it might slightly differ from the code presented in this entry.</p> +<p>If you feel adventurous, Cheat Engine has different options for scanning floating point types: &quot;rounded (default)&quot;, &quot;rounded (extreme)&quot;, and truncated. Optionally, it can scan for &quot;simple values only&quot;. You could go ahead and toy around with these!</p> +<p>We didn't touch on types with different lengths, such as strings. You could support UTF-8, UTF-16, or arbitrary byte sequences. This post also didn't cover scanning for multiple things at once, known as &quot;groupscan commands&quot;, although from what I can tell, these are just a nice way to scan for arbitrary byte sequences.</p> +<p>We also didn't look into supporting different the same scan with different alignments. All these things may be worth exploring depending on your requirements. You could even get rid of such genericity and go with something way simpler. Supporting <code>i32</code>, <code>f32</code> and <code>f64</code> is enough to complete the Cheat Engine tutorial. But I wanted something more powerful, although my solution currently can't scan for a sequence such as &quot;exact type, unknown, exact matching the unknown&quot;. So yeah.</p> +<p>In the next post, we'll tackle the fifth step of the tutorial: Code finder. Cheat Engine attaches its debugger to the process for this one, and then replaces the instruction that performs the write with a different no-op so that nothing is written anymore. This will be quite the challenge!</p> +<h3 id="footnotes">Footnotes</h3> +<div class="footnote-definition" id="1"><sup class="footnote-definition-label">1</sup> +<p><a href="https://doc.rust-lang.org/stable/std/ops/trait.Drop.html#copy-and-drop-are-exclusive"><code>Copy</code> and <code>Drop</code> are exclusive</a>. See also <a href="https://doc.rust-lang.org/stable/error-index.html#E0184">E0184</a>.</p> +</div> +<div class="footnote-definition" id="2"><sup class="footnote-definition-label">2</sup> +<p>If you added more scan types that require additional bounds, make sure to add them too. For example, the &quot;decreased by&quot; scan requires the type to <code>impl Sub</code>.</p> +</div> +<div class="footnote-definition" id="3"><sup class="footnote-definition-label">3</sup> +<p>This is a good time to remind you to read the documentation. It is of special importance when dealing with <code>unsafe</code> methods; I recommend reading it a couple times.</p> +</div> +<div class="footnote-definition" id="4"><sup class="footnote-definition-label">4</sup> +<p>Even with this option, it would not be a bad idea to make the trait <code>unsafe</code>.</p> +</div> +<div class="footnote-definition" id="5"><sup class="footnote-definition-label">5</sup> +<p>Not for long. As we will find out later, this approach has its limitations.</p> +</div> +<div class="footnote-definition" id="6"><sup class="footnote-definition-label">6</sup> +<p>We can still perform the pointer dereference when we know it's aligned. This would likely be an optimization, although it would definitely complicate the code more.</p> +</div> +<div class="footnote-definition" id="7"><sup class="footnote-definition-label">7</sup> +<p>It <em>would</em> work if you scanned for unknown values and then checked for decreased values repeatedly. But we can't just leave exact scan broken!</p> +</div> +<div class="footnote-definition" id="8"><sup class="footnote-definition-label">8</sup> +<p>Unfortunately, this makes some optimizations harder or even impossible to perform. Providing specialized functions for types where the size is known at compile time could be worth doing. Programming is all tradeoffs.</p> +</div> +<div class="footnote-definition" id="9"><sup class="footnote-definition-label">9</sup> +<p><a href="https://blog.rust-lang.org/2021/02/26/const-generics-mvp-beta.html">Rust 1.51</a>, which was not out at the time of writing, would make it a lot easier to allow scanning for fixed-length sequences of bytes, thanks to const generics.</p> +</div> +<div class="footnote-definition" id="10"><sup class="footnote-definition-label">10</sup> +<p>Workarounds do exist, such as <a href="https://crates.io/crates/dyn-clone">dtolnay's <code>dyn-clone</code></a>. But I would rather not go that route.</p> +</div> +</content> + </entry> + <entry xml:lang="en"> <title>Writing our own Cheat Engine: Unknown initial value</title> <published>2021-02-19T00:00:00+00:00</published> <updated>2021-02-19T00:00:00+00:00</updated>@@ -17,6 +430,7 @@ <ul>
<li><a href="/blog/woce-1">Part 1: Introduction</a> (start here if you're new to the series!)</li> <li><a href="/blog/woce-2">Part 2: Exact Value scanning</a></li> <li>Part 3: Unknown initial value</li> +<li><a href="/blog/woce-4">Part 4: Floating points</a></li> </ul> <p>In part 2 we left off with a bit of a cliff-hanger. Our little program is now able to scan for an exact value, remember the couple hundred addresses pointing to said value, and perform subsequent scans to narrow the list of addresses down until we're left with a handful of them.</p> <p>However, it is not always the case that you have an exact value to work with. The best you can do in these cases is guess what the software might be storing. For example, it could be a floating point for your current movement speed in a game, or an integer for your current health.</p>@@ -339,7 +753,7 @@ <p><code>Value</code> is very much like <code>CandidateLocations</code>, except that it stores a value to compare against and not an address. Here we can either have an exact value, or an older copy of the memory. Again, keeping a copy of the entire memory chunk when all we need is a handful of values is inefficient. You could keep a mapping from addresses to values if you don't have too many. Or you could shrink and fragment the copied memory in a more optimal way. There's a lot of room for improvement!</p>
<p>What if, despite all of the efforts above, we still don't have enough RAM to store all this information? The Cheat Engine Tutorial doesn't use a lot of memory, but as soon as you try scanning bigger programs, like games, you may find yourself needing several gigabytes worth of memory to remember all the found values in order to compare them in subsequent scans. You may even need to consider dumping all the regions to a file and read from it to run the comparisons. For example, running a scan for &quot;unknown value&quot; in Cheat Engine brings its memory up by the same amount of memory used by the process scanned (which makes sense), but as soon as I ran a scan for &quot;unchanged value&quot; over the misaligned values, Cheat Engine's disk usage skyrocketed to 1GB/s (!) for several seconds on my SSD. After it finished, memory usage went down to normal. It was very likely writing out all candidate locations to disk.</p> <h2 id="finale">Finale</h2> <p>There is a lot of things to learn from Cheat Engine just by observing its behaviour, and we're only scratching its surface.</p> -<p>In the next post, we'll tackle the fourth step of the tutorial: Floating points. So far, we have only been working with <code>i32</code> for simplicity. We will need to update our code to be able to account for different data types, which will make it easy to support other types like <code>i16</code>, <code>i64</code>, or even strings, represented as an arbitrary sequence of bytes.</p> +<p>In the <a href="/blog/woce-4">next post</a>, we'll tackle the fourth step of the tutorial: Floating points. So far, we have only been working with <code>i32</code> for simplicity. We will need to update our code to be able to account for different data types, which will make it easy to support other types like <code>i16</code>, <code>i64</code>, or even strings, represented as an arbitrary sequence of bytes.</p> <p>As usual, you can <a href="https://github.com/lonami/memo">obtain the code for this post</a> over at my GitHub. You can run <code>git checkout step3</code> after cloning the repository to get the right version of the code. This version is a bit cleaner than the one presented in the blog, and contains some of the things described in the <a href="https://lonami.dev/blog/woce-3/#going-beyond">Going beyond</a> section. Until next time!</p> <h3 id="footnotes">Footnotes</h3> <div class="footnote-definition" id="1"><sup class="footnote-definition-label">1</sup>@@ -370,6 +784,7 @@ <ul>
<li><a href="/blog/woce-1">Part 1: Introduction</a> (start here if you're new to the series!)</li> <li>Part 2: Exact Value scanning</li> <li><a href="/blog/woce-3">Part 3: Unknown initial value</a></li> +<li><a href="/blog/woce-4">Part 4: Floating points</a></li> </ul> <p>In the introduction, we spent a good deal of time enumerating all running processes just so we could find out the pid we cared about. With the pid now in our hands, we can do pretty much anything to its corresponding process.</p> <p>It's now time to read the process' memory and write to it. If our process was a single-player game, this would enable us to do things like setting a very high value on the player's current health pool, making us invincible. This technique will often not work for multi-player games, because the server likely knows your true current health (the most you could probably do is make the client render an incorrect value). However, if the server is crappy and it trusts the client, then you're still free to mess around with your current health.</p>@@ -747,7 +1162,7 @@ <p>Isn't that active <em>Next</em> button just beautiful?</p>
<h2 id="finale">Finale</h2> <p>This post somehow ended up being longer than part one, but look at what we've achieved! We completed a step of the Cheat Engine Tutorial <em>without using Cheat Engine</em>. Just pure Rust. Figuring out how a program works and reimplementing it yourself is a great way to learn what it's doing behind the scenes. And now that this code is yours, you can extend it as much as you like, without being constrained by Cheat Engine's UI. You can automate it as much as you want.</p> <p>And we're not even done. The current tutorial has nine steps, and three additional graphical levels.</p> -<p>In the next post, we'll tackle the third step of the tutorial: Unknown initial value. This will pose a challenge, because with just 2 MiB of memory, storing all the 4-byte aligned locations would require 524288 addresses (<code>usize</code>, 8 bytes). This adds up to twice as much memory as the original program (4 MiB), but that's not our main concern, having to perform over five hundred thousand API calls is!</p> +<p>In the <a href="/blog/woce-3">next post</a>, we'll tackle the third step of the tutorial: Unknown initial value. This will pose a challenge, because with just 2 MiB of memory, storing all the 4-byte aligned locations would require 524288 addresses (<code>usize</code>, 8 bytes). This adds up to twice as much memory as the original program (4 MiB), but that's not our main concern, having to perform over five hundred thousand API calls is!</p> <p>Remember that you can <a href="https://github.com/lonami/memo">obtain the code for this post</a> over at my GitHub. You can run <code>git checkout step2</code> after cloning the repository to get the right version of the code.</p> <h3 id="footnotes">Footnotes</h3> <div class="footnote-definition" id="1"><sup class="footnote-definition-label">1</sup>@@ -790,6 +1205,7 @@ <ul>
<li>Part 1: Introduction</li> <li><a href="/blog/woce-2">Part 2: Exact Value scanning</a></li> <li><a href="/blog/woce-3">Part 3: Unknown initial value</a></li> +<li><a href="/blog/woce-4">Part 4: Floating points</a></li> </ul> <p><a href="https://cheatengine.org/">Cheat Engine</a> is a tool designed to modify single player games and contains other useful tools within itself that enable its users to debug games or other applications. It comes with a memory scanner, (dis)assembler, inspection tools and a handful other things. In this series, we will be writing our own tiny Cheat Engine capable of solving all steps of the tutorial, and diving into how it all works underneath.</p> <p>Needless to say, we're doing this for private and educational purposes only. One has to make sure to not violate the EULA or ToS of the specific application we're attaching to. This series, much like cheatengine.org, does not condone the illegal use of the code shared.</p>
M
blog/index.html
→
blog/index.html
@@ -1,4 +1,4 @@
-<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=description content="Official Lonami's website"><meta name=viewport content="width=device-width, initial-scale=1.0, user-scalable=yes"><title> Lonami's Blog </title><link rel=stylesheet href=/style.css><body><article><nav class=sections><ul class=left><li><a href=/>lonami's site</a><li><a href=/blog class=selected>blog</a><li><a href=/golb>golb</a></ul><div class=right><a href=https://github.com/LonamiWebs><img src=/img/github.svg alt=github></a><a href=/blog/atom.xml><img src=/img/rss.svg alt=rss></a></div></nav><main><h1 class=title>My Blog</h1><p id=welcome onclick=pls_stop()>Welcome to my blog!<p>Here I occasionally post new entries, mostly tech related. Perhaps it's tips for a new game I'm playing, perhaps it has something to do with FFI, or perhaps I'm fighting the borrow checker (just kidding, I'm over that. Mostly).<hr><ul><li><a href=https://lonami.dev/blog/woce-3/>Writing our own Cheat Engine: Unknown initial value</a><span class=dim> [mod sw; 'windows, 'rust, 'hacking] </span><li><a href=https://lonami.dev/blog/woce-2/>Writing our own Cheat Engine: Exact Value scanning</a><span class=dim> [mod sw; 'windows, 'rust, 'hacking] </span><li><a href=https://lonami.dev/blog/woce-1/>Writing our own Cheat Engine: Introduction</a><span class=dim> [mod sw; 'windows, 'rust, 'hacking] </span><li><a href=https://lonami.dev/blog/university/>Data Mining, Warehousing and Information Retrieval</a><span class=dim> [mod algos; 'series, 'bigdata, 'databases] </span><li><a href=https://lonami.dev/blog/new-computer/>My new computer</a><span class=dim> [mod hw; 'showoff] </span><li><a href=https://lonami.dev/blog/tips-outpost/>Tips for Outpost</a><span class=dim> [mod games; 'tips] </span><li><a href=https://lonami.dev/blog/ctypes-and-windows/>Python ctypes and Windows</a><span class=dim> [mod sw; 'python, 'ffi, 'windows] </span><li><a href=https://lonami.dev/blog/pixel-dungeon/>Shattered Pixel Dungeon</a><span class=dim> [mod games; 'tips] </span><li><a href=https://lonami.dev/blog/installing-nixos-2/>Installing NixOS, Take 2</a><span class=dim> [mod sw; 'os, 'nixos] </span><li><a href=https://lonami.dev/blog/breaking-ror/>Breaking Risk of Rain</a><span class=dim> [mod games; 'tips] </span><li><a href=https://lonami.dev/blog/world-edit/>WorldEdit Commands</a><span class=dim> [mod games; 'minecraft, 'worldedit, 'tips] </span><li><a href=https://lonami.dev/blog/asyncio/>An Introduction to Asyncio</a><span class=dim> [mod sw; 'python, 'asyncio] </span><li><a href=https://lonami.dev/blog/posts/>Atemporal Blog Posts</a><span class=dim> [mod algos; 'algorithms, 'culture, 'debate, 'foodforthought, 'graphics, 'optimization] </span><li><a href=https://lonami.dev/blog/graphs/>Graphs</a><span class=dim> [mod algos; 'graphs] </span><li><a href=https://lonami.dev/blog/installing-nixos/>Installing NixOS</a><span class=dim> [mod sw; 'os, 'nixos] </span></ul><script> +<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=description content="Official Lonami's website"><meta name=viewport content="width=device-width, initial-scale=1.0, user-scalable=yes"><title> Lonami's Blog </title><link rel=stylesheet href=/style.css><body><article><nav class=sections><ul class=left><li><a href=/>lonami's site</a><li><a href=/blog class=selected>blog</a><li><a href=/golb>golb</a></ul><div class=right><a href=https://github.com/LonamiWebs><img src=/img/github.svg alt=github></a><a href=/blog/atom.xml><img src=/img/rss.svg alt=rss></a></div></nav><main><h1 class=title>My Blog</h1><p id=welcome onclick=pls_stop()>Welcome to my blog!<p>Here I occasionally post new entries, mostly tech related. Perhaps it's tips for a new game I'm playing, perhaps it has something to do with FFI, or perhaps I'm fighting the borrow checker (just kidding, I'm over that. Mostly).<hr><ul><li><a href=https://lonami.dev/blog/woce-4/>Writing our own Cheat Engine: Floating points</a><span class=dim> [mod sw; 'windows, 'rust, 'hacking] </span><li><a href=https://lonami.dev/blog/woce-3/>Writing our own Cheat Engine: Unknown initial value</a><span class=dim> [mod sw; 'windows, 'rust, 'hacking] </span><li><a href=https://lonami.dev/blog/woce-2/>Writing our own Cheat Engine: Exact Value scanning</a><span class=dim> [mod sw; 'windows, 'rust, 'hacking] </span><li><a href=https://lonami.dev/blog/woce-1/>Writing our own Cheat Engine: Introduction</a><span class=dim> [mod sw; 'windows, 'rust, 'hacking] </span><li><a href=https://lonami.dev/blog/university/>Data Mining, Warehousing and Information Retrieval</a><span class=dim> [mod algos; 'series, 'bigdata, 'databases] </span><li><a href=https://lonami.dev/blog/new-computer/>My new computer</a><span class=dim> [mod hw; 'showoff] </span><li><a href=https://lonami.dev/blog/tips-outpost/>Tips for Outpost</a><span class=dim> [mod games; 'tips] </span><li><a href=https://lonami.dev/blog/ctypes-and-windows/>Python ctypes and Windows</a><span class=dim> [mod sw; 'python, 'ffi, 'windows] </span><li><a href=https://lonami.dev/blog/pixel-dungeon/>Shattered Pixel Dungeon</a><span class=dim> [mod games; 'tips] </span><li><a href=https://lonami.dev/blog/installing-nixos-2/>Installing NixOS, Take 2</a><span class=dim> [mod sw; 'os, 'nixos] </span><li><a href=https://lonami.dev/blog/breaking-ror/>Breaking Risk of Rain</a><span class=dim> [mod games; 'tips] </span><li><a href=https://lonami.dev/blog/world-edit/>WorldEdit Commands</a><span class=dim> [mod games; 'minecraft, 'worldedit, 'tips] </span><li><a href=https://lonami.dev/blog/asyncio/>An Introduction to Asyncio</a><span class=dim> [mod sw; 'python, 'asyncio] </span><li><a href=https://lonami.dev/blog/posts/>Atemporal Blog Posts</a><span class=dim> [mod algos; 'algorithms, 'culture, 'debate, 'foodforthought, 'graphics, 'optimization] </span><li><a href=https://lonami.dev/blog/graphs/>Graphs</a><span class=dim> [mod algos; 'graphs] </span><li><a href=https://lonami.dev/blog/installing-nixos/>Installing NixOS</a><span class=dim> [mod sw; 'os, 'nixos] </span></ul><script> const WELCOME_EN = 'Welcome to my blog!' const WELCOME_ES = '¡Bienvenido a mi blog!' const APOLOGIES = "ok sorry i'll stop"
M
blog/woce-1/index.html
→
blog/woce-1/index.html
@@ -1,4 +1,4 @@
-<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=description content="Official Lonami's website"><meta name=viewport content="width=device-width, initial-scale=1.0, user-scalable=yes"><title> Writing our own Cheat Engine: Introduction | Lonami's Blog </title><link rel=stylesheet href=/style.css><body><article><nav class=sections><ul class=left><li><a href=/>lonami's site</a><li><a href=/blog class=selected>blog</a><li><a href=/golb>golb</a></ul><div class=right><a href=https://github.com/LonamiWebs><img src=/img/github.svg alt=github></a><a href=/blog/atom.xml><img src=/img/rss.svg alt=rss></a></div></nav><main><h1 class=title>Writing our own Cheat Engine: Introduction</h1><div class=time><p>2021-02-07<p>last updated 2021-02-19</div><p>This is part 1 on the <em>Writing our own Cheat Engine</em> series:<ul><li>Part 1: Introduction<li><a href=/blog/woce-2>Part 2: Exact Value scanning</a><li><a href=/blog/woce-3>Part 3: Unknown initial value</a></ul><p><a href=https://cheatengine.org/>Cheat Engine</a> is a tool designed to modify single player games and contains other useful tools within itself that enable its users to debug games or other applications. It comes with a memory scanner, (dis)assembler, inspection tools and a handful other things. In this series, we will be writing our own tiny Cheat Engine capable of solving all steps of the tutorial, and diving into how it all works underneath.<p>Needless to say, we're doing this for private and educational purposes only. One has to make sure to not violate the EULA or ToS of the specific application we're attaching to. This series, much like cheatengine.org, does not condone the illegal use of the code shared.<p>Cheat Engine is a tool for Windows, so we will be developing for Windows as well. However, you can also <a href=https://stackoverflow.com/q/12977179/4759433>read memory from Linux-like systems</a>. <a href=https://github.com/scanmem/scanmem>GameConqueror</a> is a popular alternative to Cheat Engine on Linux systems, so if you feel adventurous, you could definitely follow along too! The techniques shown in this series apply regardless of how we read memory from a process. You will learn a fair bit about doing FFI in Rust too.<p>We will be developing the application in Rust, because it enables us to interface with the Windows API easily, is memory safe (as long as we're careful with <code>unsafe</code>!), and is speedy (we will need this for later steps in the Cheat Engine tutorial). You could use any language of your choice though. For example, <a href=https://lonami.dev/blog/ctypes-and-windows/>Python also makes it relatively easy to use the Windows API</a>. You don't need to be a Rust expert to follow along, but this series assumes some familiarity with C-family languages. Slightly advanced concepts like the use of <code>unsafe</code> or the <code>MaybeUninit</code> type will be briefly explained. What a <code>fn</code> is or what <code>let</code> does will not be explained.<p><a href=https://github.com/cheat-engine/cheat-engine/>Cheat Engine's source code</a> is mostly written in Pascal and C. And it's <em>a lot</em> of code, with a very flat project structure, and files ranging in the thousand lines of code each. It's daunting<sup class=footnote-reference><a href=#1>1</a></sup>. It's a mature project, with a lot of knowledge encoded in the code base, and a lot of features like distributed scanning or an entire disassembler. Unfortunately, there's not a lot of comments. For these reasons, I'll do some guesswork when possible as to how it's working underneath, rather than actually digging into what Cheat Engine is actually doing.<p>With that out of the way, let's get started!<h2 id=welcome-to-the-cheat-engine-tutorial>Welcome to the Cheat Engine Tutorial</h2><details open><summary>Cheat Engine Tutorial: Step 1</summary> <blockquote><p>This tutorial will teach you the basics of cheating in video games. It will also show you foundational aspects of using Cheat Engine (or CE for short). Follow the steps below to get started.<ol><li>Open Cheat Engine if it currently isn't running.<li>Click on the "Open Process" icon (it's the top-left icon with the computer on it, below "File".).<li>With the Process List window now open, look for this tutorial's process in the list. It will look something like > "00001F98-Tutorial-x86_64.exe" or "0000047C-Tutorial-i386.exe". (The first 8 numbers/letters will probably be different.)<li>Once you've found the process, click on it to select it, then click the "Open" button. (Don't worry about all the > other buttons right now. You can learn about them later if you're interested.)</ol><p>Congratulations! If you did everything correctly, the process window should be gone with Cheat Engine now attached to the > tutorial (you will see the process name towards the top-center of CE).<p>Click the "Next" button below to continue, or fill in the password and click the "OK" button to proceed to that step.)<p>If you're having problems, simply head over to forum.cheatengine.org, then click on "Tutorials" to view beginner-friendly > guides!</blockquote></details><h2 id=enumerating-processes>Enumerating processes</h2><p>Our first step is attaching to the process we want to work with. But we need a way to find that process in the first place! Having to open the task manager, look for the process we care about, noting down the process ID (PID), and slapping it in the source code is not satisfying at all. Instead, let's enumerate all the processes from within the program, and let the user select one by typing its name.<p>From a quick <a href=https://ddg.gg/winapi%20enumerate%20all%20processes>DuckDuckGo search</a>, we find an official tutorial for <a href=https://docs.microsoft.com/en-us/windows/win32/psapi/enumerating-all-processes>Enumerating All Processes</a>, which leads to the <a href=https://docs.microsoft.com/en-us/windows/win32/api/psapi/nf-psapi-enumprocesses><code>EnumProcesses</code></a> call. Cool! Let's slap in the <a href=https://crates.io/crates/winapi><code>winapi</code></a> crate on <code>Cargo.toml</code>, because I don't want to write all the definitions by myself:<pre><code class=language-toml data-lang=toml>[dependencies] +<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=description content="Official Lonami's website"><meta name=viewport content="width=device-width, initial-scale=1.0, user-scalable=yes"><title> Writing our own Cheat Engine: Introduction | Lonami's Blog </title><link rel=stylesheet href=/style.css><body><article><nav class=sections><ul class=left><li><a href=/>lonami's site</a><li><a href=/blog class=selected>blog</a><li><a href=/golb>golb</a></ul><div class=right><a href=https://github.com/LonamiWebs><img src=/img/github.svg alt=github></a><a href=/blog/atom.xml><img src=/img/rss.svg alt=rss></a></div></nav><main><h1 class=title>Writing our own Cheat Engine: Introduction</h1><div class=time><p>2021-02-07<p>last updated 2021-02-19</div><p>This is part 1 on the <em>Writing our own Cheat Engine</em> series:<ul><li>Part 1: Introduction<li><a href=/blog/woce-2>Part 2: Exact Value scanning</a><li><a href=/blog/woce-3>Part 3: Unknown initial value</a><li><a href=/blog/woce-4>Part 4: Floating points</a></ul><p><a href=https://cheatengine.org/>Cheat Engine</a> is a tool designed to modify single player games and contains other useful tools within itself that enable its users to debug games or other applications. It comes with a memory scanner, (dis)assembler, inspection tools and a handful other things. In this series, we will be writing our own tiny Cheat Engine capable of solving all steps of the tutorial, and diving into how it all works underneath.<p>Needless to say, we're doing this for private and educational purposes only. One has to make sure to not violate the EULA or ToS of the specific application we're attaching to. This series, much like cheatengine.org, does not condone the illegal use of the code shared.<p>Cheat Engine is a tool for Windows, so we will be developing for Windows as well. However, you can also <a href=https://stackoverflow.com/q/12977179/4759433>read memory from Linux-like systems</a>. <a href=https://github.com/scanmem/scanmem>GameConqueror</a> is a popular alternative to Cheat Engine on Linux systems, so if you feel adventurous, you could definitely follow along too! The techniques shown in this series apply regardless of how we read memory from a process. You will learn a fair bit about doing FFI in Rust too.<p>We will be developing the application in Rust, because it enables us to interface with the Windows API easily, is memory safe (as long as we're careful with <code>unsafe</code>!), and is speedy (we will need this for later steps in the Cheat Engine tutorial). You could use any language of your choice though. For example, <a href=https://lonami.dev/blog/ctypes-and-windows/>Python also makes it relatively easy to use the Windows API</a>. You don't need to be a Rust expert to follow along, but this series assumes some familiarity with C-family languages. Slightly advanced concepts like the use of <code>unsafe</code> or the <code>MaybeUninit</code> type will be briefly explained. What a <code>fn</code> is or what <code>let</code> does will not be explained.<p><a href=https://github.com/cheat-engine/cheat-engine/>Cheat Engine's source code</a> is mostly written in Pascal and C. And it's <em>a lot</em> of code, with a very flat project structure, and files ranging in the thousand lines of code each. It's daunting<sup class=footnote-reference><a href=#1>1</a></sup>. It's a mature project, with a lot of knowledge encoded in the code base, and a lot of features like distributed scanning or an entire disassembler. Unfortunately, there's not a lot of comments. For these reasons, I'll do some guesswork when possible as to how it's working underneath, rather than actually digging into what Cheat Engine is actually doing.<p>With that out of the way, let's get started!<h2 id=welcome-to-the-cheat-engine-tutorial>Welcome to the Cheat Engine Tutorial</h2><details open><summary>Cheat Engine Tutorial: Step 1</summary> <blockquote><p>This tutorial will teach you the basics of cheating in video games. It will also show you foundational aspects of using Cheat Engine (or CE for short). Follow the steps below to get started.<ol><li>Open Cheat Engine if it currently isn't running.<li>Click on the "Open Process" icon (it's the top-left icon with the computer on it, below "File".).<li>With the Process List window now open, look for this tutorial's process in the list. It will look something like > "00001F98-Tutorial-x86_64.exe" or "0000047C-Tutorial-i386.exe". (The first 8 numbers/letters will probably be different.)<li>Once you've found the process, click on it to select it, then click the "Open" button. (Don't worry about all the > other buttons right now. You can learn about them later if you're interested.)</ol><p>Congratulations! If you did everything correctly, the process window should be gone with Cheat Engine now attached to the > tutorial (you will see the process name towards the top-center of CE).<p>Click the "Next" button below to continue, or fill in the password and click the "OK" button to proceed to that step.)<p>If you're having problems, simply head over to forum.cheatengine.org, then click on "Tutorials" to view beginner-friendly > guides!</blockquote></details><h2 id=enumerating-processes>Enumerating processes</h2><p>Our first step is attaching to the process we want to work with. But we need a way to find that process in the first place! Having to open the task manager, look for the process we care about, noting down the process ID (PID), and slapping it in the source code is not satisfying at all. Instead, let's enumerate all the processes from within the program, and let the user select one by typing its name.<p>From a quick <a href=https://ddg.gg/winapi%20enumerate%20all%20processes>DuckDuckGo search</a>, we find an official tutorial for <a href=https://docs.microsoft.com/en-us/windows/win32/psapi/enumerating-all-processes>Enumerating All Processes</a>, which leads to the <a href=https://docs.microsoft.com/en-us/windows/win32/api/psapi/nf-psapi-enumprocesses><code>EnumProcesses</code></a> call. Cool! Let's slap in the <a href=https://crates.io/crates/winapi><code>winapi</code></a> crate on <code>Cargo.toml</code>, because I don't want to write all the definitions by myself:<pre><code class=language-toml data-lang=toml>[dependencies] winapi = { version = "0.3.9", features = ["psapi"] } </code></pre><p>Because <a href=https://docs.microsoft.com/en-us/windows/win32/api/psapi/nf-psapi-enumprocesses><code>EnumProcesses</code></a> is in <code>Psapi.h</code> (you can see this in the online page of its documentation), we know we'll need the <code>psapi</code> crate feature. Another option is to search for it in the <a href=https://docs.rs/winapi/><code>winapi</code> documentation</a> and noting down the parent module where its stored.<p>The documentation for the method has the following remark:<blockquote><p>It is a good idea to use a large array, because it is hard to predict how many processes there will be at the time you call <strong>EnumProcesses</strong>.</blockquote><p><em>Sidenote: reading the documentation for the methods we'll use from the Windows API is extremely important. There's a lot of gotchas involved, so we need to make sure we're extra careful.</em><p>1024 is a pretty big number, so let's go with that:<pre><code class=language-rust data-lang=rust>use std::io; use std::mem;
M
blog/woce-2/index.html
→
blog/woce-2/index.html
@@ -1,4 +1,4 @@
-<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=description content="Official Lonami's website"><meta name=viewport content="width=device-width, initial-scale=1.0, user-scalable=yes"><title> Writing our own Cheat Engine: Exact Value scanning | Lonami's Blog </title><link rel=stylesheet href=/style.css><body><article><nav class=sections><ul class=left><li><a href=/>lonami's site</a><li><a href=/blog class=selected>blog</a><li><a href=/golb>golb</a></ul><div class=right><a href=https://github.com/LonamiWebs><img src=/img/github.svg alt=github></a><a href=/blog/atom.xml><img src=/img/rss.svg alt=rss></a></div></nav><main><h1 class=title>Writing our own Cheat Engine: Exact Value scanning</h1><div class=time><p>2021-02-12<p>last updated 2021-02-19</div><p>This is part 2 on the <em>Writing our own Cheat Engine</em> series:<ul><li><a href=/blog/woce-1>Part 1: Introduction</a> (start here if you're new to the series!)<li>Part 2: Exact Value scanning<li><a href=/blog/woce-3>Part 3: Unknown initial value</a></ul><p>In the introduction, we spent a good deal of time enumerating all running processes just so we could find out the pid we cared about. With the pid now in our hands, we can do pretty much anything to its corresponding process.<p>It's now time to read the process' memory and write to it. If our process was a single-player game, this would enable us to do things like setting a very high value on the player's current health pool, making us invincible. This technique will often not work for multi-player games, because the server likely knows your true current health (the most you could probably do is make the client render an incorrect value). However, if the server is crappy and it trusts the client, then you're still free to mess around with your current health.<p>Even if we don't want to write to the process' memory, reading is still very useful. Maybe you could enhance your experience by making a custom overlay that displays useful information, or something that makes noise if it detects the life is too low, or even simulating a keyboard event to automatically recover some mana when you're running low.<p>Be warned about anti-cheat systems. Anything beyond a basic game is likely to have some protection measures in place, making the analysis more difficult (perhaps the values are scrambled in memory), or even pinging the server if it detects something fishy.<p><strong>I am not responsible for any bans!</strong> Use your brain before messing with online games, and don't ruin the fun for everyone else. If you get caught for cheating, I don't want to know about it.<p>Now that all <a href=https://www.urbandictionary.com/define.php?term=script%20kiddie>script kiddies</a> have left the room, let's proceed with the post.<h2 id=exact-value-scanning>Exact Value scanning</h2><details open><summary>Cheat Engine Tutorial: Step 2</summary> <blockquote><p>Now that you have opened the tutorial with Cheat Engine let's get on with the next step.<p>You can see at the bottom of this window is the text Health: xxx. Each time you click 'Hit me' your health gets decreased.<p>To get to the next step you have to find this value and change it to 1000<p>To find the value there are different ways, but I'll tell you about the easiest, 'Exact Value': First make sure value type is set to at least 2-bytes or 4-bytes. 1-byte will also work, but you'll run into an easy to fix problem when you've found the address and want to change it. The 8-byte may perhaps works if the bytes after the address are 0, but I wouldn't take the bet. Single, double, and the other scans just don't work, because they store the value in a different way.<p>When the value type is set correctly, make sure the scantype is set to 'Exact Value'. Then fill in the number your health is in the value box. And click 'First Scan'. After a while (if you have a extremely slow pc) the scan is done and the results are shown in the list on the left<p>If you find more than 1 address and you don't know for sure which address it is, click 'Hit me', fill in the new health value into the value box, and click 'Next Scan'. Repeat this until you're sure you've found it. (that includes that there's only 1 address in the list.....)<p>Now double click the address in the list on the left. This makes the address pop-up in the list at the bottom, showing you the current value. Double click the value, (or select it and press enter), and change the value to 1000.<p>If everything went ok the next button should become enabled, and you're ready for the next step.<p>Note: If you did anything wrong while scanning, click "New Scan" and repeat the scanning again. Also, try playing around with the value and click 'hit me'</blockquote></details><h2 id=our-first-scan>Our First Scan</h2><p>The Cheat Engine tutorial talks about "value types" and "scan types" like "exact value".<p>The <strong>value types</strong> will help us narrow down <em>what</em> we're looking for. For example, the integer type <code>i32</code> is represented in memory as 32 bits, or 4 bytes. However, <code>f32</code> is <em>also</em> represented by 4 bytes, and so is <code>u32</code>. Or perhaps the 4 bytes represent RGBA values of a color! So any 4 bytes in memory can be interpreted in many ways, and it's up to us to decide which way we interpret the bytes in.<p>When programming, numbers which are 32-bit wide are common, as they're a good (and fast) size to work with. Scanning for this type is often a good bet. For positive numbers, <code>i32</code> is represented the same as <code>u32</code> in memory, so even if the value turns out to not be signed, the scan is likely to work. Focusing on <code>i32</code> will save us from scanning for <code>f32</code> or even other types, like interpreting 8 bytes for <code>i64</code>, <code>f64</code>, or less bytes like <code>i16</code>.<p>The <strong>scan types</strong> will help us narrow down <em>how</em> we're looking for a value. Scanning for an exact value means what you think it does: interpret all 4 bytes in the process' memory as our value type, and check if they exactly match our value. This will often yield a lot of candidates, but it will be enough to get us started. Variations of the exact scan include checking for all values below a threshold, above, in between, or even just… unknown.<p>What's the point of scanning for unknown values if <em>everything</em> in memory is unknown? Sometimes you don't have a concrete value. Maybe your health pool is a bar and it nevers tell you how much health you actually have, just a visual indicator of your percentage left, even if the health is not stored as a percentage. As we will find later on, scanning for unknown values is more useful than it might appear at first.<p>We can access the memory of our own program by guessing random pointers and trying to read from them. But Windows isolates the memory of each program, so no pointer we could ever guess will let us read from the memory of another process. Luckily for us, searching for "read process memory winapi" leads us to the <a href=https://docs.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-readprocessmemory><code>ReadProcessMemory</code></a> function. Spot on.<pre><code class=language-rust data-lang=rust>pub fn read_memory(&self, addr: usize, n: usize) -> io::Result<Vec<u8>> { +<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=description content="Official Lonami's website"><meta name=viewport content="width=device-width, initial-scale=1.0, user-scalable=yes"><title> Writing our own Cheat Engine: Exact Value scanning | Lonami's Blog </title><link rel=stylesheet href=/style.css><body><article><nav class=sections><ul class=left><li><a href=/>lonami's site</a><li><a href=/blog class=selected>blog</a><li><a href=/golb>golb</a></ul><div class=right><a href=https://github.com/LonamiWebs><img src=/img/github.svg alt=github></a><a href=/blog/atom.xml><img src=/img/rss.svg alt=rss></a></div></nav><main><h1 class=title>Writing our own Cheat Engine: Exact Value scanning</h1><div class=time><p>2021-02-12<p>last updated 2021-02-19</div><p>This is part 2 on the <em>Writing our own Cheat Engine</em> series:<ul><li><a href=/blog/woce-1>Part 1: Introduction</a> (start here if you're new to the series!)<li>Part 2: Exact Value scanning<li><a href=/blog/woce-3>Part 3: Unknown initial value</a><li><a href=/blog/woce-4>Part 4: Floating points</a></ul><p>In the introduction, we spent a good deal of time enumerating all running processes just so we could find out the pid we cared about. With the pid now in our hands, we can do pretty much anything to its corresponding process.<p>It's now time to read the process' memory and write to it. If our process was a single-player game, this would enable us to do things like setting a very high value on the player's current health pool, making us invincible. This technique will often not work for multi-player games, because the server likely knows your true current health (the most you could probably do is make the client render an incorrect value). However, if the server is crappy and it trusts the client, then you're still free to mess around with your current health.<p>Even if we don't want to write to the process' memory, reading is still very useful. Maybe you could enhance your experience by making a custom overlay that displays useful information, or something that makes noise if it detects the life is too low, or even simulating a keyboard event to automatically recover some mana when you're running low.<p>Be warned about anti-cheat systems. Anything beyond a basic game is likely to have some protection measures in place, making the analysis more difficult (perhaps the values are scrambled in memory), or even pinging the server if it detects something fishy.<p><strong>I am not responsible for any bans!</strong> Use your brain before messing with online games, and don't ruin the fun for everyone else. If you get caught for cheating, I don't want to know about it.<p>Now that all <a href=https://www.urbandictionary.com/define.php?term=script%20kiddie>script kiddies</a> have left the room, let's proceed with the post.<h2 id=exact-value-scanning>Exact Value scanning</h2><details open><summary>Cheat Engine Tutorial: Step 2</summary> <blockquote><p>Now that you have opened the tutorial with Cheat Engine let's get on with the next step.<p>You can see at the bottom of this window is the text Health: xxx. Each time you click 'Hit me' your health gets decreased.<p>To get to the next step you have to find this value and change it to 1000<p>To find the value there are different ways, but I'll tell you about the easiest, 'Exact Value': First make sure value type is set to at least 2-bytes or 4-bytes. 1-byte will also work, but you'll run into an easy to fix problem when you've found the address and want to change it. The 8-byte may perhaps works if the bytes after the address are 0, but I wouldn't take the bet. Single, double, and the other scans just don't work, because they store the value in a different way.<p>When the value type is set correctly, make sure the scantype is set to 'Exact Value'. Then fill in the number your health is in the value box. And click 'First Scan'. After a while (if you have a extremely slow pc) the scan is done and the results are shown in the list on the left<p>If you find more than 1 address and you don't know for sure which address it is, click 'Hit me', fill in the new health value into the value box, and click 'Next Scan'. Repeat this until you're sure you've found it. (that includes that there's only 1 address in the list.....)<p>Now double click the address in the list on the left. This makes the address pop-up in the list at the bottom, showing you the current value. Double click the value, (or select it and press enter), and change the value to 1000.<p>If everything went ok the next button should become enabled, and you're ready for the next step.<p>Note: If you did anything wrong while scanning, click "New Scan" and repeat the scanning again. Also, try playing around with the value and click 'hit me'</blockquote></details><h2 id=our-first-scan>Our First Scan</h2><p>The Cheat Engine tutorial talks about "value types" and "scan types" like "exact value".<p>The <strong>value types</strong> will help us narrow down <em>what</em> we're looking for. For example, the integer type <code>i32</code> is represented in memory as 32 bits, or 4 bytes. However, <code>f32</code> is <em>also</em> represented by 4 bytes, and so is <code>u32</code>. Or perhaps the 4 bytes represent RGBA values of a color! So any 4 bytes in memory can be interpreted in many ways, and it's up to us to decide which way we interpret the bytes in.<p>When programming, numbers which are 32-bit wide are common, as they're a good (and fast) size to work with. Scanning for this type is often a good bet. For positive numbers, <code>i32</code> is represented the same as <code>u32</code> in memory, so even if the value turns out to not be signed, the scan is likely to work. Focusing on <code>i32</code> will save us from scanning for <code>f32</code> or even other types, like interpreting 8 bytes for <code>i64</code>, <code>f64</code>, or less bytes like <code>i16</code>.<p>The <strong>scan types</strong> will help us narrow down <em>how</em> we're looking for a value. Scanning for an exact value means what you think it does: interpret all 4 bytes in the process' memory as our value type, and check if they exactly match our value. This will often yield a lot of candidates, but it will be enough to get us started. Variations of the exact scan include checking for all values below a threshold, above, in between, or even just… unknown.<p>What's the point of scanning for unknown values if <em>everything</em> in memory is unknown? Sometimes you don't have a concrete value. Maybe your health pool is a bar and it nevers tell you how much health you actually have, just a visual indicator of your percentage left, even if the health is not stored as a percentage. As we will find later on, scanning for unknown values is more useful than it might appear at first.<p>We can access the memory of our own program by guessing random pointers and trying to read from them. But Windows isolates the memory of each program, so no pointer we could ever guess will let us read from the memory of another process. Luckily for us, searching for "read process memory winapi" leads us to the <a href=https://docs.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-readprocessmemory><code>ReadProcessMemory</code></a> function. Spot on.<pre><code class=language-rust data-lang=rust>pub fn read_memory(&self, addr: usize, n: usize) -> io::Result<Vec<u8>> { todo!() } </code></pre><p>Much like trying to dereference a pointer pointing to released memory or even null, reading from an arbitrary address can fail for the same reasons (and more). We will want to signal this with <code>io::Result</code>. It's funny to note that, even though we're doing something that seems wildly unsafe (reading arbitrary memory, even if the other process is mutating it at the same time), the function is perfectly safe. If we cannot read something, it will return <code>Err</code>, but if it succeeds, it has taken a snapshot of the memory of the process, and the returned value will be correctly initialized.<p>The function will be defined inside our <code>impl Process</code>, since it conveniently holds an open handle to the process in question. It takes <code>&self</code>, because we do not need to mutate anything in the <code>Process</code> instance. After adding the <code>memoryapi</code> feature to <code>Cargo.toml</code>, we can perform the call:<pre><code class=language-rust data-lang=rust>let mut buffer = Vec::<u8>::with_capacity(n);@@ -256,4 +256,4 @@ )
</code></pre><p>Behold:<pre><code>Now have 1 location(s) Enter new memory value: 1000 Written 4 bytes to [15d8b90] -</code></pre><p><img src=https://user-images.githubusercontent.com/6297805/107829541-3f4f2d00-6d8a-11eb-87c4-e2f2d505afbc.png alt="Tutorial complete with memo"><p>Isn't that active <em>Next</em> button just beautiful?<h2 id=finale>Finale</h2><p>This post somehow ended up being longer than part one, but look at what we've achieved! We completed a step of the Cheat Engine Tutorial <em>without using Cheat Engine</em>. Just pure Rust. Figuring out how a program works and reimplementing it yourself is a great way to learn what it's doing behind the scenes. And now that this code is yours, you can extend it as much as you like, without being constrained by Cheat Engine's UI. You can automate it as much as you want.<p>And we're not even done. The current tutorial has nine steps, and three additional graphical levels.<p>In the next post, we'll tackle the third step of the tutorial: Unknown initial value. This will pose a challenge, because with just 2 MiB of memory, storing all the 4-byte aligned locations would require 524288 addresses (<code>usize</code>, 8 bytes). This adds up to twice as much memory as the original program (4 MiB), but that's not our main concern, having to perform over five hundred thousand API calls is!<p>Remember that you can <a href=https://github.com/lonami/memo>obtain the code for this post</a> over at my GitHub. You can run <code>git checkout step2</code> after cloning the repository to get the right version of the code.<h3 id=footnotes>Footnotes</h3><div class=footnote-definition id=1><sup class=footnote-definition-label>1</sup><p>I did in fact use an online tool to spell it out for me.</div><div class=footnote-definition id=2><sup class=footnote-definition-label>2</sup><p>16 GiB is good enough for my needs. I don't think I'll ever upgrade to 16 EiB.</div><div class=footnote-definition id=3><sup class=footnote-definition-label>3</sup><p>Every address we query should have a corresponding region, even if it's not allocated or we do not have access. This is why we can query for the memory address zero to get its corresponding region.</div><div class=footnote-definition id=4><sup class=footnote-definition-label>4</sup><p>Another option is to <a href=https://docs.microsoft.com/en-us/windows/win32/api/sysinfoapi/nf-sysinfoapi-getsysteminfo><code>GetSystemInfo</code></a> to determine the <code>lpMinimumApplicationAddress</code> and <code>lpMaximumApplicationAddress</code> and only work within bounds.</div><div class=footnote-definition id=5><sup class=footnote-definition-label>5</sup><p>Memory regions are page-aligned, which is a large power of two. Our alignment of 4 is much lower than this, so we're guaranteed to start off at an aligned address.</div><div class=footnote-definition id=6><sup class=footnote-definition-label>6</sup><p>If it turns out that the value was actually misaligned, we will miss it. You will notice this if, after going through the whole process, there are no results. It could mean that either the value type is wrong, or the value type is misaligned. In the worst case, the value is not stored directly but is rather computed with something like <code>maximum - stored</code>, or XORed with some magic value, or a myriad other things.</div><div class=footnote-definition id=7><sup class=footnote-definition-label>7</sup><p>You could do this without getting hit, and just keep on repeating the scan for the same value over and over again. This does work, but the results are suboptimal, because there are also many other values that didn't change. Scanning for a changed value is a better option.</div><div class=footnote-definition id=8><sup class=footnote-definition-label>8</sup><p>You could actually just go ahead and try to modify the memory at the hundred addresses you just found, although don't be surprised if the program starts to misbehave!</div><div class=footnote-definition id=9><sup class=footnote-definition-label>9</sup><p>Okay, we cannot fit infinity in an <code>i32</code>. However, we can fit sufficiently large numbers. Like <code>1000</code>, which is enough to complete the tutorial.</div></main><footer><div><p>Share your thoughts, or simply come hang with me <a href=https://t.me/LonamiWebs><img src=/img/telegram.svg alt=Telegram></a> <a href=mailto:totufals@hotmail.com><img src=/img/mail.svg alt=Mail></a></div></footer></article><p class=abyss>Glaze into the abyss… Oh hi there!+</code></pre><p><img src=https://user-images.githubusercontent.com/6297805/107829541-3f4f2d00-6d8a-11eb-87c4-e2f2d505afbc.png alt="Tutorial complete with memo"><p>Isn't that active <em>Next</em> button just beautiful?<h2 id=finale>Finale</h2><p>This post somehow ended up being longer than part one, but look at what we've achieved! We completed a step of the Cheat Engine Tutorial <em>without using Cheat Engine</em>. Just pure Rust. Figuring out how a program works and reimplementing it yourself is a great way to learn what it's doing behind the scenes. And now that this code is yours, you can extend it as much as you like, without being constrained by Cheat Engine's UI. You can automate it as much as you want.<p>And we're not even done. The current tutorial has nine steps, and three additional graphical levels.<p>In the <a href=/blog/woce-3>next post</a>, we'll tackle the third step of the tutorial: Unknown initial value. This will pose a challenge, because with just 2 MiB of memory, storing all the 4-byte aligned locations would require 524288 addresses (<code>usize</code>, 8 bytes). This adds up to twice as much memory as the original program (4 MiB), but that's not our main concern, having to perform over five hundred thousand API calls is!<p>Remember that you can <a href=https://github.com/lonami/memo>obtain the code for this post</a> over at my GitHub. You can run <code>git checkout step2</code> after cloning the repository to get the right version of the code.<h3 id=footnotes>Footnotes</h3><div class=footnote-definition id=1><sup class=footnote-definition-label>1</sup><p>I did in fact use an online tool to spell it out for me.</div><div class=footnote-definition id=2><sup class=footnote-definition-label>2</sup><p>16 GiB is good enough for my needs. I don't think I'll ever upgrade to 16 EiB.</div><div class=footnote-definition id=3><sup class=footnote-definition-label>3</sup><p>Every address we query should have a corresponding region, even if it's not allocated or we do not have access. This is why we can query for the memory address zero to get its corresponding region.</div><div class=footnote-definition id=4><sup class=footnote-definition-label>4</sup><p>Another option is to <a href=https://docs.microsoft.com/en-us/windows/win32/api/sysinfoapi/nf-sysinfoapi-getsysteminfo><code>GetSystemInfo</code></a> to determine the <code>lpMinimumApplicationAddress</code> and <code>lpMaximumApplicationAddress</code> and only work within bounds.</div><div class=footnote-definition id=5><sup class=footnote-definition-label>5</sup><p>Memory regions are page-aligned, which is a large power of two. Our alignment of 4 is much lower than this, so we're guaranteed to start off at an aligned address.</div><div class=footnote-definition id=6><sup class=footnote-definition-label>6</sup><p>If it turns out that the value was actually misaligned, we will miss it. You will notice this if, after going through the whole process, there are no results. It could mean that either the value type is wrong, or the value type is misaligned. In the worst case, the value is not stored directly but is rather computed with something like <code>maximum - stored</code>, or XORed with some magic value, or a myriad other things.</div><div class=footnote-definition id=7><sup class=footnote-definition-label>7</sup><p>You could do this without getting hit, and just keep on repeating the scan for the same value over and over again. This does work, but the results are suboptimal, because there are also many other values that didn't change. Scanning for a changed value is a better option.</div><div class=footnote-definition id=8><sup class=footnote-definition-label>8</sup><p>You could actually just go ahead and try to modify the memory at the hundred addresses you just found, although don't be surprised if the program starts to misbehave!</div><div class=footnote-definition id=9><sup class=footnote-definition-label>9</sup><p>Okay, we cannot fit infinity in an <code>i32</code>. However, we can fit sufficiently large numbers. Like <code>1000</code>, which is enough to complete the tutorial.</div></main><footer><div><p>Share your thoughts, or simply come hang with me <a href=https://t.me/LonamiWebs><img src=/img/telegram.svg alt=Telegram></a> <a href=mailto:totufals@hotmail.com><img src=/img/mail.svg alt=Mail></a></div></footer></article><p class=abyss>Glaze into the abyss… Oh hi there!
M
blog/woce-3/index.html
→
blog/woce-3/index.html
@@ -1,4 +1,4 @@
-<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=description content="Official Lonami's website"><meta name=viewport content="width=device-width, initial-scale=1.0, user-scalable=yes"><title> Writing our own Cheat Engine: Unknown initial value | Lonami's Blog </title><link rel=stylesheet href=/style.css><body><article><nav class=sections><ul class=left><li><a href=/>lonami's site</a><li><a href=/blog class=selected>blog</a><li><a href=/golb>golb</a></ul><div class=right><a href=https://github.com/LonamiWebs><img src=/img/github.svg alt=github></a><a href=/blog/atom.xml><img src=/img/rss.svg alt=rss></a></div></nav><main><h1 class=title>Writing our own Cheat Engine: Unknown initial value</h1><div class=time><p>2021-02-19</div><p>This is part 3 on the <em>Writing our own Cheat Engine</em> series:<ul><li><a href=/blog/woce-1>Part 1: Introduction</a> (start here if you're new to the series!)<li><a href=/blog/woce-2>Part 2: Exact Value scanning</a><li>Part 3: Unknown initial value</ul><p>In part 2 we left off with a bit of a cliff-hanger. Our little program is now able to scan for an exact value, remember the couple hundred addresses pointing to said value, and perform subsequent scans to narrow the list of addresses down until we're left with a handful of them.<p>However, it is not always the case that you have an exact value to work with. The best you can do in these cases is guess what the software might be storing. For example, it could be a floating point for your current movement speed in a game, or an integer for your current health.<p>The problem with this is that there are far too many possible locations storing our desired value. If you count misaligned locations, this means there is a different location to address every single byte in memory. A program with one megabyte of memory already has a <em>million</em> of addresses. Clearly, we need to do better than performing one million memory reads<sup class=footnote-reference><a href=#1>1</a></sup>.<p>This post will shift focus a bit from using <code>winapi</code> to possible techniques to perform the various scans.<h2 id=unknown-initial-value>Unknown initial value</h2><details open><summary>Cheat Engine Tutorial: Step 3</summary> <blockquote><p>Ok, seeing that you've figured out how to find a value using exact value let's move on to the next step.<p>First things first though. Since you are doing a new scan, you have to click on New Scan first, to start a new scan. (You may think this is straighforward, but you'd be surprised how many people get stuck on that step) I won't be explaining this step again, so keep this in mind Now that you've started a new scan, let's continue<p>In the previous test we knew the initial value so we could do a exact value, but now we have a status bar where we don't know the starting value. We only know that the value is between 0 and 500. And each time you click 'hit me' you lose some health. The amount you lose each time is shown above the status bar.<p>Again there are several different ways to find the value. (like doing a decreased value by... scan), but I'll only explain the easiest. "Unknown initial value", and decreased value. Because you don't know the value it is right now, a exact value wont do any good, so choose as scantype 'Unknown initial value', again, the value type is 4-bytes. (most windows apps use 4-bytes)click first scan and wait till it's done.<p>When it is done click 'hit me'. You'll lose some of your health. (the amount you lost shows for a few seconds and then disappears, but you don't need that) Now go to Cheat Engine, and choose 'Decreased Value' and click 'Next Scan' When that scan is done, click hit me again, and repeat the above till you only find a few.<p>We know the value is between 0 and 500, so pick the one that is most likely the address we need, and add it to the list. Now change the health to 5000, to proceed to the next step.</blockquote></details><h2 id=dense-memory-locations>Dense memory locations</h2><p>The key thing to notice here is that, when we read memory from another process, we do so over <em>entire regions</em>. A memory region is represented by a starting offset, a size, and a bunch of other things like protection level.<p>When running the first scan for an unknown value, all we need to remember is the starting offset and size for every single region. All the candidate locations that could point to our value fall within this range, so it is enough for us to store the range definition, and not every location within it.<p>To gain a better understanding of what this means, let's come up with a more specific scenario. With our current approach of doing things, we store an address (<code>usize</code>) for every location pointing to our desired value. In the case of unknown values, all locations are equally valid, since we don't know what value they should point to yet, and any value they point to is good. With this representation, we would end up with a very large vector:<pre><code class=language-rust data-lang=rust>let locations = vec![0x2000, 0x2001, ..., 0x20ff, 0x2100]; +<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=description content="Official Lonami's website"><meta name=viewport content="width=device-width, initial-scale=1.0, user-scalable=yes"><title> Writing our own Cheat Engine: Unknown initial value | Lonami's Blog </title><link rel=stylesheet href=/style.css><body><article><nav class=sections><ul class=left><li><a href=/>lonami's site</a><li><a href=/blog class=selected>blog</a><li><a href=/golb>golb</a></ul><div class=right><a href=https://github.com/LonamiWebs><img src=/img/github.svg alt=github></a><a href=/blog/atom.xml><img src=/img/rss.svg alt=rss></a></div></nav><main><h1 class=title>Writing our own Cheat Engine: Unknown initial value</h1><div class=time><p>2021-02-19</div><p>This is part 3 on the <em>Writing our own Cheat Engine</em> series:<ul><li><a href=/blog/woce-1>Part 1: Introduction</a> (start here if you're new to the series!)<li><a href=/blog/woce-2>Part 2: Exact Value scanning</a><li>Part 3: Unknown initial value<li><a href=/blog/woce-4>Part 4: Floating points</a></ul><p>In part 2 we left off with a bit of a cliff-hanger. Our little program is now able to scan for an exact value, remember the couple hundred addresses pointing to said value, and perform subsequent scans to narrow the list of addresses down until we're left with a handful of them.<p>However, it is not always the case that you have an exact value to work with. The best you can do in these cases is guess what the software might be storing. For example, it could be a floating point for your current movement speed in a game, or an integer for your current health.<p>The problem with this is that there are far too many possible locations storing our desired value. If you count misaligned locations, this means there is a different location to address every single byte in memory. A program with one megabyte of memory already has a <em>million</em> of addresses. Clearly, we need to do better than performing one million memory reads<sup class=footnote-reference><a href=#1>1</a></sup>.<p>This post will shift focus a bit from using <code>winapi</code> to possible techniques to perform the various scans.<h2 id=unknown-initial-value>Unknown initial value</h2><details open><summary>Cheat Engine Tutorial: Step 3</summary> <blockquote><p>Ok, seeing that you've figured out how to find a value using exact value let's move on to the next step.<p>First things first though. Since you are doing a new scan, you have to click on New Scan first, to start a new scan. (You may think this is straighforward, but you'd be surprised how many people get stuck on that step) I won't be explaining this step again, so keep this in mind Now that you've started a new scan, let's continue<p>In the previous test we knew the initial value so we could do a exact value, but now we have a status bar where we don't know the starting value. We only know that the value is between 0 and 500. And each time you click 'hit me' you lose some health. The amount you lose each time is shown above the status bar.<p>Again there are several different ways to find the value. (like doing a decreased value by... scan), but I'll only explain the easiest. "Unknown initial value", and decreased value. Because you don't know the value it is right now, a exact value wont do any good, so choose as scantype 'Unknown initial value', again, the value type is 4-bytes. (most windows apps use 4-bytes)click first scan and wait till it's done.<p>When it is done click 'hit me'. You'll lose some of your health. (the amount you lost shows for a few seconds and then disappears, but you don't need that) Now go to Cheat Engine, and choose 'Decreased Value' and click 'Next Scan' When that scan is done, click hit me again, and repeat the above till you only find a few.<p>We know the value is between 0 and 500, so pick the one that is most likely the address we need, and add it to the list. Now change the health to 5000, to proceed to the next step.</blockquote></details><h2 id=dense-memory-locations>Dense memory locations</h2><p>The key thing to notice here is that, when we read memory from another process, we do so over <em>entire regions</em>. A memory region is represented by a starting offset, a size, and a bunch of other things like protection level.<p>When running the first scan for an unknown value, all we need to remember is the starting offset and size for every single region. All the candidate locations that could point to our value fall within this range, so it is enough for us to store the range definition, and not every location within it.<p>To gain a better understanding of what this means, let's come up with a more specific scenario. With our current approach of doing things, we store an address (<code>usize</code>) for every location pointing to our desired value. In the case of unknown values, all locations are equally valid, since we don't know what value they should point to yet, and any value they point to is good. With this representation, we would end up with a very large vector:<pre><code class=language-rust data-lang=rust>let locations = vec![0x2000, 0x2001, ..., 0x20ff, 0x2100]; </code></pre><p>This representation is dense. Every single number in the range <code>0x2000..=0x2100</code> is present. So why bother storing the values individually when the range is enough?:<pre><code class=language-rust data-lang=rust>let locations = EntireRegion { range: 0x2000..=0x2100 }; </code></pre><p>Much better! With two <code>usize</code>, one for the starting location and another for the end, we can indicate that we care about all the locations falling in that range.<p>In fact, some accessible memory regions immediately follow eachother, so we could even compact this further and merge regions which are together. But due to their potential differences with regards to protection levels, we will not attempt to merge regions.<p>We don't want to get rid of the old way of storing locations, because once we start narrowing them down, we will want to go back to storing just a few candidates. To keep things tidy, let's introduce a new <code>enum</code> representing either possibility:<pre><code class=language-rust data-lang=rust>use std::ops::Range;@@ -210,4 +210,4 @@ let bytes = &new_memory[base..base + 4];
let new = i32::from_ne_bytes([bytes[0], bytes[1], bytes[2], bytes[3]]); (addr, old, new) }), -</code></pre><h2 id=going-beyond>Going beyond</h2><p>Let's take a look at other possible <code>Scan</code> types. Cheat Engine supports the following initial scan types:<ul><li>Exact Value<li>Bigger than…<li>Smaller than…<li>Value between…<li>Unknown initial value</ul><p>"Bigger than" and "Smaller than" can both be represented by "Value between", so it's pretty much just three.<p>For subsequent scans, in addition to the scan types described above, we find:<ul><li>Increased value<li>Increased value by…<li>Decreased value<li>Decreased value by…<li>Changed value<li>Unchanged value</ul><p>Not only does Cheat Engine provide all of these scans, but all of them can also be negated. For example, "find values that were not increased by 7". One could imagine to also support things like "increased value by range". For the increased and decreased scans, Cheat Engine also supports "at least xx%", so that if the value changed within the specified percentage interval, it will be considered.<p>What about <code>CandidateLocations</code>? I can't tell you how Cheat Engine stores these, but I can tell you that <code>CandidateLocations::Discrete</code> can still be quite inefficient. Imagine you've started with a scan for unknown values and then ran a scan for unchanged valueus. Most values in memory will have been unchanged, but with our current implementation, we are now storing an entire <code>usize</code> address for each of these. One option would be to introduce <code>CandidateLocations::Sparse</code>, which would be a middle ground. You could implement it like <code>Dense</code> and include a vector of booleans telling you which values to consider, or go smaller and use a bitstring or bit vector. You could use a sparse vector data structure.<p><code>Value</code> is very much like <code>CandidateLocations</code>, except that it stores a value to compare against and not an address. Here we can either have an exact value, or an older copy of the memory. Again, keeping a copy of the entire memory chunk when all we need is a handful of values is inefficient. You could keep a mapping from addresses to values if you don't have too many. Or you could shrink and fragment the copied memory in a more optimal way. There's a lot of room for improvement!<p>What if, despite all of the efforts above, we still don't have enough RAM to store all this information? The Cheat Engine Tutorial doesn't use a lot of memory, but as soon as you try scanning bigger programs, like games, you may find yourself needing several gigabytes worth of memory to remember all the found values in order to compare them in subsequent scans. You may even need to consider dumping all the regions to a file and read from it to run the comparisons. For example, running a scan for "unknown value" in Cheat Engine brings its memory up by the same amount of memory used by the process scanned (which makes sense), but as soon as I ran a scan for "unchanged value" over the misaligned values, Cheat Engine's disk usage skyrocketed to 1GB/s (!) for several seconds on my SSD. After it finished, memory usage went down to normal. It was very likely writing out all candidate locations to disk.<h2 id=finale>Finale</h2><p>There is a lot of things to learn from Cheat Engine just by observing its behaviour, and we're only scratching its surface.<p>In the next post, we'll tackle the fourth step of the tutorial: Floating points. So far, we have only been working with <code>i32</code> for simplicity. We will need to update our code to be able to account for different data types, which will make it easy to support other types like <code>i16</code>, <code>i64</code>, or even strings, represented as an arbitrary sequence of bytes.<p>As usual, you can <a href=https://github.com/lonami/memo>obtain the code for this post</a> over at my GitHub. You can run <code>git checkout step3</code> after cloning the repository to get the right version of the code. This version is a bit cleaner than the one presented in the blog, and contains some of the things described in the <a href=https://lonami.dev/blog/woce-3/#going-beyond>Going beyond</a> section. Until next time!<h3 id=footnotes>Footnotes</h3><div class=footnote-definition id=1><sup class=footnote-definition-label>1</sup><p>Well, technically, we will perform a million memory reads<sup class=footnote-reference><a href=#5>5</a></sup>. The issue here is the million calls to <code>ReadProcessMemory</code>, not reading memory per se.</div><div class=footnote-definition id=2><sup class=footnote-definition-label>2</sup><p>Not currently. After a basic implementation works, writing each implementation by hand and fine-tuning them by treating each of them as a special case could yield significant speed improvements. So although it would be a lot of work, this option shouldn't be ruled out completely.</div><div class=footnote-definition id=3><sup class=footnote-definition-label>3</sup><p>You could ask the candidate locations where one should read, which would still keep the code reasonably simple.</div><div class=footnote-definition id=4><sup class=footnote-definition-label>4</sup><p>You could also optimize for this case by determining both the smallest and largest address, and reading enough to cover them both. Or apply additional heuristics to only do so if the ratio of the size you're reading compared to the size you need isn't too large and abort the joint read otherwise. There is a lot of room for optimization here.</div><div class=footnote-definition id=5><sup class=footnote-definition-label>5</sup><p>(A footnote in a footnote?) The machine registers, memory cache and compiler will all help lower this cost, so the generated executable might not actually need that many reads from RAM. But that's getting way too deep into the details now.</div></main><footer><div><p>Share your thoughts, or simply come hang with me <a href=https://t.me/LonamiWebs><img src=/img/telegram.svg alt=Telegram></a> <a href=mailto:totufals@hotmail.com><img src=/img/mail.svg alt=Mail></a></div></footer></article><p class=abyss>Glaze into the abyss… Oh hi there!+</code></pre><h2 id=going-beyond>Going beyond</h2><p>Let's take a look at other possible <code>Scan</code> types. Cheat Engine supports the following initial scan types:<ul><li>Exact Value<li>Bigger than…<li>Smaller than…<li>Value between…<li>Unknown initial value</ul><p>"Bigger than" and "Smaller than" can both be represented by "Value between", so it's pretty much just three.<p>For subsequent scans, in addition to the scan types described above, we find:<ul><li>Increased value<li>Increased value by…<li>Decreased value<li>Decreased value by…<li>Changed value<li>Unchanged value</ul><p>Not only does Cheat Engine provide all of these scans, but all of them can also be negated. For example, "find values that were not increased by 7". One could imagine to also support things like "increased value by range". For the increased and decreased scans, Cheat Engine also supports "at least xx%", so that if the value changed within the specified percentage interval, it will be considered.<p>What about <code>CandidateLocations</code>? I can't tell you how Cheat Engine stores these, but I can tell you that <code>CandidateLocations::Discrete</code> can still be quite inefficient. Imagine you've started with a scan for unknown values and then ran a scan for unchanged valueus. Most values in memory will have been unchanged, but with our current implementation, we are now storing an entire <code>usize</code> address for each of these. One option would be to introduce <code>CandidateLocations::Sparse</code>, which would be a middle ground. You could implement it like <code>Dense</code> and include a vector of booleans telling you which values to consider, or go smaller and use a bitstring or bit vector. You could use a sparse vector data structure.<p><code>Value</code> is very much like <code>CandidateLocations</code>, except that it stores a value to compare against and not an address. Here we can either have an exact value, or an older copy of the memory. Again, keeping a copy of the entire memory chunk when all we need is a handful of values is inefficient. You could keep a mapping from addresses to values if you don't have too many. Or you could shrink and fragment the copied memory in a more optimal way. There's a lot of room for improvement!<p>What if, despite all of the efforts above, we still don't have enough RAM to store all this information? The Cheat Engine Tutorial doesn't use a lot of memory, but as soon as you try scanning bigger programs, like games, you may find yourself needing several gigabytes worth of memory to remember all the found values in order to compare them in subsequent scans. You may even need to consider dumping all the regions to a file and read from it to run the comparisons. For example, running a scan for "unknown value" in Cheat Engine brings its memory up by the same amount of memory used by the process scanned (which makes sense), but as soon as I ran a scan for "unchanged value" over the misaligned values, Cheat Engine's disk usage skyrocketed to 1GB/s (!) for several seconds on my SSD. After it finished, memory usage went down to normal. It was very likely writing out all candidate locations to disk.<h2 id=finale>Finale</h2><p>There is a lot of things to learn from Cheat Engine just by observing its behaviour, and we're only scratching its surface.<p>In the <a href=/blog/woce-4>next post</a>, we'll tackle the fourth step of the tutorial: Floating points. So far, we have only been working with <code>i32</code> for simplicity. We will need to update our code to be able to account for different data types, which will make it easy to support other types like <code>i16</code>, <code>i64</code>, or even strings, represented as an arbitrary sequence of bytes.<p>As usual, you can <a href=https://github.com/lonami/memo>obtain the code for this post</a> over at my GitHub. You can run <code>git checkout step3</code> after cloning the repository to get the right version of the code. This version is a bit cleaner than the one presented in the blog, and contains some of the things described in the <a href=https://lonami.dev/blog/woce-3/#going-beyond>Going beyond</a> section. Until next time!<h3 id=footnotes>Footnotes</h3><div class=footnote-definition id=1><sup class=footnote-definition-label>1</sup><p>Well, technically, we will perform a million memory reads<sup class=footnote-reference><a href=#5>5</a></sup>. The issue here is the million calls to <code>ReadProcessMemory</code>, not reading memory per se.</div><div class=footnote-definition id=2><sup class=footnote-definition-label>2</sup><p>Not currently. After a basic implementation works, writing each implementation by hand and fine-tuning them by treating each of them as a special case could yield significant speed improvements. So although it would be a lot of work, this option shouldn't be ruled out completely.</div><div class=footnote-definition id=3><sup class=footnote-definition-label>3</sup><p>You could ask the candidate locations where one should read, which would still keep the code reasonably simple.</div><div class=footnote-definition id=4><sup class=footnote-definition-label>4</sup><p>You could also optimize for this case by determining both the smallest and largest address, and reading enough to cover them both. Or apply additional heuristics to only do so if the ratio of the size you're reading compared to the size you need isn't too large and abort the joint read otherwise. There is a lot of room for optimization here.</div><div class=footnote-definition id=5><sup class=footnote-definition-label>5</sup><p>(A footnote in a footnote?) The machine registers, memory cache and compiler will all help lower this cost, so the generated executable might not actually need that many reads from RAM. But that's getting way too deep into the details now.</div></main><footer><div><p>Share your thoughts, or simply come hang with me <a href=https://t.me/LonamiWebs><img src=/img/telegram.svg alt=Telegram></a> <a href=mailto:totufals@hotmail.com><img src=/img/mail.svg alt=Mail></a></div></footer></article><p class=abyss>Glaze into the abyss… Oh hi there!
A
blog/woce-4/index.html
@@ -0,0 +1,198 @@
+<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=description content="Official Lonami's website"><meta name=viewport content="width=device-width, initial-scale=1.0, user-scalable=yes"><title> Writing our own Cheat Engine: Floating points | Lonami's Blog </title><link rel=stylesheet href=/style.css><body><article><nav class=sections><ul class=left><li><a href=/>lonami's site</a><li><a href=/blog class=selected>blog</a><li><a href=/golb>golb</a></ul><div class=right><a href=https://github.com/LonamiWebs><img src=/img/github.svg alt=github></a><a href=/blog/atom.xml><img src=/img/rss.svg alt=rss></a></div></nav><main><h1 class=title>Writing our own Cheat Engine: Floating points</h1><div class=time><p>2021-02-22</div><p>This is part 4 on the <em>Writing our own Cheat Engine</em> series:<ul><li><a href=/blog/woce-1>Part 1: Introduction</a> (start here if you're new to the series!)<li><a href=/blog/woce-2>Part 2: Exact Value scanning</a><li><a href=/blog/woce-3>Part 3: Unknown initial value</a><li>Part 4: Floating points</ul><p>In part 3 we did a fair amount of plumbing in order to support scan modes beyond the trivial "exact value scan". As a result, we have abstracted away the <code>Scan</code>, <code>CandidateLocations</code> and <code>Value</code> types as a separate <code>enum</code> each. Scanning for changed memory regions in an opened process can now be achieved with three lines of code:<pre><code class=language-rust data-lang=rust>let regions = process.memory_regions(); +let first_scan = process.scan_regions(&regions, Scan::InRange(0, 500)); +let second_scan = process.rescan_regions(&first_scan, Scan::DecreasedBy(7)); +</code></pre><p>How's that for programmability? No need to fire up Cheat Engine's GUI anymore!<p>The <code>first_scan</code> in the example above remembers all the found <code>Value</code> within the range specified by <code>Scan</code>. Up until now, we have only worked with <code>i32</code>, so that's the type the scans expect and what they work with.<p>Now it's time to introduce support for different types, like <code>f32</code>, <code>i64</code>, or even more atypical ones, like arbitrary sequences of bytes (think of strings) or even numbers in big-endian.<p>Tighten your belt, because this post is quite the ride. Let's get right into it!<h2 id=floating-points>Floating points</h2><details open><summary>Cheat Engine Tutorial: Step 4</summary> <blockquote><p>In the previous tutorial we used bytes to scan, but some games store information in so called 'floating point' notations. (probably to prevent simple memory scanners from finding it the easy way). A floating point is a value with some digits behind the point. (like 5.12 or 11321.1)<p>Below you see your health and ammo. Both are stored as Floating point notations, but health is stored as a float and ammo is stored as a double. Click on hit me to lose some health, and on shoot to decrease your ammo with 0.5<p>You have to set BOTH values to 5000 or higher to proceed.<p>Exact value scan will work fine here, but you may want to experiment with other types too.<p>Hint: It is recommended to disable "Fast Scan" for type double</blockquote></details><h2 id=generic-values>Generic values</h2><p>The <code>Value</code> enumeration holds scanned values, and is currently hardcoded to store <code>i32</code>. The <code>Scan</code> type also holds a value, the value we want to scan for. Changing it to support other types is trivial:<pre><code class=language-rust data-lang=rust>pub enum Scan<T> { + Exact(T), + Unknown, + Decreased, + // ...other variants... +} + +pub enum Value<T> { + Exact(T), + AnyWithin(Vec<u8>), +} +</code></pre><p><code>AnyWithin</code> is the raw memory, and <code>T</code> can be interpreted from any sequence of bytes thanks to our friend <a href=https://doc.rust-lang.org/stable/std/mem/fn.transmute.html><code>mem::transmute</code></a>. This change alone is enough to store an arbitrary <code>T</code>! So we're done now? Not really, no.<p>First of all, we need to update all the places where <code>Scan</code> or <code>Value</code> are used. Our first stop is the scanned <code>Region</code>, which holds the found <code>Value</code>:<pre><code class=language-rust data-lang=rust>pub struct Region<T> { + pub info: MEMORY_BASIC_INFORMATION, + pub locations: CandidateLocations, + pub value: Value<T>, +} +</code></pre><p>Then, we need to update everywhere <code>Region</code> is used, and on and on… All in all this process is just repeating <code>cargo check</code>, letting the compiler vent on you, and taking good care of it by fixing the errors. It's quite reassuring to know you will not miss a single place. Thank you, compiler!<p>But wait, how could scanning for a decreased value work for any <code>T</code>? The type is not <code>Ord</code>, we should add some trait bounds. And also, what happens if the type is not <code>Copy</code>? It could implement <code>Drop</code><sup class=footnote-reference><a href=#1>1</a></sup>, and we will be transmuting from raw bytes, which would trigger the <code>Drop</code> implementation when we're done with the value! Not memory safe at all! And how could we possibly cast raw memory to the type without knowing its siz– oh nevermind, <a href=https://doc.rust-lang.org/stable/std/marker/trait.Sized.html><code>T</code> is already <code>Sized</code> by default</a>. But anyway, we need the other bounds.<p>In order to not repeat ourselves, we will implement a new <code>trait</code>, let's say <code>Scannable</code>, which requires all other bounds:<pre><code class=language-rust data-lang=rust>pub trait Scannable: Copy + PartialEq + PartialOrd {} + +impl<T: Copy + PartialEq + PartialOrd> Scannable for T {} +</code></pre><p>And fix our definitions:<pre><code class=language-rust data-lang=rust>pub enum Scan<T: Scannable> { ... } +pub enum Value<T: Scannable> { ... } +pub struct Region<T: Scannable> { ... } + +// ...and the many other places referring to T +</code></pre><p>Every type which is <code>Copy</code>, <code>PartialEq</code> and <code>PartialOrd</code> can be scanned over<sup class=footnote-reference><a href=#2>2</a></sup>, because we <code>impl Scan for T</code> where the bounds are met. Unfortunately, we cannot require <code>Eq</code> or <code>Ord</code> because the floating point types do not implement it.<h2 id=transmuting-memory>Transmuting memory</h2><p>Also known as reinterpreting a bunch of bytes as something else, or perhaps it stands for "summoning the demon":<blockquote><p><code>transmute</code> is <strong>incredibly</strong> unsafe. There are a vast number of ways to cause <a href=https://doc.rust-lang.org/stable/reference/behavior-considered-undefined.html>undefined behavior</a> with this function. <code>transmute</code> should be the absolute last resort.</blockquote><p>Types like <code>i32</code> define methods such as <a href=https://doc.rust-lang.org/stable/std/primitive.i32.html#method.from_ne_bytes><code>from_ne_bytes</code></a> and <a href=https://doc.rust-lang.org/stable/std/primitive.i32.html#method.to_ne_bytes><code>to_ne_bytes</code></a> which convert raw bytes from and into its native representation. This is all really nice, but unfortunately, there's no standard trait in the Rust's standard library to "interpret a type <code>T</code> as the byte sequence of its native representation". <code>transmute</code>, however, does exist, and similar to any other <code>unsafe</code> function, it's safe to call <strong>as long as we respect its invariants</strong>. What are these invariants<sup class=footnote-reference><a href=#3>3</a></sup>?<blockquote><p>Both types must have the same size</blockquote><p>Okay, we can just assert that the window length matches the type's length. What else?<blockquote><p>Neither the original, nor the result, may be an <a href=https://doc.rust-lang.org/nomicon/what-unsafe-does.html>invalid value</a>.</blockquote><p>What's an invalid value?<blockquote><ul><li>a <code>bool</code> that isn't 0 or 1<li>an <code>enum</code> with an invalid discriminant<li>a null <code>fn</code> pointer<li>a <code>char</code> outside the ranges [0x0, 0xD7FF] and [0xE000, 0x10FFFF]<li>a <code>!</code> (all values are invalid for this type)<li>an integer (<code>i*</code>/<code>u*</code>), floating point value (<code>f*</code>), or raw pointer read from uninitialized memory, or uninitialized memory in a <code>str</code>.<li>a reference/<code>Box</code> that is dangling, unaligned, or points to an invalid value.<li>a wide reference, <code>Box</code>, or raw pointer that has invalid metadata: <ul><li><code>dyn Trait</code> metadata is invalid if it is not a pointer to a vtable for <code>Trait</code> that matches the actual dynamic trait the pointer or reference points to<li>slice metadata is invalid if the length is not a valid <code>usize</code> (i.e., it must not be read from uninitialized memory)</ul><li>a type with custom invalid values that is one of those values, such as a <code>NonNull</code> that is null. (Requesting custom invalid values is an unstable feature, but some stable libstd types, like <code>NonNull</code>, make use of it.)</ul></blockquote><p>Okay, that's actually an awful lot. Types like <code>bool</code> implement all the trait bounds we defined, and it would be insta-UB to ever try to cast them from arbitrary bytes. The same goes for <code>char</code>, and all <code>enum</code> are out of our control, too. At least we're safe on the "memory is initialized" front.<p>Dang it, I really wanted to use <code>transmute</code>! But if we were to use it for arbitrary types, it would trigger undefined behaviour sooner than later.<p>We have several options here:<ul><li>Make it an <code>unsafe trait</code>. Implementors will be responsible for ensuring that the type they're implementing it for can be safely transmuted from and into.<li><a href=https://rust-lang.github.io/api-guidelines/future-proofing.html>Seal the <code>trait</code></a> and implement it only for types we know are safe<sup class=footnote-reference><a href=#4>4</a></sup>, like <code>i32</code>.<li>Add methods to the <code>trait</code> definition that do the conversion of the type into its native representation.</ul><p>We will go with the first option<sup class=footnote-reference><a href=#5>5</a></sup>, because I really want to use <code>transmute</code>, and I want users to be able to implement the trait on their own types.<p>In any case, we need to change our <code>impl</code> to something more specific, in order to prevent it from automatically implementing the trait for types for which their memory representation has invalid values. So we get rid of this:<pre><code class=language-rust data-lang=rust>pub trait Scannable: Copy + PartialEq + PartialOrd {} + +impl<T: Copy + PartialEq + PartialOrd> Scannable for T {} +</code></pre><p>And replace it with this:<pre><code class=language-rust data-lang=rust>pub unsafe trait Scannable: Copy + PartialEq + PartialOrd {} + +macro_rules! impl_many { + ( unsafe impl $trait:tt for $( $ty:ty ),* ) => { + $( unsafe impl $trait for $ty {} )* + }; +} + +// SAFETY: all these types respect `Scannable` invariants. +impl_many!(unsafe impl Scannable for i8, u8, i16, u16, i32, u32, i64, u64, f32, f64); +</code></pre><p>Making a small macro for things like these is super useful. You could of course write <code>unsafe impl Scannable for T</code> for all ten <code>T</code> as well, but that introduces even more <code>unsafe</code> to read. Last but not least, let's replace the hardcoded <code>i32::from_ne_bytes</code> and <code>i32::to_ne_bytes</code> with <code>mem::transmute</code>.<p>All the <code>windows(4)</code> need to be replaced with <code>windows(mem::size_of::<T>())</code> because the size may no longer be <code>4</code>. All the <code>i32::from_ne_bytes(...)</code> need to be replaced with <code>mem::transmute::<_, T>(...)</code>. We explicitly write out <code>T</code> to make sure the compiler doesn't accidentally infer something we didn't intend.<p>And… it doesn't work at all. We're working with byte slices of arbitrary length. We cannot transmute a <code>&[]</code> type, which is 16 bytes (8 for the pointer and 8 for the length), to our <code>T</code>. My plan to use transmute can't possibly work here. Sigh.<h2 id=not-quite-transmuting-memory>Not quite transmuting memory</h2><p>Okay, we can't transmute, because we don't have a sized value, we only have a slice of bytes pointing somewhere else. What we <em>could</em> do is reinterpret the pointer to those bytes as a different type, and then dereference it! This is still a form of "transmutation", just without using <code>transmute</code>.<pre><code class=language-rust data-lang=rust>let value = unsafe { *(window.as_ptr() as *const T) }; +</code></pre><p>Woop! You can compile this and test it out on the step 2 and 3 of the tutorial, using <code>i32</code>, and it will still work! Something troubles me, though. Can you see what it is?<p>When we talked about invalid values, it had a note about unaligned references:<blockquote><p>a reference/<code>Box</code> that is dangling, unaligned, or points to an invalid value.</blockquote><p>Our <code>window</code> is essentially a reference to <code>T</code>. The only difference is we're working at the pointer level, but they're pretty much references. Let's see what the documentation for <a href=https://doc.rust-lang.org/std/primitive.pointer.html><code>pointer</code></a> has to say as well, since we're dereferencing pointers:<blockquote><p>when a raw pointer is dereferenced (using the <code>*</code> operator), it must be non-null and aligned.</blockquote><p>It must be aligned. The only reason why our data is aligned is because we are also performing a "fast scan", so we only look at aligned locations. This is a time bomb waiting to blow up. Is there any other way to <a href=https://doc.rust-lang.org/std/ptr/fn.read.html><code>read</code></a> from a pointer which is safer?<blockquote><p><code>src</code> must be properly aligned. Use <a href=https://doc.rust-lang.org/std/ptr/fn.read_unaligned.html><code>read_unaligned</code></a> if this is not the case.</blockquote><p>Bingo! Both <code>read</code> and <code>read_unaligned</code>, unlike dereferencing the pointer, will perform a copy, but if it can make the code less prone to blowing up, I'll take it<sup class=footnote-reference><a href=#6>6</a></sup>. Let's change the code one more time:<pre><code class=language-rust data-lang=rust>let current = unsafe { window.as_ptr().cast::<T>().read_unaligned() }; +</code></pre><p>I prefer to avoid type annotations in variables where possible, which is why I use the <a href=https://www.reddit.com/r/rust/comments/3fimgp/why_double_colon_rather_that_dot/ctozkd0/>turbofish</a> so often. You can get rid of the cast and use a type annotation instead, but make sure the type is known, otherwise it will think it's <code>u8</code> because <code>window</code> is a <code>&[u8]</code>.<p>Now, this is all cool and good. You can replace <code>i32</code> with <code>f32</code> for <code>T</code> and you'll be able to get halfway done with the step 4 of Cheat Engine's tutorial. Unfortunately, as it is, this code is not enough to complete step 4 with exact scans<sup class=footnote-reference><a href=#7>7</a></sup>. You see, comparing floating point values is not as simple as checking for bitwise equality. We were actually really lucky that the <code>f32</code> part works! But the values in the <code>f64</code> part are not as precise as our inputs, so our exact scan fails.<p>Using a fixed type parameter is pretty limiting as well. On the one hand, it is nice that, if you scan for <code>i32</code>, the compiler statically guarantees that subsequent scans will also happen on <code>i32</code> and thus be compatible. On the other, this requires us to know the type at compile time, which for an interactive program, is not possible. While we <em>could</em> create different methods for each supported type and, at runtime, decide to which we should jump, I am not satisfied with that solution. It also means we can't switch from scanning an <code>u32</code> to an <code>i32</code>, for whatever reason.<p>So we need to work around this once more.<h2 id=rethinking-the-scans>Rethinking the scans</h2><p>What does our scanning function need, really? It needs a way to compare two chunks of memory as being equal or not (as we have seen, this isn't trivial with types such as floating point numbers) and, for other types of scans, it needs to be able to produce an ordering, or calculate a difference.<p>Instead of having a our trait require the bounds <code>PartialEq</code> and <code>PartialOrd</code>, we can define our own methods to compare <code>Self</code> with <code>&[u8]</code>. It still should be <code>Clone</code>, so we can pass it around without worrying about lifetimes:<pre><code class=language-rust data-lang=rust>// Callers must `assert_eq!(memory.len(), mem::size_of::<Self>())`. +unsafe fn eq(&self, memory: &[u8]) -> bool; +unsafe fn cmp(&self, memory: &[u8]) -> Ordering; +</code></pre><p>This can be trivially implemented for all integer types:<pre><code class=language-rust data-lang=rust>macro_rules! impl_scannable_for_int { + ( $( $ty:ty ),* ) => { + $( + // SAFETY: caller is responsible to `assert_eq!(memory.len(), mem::size_of::<T>())` + impl Scannable for $ty { + unsafe fn eq(&self, memory: &[u8]) -> bool { + let other = unsafe { memory.as_ptr().cast::<$ty>().read_unaligned() }; + *self == other + } + + unsafe fn cmp(&self, memory: &[u8]) -> Ordering { + let other = unsafe { memory.as_ptr().cast::<$ty>().read_unaligned() }; + <$ty as Ord>::cmp(self, &other) + } + } + )* + }; +} + +impl_scannable_for_int!(i8, u8, i16, u16, i32, u32, i64, u64); +</code></pre><p>The funny <code><$ty as Ord></code> is because I decided to call the method <code>Scannable::cmp</code>, so I have to disambiguate between it and <code>Ord::cmp</code>. We can go ahead and update the code using <code>Scannable</code> to use these new functions instead.<p>Now, you may have noticed I only implemented it for the integer types. That's because floats need some extra care. Unfortunately, floating point types do not have any form of "precision" embedded in them, so we can't accurately say "compare these floats to the precision level the user specified". What we can do, however, is drop a few bits from the mantissa, so "relatively close" quantities are considered equal. It's definitely not as good as comparing floats to the user's precision, but it will get the job done.<p>I'm going to arbitrarily say that we are okay comparing with "half" the precision. We can achieve that by masking half of the bits from the mantissa to zero:<pre><code class=language-rust data-lang=rust> +macro_rules! impl_scannable_for_float { + ( $( $ty:ty : $int_ty:ty ),* ) => { + $( + #[allow(unused_unsafe)] // mind you, it is necessary + impl Scannable for $ty { + unsafe fn eq(&self, memory: &[u8]) -> bool { + const MASK: $int_ty = !((1 << (<$ty>::MANTISSA_DIGITS / 2)) - 1); + + // SAFETY: caller is responsible to `assert_eq!(memory.len(), mem::size_of::<T>())` + let other = unsafe { memory.as_ptr().cast::<$ty>().read_unaligned() }; + let left = <$ty>::from_bits(self.to_bits() & MASK); + let right = <$ty>::from_bits(other.to_bits() & MASK); + left == right + } + + ... + } + )* + }; +} + +impl_scannable_for_float!(f32: u32, f64: u64); +</code></pre><p>You may be wondering what's up with that weird <code>MASK</code>. Let's visualize it with a <a href=https://en.wikipedia.org/wiki/Bfloat16_floating-point_format><code>f16</code></a>. This type has 16 bits, 1 for sign, 5 for exponent, and 10 for the mantissa:<pre><code>S EEEEE MMMMMMMMMM +</code></pre><p>If we substitute the constant with the numeric value and operate:<pre><code class=language-rust data-lang=rust>!((1 << (10 / 2)) - 1) +!((1 << 5) - 1) +!(0b00000000_00100000 - 1) +!(0b00000000_00011111) +0b11111111_11100000 +</code></pre><p>So effectively, half of the mantisssa bit will be masked to 0. For the <code>f16</code> example, this makes us lose 5 bits of precision. Comparing two floating point values with their last five bits truncated is equivalent to checking if they are "roughly equal"!<p>When Cheat Engine scans for floating point values, several additional settings show, and one such option is "truncated". I do not know if it behaves like this, but it might.<p>Let's try this out:<pre><code class=language-rust data-lang=rust>#[test] +fn f32_roughly_eq() { + let left = 0.25f32; + let right = 0.25000123f32; + let memory = unsafe { mem::transmute::<_, [u8; 4]>(right) }; + assert_ne!(left, right); + assert!(unsafe { Scannable::eq(&left, &memory) }); +} +</code></pre><pre><code>>cargo test f32_roughly_eq + +running 1 test +test scan::candidate_location_tests::f32_roughly_eq ... ok +</code></pre><p>Huzzah! The <code>assert_ne!</code> makes sure that a normal comparision would fail, and then we <code>assert!</code> that our custom one passes the test. When the user performs an exact scan, the code will be more tolerant to the user's less precise inputs, which overall should result in a nicer experience.<h2 id=dynamically-sized-scans>Dynamically sized scans</h2><p>The second problem we need to solve is the possibility of the size not being known at compile time<sup class=footnote-reference><a href=#8>8</a></sup>. While we can go as far as scanning over strings of a known length, this is rather limiting, because we need to know the length at compile time<sup class=footnote-reference><a href=#9>9</a></sup>. Heap allocated objects are another problem, because we don't want to compare the memory representation of the stack object, but likely the memory where they point to (such as <code>String</code>).<p>Instead of using <code>mem::size_of</code>, we can add a new method to our <code>Scannable</code>, <code>size</code>, which will tell us the size required of the memory view we're comparing against:<pre><code class=language-rust data-lang=rust>unsafe impl Scannable { + ... + + fn size(&self) -> usize; +} +</code></pre><p>It is <code>unsafe</code> to implement, because we are relying on the returned value to be truthful and unchanging. It should be safe to call, because it cannot have any invariants. Unfortunately, signaling "unsafe to implement" is done by marking the entire trait as <code>unsafe</code>, since "unsafe to call" is reserved for <code>unsafe fn</code>, and even though the rest of methods are not necessarily unsafe to implement, they're treated as such.<p>At the moment, <code>Scannable</code> cannot be made into a trait object because it is <a href=https://doc.rust-lang.org/stable/error-index.html#E0038>not object safe</a>. This is caused by the <code>Clone</code> requirement on all <code>Scannable</code> object, which in turn needs the types to be <code>Sized</code> because <code>clone</code> returns <code>Self</code>. Because of this, the size must be known.<p>However, we <em>can</em> move the <code>Clone</code> requirement to the methods that need it! This way, <code>Scannable</code> can remain object safe, enabling us to do the following:<pre><code class=language-rust data-lang=rust>unsafe impl<T: AsRef<dyn Scannable> + AsMut<dyn Scannable>> Scannable for T { + unsafe fn eq(&self, memory: &[u8]) -> bool { + self.as_ref().eq(memory) + } + + unsafe fn cmp(&self, memory: &[u8]) -> Ordering { + self.as_ref().cmp(memory) + } + + fn mem_view(&self) -> &[u8] { + self.as_ref().mem_view() + } + + fn size(&self) -> usize { + self.as_ref().size() + } +} +</code></pre><p>Any type which can be interpreted as a reference to <code>Scannable</code> is also a scannable! This enables us to perform scans over <code>Box<dyn i32></code>, where the type is known at runtime! Or rather, it would, if <code>Box<dyn T></code> implemented <code>Clone</code>, which it can't<sup class=footnote-reference><a href=#10>10</a></sup> because that's what prompted this entire issue. Dang it! I can't catch a breath today!<p>Okay, let's step back. Why did we need our scannables to be clone in the first place? When we perform exact scans, we store the original value in the region, which we don't own, so we clone it. But what if we <em>did</em> own the value? Instead of taking the <code>Scan</code> by reference, which holds <code>T: Scannable</code>, we could take it by value. If we get rid of all the <code>Clone</code> bounds and update <code>Scan::run</code> to take <code>self</code>, along with updating all the things that take a <code>Region</code> to take them by value as well, it should all work out.<p>But it does not. If we take <code>Scan</code> by value, with it not being <code>Clone</code>, we simply can't use it to scan over multiple regions. After the first region, we have lost the <code>Scan</code>.<p>Let's take a second step back. We are scanning memory, and we want to compare memory, but we want to treat the memory with different semantics (for example, if we treat it as <code>f32</code>, we want to check for rough equality). Instead of storing the <em>value</em> itself, we could store its <em>memory representation</em>, and when we compare memory representations, we can do so under certain semantics.<p>First off, let's revert getting rid of all <code>Clone</code>. Wherever we stored a <code>T</code>, we will now store a <code>Vec<u8></code>. We will still use a type parameter to represent the "implementations of <code>Scannable</code>". For this to work, our definitions need to use <code>T</code> somewhere, or else the compiler refuses to compile the code with error <a href=https://doc.rust-lang.org/stable/error-index.html#E0392>E0392</a>. For this, I will stick a <a href=https://doc.rust-lang.org/stable/std/marker/struct.PhantomData.html><code>PhantomData</code></a> in the <code>Exact</code> variant. It's a bit pointless to include it in all variants, and <code>Exact</code> seems the most appropriated:<pre><code class=language-rust data-lang=rust>pub enum Scan<T: Scannable> { + Exact(Vec<u8>, PhantomData<T>), + Unknown, + ... +} +</code></pre><p>This keeps in line with <code>Value</code>:<pre><code class=language-rust data-lang=rust>pub enum Value<T: Scannable> { + Exact(Vec<u8>, PhantomData<T>), + ... +} +</code></pre><p>Our <code>Scannable</code> will no longer work on <code>T</code> and <code>&[u8]</code>. Instead, it will work on two <code>&[u8]</code>. We will also need a way to interpret a <code>T</code> as <code>&[u8]</code>, which we can achieve with a new method, <code>mem_view</code>. This method interprets the raw memory representation of <code>self</code> as its raw bytes. It also lets us get rid of <code>size</code>, because we can simply do <code>mem_view().len()</code>. It's still <code>unsafe</code> to implement, because it should return the same length every time:<pre><code class=language-rust data-lang=rust>pub unsafe trait Scannable { + // Callers must `assert_eq!(left.len(), right.len(), self.mem_view().len())`. + unsafe fn eq(left: &[u8], right: &[u8]) -> bool; + unsafe fn cmp(left: &[u8], right: &[u8]) -> Ordering; + fn mem_view(&self) -> &[u8]; +} +</code></pre><p>But now we can't use it in trait object, so the following no longer works:<pre><code class=language-rust data-lang=rust>unsafe impl<T: AsRef<dyn Scannable> + AsMut<dyn Scannable>> Scannable for T { + ... +} +</code></pre><p>Ugh! Well, to be fair, we no longer have a "scannable" at this point. It's more like a scan mode that tells us how memory should be compared according to a certain type. Let's split the trait into two: one for the scan mode, and other for "things which are scannable":<pre><code class=language-rust data-lang=rust>pub trait ScanMode { + unsafe fn eq(left: &[u8], right: &[u8]) -> bool; + unsafe fn cmp(left: &[u8], right: &[u8]) -> Ordering; +} + +pub unsafe trait Scannable { + type Mode: ScanMode; + + fn mem_view(&self) -> &[u8]; +} +</code></pre><p>Note that we have an associated <code>type Mode</code> which contains the corresponding <code>ScanMode</code>. If we used a trait bound such as <code>Scannable: ScanMode</code>, we'd be back to square one: it would inherit the method definitions that don't use <code>&self</code> and thus cannot be used as trait objects.<p>With these changes, it is possible to implement <code>Scannable</code> for any <code>dyn Scannable</code>:<pre><code class=language-rust data-lang=rust>unsafe impl<T: ScanMode + AsRef<dyn Scannable<Mode = Self>>> Scannable for T { + type Mode = Self; + + fn mem_view(&self) -> &[u8] { + self.as_ref().mem_view() + } +} +</code></pre><p>We do have to adjust a few places of the code to account for both <code>Scannable</code> and <code>ScanMode</code>, but all in all, it's pretty straightforward. Things like <code>Value</code> don't need to store the <code>Scannable</code> anymore, just a <code>Vec<u8></code>. It also doesn't need the <code>ScanMode</code>, because it's not going to be scanning anything on its own. This applies transitively to <code>Region</code> which was holding a <code>Value</code>.<p><code>Value</code> <em>does</em> need to be updated to store the size of the region we are scanning for, however, because we need that information when running a subsequent scan. For all <code>Scan</code> that don't have a explicit thing to scan for (like <code>Decreased</code>), the <code>size</code> also needs to be stored in them.<p>Despite all our efforts, we're still unable to return an <code>Scannable</code> chosen at runtime.<pre><code class=language-rust data-lang=rust>fn prompt_user_for_scan() -> Scan<Box<dyn Scannable<Mode = ???>>> { + todo!() +} +</code></pre><p>As far as I can tell, there's simply no way to specify that type. We want to return a type which is scannable, which has itself (which is also a <code>ScanMode</code>) as the corresponding mode. Even if we just tried to return the mode, we simply can't, because it's not object-safe. Is this the end of the road?<h2 id=specifying-the-scan-mode>Specifying the scan mode</h2><p>We need a way to pass an arbitrary scan mode to our <code>Scan</code>. This scan mode should go in tandem with <code>Scannable</code> types, because it would be unsafe otherwise. We've seen that using a type just doesn't cut it. What else can we do?<p>Using an enumeration is a no-go, because I want users to be able to extend it further. I also would like to avoid having to update the <code>enum</code> and all the matches every time I come up with a different type combination. And it could get pretty complicated if I ever built something dynamically, such as letting the user combine different scans in one pass.<p>So what if we make <code>Scannable</code> return a value that implements the functions we need?<pre><code class=language-rust data-lang=rust>pub struct ScanMode { + eq: unsafe fn(left: &[u8], right: &[u8]) -> bool, + cmp: unsafe fn(left: &[u8], right: &[u8]) -> Ordering, +} +</code></pre><p>It's definitely… non-conventional. But hey, now we're left with the <code>Scannable</code> trait, which is object-safe, and does not have any type parameters!<pre><code class=language-rust data-lang=rust>pub unsafe trait Scannable { + fn mem_view(&self) -> &[u8]; + fn scan_mode(&self) -> ScanMode; +} +</code></pre><p>It is a bit weird, but defining local functions and using those in the returned value is a nice way to keep things properly scoped:<pre><code class=language-rust data-lang=rust>macro_rules! impl_scannable_for_int { + ( $( $ty:ty ),* ) => { + $( + unsafe impl Scannable for $ty { + fn mem_view(&self) -> &[u8] { + unsafe { std::slice::from_raw_parts(self as *const _ as *const u8, mem::size_of::<$ty>()) } + } + + fn scan_mode(&self) -> ScanMode { + unsafe fn eq(left: &[u8], right: &[u8]) -> bool { + ... + } + + unsafe fn cmp(left: &[u8], right: &[u8]) -> Ordering { + ... + } + + ScanMode { eq, cmp } + } + } + )* + }; +} +</code></pre><p>Our <code>Scan</code> needs to store the <code>Scannable</code> type, and not just the memory, once again. For variants that don't need any value, they can store the <code>ScanMode</code> and size instead.<p>Does this solution work? Yes! It's possible to return a <code>Box<dyn Scannable></code> from a function, and underneath, it may be using any type which is <code>Scannable</code>. Is this the best solution? Well, that's hard to say. This is <em>one</em> of the possible solutions.<p>We have been going around in circles for quite some time now, so I'll leave it there. It's a solution, which may not be pretty, but it works. With these changes, the code is capable of completing all of the steps in the Cheat Engine tutorial up until point!<h2 id=finale>Finale</h2><p>If there's one lesson to learn from this post, it's that there is often no single correct solution to a problem. We could have approached the scan types in many, many ways (and we tried quite a few!), but in the end, choosing one option or the other comes down to your (sometimes self-imposed) requirements.<p>You may <a href=https://github.com/lonami/memo>obtain the code for this post</a> over at my GitHub. You can run <code>git checkout step4</code> after cloning the repository to get the right version of the code. The code has gone through a lot of iterations, and I'd still like to polish it a bit more, so it might slightly differ from the code presented in this entry.<p>If you feel adventurous, Cheat Engine has different options for scanning floating point types: "rounded (default)", "rounded (extreme)", and truncated. Optionally, it can scan for "simple values only". You could go ahead and toy around with these!<p>We didn't touch on types with different lengths, such as strings. You could support UTF-8, UTF-16, or arbitrary byte sequences. This post also didn't cover scanning for multiple things at once, known as "groupscan commands", although from what I can tell, these are just a nice way to scan for arbitrary byte sequences.<p>We also didn't look into supporting different the same scan with different alignments. All these things may be worth exploring depending on your requirements. You could even get rid of such genericity and go with something way simpler. Supporting <code>i32</code>, <code>f32</code> and <code>f64</code> is enough to complete the Cheat Engine tutorial. But I wanted something more powerful, although my solution currently can't scan for a sequence such as "exact type, unknown, exact matching the unknown". So yeah.<p>In the next post, we'll tackle the fifth step of the tutorial: Code finder. Cheat Engine attaches its debugger to the process for this one, and then replaces the instruction that performs the write with a different no-op so that nothing is written anymore. This will be quite the challenge!<h3 id=footnotes>Footnotes</h3><div class=footnote-definition id=1><sup class=footnote-definition-label>1</sup><p><a href=https://doc.rust-lang.org/stable/std/ops/trait.Drop.html#copy-and-drop-are-exclusive><code>Copy</code> and <code>Drop</code> are exclusive</a>. See also <a href=https://doc.rust-lang.org/stable/error-index.html#E0184>E0184</a>.</div><div class=footnote-definition id=2><sup class=footnote-definition-label>2</sup><p>If you added more scan types that require additional bounds, make sure to add them too. For example, the "decreased by" scan requires the type to <code>impl Sub</code>.</div><div class=footnote-definition id=3><sup class=footnote-definition-label>3</sup><p>This is a good time to remind you to read the documentation. It is of special importance when dealing with <code>unsafe</code> methods; I recommend reading it a couple times.</div><div class=footnote-definition id=4><sup class=footnote-definition-label>4</sup><p>Even with this option, it would not be a bad idea to make the trait <code>unsafe</code>.</div><div class=footnote-definition id=5><sup class=footnote-definition-label>5</sup><p>Not for long. As we will find out later, this approach has its limitations.</div><div class=footnote-definition id=6><sup class=footnote-definition-label>6</sup><p>We can still perform the pointer dereference when we know it's aligned. This would likely be an optimization, although it would definitely complicate the code more.</div><div class=footnote-definition id=7><sup class=footnote-definition-label>7</sup><p>It <em>would</em> work if you scanned for unknown values and then checked for decreased values repeatedly. But we can't just leave exact scan broken!</div><div class=footnote-definition id=8><sup class=footnote-definition-label>8</sup><p>Unfortunately, this makes some optimizations harder or even impossible to perform. Providing specialized functions for types where the size is known at compile time could be worth doing. Programming is all tradeoffs.</div><div class=footnote-definition id=9><sup class=footnote-definition-label>9</sup><p><a href=https://blog.rust-lang.org/2021/02/26/const-generics-mvp-beta.html>Rust 1.51</a>, which was not out at the time of writing, would make it a lot easier to allow scanning for fixed-length sequences of bytes, thanks to const generics.</div><div class=footnote-definition id=10><sup class=footnote-definition-label>10</sup><p>Workarounds do exist, such as <a href=https://crates.io/crates/dyn-clone>dtolnay's <code>dyn-clone</code></a>. But I would rather not go that route.</div></main><footer><div><p>Share your thoughts, or simply come hang with me <a href=https://t.me/LonamiWebs><img src=/img/telegram.svg alt=Telegram></a> <a href=mailto:totufals@hotmail.com><img src=/img/mail.svg alt=Mail></a></div></footer></article><p class=abyss>Glaze into the abyss… Oh hi there!
M
sitemap.xml
→
sitemap.xml
@@ -205,6 +205,10 @@ <loc>https://lonami.dev/blog/woce-3/</loc>
<lastmod>2021-02-19</lastmod> </url> <url> + <loc>https://lonami.dev/blog/woce-4/</loc> + <lastmod>2021-02-22</lastmod> + </url> + <url> <loc>https://lonami.dev/blog/world-edit/</loc> <lastmod>2018-07-11</lastmod> </url>