AwesomeNumbersKit Benchmarks

[mgriebling]

I was going to use ANK in a project of mine but noticed there were no benchmarks.

So, I did some of my own and, if ok, thought I'd publish them for others:

At first I thought ANK's poor performance was due to generics but the addition and shift numbers are very close to Apple's unreleased UInt128 numbers. So, not sure why multiplication, division, and string operations took such big hits.

Note: These were using the ANKFullWidthKit numbers. I don't have any other UInt256 or higher libraries to compare against so haven't done those benchmarks yet.

[oscbyspro]

I don't mind. I haven't included benchmarks because I haven't measured it against any other implementation, so I appreciate that you did. I'm curious, however, about that string conversion. Did you use the correct encoding/decoding methods? Because I'm aware that the performance is garbage for Apple's default implementation (#60).

Edit: for clarity, I'm asking which pair of methods you used:

 String(_:radix:) ❌
UInt256(_:radix:) ❌

 String(encoding:radix:) ✅
UInt256(decoding:radix:) ✅

UInt256.decodeBigEndianText(_:radix:) ✅
UInt256.encodeBigEndianText(_:radix:uppercase:) ✅

[mgriebling]

I used the UInt128(_:radix:) for the from string and String(_:radix:) (more specifically the .description) for the to string. I reran the benchmarks using your preferred methods and had the following results:

The from string conversion now looks much better. Just a suggestion, but maybe you could override Apple's default methods instead of introducing new methods.

[oscbyspro]

If you don't mind me asking, did you benchmark using Instruments or Xcode with the "code coverage" setting enabled? I measured a rework and observed 4x slower shifts, 3x slower multiplication, and 6x slower division than expected - because this setting was on by default. It would be nice to know whether your measurements are related to (#113).

[mgriebling]

I used a very simple custom test bench:

func testPerformanceUInt128Divide() {
    // Divide is 40X faster than UInt128 from Gerber, Apple is 1.5X faster
    let x = UInt128("123456789012345678901234567890")!
    let y = UInt128(100_000_000)
    measure(name: "ANK   Divide     ") {
        for _ in 1...1000 {
            let _ = x / y
        }
    }
}

where the measure function is defined as:

@discardableResult
func measure<A>(name: String = "", _ block: () -> A) -> A {
    let startTime = CACurrentMediaTime()
    let result = block()
    let timeElapsed = CACurrentMediaTime() - startTime
    print("Time: \(name) - \(timeElapsed)")
    return result
}

The same measurement approach was used for both libraries with identical compiler/optimization settings (most likely unoptimized). Since this was a comparative test, I didn't think the actual settings would matter. I didn't use instruments or code coverage.

[oscbyspro]

Thanks! I'll think about it. I initially decided on doing it this way because I thought it would be much easier to say "you're using the wrong method" than it would be to explain why it is slow in some situations (the overload would not be available to a function generic over Swift.FixedWidthInteger, for example).

[mgriebling]

Is there a reason you didn't use Swift's FixedWidthInteger instead of rolling your own? This would be in place of your ANKFullWidth.

What special purpose does the ANKSigned provide. It seems the ANKFullWidth could be signed as well?
All these designations are a bit confusing since they don't really line up with Apple's fixed-width signed & unsigned integers.

[oscbyspro]

I thought adding single digit arithmetic sounded cool, and that required a new associated type, etc.

Edit: Yes, ANKFullWidth is signed if its High component is signed.

[mgriebling]

Yes, that is a nice feature. How about using Apple's as a baseline protocol and extending that?

Anyway, really nice package. I'll try it out in my current project since I really need larger integers than just UInt128.

[oscbyspro]

I’d consider it, if I were on Apple’s payroll. But I’m not, so I’m building something tailored to my needs.

Edit: cheers 🍻

[mgriebling]

No problem. Thanks for making this available.