Geekbench AI

[Darkman.X]

Geekbench hat einen weiteren Benchmark veröffentlicht:

Geekbench AI

Es nutzt die Frameworks "ONNX" (CPU und DirectML), "OpenVINO" und "TensorFlow Lite" (soll ein herstellerspezifisches Framework für z.B. "Samsung ENN", "ArmNN" und "Qualcomm QNN" sein).
(Ich habe ehrlich gesagt keine Ahnung davon, die Infos stammen vom Blog, Link siehe weiter unten.)

Wenn man sich die GUI anschaut, dann scheint es eine Weiterentwicklung vom "Geekbench ML 0.6.0 for Windows (https://www.forum-3dcenter.org/vbulletin/showthread.php?t=617789)" zu sein, mit mehr unterstützen Frameworks.
Ich habe aber keine Ahnung welches GPU-Backend auf Nicht-Windows-Betriebssystemen genutzt wird, falls überhaupt eine Alternative zu DirectML implementiert wurde.

Der Blog-Eintrag zum neuen Benchmark:
https://www.geekbench.com/blog/2024/08/geekbench-ai/

Download:
https://www.geekbench.com/ai/download/
(macOS, Windows, Linux, Android, iOS)

-------------------------------------------------------------------------------------

Hier meine Werte.

OS: Windows 11 23H2
CPU: Intel Core i9-14900K mit Intel-Performance-Profil (253 W & 307 A)
GPU: MSI GeForce RTX 4090 Gaming X Trio @ Werks-OC

AI Framework: ONNX
AI Backend: CPU
Single Precision Score: 5150
Half Precision Score: 943
Quantized Score: 6006
https://browser.geekbench.com/ai/v1/7972

------

AI Framework: ONNX
AI Backend: DirectML
Single Precision Score: 37843
Half Precision Score: 52682
Quantized Score: 27254
https://browser.geekbench.com/ai/v1/7994

------

AI Framework: OpenVINO
AI Backend: CPU
Single Precision Score: 5689
Half Precision Score: 5808
Quantized Score: 16374
https://browser.geekbench.com/ai/v1/8016

------

Hier nochmal als Tabelle. Ich kann mich nicht entscheiden welche Darstellung mir mehr gefällt :(

|ONNX - CPU (https://browser.geekbench.com/ai/v1/7972)|ONNX - DirectML (https://browser.geekbench.com/ai/v1/7994)|OpenVINO - CPU (https://browser.geekbench.com/ai/v1/8016)
Single Precision Score|5150|37843|5689
Half Precision Score|943|52682|5808
Quantized Score|6006|27254|16374
.

[Darkman.X]

Und hier die Werte vom Smartphone.

Modell: Samsung Galaxy S21 5G
OS: Android 14
CPU: Samsung Exynos 2100

AI Framework: TensorFlow Lite
AI Backend: CPU
Single Precision Score: 841
Half Precision Score: 826
Quantized Score: 1488
https://browser.geekbench.com/ai/v1/13680

------

AI Framework: TensorFlow Lite
AI Backend: GPU
Single Precision Score: 414
Half Precision Score: 594
Quantized Score: 650
https://browser.geekbench.com/ai/v1/14047

------

AI Framework: TensorFlow Lite
AI Backend: NNAPI
Single Precision Score: 431
Half Precision Score: 551
Quantized Score: 667
https://browser.geekbench.com/ai/v1/13844

------

Und nochmal als Tabelle:

|TensorFlow Lite - CPU (https://browser.geekbench.com/ai/v1/13680)|TensorFlow Lite - GPU (https://browser.geekbench.com/ai/v1/14047)|TensorFlow Lite - NNAPI (https://browser.geekbench.com/ai/v1/13844)
Single Precision Score|841|414|431
Half Precision Score|826|594|551
Quantized Score|1488|650|667

[misterh]

OS: Windows 11 PRO
CPU: Intel Core i7-12700K @ 5,1/4,1/4,4
GPU: RTX 4090 (STD)

AI Framework: ONNX
AI Backend: CPU
Single Precision Score: 4217
Half Precision Score: 956
Quantized Score: 5929
https://browser.geekbench.com/ai/v1/9171

AI Framework: ONNX
AI Backend: DirectML
Single Precision Score: 35633
Half Precision Score: 48239
Quantized Score: 26716
https://browser.geekbench.com/ai/v1/9194

AI Framework: OpenVINO
AI Backend: CPU
Single Precision Score: 5964
Half Precision Score: 6005
Quantized Score: 14418
https://browser.geekbench.com/ai/v1/9205

[Nuon]

OS: Windows 11 home
CPU: AMD Ryzen 9 5900X @ 87W
GPU: RTX 4080 @ PT 70%

AI Framework: ONNX
AI Backend: CPU
Single Precision Score: 3279
Half Precision Score: 799
Quantized Score: 3522
https://browser.geekbench.com/ai/v1/9714

AI Framework: ONNX
AI Backend: DirectML
Single Precision Score: 31502
Half Precision Score: 45687
Quantized Score: 23826
https://browser.geekbench.com/ai/v1/9747

AI Framework: OpenVINO
AI Backend: CPU
Single Precision Score: 4218
Half Precision Score: 4266
Quantized Score: 6837
https://browser.geekbench.com/ai/v1/9764

[Atma]

Die CPU Werte sind irgendwie furchtbar inkonsistent.

OS: Windows 11 Pro
CPU: i9-14900KS @ 5,5/4,5/5,0 GHz
GPU: GeForce RTX 4090 @ 2700/1500 MHz

AI Framewok: ONNX
AI Backend: CPU
Single Precision Score: 5412
Half Precision Score: 964
Quantized Score: 6236

https://browser.geekbench.com/ai/v1/12414

AI Framewok: OpenVINO
AI Backend: CPU
Single Precision Score: 6097
Half Precision Score: 6116
Quantized Score: 15801

https://browser.geekbench.com/ai/v1/12486

AI Framewok: ONNX
AI Backend: DirectML
Single Precision Score: 36129
Half Precision Score: 48866
Quantized Score: 26589

https://browser.geekbench.com/ai/v1/12144

[misterh]

OS: MacOS 15.0
CPU + GPU : M3 MAX 16" (16/40 + 48GB)

AI Framework: Core ML
AI Backend: CPU
Single Precision Score: 4831
Half Precision Score: 7536
Quantized Score: 6457
https://browser.geekbench.com/ai/v1/14931

AI Framework: Core ML
AI Backend: GPU
Single Precision Score: 17670
Half Precision Score: 19686
Quantized Score: 14892
https://browser.geekbench.com/ai/v1/14979

AI Framework: Core ML
AI Backend: Neutral Engine
Single Precision Score: 4764
Half Precision Score: 24117
Quantized Score: 25908
https://browser.geekbench.com/ai/v1/15567

[Man From Atlantis]

OS: Windows 11 Home 23H2
CPU: 5800X3D@CO -30, PPT/TDC/EDC 124/75/125,
DRAM: 4*8GB 4000MHz CL16
GPU: Palit RTX 4090 GameRock OC 3000/24000MHz at 1100mV

AI Framework: ONNX
AI Backend: CPU
Single Precision Score: 4352
Half Precision Score: 944
Quantized Score: 3987
https://browser.geekbench.com/ai/v1/16241

------

AI Framework: ONNX
AI Backend: DirectML
Single Precision Score: 37629
Half Precision Score: 51142
Quantized Score: 28079
https://browser.geekbench.com/ai/v1/16257

------

AI Framework: OpenVINO
AI Backend: CPU
Single Precision Score: 5724
Half Precision Score: 5794
Quantized Score: 8106
https://browser.geekbench.com/ai/v1/16275

[misterh]

OS: Windows 11 Home
CPU: Snapdragon X Elite - X1E78100

AI Framework: ONNX
AI Backend: CPU
Single Precision Score: 1437
Half Precision Score: 1391
Quantized Score: 3067
https://browser.geekbench.com/ai/v1/36101

AI Framework: ONNX
AI Backend: QNN
Single Precision Score: 1756
Half Precision Score: 10815
Quantized Score: 20301
https://browser.geekbench.com/ai/v1/36106

[Daredevil]

iOS + CoreML

CPU:
iPhone 15Pro: SP.3935-HP.6836-QS.5590
iPhone 16Pro: SP.4529-HP.7743-QS.6185

GPU:
iPhone 15Pro: SP.5002-HP.5922-QS.5155
iPhone 16Pro: SP.5756-HP.6585-QS.6055

Neural Engine:
iPhone 15Pro: SP.3950-HP.24408-QS.32234
iPhone 16Pro: SP.4493-HP.32210-QS.44479

[Daredevil]

Core Ultra 245K / P-Core 55x / E-Core 51x / Cache 42x / DDR7600Cl42

CPU OpenVINO: SP.5587-HP.5587-QS.13398
https://browser.geekbench.com/ai/v1/155854

iGPU OpenVINO: SP.5442-HP.8191-QS.12885
https://browser.geekbench.com/ai/v1/155860

NPU OpenVINO: SP.5542-HP.9007-QS.12857
https://browser.geekbench.com/ai/v1/155866