firewars
2003-02-12, 07:24:27
Von HotHardware.com:
By now, many of you are probably scrambling to download a copy of Futuremark's (formerly MadOnion) newly released 3DMark2003 DirectX benchmark. For about two years now, 3DMark2001 has been a tool almost universally used to gauge the DirectX video performance of system. This latest release seems to have caused a bit of a stir over at NVIDIA. We got our hands on a document published by the folks at NVIDIA, that aims to discredit the validity of 3DMark2003 as a "gamer's benchmark"...
NVIDIA:
"3DMark03 combines custom artwork with a custom rendering engine that creates a set of demo scenes that, while pretty, have very little to do with actual games. It is much better termed a demo than a benchmark. The examples included in this report illustrate that 3DMark03 does not represent games, can never be used as a stand-in for games, and should not be used as a gamers’ benchmark."
The statement above is a direct quote taken from NVIDIA's report on 3DMark2003. This is not a "new" argument, however. We here at HotHardware get statements similar to this one sent to us after virtually every video card review we write. There were, are and always will be a group of people that put very little stock in synthetic benchmarks. Throughout the report, NVIDIA makes a case against all the "game" modules that comprise 3DMark2003...
NVIDIA:
"Unfortunately, Futuremark chose a flight simulation scene for this test (game 1). This genre of games is not only a small fraction of the game market (approximately 1%), but utilizes a simplistic rendering style common to this genre. Further, the specific scene chosen is a high altitude flight simulation, which is indicative of only a small fraction of that 1%."
"For all intents and purposes game tests 2 and 3 are the same test. They use the same rendering paths and the same feature set. The sole difference in these tests appears to be the artwork. This fact alone raises some questions about breadth of game genres addressed by 3DMark03. --- These two tests attempt to duplicate the “Z-first” rendering style used in the upcoming first-person shooter game, “Doom 3”. They have a “Doom-like” look, but use a bizarre rendering method that is far from Doom 3 or any other known game application."
"Finally, the choice of pixel shaders in game tests 2 and 3 is also odd. These tests use ps1.4 for all the pixel shaders in the scenes. Fallback versions of the pixel shaders are provided in ps1.1 for hardware that doesn’t support ps1.4. Conspicuously absent from these scenes, however, is any ps1.3 pixel shaders. Current DirectX 8.0 (DX8) games, such as Tiger Woods and Unreal Tournament 2003, all use ps1.1 and ps1.3 pixel shaders. Few, if any, are using ps1.4."
"This year’s 3DMark has a new nature scene (game 4). It is intended to represent the new DirectX 9.0 (DX9) applications targeted for release this year. The key issue with this game scene is that it is barely DX9."
As you can see, NVIDIA takes issue with almost every aspect of Futuremark's latest benchmark. Why would they do this? I'm sure time will tell. Unfortunately for us, we haven't spent enough time with 3DMark2003 to have a truly educated opinion. My hypothesis would be that NVIDIA's legacy hardware lacks true support for ps1.4, which means performance will suffer in this benchmark as ps1.1 will be used as a fallback version rather than ps1.3...the version most NVIDIA hardware supports. The GeForceFX seems to perform well in 3DMark2003, but until we have a card of our own to test we can't really say for sure. It's the GeForce 4 an older product lines that 3DMark2003 may shine a negatice light on. After reading this report, we asked the folks at ATi what they thought of 3DMark2003, and they seemed pleased overall with Futuremark's latest.
NVIDIA:
"So, where do you find a true gamers’ benchmark? How about running actual games? Most popular games include a benchmark mode for just this purpose. Doom3, Unreal Tournament 2003, and Serious Sam Second Encounter are all far better indicators of current and upcoming game performance."
Pretty strong words, huh? It seems to us that this is a bit of damage control on NVIDIA's part, but we're sure there are going to be some that will totally agree with NVIDIA's stance on this situation. Why wouldn't 3DMark2003 fallback to ps1.3, if support for ps1.4 wasn't present? Interesting question.
Nun, was meint ihr, woran es liegt?
By now, many of you are probably scrambling to download a copy of Futuremark's (formerly MadOnion) newly released 3DMark2003 DirectX benchmark. For about two years now, 3DMark2001 has been a tool almost universally used to gauge the DirectX video performance of system. This latest release seems to have caused a bit of a stir over at NVIDIA. We got our hands on a document published by the folks at NVIDIA, that aims to discredit the validity of 3DMark2003 as a "gamer's benchmark"...
NVIDIA:
"3DMark03 combines custom artwork with a custom rendering engine that creates a set of demo scenes that, while pretty, have very little to do with actual games. It is much better termed a demo than a benchmark. The examples included in this report illustrate that 3DMark03 does not represent games, can never be used as a stand-in for games, and should not be used as a gamers’ benchmark."
The statement above is a direct quote taken from NVIDIA's report on 3DMark2003. This is not a "new" argument, however. We here at HotHardware get statements similar to this one sent to us after virtually every video card review we write. There were, are and always will be a group of people that put very little stock in synthetic benchmarks. Throughout the report, NVIDIA makes a case against all the "game" modules that comprise 3DMark2003...
NVIDIA:
"Unfortunately, Futuremark chose a flight simulation scene for this test (game 1). This genre of games is not only a small fraction of the game market (approximately 1%), but utilizes a simplistic rendering style common to this genre. Further, the specific scene chosen is a high altitude flight simulation, which is indicative of only a small fraction of that 1%."
"For all intents and purposes game tests 2 and 3 are the same test. They use the same rendering paths and the same feature set. The sole difference in these tests appears to be the artwork. This fact alone raises some questions about breadth of game genres addressed by 3DMark03. --- These two tests attempt to duplicate the “Z-first” rendering style used in the upcoming first-person shooter game, “Doom 3”. They have a “Doom-like” look, but use a bizarre rendering method that is far from Doom 3 or any other known game application."
"Finally, the choice of pixel shaders in game tests 2 and 3 is also odd. These tests use ps1.4 for all the pixel shaders in the scenes. Fallback versions of the pixel shaders are provided in ps1.1 for hardware that doesn’t support ps1.4. Conspicuously absent from these scenes, however, is any ps1.3 pixel shaders. Current DirectX 8.0 (DX8) games, such as Tiger Woods and Unreal Tournament 2003, all use ps1.1 and ps1.3 pixel shaders. Few, if any, are using ps1.4."
"This year’s 3DMark has a new nature scene (game 4). It is intended to represent the new DirectX 9.0 (DX9) applications targeted for release this year. The key issue with this game scene is that it is barely DX9."
As you can see, NVIDIA takes issue with almost every aspect of Futuremark's latest benchmark. Why would they do this? I'm sure time will tell. Unfortunately for us, we haven't spent enough time with 3DMark2003 to have a truly educated opinion. My hypothesis would be that NVIDIA's legacy hardware lacks true support for ps1.4, which means performance will suffer in this benchmark as ps1.1 will be used as a fallback version rather than ps1.3...the version most NVIDIA hardware supports. The GeForceFX seems to perform well in 3DMark2003, but until we have a card of our own to test we can't really say for sure. It's the GeForce 4 an older product lines that 3DMark2003 may shine a negatice light on. After reading this report, we asked the folks at ATi what they thought of 3DMark2003, and they seemed pleased overall with Futuremark's latest.
NVIDIA:
"So, where do you find a true gamers’ benchmark? How about running actual games? Most popular games include a benchmark mode for just this purpose. Doom3, Unreal Tournament 2003, and Serious Sam Second Encounter are all far better indicators of current and upcoming game performance."
Pretty strong words, huh? It seems to us that this is a bit of damage control on NVIDIA's part, but we're sure there are going to be some that will totally agree with NVIDIA's stance on this situation. Why wouldn't 3DMark2003 fallback to ps1.3, if support for ps1.4 wasn't present? Interesting question.
Nun, was meint ihr, woran es liegt?