If the selected floating-point hardware includes the NEON
extension (e.g. -mfpu=‘neon’), note that floating-point
operations are not generated by GCC's auto-vectorization
pass unless -funsafe-math-optimizations is also specified.
This is because NEON hardware does not fully implement the
IEEE 754 standard for floating-point arithmetic (in
particular denormal values are treated as zero), so the use
of NEON instructions may lead to a loss of precision.
This is a pretty interesting email chain and a fairly good case study for those of us who aim for zealous adherence to standards and rules vs. providing a practical and useful output for users. I know I tend to go zealous by default and often have to remind myself that I'm not just writing software to satisfy my own academic ego.
Here's an interesting anecdote for -ffast-math: the Kahan summation algorithm [1] can be used to reduce numerical errors in sums.
Now have a look at the codegen [2] without and with -ffast-math.
With -ffast-math the compiler rewrites the Kahan summation algorithm to a simple sum, completely screwing the code's semantic (since we told the compiler to do so!). Keep this in mind.
Me too. Worse, at least for as recent of clang and GCC that I've tested, it doesn't warn when you do things that -ffinite-math-only makes tautological. So "if (x!=x)" happily compiles to nothing without warning when x is a float. (You get a warning if using integers)
This is an odd statement. I'm not aware of Linus ever being contracted to port video games to Linux. The Quake 3 Linux port was originally worked on by Loki Software, and then when they went out of business it was handled by Timothee Besset.
>Quake 3 port to linux in 1996? Can this sentence be read otherwise? Did he possibly mean Quake 1?
The mail is from July 2001. Q3 was released in early 1999, that's already 2 years and 3 months.
Now, Q2 was released in 1997. Q3 could have started in development immediately afterwards, so that pushes it to 4 years.
Not that much of a stretch to call them 'five years', especially casually speaking. And if you're a little older (over 30) it's even easier to have a fuzzier feeling for time (for a 20 year old, 5 years is 25% of their life -- for a 30 year old just 16%, and even whole decades start to have less defined edges, especially combined work/family routine).
I'm pretty sure it was a typo, Quake 3 didn't have a software renderer, so I wouldn't think they would get a doubling of framerate from that optimization (unless they were artificially lowering res to make things CPU bound), but Quake 2 did, so it could.
> I'm guessing he meant Quake 2. He mentions x86 and [DEC] Alpha. Did a Q3 port to the Alpha even exist?
I can't seem to find any evidence of one, so I am thinking it must be Q2, which was ported to Linux and NT Alpha. Jeez, just seeing that gave me a pang of nostalgia; I never got to work on an Alpha machine (spent plenty of time on SPARC though) but I always wanted to. I remember people being really excited about Alpha, thinking it was The Future™.
That was back in the days where it was hard to imagine that in 20 years the vast majority of computers would be either running on 8086 or Acorn descendants...
Maybe instead of --fast-math we need a new type aprox_float to indicate, we (the programmes) allow this variable to take some shortcuts during computation. While we promise not to do things like div0.
Edit: we must agree on a max deviation/error for this type.
If you can figure out a sensible way to formally specify what you want, that would be a good idea. But that gets very complex very quickly. E.g. if you allow the compiler to compute any algebraically equivalent expression in place of the expression you wrote, it can just add FLOAT_MAX and then subtract FLOAT_MAX (which mathematically should make no difference, right?), and then if it knows the input was reasonably small then it can optimize the result to 0.
That's actually how GPU shader languages work, albeit in reverse. HLSL/GLSL always allow floating point shortcuts unless they could affect the value of a variable declared with the "precise" qualifier.
"Big companies that have billion-dollar fabs spend time optimizing their chips that take several years to design for _games_. Not for IEEE traditional Fortran-kind math."
Then again, GPUs now do strictly follow IEEE and certainly not for games sake. That didnt use to be the case.
In fact they even support denormal numbers at full speed (not even Intel does that).
How so? Mobile chips are based on the same architecture, aren't they?
They often lag behind or are cut down (fewer pipelines, lower clocks) for lower power consumption, but they aren't entirely different designs as far as I can tell.
Mobile chips are whatever architecture the manufacturer has implemented underneath. There's a bunch of them - Mali, Broadcom, IMX et al - and they're famous for ignoring the industry standards. Just how badly is something you find out for yourself when trying to write GL applications on embedded devices.
I mean mobile as in phones/tablets , a market dominated by arm,Qualcomm and imagination technology, with mobile-only designs. Nvidia has very little market share there.
In my experience, -ffast-math has become vastly more useful over the last 10-15 years. I suppose this was a somewhat reasonable discussion to have in 2001, when most performance critical code still heavily relied on hand-optimized assembly, but nowadays -ffast-math is often the only reasonably convenient way to get compilers to autovectorize code properly, use FMA instructions, etc. I have production code that literally runs 4x faster purely by adding -ffast-math due to autovectorization (and subsequent ILP and other improvements).
The point about memory bandwidth and general algorithmic factors outweighing raw FPU performance holds true with pretty much all software even today. (And if you genuinely need raw FPU performance that badly then you're already running that FPU crunching on a GPU.)
More about data hazard. You can arrange your access pattern pretty well, but fast-math helps to avoid data dependency therefore better instruction level parallelism and pipelining.
Linus is very pragmatic in this exchange, in fact, hard numerical code, as the other person is implying in the discussion, is normally not following IEEE. Especially around zero. If your code is losing precision when you divide by something very big resulting in a zero value, you do not want to rise an exception, you just want to have zero.
Numerical code is normally not following the full IEEE and ignoring the underflow and inexact exceptions.
IEEE is a lot about tracking precision loss but in numerical code, precision loss is acceptable and needed (for speed) in many situations. But knowledge of the precision loss is for some people very important, this is the raison d'être of the IEEE standard.
The IEEE standard for floating point numbers has certain mathematical requirements that makes it a bit more "correct" but possibly also a bit slower. What Linus is saying [1] is that you don't need that in most applications [2] and definitely not when you specify -ffast-math
