This argument comes up against haskell over and over again and I think it is because it has some merit. Type level hackery and monad wizardry are great but it just feels like a whole bunch of smart people showing off about how clever they can be with avoiding state and pushing it all into some high level categorical constructs.
I've criticized Haskell many times, and being criticized many times of that.
You see, I would love to love Haskell but in practice the language just always turns out to be an overly intellectual exercise. Real world programs need to manage state, preferably in a very controlled way, but the bottom line is that they need to do that in some obvious and easy way.
It doesn't help if you explain me that you really don't need to explicitly store state anywhere, that you can just use monads to carry it over from computation to computation. Monads are an interesting abstraction per se but I don't want to use such heavy-weight artillery to make a mere assignment, to write a pointer to memory. I want my language to put some effort in protecting myself from the hazards of unintended concurrency but when I want to do it, the language must obey.
That said, Clojure is just perfect. For now, Clojure is my Blub.
You never want "to make a mere assignment", nor "to write a pointer to memory". These are not problems you want solved, they are parts of a possible solution.
Haskell don't need assignments nor memory writes. It solves problems (including state management) differently. You may not like that, but don't say that not using some specific low-level feature is bad by itself.
It doesn't help to explain state in a very sophisticated way because eventually a memory write is what you're after.
I'm programming on a real computer that has memory and a cpu and in order for theory and practice to meet, there must be in the language a concept of variables that somehow map to the concept of reading and writing memory. Clojure doesn't have memory access either but it has mutable containers with very controlled, concurrent accessors.
Assignments and memory writes are very much problems I want to be solved and I'm just unhappy because Haskell doesn't want to help me with that except by strictly with its own doctrine.
In Clojure, I can partition the program state into mutable vars and refs at appropriate levels, depending on the problem and my architecture. Most of my code is pure and in general, I don't use many of those mutable containers, but I do.
If Clojure didn't have a solution for that it would remain an academic toy language.
> It doesn't help to explain state in a very sophisticated way because eventually a memory write is what you're after.
I'm not sure about memory writes, but people who adopt this 'realistic' view that 'look it's a machine, it's not math', are often in a state of confusion about the workings of their own compilers.
I saw this in action recently when I was fiddling semi-competently with the 'threadring' benchmark (http://shootout.alioth.debian.org/u64q/performance.php?test=...). What the requisite program is actually doing is to calculate a certain pure function of input (something like `x mod 503 - 1`). This would be lightning fast as ... well:
main = head <$> getArgs >>= print . f
where f n = n `mod` 503 - 1
But the terms of the benchmark are that you do this in an amusing way where the steps involve passing news from one thread to another in a ring. In Haskell this is done with `forkIO` and that sort of thing.
When compiled with any of GHC's normal flags, my attempts would chug along on big inputs evidently actually doing what the terms of the benchmark required: 503 threads were passing a hot potato around in my cpu.
When I compiled with the 'via C' flag, by contrast, it was lightning fast, a little slower than the pure function mentioned above.
The reason was plain: the C compiler recognized that I was writing a pure function and optimized away the ridiculous division of threads that was imposed by the terms of the benchmark. I'm pretty sure the 'winning' C program is employing this advantage though the program text is legit, but I don't have the C chops to prove it. (In any case the benchmark seems to have been deprecated, rightly -- wish I'd realized that before spending several hours working on it!)
MORAL: if you really want forking processes, memory reads etc. -- if you really want to "get your hands on the real machine with real CPU" -- then .... use GHC, and tell it that's what you want.
Otherwise, you're only imagining it -- and maybe your compiler is doing it, maybe it isn't; it's none of your business.
> the benchmark seems to have been deprecated, rightly
Well, not actually deprecated as such - but neither thread-ring nor chameneos-redux are included in the which-programming-languages-are-fastest summary.
You're tying up the concept of computation with the implementation of computation that we commonly see (the Von Neumann architecture). Yes, whatever code you write in whatever language will eventually compile down to something that changes bytes in memory. But the way that you express that computation is not required to have such a concept.
It sounds like Haskell just isn't your style, but you're saying that Haskell is bad for "real-world" programming. And it sounds like you've never really used Haskell seriously (just a guess, most Haskellers I've met get over their monad fixation within a few months, yet you describe them as "artillery"). If you like Clojure so much, why not go champion that instead of complaining about Haskell?
Besides, if you use "how easy is it to implement 'memoize'" as a benchmark then you'll always end up choosing a Lisp.
It's not the style; it's the real world where Haskell and me have never got together very well.
I would like to write in the concise, immutable, lazy, functional, and strongly typed style of Haskell and that's fine with (nontrivial) toy programs and interesting algorithm implementations, but it's the real world requirements and state management where I've always found Haskell unnecessarily burdensome to deal with.
My benchmark for any language is, as a visual person, to first get stuff on the screen and then get it moving. With Haskell I always end up writing more code to deal with the state than I would like to. And by judging other people's code to do similar things (like a vector graphics on the screen in a game or demo) it doesn't seem to get too pretty there either.
Haskell would be a more practical and smoother language by a tenfold if it just went ahead and accepted that I want to do impure things occasionally, and gave me pragmatic tools to do that. Don't say monads, they feel phony given the simplicity of what I need from them. I chose the word artillery because monads are a Goldbergesque way to achieve writing a pointer to some place in memory.
FYI, I do champion Clojure much more than complain about Haskell, but I occasionally do the latter as well in order to reconsider my criticism. And I retry with Haskell from time to time, to see if the feel of the language would finally make me overcome the resistance.
I've spent the last 15 years programming professionally various imperative and object-oriented languages.
I think you're being too harsh on Haskell here. Certainly if you just want to quickly hack something together than Haskell is not optimal. Similarly, if you just want to create a web page that has one dynamic element in it, then a web framework is overkill and you might as well just use an inline PHP script.
But the real challenges in programming are always managing complexity over time. I am far from a Haskell expert, but I am definitely an expert in many types of bugs that programmers experience in the real world. And I feel pretty confident saying that at least 90% of bugs that crop up in the real world programs are due to a mismanagement of state. Different people have developed different techniques to combat this, but pure functional programming is by far the most powerful. The TDD/BDD community often touts testing as a guarantor of quality, but as useful as testing is, it pales in comparison to the assurances of mathematical provable functions.
With Haskell you spend more time developing, but you get the greatest possible assurances that A) your code is bug free and B) it will remain bug free even as the program as a whole changes.
It's absolutely true that most development these days is low-budget high-priority stuff where time to market trumps long-term maintainability. However to suggest that Haskell--the strongest bastion of functional purity, and one of the few places where the mathematics behind computer science are being advanced--should water itself down to be more like 1000 other programming languages designed for getting things done™ is just plain silly. If you don't have use for Haskell's purity than don't use it.
Supporting Haskell and complaining about Haskell are both valid activities. There's plenty to complain about Clojure. However, the post does point out quite succinctly that while Haskell features might be incredible for simplifying many kinds of problems, it's shocking how many simple problems become staggeringly complex in Haskell.
"while $Language features might be incredible for simplifying many kinds of problems, it's shocking how many simple problems become staggeringly complex in $Language."
Replace "$Language" with almost any computer language in existence and that statement is true (given appropriately chosen values of "simple problems" and "staggeringly complex").
So it reduces to, "For any given language some things are simple and others complex. For other languages what is simple and complex differ".
> So it reduces to, "For any given language some things are simple and others complex. For other languages what is simple and complex differ".
Though that might be the case, what doesn't change with languages is what users typically consider as simple problems. So "simple problems" are specific to user expectations, not language features. So I think the parent's point still stands.
"what doesn't change with languages is what users typically consider as simple problems.".
If you meant users of the language (vs users of an app built with the langauge) you are wrong.
Consider Text Processing (awk vs Java , even though Java has Text processing libraries), Distributed Computation (Erlang vs C - though you can do DC with C ) , "close to the metal" coding (C vs Scheme), Continuation Passing Style (again C vs Scheme, with a different "winner"), Array processing (J vs Java), statistical code (R vs C++).
What users consider "simple problems" does change with the language used, contra what you said above.
And just like if you start with "Continuation Passing Style is easy" as a criterion , Scheme beats C, or "customized memory management" makes C win over Scheme, if you use "ease of memoizing a function" as a selection criterion you'll end up with a Lisp (over Haskell or C or Java).
Change the criterion and a different language rises to the top. As I said earlier this isn't saying much.
The point was, to make any language "win", just choose one of its strengths as a deciding criterion and take another language without that feature as a punching bag. So to make Haskell "win", All I have to do is state "monadic programming should be easy" or "effectful programming should be clearly distinguished from the pure functional core by the type system." or "Using typeclasses to structure my code should be trivial"
And Haskell "wins" by default over other languages.
That said, fwiw, in my experience the "sweet spot" of Haskell is in large + heavily algorithmic + complex code bases, where its powerful type system shines. It is very fast without sacrificing abstraction. I've built some large machine learning systems with it and I couldn't be happier.
I know other people who are building trading systems in it and they seem delighted too.
As I said, just my experience and I making no claim about how other languages are losers somehow because they have different goals.
This is what I use it for. My program ends up being as long as Perl, but as fast as C. For me, that's the best of both worlds.
For gluing libraries together, it's not the best, but in the last few months it's gotten quite good. There is hardly anything I want to do that would involve reinventing any wheels that I already have in $other_language.
"I've built some large machine learning systems with it and I couldn't be happier."
Does Haskell have good matrix libraries? That seems to be at the core of a lot of machine learning tasks, and when I briefly checked a few years back, I didn't see any Haskell native matrix libraries. I think there were LAPACK bindings, maybe.
> This argument comes up against haskell over and over again and I think it is because it has some merit.
Yes and no. As with many other things, there's a tradeoff between Haskell's purity and referential transparency and side-effecting constructs such as storing stuff in a local cache, which is what memoization does. You can't litter printfs or cache calls throughout your pure functions either, "for some reason". It makes sense that's it's a pain to do in a (non-IO/state) haskell function, because those are very much about mixing side-effects into the code.
A better solution would probably be to integrate memoization at the runtime level, with some compiler hints maybe. That way it becomes an implementation detail not an imperative side-effect, and the world is pure again.
Anyway yeah, if you can't bear with Haskell's constraints you can just use something else, such as MLs (in order not to give up static type safety). Whoop the fucking do.
You can litter printfs in your code, you'll just get a lot of hate if you do it in library code that you share. It's fine when debugging to write a "debug printf" function using "unsafePerformIO". Yes, it mixes side effects into pure code, but that's fine for debugging purposes.
-- totally OK! go ahead! it won't crash or anything!
debugPrint :: String -> a -> a
debugPrint s x = unsafePerformIO (putStrLn s >> return x)
You could also implement a memoization function using "unsafePerformIO". No, you won't get a visit from the FP purity police for doing so, it's perfectly fine. Why? Using "unsafePerformIO" in a function does NOT mean that that function is impure. It merely means that the compiler will not inform you whether the function is pure or impure -- the compiler's safety checks are bypassed when you use the function.
Here's how I'd type the memoize function:
memoize :: Ord a => (a -> b) -> IO (a -> b)
Notice how it gives you a pure function out. No need to do stuff at the runtime level, or write compiler hints. It just works. If you want to memoize some nasty function:
-- Totally fine! Unless nastyFunction is polymorphic...
nastyFunction :: Paper -> PublishedBook
nastyFunctionMemoized = unsafePerformIO (memoize nastyFunction)
If you're new to Haskell, don't write memoize yourself if you want a version that works with polymorphic functions as that requires some more advanced techniques. Most people who need a memoized function don't need a polymorphic one though.
> You can litter printfs in your code, you'll just get a lot of hate if you do it in library code that you share. It's fine when debugging to write a "debug printf" function using "unsafePerformIO".
Well obviously but I considered we were ignoring such "hacks" as unsafePerformIO. If you introduce it, then all of that is trivial to do so the discussion has no reason to be.
The function "unsafePerformIO" is not a hack, it is the tool that allows you to write safe functions with the correct, pure type signature even though the compiler cannot prove that the function is safe.
For example, suppose you write "memoize" without using "unsafePerformIO". Here's the resulting type signature:
badMemoize :: Ord a => (a -> b) -> IO (a -> IO b)
The resulting output has type "a -> IO b", but it doesn't actually do IO and it has no side effects! It's a normal, pure, referentially transparent function, and it shouldn't be in the IO Monad. The correct signature is "a -> b", just like the input. The "unsafePerformIO" function is a non-hack used by careful programmers to expose pure interfaces to code that uses impure constructs.
See the FFI tutorials for more examples of using unsafePerformIO.
(Of course, unsafe does mean unsafe... you can easily cause segfaults with that thing, you have to use it correctly.)
> The function "unsafePerformIO" is not a hack, it is the tool that allows you to write safe functions with the correct, pure type signature even though the compiler cannot prove that the function is safe.
It is a hack, since it subverts the type system. It allows you to write safe functions with a pure (and potentially lying) type signature, but it also has no problem letting you create completely unsafe and broken functions.
> The resulting output has type "a -> IO b", but it doesn't actually do IO and it has no side effects!
Of course it does in its implementation, you do have to fill the memoization cache on a miss don't you?
> The "unsafePerformIO" function is a non-hack used by careful programmers to expose pure interfaces to code that uses impure constructs.
I don't agree with the first part of your declaration. At all.
> Of course, unsafe does mean unsafe... you can easily cause segfaults with that thing, you have to use it correctly.
And that's why it's a hack, just as using bit shifting instead of multiplication or division is a hack as it relies on the implementation details of the language and the hardware infrastructure. Yeah careful programmers can use it to wring out the latest drop of performance out of their software (assuming there is no optimizing compiler on the platform), but it's still a hack.
It seems to me that there is an element of hypocrisy in some of the complaints above; it is as if they were saying:
> In Haskell, it's a nightmare to define a pure function with IO. You have to use that dreadful hack `unsafePerformIO`! In practical languages, by contrast, we are spared this boilerplate, since basically every line is already unsafePerformIO.
> It seems to me that there is an element of hypocrisy in some of the complaints above; it is as if they were saying
Apart from the "nightmare" part, this statement you made up happens to be factually true. What is your issue with it? It's exactly what happens: in most other langages, every line is indeed unsafePerformIO. Isn't that fact one of the reasons Haskellers give for using haskell in the first place?
We use Haskell so that 99% of the time, the compiler can prove our program safe. Then, the other 1% of the time, we use our human intuition to decide to "gamble" and disagree with the compiler.
In most languages, 100% of your code is that same gamble.
let x = [1..]
in do print (x !! 1000000)
print (x !! 1000000)
Element #1000000 (counting from zero) of the list "x" is printed out twice. The first time, "x" is a thunk. That thunk is evaluated, and the runtime stores the resulting value back into "x". The second time, "x" is a constructor, so it does not need to be evaluated. You'll see the memory usage spike after the first "print" to account for the million cons cells allocated.
This is no different than the interface provided by the memoization function. What distinguishes pure from impure is not memory access -- memory MUST always be modified in order to perform calculation. I recommend reading the Haskell report section on semantics.
AFAICS, there is no unsafePerformIO in any of the above except for stringtable-atom.
AIUI, it's possible to implement generalized memoization in haskell w/o unsafePerformIO, and both memocombinators and MemTrie do so. I take your point that if necessary, they could have been written to use unsafePerformIO, and as long as the library authors know what they're doing, and are careful, that would be OK.
So, you don't actually need to read CS papers to do memoization in haskell. You just need to know that the libraries that do it have "memo", not "memoize" in their names. This stuff, could, perhaps be easier to find.
You are repeating what I said. I was defending the use of `unsafePerformIO` as a non-'hack', and perhaps I misunderstood you to be thinking otherwise. The memocombinators library uses Array, which uses `unsafePerformIO` and friends almost 700 times in one module. MemTrie takes a different approach, one not using unsafePerformIO; maybe I wasn't clear that was my meaning. In any case, there is little need for a library, at least for the simplest idea of memoization (the ugly Reddit discussion suggests there are many things to be meant.) It is just a case of writing down what is meant, as happens a few times in slightly different ways on the reddit page, off the top of peoples' heads. So for example this one:
memo :: Eq k => (k -> v) -> k -> v
memo f = unsafePerformIO $ do
ref <- newIORef [] :: IO (IORef [(k,v)]) -- make a page for a table
return $ \k -> unsafePerformIO $ do -- when f is applied to k
memotable <- readIORef ref -- get out the table
case lookup k memotable of -- see if k has been done
Just x -> return x -- if it has, we're done!
Nothing -> do -- if it hasn't
let a = f k -- calculate f k
atomicModifyIORef ref $ \t -> ((k,a):t,()) -- add it to the table before
return a -- returning f k
This is someone's brief treatment, which meets most of the definitions people were using. The clojure procedure is more powerful; as far as I can see, getting something similar in Haskell requires a bit more subtlety.
The description you give of "type-level hackery" and "monad wizardry" leads me to believe that you don't really understand what's going on. This is dangerous, because you are expecting these things to be complicated, but they are actually so simple that you've probably already invented them in your own computer programs. The Haskell community merely gives names to ideas you already know about.
This isn't "smart people showing off about how clever they can be", it's people giving names to things that they use everyday.
The problem people have with monads is that the word "Monad" does not describe something they're used to dealing with. A computer programmer is used to concrete things like lists, trees, functions, compilers; things you could hold in your hand (or at least draw on a whiteboard), poke at, and observe.
Monads can't be understood in this way, because they are not anything concrete; "Monad" is a word to describe concrete things that, individually, are all very different. It's like explaining the concept of "Green" to someone by showing them an apple, a tennis ball, and some grass clippings. They will never understand "Green", because you can't understand what Green is by seeing three examples of them. Green is a color, not some sort of weird tennis-ball-apple-grass-clipping hybrid.
Monads are the same way. List is a Monad. Reader is a Monad. State is a Monad. But when you look at these things and try to generalize, you might not see the parts that are the same, because the parts that are the same that make them all monads are hard to see.
I think the hard part is that you hear about what defines a monad ("return" and "bind" or "return", "fmap", and "join"), and don't see how some dual or triple like that helps you write computer programs. Each set of functions is so different from the other sets that the fact that they are all called "return" and "bind" doesn't seem to matter. And that's exactly right; at some level, it doesn't matter.
You use a List as a monad to combine computations that return multiple values, and you use Cont to write your programs in a continuation-passing style. Very different. Nothing similar at all, really. On the concrete level.
The similarity does matter at a higher level, though; noticing similarities means you can write parts of your program in advance, and just tweak those parts for each specific case.
An example is liftM2; it abstracts away the tedium of using a calling a two-arg function with monad-like semantics. If there were no similarities between List and Maybe, you'd have to write something like this:
liftList2 f xs ys = concat $ (\x ->
concat $ (\y -> [f x y])
`map` ys)
`map` xs
-- this doesn't compile; I want to use more descriptive words than the Data.Maybe API does :)
liftMaybe2 f x y = if has x then extract x -> \x'
if has y then extract y -> \y'
makeMaybe f x' y'
else Nothing
else Nothing
(Here's how to use the functions, in case you aren't sure:
These seem very similar. There is the same sort of nesting in both structures, we seem to be saying the same thing twice in both functions. We seem to be applying functions to values "inside" each of these things.
Maybe we can abstract those things away somehow, and make the two functions look a little more similar:
newList x = [x]
fmapList xs f = f `map` xs --
joinList xs = concat xs
liftList2 f xs ys = joinList $ fmapList xs $ \x ->
joinList $ fmapList ys $ \y ->
newList $ f x y
-- hey, the code looks nicer already!
newMaybe x = Just x
fmapMaybe Nothing _ = Nothing
fmapMaybe (Just x) f = Just $ f x
joinMaybe Nothing = Nothing
joinMaybe (Just x) = x
liftMaybe2 f x y = joinMaybe $ fmapMaybe x $ \x' ->
joinMaybe $ fmapMaybe y $ \y' ->
newMaybe $ f x' y'
Now that we've given List and Maybe some common terminology, it's very hard not to see how these lift2 functions are nearly identical. We just need to do some refactoring, and then we can use the same definition for both!
class Monad m where
new :: a -> m a
fmap :: m a -> (a -> b) -> m b
join :: m (m a) -> m a
instance Monad [] where -- haskell98 spells List as []
new = newList
fmap = fmapList
join = joinList
instance Monad Maybe where
new = newMaybe
fmap = fmapMaybe
join = joinMaybe
liftM2 f x y = join $ fmap x $ \x' ->
join $ fmap y $ \y' ->
new $ f x' y'
And there you go, that's what a Monad is. It's just refactoring, so we have less code to maintain! That's what any programmer would do, not just super-smart PhDs. You do the same thing every day!
Agreed that abstraction is itself the problem - very abstract things are more difficult to understand than concrete things, because there's nothing to grab onto.
I know you're trying, but you're going to lose the beginner as soon as you get to the code example - it's just too terse and foreign. Remember that people who don't know monads also don't understand Haskell very well, so they need to know your intentions before trying to read the code. You have to explain this in English.
First you'd have to explain what the comma function does (creates a pair), what liftList2 does (takes the cartesian product and applies an operator to each pair - oops, that assumes the reader knows what cartesian product means!) and what the liftMaybe2 does (applies a binary operator if both arguments are not null - er, I mean Nothing).
In Java all these methods would have JavaDoc, so you have some clue what the author intends before reading the code. I think readable Haskell would probably need five times as many lines of comments as actual code since so many functions are one-liners.
It's much like math: if you really want to explain things so a beginner will get it, you need half a page of text and maybe some diagrams per equation.
Agreed. I didn't want to spend too much time on this, since sigfpe already beat me to "you invented mondas" by like 8 years :)
I think the key point I missed is explaining why a sequence like this is join . fmap . return . f. Mapping over a list makes sense, but would would I pick that abstraction for null/not null? Why "join"? Why not if/then?
But on the other hand, I think the main idea, "it's just programming", sticks.
One day, a Haskell programmer was writing some code. He wrote code that felt similar, and he looked for the similarities. He noticed, "hey, these are both containers", and "oh look, i'm applying a function inside the container", and "hmm, i always have to remove a level of nesting". Some thinking later, and the return/fmap/join abstraction appeared, and with even more thinking, the monad.
This is how programmers should always approach programming. Not just "genius PhD students". Just because some PHP programmer doesn't do this much thinking for his login page, doesn't mean you shouldn't.
I have a degree in mathematics so don't try to explain to me what abstract is. I'm no stranger to chasing diagrams and figuring out if something is a natural transformation. The problem is that haskell overlays all this heavy duty math on top of state computations and hopes that by throwing enough examples at the reader, kinda like you did, the reader will figure out that "return", "join", etc. are very special kinds of natural transformations. The fact remains that you are using categorical constructs to program and to me this is just a horrible idea because it confounds a whole bunch of ideas and is basically using a hammer to kill a fly. Just because every concept in programming can be coerced into being a monad doesn't mean it should and that's exactly what haskell does and don't even get me started on arrows. Semantic distinctions between concepts in programming are a good thing because they show beginners boundaries and allow them to apply the concept and gain mastery. There is such a thing as too much abstraction when you are trying to be practical because after all everything in mathematics is basically some kind of category but when you are integrating a function you are not thinking of limits and colimits. Haskell doesn't have many semantic distinctions between it's underlying concepts. Everything is a function, a type or a type constructor and that's it. Sure it's theoretically very beautiful but it's not very practical. There is no way a language with such constructs is going to gain any kind of followers other than people who already have a PhD and eat natural transformations for breakfast.
You didn't read my post. Monads are just refactoring with a funny name. That's it.
Unless the deepest abstraction you are capable of understanding is cut-n-paste, this is not particularly far-fetched. I don't have a PhD in math, and I don't have any trouble writing "real applications" in Haskell.
